systemd split networkd package
cover systemd split networkd package
First of all, this patch series fixes a bug in systemd.bbclass that forces service files with an Also= relationship to be in the same package. This may often be the case, but as long as the RDEPENDS between the packages are set correctly, it is no problem to distribute the service files over several packages.
If service files referenced with Also= are no longer automatically added to the list of FILES, build errors caused by unpacked files are expected. A bitbake world has shown that there are only two recipes in oe-core that use automatic packing: busybox and dropbear. There are 2 patches that fix this.
Also= relation are probably most often used for sockets. Consequently the systemd.bbclass should also allow to split socket files into separate packages. However, that's proably practically not used and would cause tons of packaging errors. Therefore the automatic packaging feature for sockets is not changed.
The latest patch splits systemd-networkd into its own package. This adds a little extra complexity, but splitting systemd into packages is a good step towards making Yocto more usable in binary form.
Adrian Freihofer (5):
busybox: add klogd service to FILES
dropbear: add dropbearkey.service to FILES systemd.bbclass: refactor adding files systemd.bbclass: do not automatically add Also services
systemd: split networkd into its own package
meta/classes-recipe/systemd.bbclass | 28 ++++++++----------- meta/recipes-core/busybox/busybox.inc | 9 +++++- .../recipes-core/dropbear/dropbear_2024.86.bb | 2 ++ meta/recipes-core/systemd/systemd_256.8.bb | 27 ++++++++++++++++-- 4 files changed, 46 insertions(+), 20 deletions(-)
Signature · 1 line
2.47.0
PATCH 1/5 busybox: add klogd service to FILES
From: Adrian Freihofer <[email protected]>
busybox syslog.service file contains:
Also=busybox-klogd.service
That's why busybox-klog.service got automatically added to FILES:busybox-syslog. Since this incorrect auto adding feature gets dropped from systemd.bbclass this needs to be added explicitly.
--- meta/recipes-core/busybox/busybox.inc | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
meta/recipes-core/busybox/busybox.inc +8 -1
@@ -24,7 +24,14 @@ EXTRA_OEMAKE = "CC='${CC}' LD='${CCLD}' V=1 ARCH=${TARGET_ARCH} CROSS_COMPILE=${ PACKAGES =+ "${PN}-httpd ${PN}-udhcpd ${PN}-udhcpc ${PN}-syslog ${PN}-mdev ${PN}-hwclock" FILES:${PN}-httpd = "${sysconfdir}/init.d/busybox-httpd /srv/www" -FILES:${PN}-syslog = "${sysconfdir}/init.d/syslog* ${sysconfdir}/syslog-startup.conf* ${sysconfdir}/syslog.conf* ${systemd_system_unitdir}/syslog.service ${sysconfdir}/default/busybox-syslog" +FILES:${PN}-syslog = "\ + ${sysconfdir}/init.d/syslog* \ + ${sysconfdir}/syslog-startup.conf* \ + ${sysconfdir}/syslog.conf* \ + ${systemd_system_unitdir}/syslog.service \ + ${systemd_system_unitdir}/busybox-klogd.service \ + ${sysconfdir}/default/busybox-syslog \ +" FILES:${PN}-mdev = "${sysconfdir}/init.d/mdev ${sysconfdir}/mdev.conf ${sysconfdir}/mdev/*" FILES:${PN}-udhcpd = "${sysconfdir}/init.d/busybox-udhcpd" FILES:${PN}-udhcpc = "${sysconfdir}/udhcpc.d ${datadir}/udhcpc"
Signature · 1 line
2.47.0
PATCH 2/5 dropbear: add dropbearkey.service to FILES
In reply to Adrian FreihoferFrom: Adrian Freihofer <[email protected]>
dropbear.socket file contains:
Also=dropbearkey.service
That's why dropbearkey.service got automatically added to FILES. Since this incorrect auto adding feature gets dropped from systemd.bbclass this needs to be added explicitly.
--- meta/recipes-core/dropbear/dropbear_2024.86.bb | 2 ++ 1 file changed, 2 insertions(+)
meta/recipes-core/dropbear/dropbear_2024.86.bb +2 -0
@@ -129,3 +129,5 @@ pkg_postrm:${PN} () { } CONFFILES:${PN} = "${sysconfdir}/default/dropbear" + +FILES:${PN} += "${systemd_system_unitdir}/dropbearkey.service"
Signature · 1 line
2.47.0
PATCH 3/5 systemd.bbclass: refactor adding files
In reply to Adrian FreihoferFrom: Adrian Freihofer <[email protected]>
The keys variable was intended as an array of keys. But it looks like this has not been used for more than 10 years now. Adding files automatically to packages needs probably anyway very specific code rather than a generic loop. Lets simplify this a bit. Using python code should also not be slower for these usually small files.
--- meta/classes-recipe/systemd.bbclass | 28 ++++++++++++---------------- 1 file changed, 12 insertions(+), 16 deletions(-)
meta/classes-recipe/systemd.bbclass +12 -16
@@ -124,29 +124,26 @@ python systemd_populate_packages() { return appended # Add systemd files to FILES:*-systemd, parse for Also= and follow recursive - def systemd_add_files_and_parse(pkg_systemd, path, service, keys): + def systemd_add_files_and_parse(pkg_systemd, path, service): # avoid infinite recursion if systemd_append_file(pkg_systemd, oe.path.join(path, service)): fullpath = oe.path.join(d.getVar("D"), path, service) if service.find('.service') != -1: # for *.service add *@.service service_base = service.replace('.service', '') - systemd_add_files_and_parse(pkg_systemd, path, service_base + '@.service', keys) + systemd_add_files_and_parse(pkg_systemd, path, service_base + '@.service') if service.find('.socket') != -1: # for *.socket add *.service and *@.service service_base = service.replace('.socket', '') - systemd_add_files_and_parse(pkg_systemd, path, service_base + '.service', keys) - systemd_add_files_and_parse(pkg_systemd, path, service_base + '@.service', keys) - for key in keys.split(): - # recurse all dependencies found in keys ('Also';'Conflicts';..) and add to files - cmd = "grep %s %s | sed 's,%s=,,g' | tr ',' '\\n'" % (key, shlex.quote(fullpath), key) - pipe = os.popen(cmd, 'r') - line = pipe.readline() - while line: - line = line.replace('\n', '') - systemd_add_files_and_parse(pkg_systemd, path, line, keys) - line = pipe.readline() - pipe.close() + systemd_add_files_and_parse(pkg_systemd, path, service_base + '.service') + systemd_add_files_and_parse(pkg_systemd, path, service_base + '@.service') + # Add all units which have an Also= referring a unit in this package to this package as well. + with open(fullpath, 'r') as unit_f: + for line in unit_f: + if line.startswith('Also'): + also_unit = line.split('=', 1)[1].strip() + bb.warn("also: %s" % also_unit) + systemd_add_files_and_parse(pkg_systemd, path, also_unit) # Check service-files and call systemd_add_files_and_parse for each entry def systemd_check_services():
@@ -155,7 +152,6 @@ python systemd_populate_packages() { searchpaths.append(d.getVar("systemd_user_unitdir")) systemd_packages = d.getVar('SYSTEMD_PACKAGES') - keys = 'Also' # scan for all in SYSTEMD_SERVICE[] for pkg_systemd in systemd_packages.split(): for service in get_package_var(d, 'SYSTEMD_SERVICE', pkg_systemd).split():
@@ -179,7 +175,7 @@ python systemd_populate_packages() { break if path_found != '': - systemd_add_files_and_parse(pkg_systemd, path_found, service, keys) + systemd_add_files_and_parse(pkg_systemd, path_found, service) else: bb.fatal("Didn't find service unit '{0}', specified in SYSTEMD_SERVICE:{1}. {2}".format( service, pkg_systemd, "Also looked for service unit '{0}'.".format(base) if base is not None else ""))
Signature · 1 line
2.47.0
PATCH 4/5 systemd.bbclass: do not automatically add Also services
In reply to Adrian FreihoferFrom: Adrian Freihofer <[email protected]>
Starting a systemd service “Also” does not mean that both services must be in the same package. However, the systemd.bbclass enforces this.
Example:
a.service:
[Install]
Also=b.service
If a.service is packed in package A, b.service is automatically packed into package A as well. This happens even if b.service is explicitly added to package B using FILES and SYSTEMD_SERVICE variables.
The automatic packing of socket files with the corresponding service files is probably a widely used feature of systemd.bbclass. This bahavior does not change.
Adding regular service files to a package just because it is another service in the same package that "Also" uses the service is a bug that this commit fixes.
--- meta/classes-recipe/systemd.bbclass | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-)
meta/classes-recipe/systemd.bbclass +7 -7
@@ -132,18 +132,18 @@ python systemd_populate_packages() { # for *.service add *@.service service_base = service.replace('.service', '') systemd_add_files_and_parse(pkg_systemd, path, service_base + '@.service') + # Add the socket unit which is referred by the Also= in this service file to the same package. + with open(fullpath, 'r') as unit_f: + for line in unit_f: + if line.startswith('Also'): + also_unit = line.split('=', 1)[1].strip() + if also_unit.find('.socket') != -1: + systemd_add_files_and_parse(pkg_systemd, path, also_unit) if service.find('.socket') != -1: # for *.socket add *.service and *@.service service_base = service.replace('.socket', '') systemd_add_files_and_parse(pkg_systemd, path, service_base + '.service') systemd_add_files_and_parse(pkg_systemd, path, service_base + '@.service') - # Add all units which have an Also= referring a unit in this package to this package as well. - with open(fullpath, 'r') as unit_f: - for line in unit_f: - if line.startswith('Also'): - also_unit = line.split('=', 1)[1].strip() - bb.warn("also: %s" % also_unit) - systemd_add_files_and_parse(pkg_systemd, path, also_unit) # Check service-files and call systemd_add_files_and_parse for each entry def systemd_check_services():
Signature · 1 line
2.47.0
PATCH 5/5 systemd: split networkd into its own package
In reply to Adrian FreihoferFrom: Adrian Freihofer <[email protected]>
Support of images with different network management implementations without having to recompile systemd and other components.
Fedora does this as well since systemd version 246.6-2:
https://src.fedoraproject.org/rpms/systemd/blob/rawhide/f/systemd.spec#_578
This proves that it is technically possible and officially supported by systemd.
--- meta/recipes-core/systemd/systemd_256.8.bb | 27 +++++++++++++++++++--- 1 file changed, 24 insertions(+), 3 deletions(-)
meta/recipes-core/systemd/systemd_256.8.bb +24 -3
@@ -412,6 +412,7 @@ PACKAGE_BEFORE_PN = "\ ${PN}-journal-remote \ ${PN}-kernel-install \ ${PN}-mime \ + ${PN}-networkd \ ${PN}-rpm-macros \ ${PN}-udev-rules \ ${PN}-vconsole-setup \
@@ -440,6 +441,7 @@ SYSTEMD_PACKAGES = "${@bb.utils.contains('PACKAGECONFIG', 'binfmt', '${PN}-binfm ${@bb.utils.contains('PACKAGECONFIG', 'microhttpd', '${PN}-journal-gatewayd', '', d)} \ ${@bb.utils.contains('PACKAGECONFIG', 'microhttpd', '${PN}-journal-remote', '', d)} \ ${@bb.utils.contains('PACKAGECONFIG', 'journal-upload', '${PN}-journal-upload', '', d)} \ + ${@bb.utils.contains('PACKAGECONFIG', 'networkd', '${PN}-networkd', '', d)} \ " SYSTEMD_SERVICE:${PN}-binfmt = "systemd-binfmt.service"
@@ -453,7 +455,7 @@ GROUPADD_PARAM:${PN} = "-r systemd-journal;" GROUPADD_PARAM:udev = "-r render" GROUPADD_PARAM:${PN} += "${@bb.utils.contains('PACKAGECONFIG', 'polkit_hostnamed_fallback', '-r systemd-hostname;', '', d)}" USERADD_PARAM:${PN} += "${@bb.utils.contains('PACKAGECONFIG', 'coredump', '--system -d / -M --shell /sbin/nologin systemd-coredump;', '', d)}" -USERADD_PARAM:${PN} += "${@bb.utils.contains('PACKAGECONFIG', 'networkd', '--system -d / -M --shell /sbin/nologin systemd-network;', '', d)}" +USERADD_PARAM:${PN}-networkd = "--system -d / -M --shell /sbin/nologin systemd-network" USERADD_PARAM:${PN} += "${@bb.utils.contains('PACKAGECONFIG', 'polkit', '--system --no-create-home --user-group --home-dir ${datadir}/polkit-1 polkitd;', '', d)}" USERADD_PARAM:${PN} += "${@bb.utils.contains('PACKAGECONFIG', 'resolved', '--system -d / -M --shell /sbin/nologin systemd-resolve;', '', d)}" USERADD_PARAM:${PN} += "${@bb.utils.contains('PACKAGECONFIG', 'timesyncd', '--system -d / -M --shell /sbin/nologin systemd-timesync;', '', d)}"
@@ -620,6 +622,27 @@ FILES:${PN}-extra-utils = "\ FILES:${PN}-mime = "${MIMEDIR}" RRECOMMENDS:${PN} += "${PN}-mime" +FILES:${PN}-networkd = "\ + ${bindir}/networkctl \ + ${datadir}/dbus-1/system-services/org.freedesktop.network1.service \ + ${datadir}/dbus-1/system.d/org.freedesktop.network1.conf \ + ${datadir}/polkit-1/actions/org.freedesktop.network1.policy \ + ${nonarch_libdir}/sysusers.d/systemd-network.conf \ + ${nonarch_libdir}/tmpfiles.d/systemd-network.conf \ + ${sysconfdir}/systemd/networkd.conf \ + ${systemd_system_unitdir}/systemd-networkd* \ + ${systemd_unitdir}/network/*.network \ + ${systemd_unitdir}/network/*.network.example \ + ${systemd_unitdir}/networkd.conf \ + ${systemd_unitdir}/systemd-networkd* \ +" +# systemd-networkd-persistent-storage.service BindsTo=systemd-networkd.service +# systemd-networkd.service has Also=systemd-networkd-wait-online.service +SYSTEMD_SERVICE:${PN}-networkd = "systemd-networkd.service" +CONFFILES:${PN}-networkd = "${sysconfdir}/systemd/networkd.conf" +RDEPENDS:${PN}-networkd += "${PN}" +RRECOMMENDS:${PN} += "${@bb.utils.contains('PACKAGECONFIG', 'networkd', '${PN}-networkd', '', d)}" + FILES:${PN}-udev-rules = "\ ${nonarch_libdir}/udev/rules.d/70-uaccess.rules \ ${nonarch_libdir}/udev/rules.d/71-seat.rules \
@@ -630,7 +653,6 @@ FILES:${PN}-udev-rules = "\ CONFFILES:${PN} = "${sysconfdir}/systemd/coredump.conf \ ${sysconfdir}/systemd/journald.conf \ ${sysconfdir}/systemd/logind.conf \ - ${sysconfdir}/systemd/networkd.conf \ ${sysconfdir}/systemd/pstore.conf \ ${sysconfdir}/systemd/resolved.conf \ ${sysconfdir}/systemd/sleep.conf \
@@ -698,7 +720,6 @@ FILES:${PN} = " ${base_bindir}/* \ ${nonarch_libdir}/modprobe.d/README \ ${datadir}/dbus-1/system.d/org.freedesktop.timedate1.conf \ ${datadir}/dbus-1/system.d/org.freedesktop.locale1.conf \ - ${datadir}/dbus-1/system.d/org.freedesktop.network1.conf \ ${datadir}/dbus-1/system.d/org.freedesktop.resolve1.conf \ ${datadir}/dbus-1/system.d/org.freedesktop.systemd1.conf \ ${@bb.utils.contains('PACKAGECONFIG', 'polkit_hostnamed_fallback', '${datadir}/dbus-1/system.d/org.freedesktop.hostname1_no_polkit.conf', '', d)} \
Signature · 1 line
2.47.0
Re: [OE-core] [PATCH 5/5] systemd: split networkd into its own package
Quoted 5 lines · Adrian Freihofer via lists.openembedded.org
From: Adrian Freihofer <[email protected]>
Support of images with different network management implementations without having to recompile systemd and other components.
Quoted 7 lines
Fedora does this as well since systemd version 246.6-2:
https://src.fedoraproject.org/rpms/systemd/blob/rawhide/f/systemd.spec#_578
This proves that it is technically possible and officially supported by systemd.Signed-off-by Adrian Freihofer <[email protected]>---
Hi Adrian,
I believe this patch or another one from this series is responsible of some warnings we can see on the autobuilder:
stdio: WARNING: core-image-full-cmdline-1.0-r0 do_rootfs: User systemd-network has never been defined
stdio: WARNING: core-image-sato-1.0-r0 do_rootfs: User systemd-network has never been defined
stdio: WARNING: core-image-sato-sdk-1.0-r0 do_rootfs: User systemd-network has never been defined
https://valkyrie.yoctoproject.org/#/builders/10/builds/638/steps/11/logs/warnings
https://valkyrie.yoctoproject.org/#/builders/64/builds/559/steps/12/logs/warnings
https://valkyrie.yoctoproject.org/#/builders/13/builds/561/steps/12/logs/warnings
Can you look at this issue please ?
Signature · 3 lines
Mathieu Dubois-Briand, Bootlin Embedded Linux and Kernel engineering https://bootlin.com