State of systemd from an integrator point of view
Hi,
this is my second attempt to use systemd with Poky for our sysmoBTS product (mostly to replace some respawn shell scripts and not to use the daemontools). I aborted the first trial because of journald and now that systemd is being merged into OE-Core I started again and I would like to share some of my issues.
* systemd as a watchdog. For our BTS I have a init script for the watchdog package and a configuration file in our BSP layer. For systemd I want the watchdog functionality provided by systemd itself. So I have a system.conf in my BSP layer an this line in my machine config:
MACHINE_ESSENTIAL_EXTRA_RDEPENDS = "\
...
${@['watchdog', ''][d.getVar('DISTRO_FEATURES_INITMAN', True) == 'systemd']} \
kernel-module-rtfifo "IMAGE_FSTYPES ?= "tar.bz2 cpio.gz ubifs ubi jffs2"
MACHINE_EXTRA_RDEPENDS = "\
...
${@['watchdog', ''][d.getVar('DISTRO_FEATURES_INITMAN', True) == 'systemd']} \
"this is obviously not a piece of beauty.. and it would be nicer if systemd itself could build a watchdog package or such...
* systemd and svsv start scripts. I am using the busybox ifplugd to configure the network on plug/un-plug of the cable. My start script ended with .sh in the name. systemd recognized the script but failed to start it. Renaming it to not end with .sh fixed the issue for me. This is more like a systemd bug than OE but I wonder if we could change update-rc.d bbclass to check the for the .sh in the start script and then complain about it?
* systemd's journald is not suitable for small devices. I think the default should be to disable the journald and do not disable the busybox log application in the systemd-compat units package. journald is dominated by memory allocations and reading procfs over and over again.
regards
holger
Hi Holger,
Quoted 5 lines · Holger Hans Peter Freyther
this is my second attempt to use systemd with Poky for our sysmoBTS product (mostly to replace some respawn shell scripts and not to use the daemontools). I aborted the first trial because of journald and now that systemd is being merged into OE-Core I started again and I would like to share some of my issues.
Great feedback, thanks Holger.
Quoted 12 lines
* systemd as a watchdog. For our BTS I have a init script for the watchdog package and a configuration file in our BSP layer. For systemd I want the watchdog functionality provided by systemd itself. So I have a system.conf in my BSP layer an this line in my machine config:
MACHINE_ESSENTIAL_EXTRA_RDEPENDS = "\ ... ${@['watchdog', ''][d.getVar('DISTRO_FEATURES_INITMAN', True) == 'systemd']} \ kernel-module-rtfifo "
IMAGE_FSTYPES?= "tar.bz2 cpio.gz ubifs ubi jffs2"
Quoted 7 lines
MACHINE_EXTRA_RDEPENDS = "\ ... ${@['watchdog', ''][d.getVar('DISTRO_FEATURES_INITMAN', True) == 'systemd']} \ "this is obviously not a piece of beauty.. and it would be nicer if systemd itself could build a watchdog package or such...
The INITMAN variable is injected into DISTRO_FEATURES, so I do find the more idiomatic base_contains more readable:
${@base_contains("DISTRO_FEATURES", "systemd", "", "watchdog", d)}
systemd could RPROVIDE="watchdog" but as the means to configure it are different that might not be ideal, and if you ever switched entirely to systemd with these conditionals it's obvious where your redundant dependencies are.
Personally, I'd stick with conditional dependencies.
Quoted 7 lines
* systemd and svsv start scripts. I am using the busybox ifplugd to configure the network on plug/un-plug of the cable. My start script ended with .sh in the name. systemd recognized the script but failed to start it. Renaming it to not end with .sh fixed the issue for me. This is more like a systemd bug than OE but I wonder if we could change update-rc.d bbclass to check the for the .sh in the start script and then complain about it?
Oh, interesting. Without any thinking I can't think of reasons why
that would fail. Filed #3900.
Quoted 4 lines
* systemd's journald is not suitable for small devices. I think the default should be to disable the journald and do not disable the busybox log application in the systemd-compat units package. journald is dominated by memory allocations and reading procfs over and over again.
Agreed, although it would be good if the journal could be tuned. IMHO it's technically great, just young.
I wonder if we can split out the bulk of the journal binaries into a separate package, so by default it's disabled and then installing systemd-journal would enable it.
Filed #3901.
Cheers,
Ross
Quoted 11 lines · Ross
Quoted 4 lines
* systemd's journald is not suitable for small devices. I think the default should be to disable the journald and do not disable the busybox log application in the systemd-compat units package. journald is dominated by memory allocations and reading procfs over and over again.
Agreed, although it would be good if the journal could be tuned. IMHO it's technically great, just young.
I wonder if we can split out the bulk of the journal binaries into a separate package, so by default it's disabled and then installing systemd-journal would enable it.
Just noticed that the disabling of syslog isn't working for me, it disables syslog-busybox but there's a syslog symlink to it, which is being started. :)
Ross
Quoted 3 lines · Ross
Just noticed that the disabling of syslog isn't working for me, it disables syslog-busybox but there's a syslog symlink to it, which is being started. :)
I haven't gotten to the part to get it started again. The symlink probably exists to deal with a real syslog package being installed. I am not sure how to deal with that. The way forward is probably to have syslog package install some kind of /etc/default/* that inhibits starting the busybox one.
regards
holger
Quoted 4 lines · Ross
The INITMAN variable is injected into
DISTRO_FEATURES, so I do find the more idiomaticbase_containsmore readable:${@base_contains("DISTRO_FEATURES", "systemd", "", "watchdog", d)}
thanks. I will update my BSP.
Quoted 7 lines
systemd could RPROVIDE="watchdog" but as the means to configure it are different that might not be ideal, and if you ever switched entirely to systemd with these conditionals it's obvious where your redundant dependencies are.
Personally, I'd stick with conditional dependencies.
true, the watchdog package can potentially do a lot more as well. I also have one watchdog related issue in my kernel/systemd. The TI Davinci wdt doesn't allow to set the interval via IOCTL.
Quoted 2 lines
Oh, interesting. Without any thinking I can't think of reasons why
that would fail. Filed #3900.
I will try to find some time toward the end to re-produce and provide feedback. I think it is a genuine bug in systemd. I am going to update the ticket then.
Quoted 2 lines
Agreed, although it would be good if the journal could be tuned. IMHO it's technically great, just young.
journald is doing a 'lot' of work for each syslog message. E.g. if one takes a look at journald-server.c:dispatch_message_real. I don't know how the messages are flowing but from what I see the only way it could be made usable is:
a.) API changes to there helpers to avoid dynamic allocations in many of the cases. E.g. re-use one dynamically allocated string.
r = get_process_cmdline(ucred->pid, 0, false, &t);
if (r >= 0) {
cmdline = strappend("_CMDLINE=", t);
free(t); if (cmdline)
IOVEC_SET_STRING(iovec[n++], cmdline);}
in here there are at least:
1.) a malloc for /proc/%lu/cmdline
2.) a malloc for the FILE* pointer by glibc (3.) a mmap for ANOM memory in the fgets callby glibc)
this example uses fgetc so it might not apply
3.) re-allocs for every character read from the file 4.) the strappend does another allocation and memcpy.
b.) Reducing the work by having the systemd send all this info with each message (probably it doesn't work because of the interface that is attempted to be used between systemd and the 'syslog'?
c.) Disabling/caching most of this information. E.g. the audit handling should probably be behind a #ifdef if AUDIT is disabled. I will probably experiment with putting a #ifdef to most of the additional information it wants to query.
Upstream will probably end up doing nothing about it.
regards
holger