Fix mask, preset, enable and restore /etc/machine-id in stateful images
v2 cover Fix mask, preset, enable and restore /etc/machine-id in stateful images
The recent changes to systemd.bbclass broke support for SYSTEMD_AUTO_ENABLE set to anything other than "enable" or "disable", whereas there are existing users for "mask" and "preset".
In addition /etc/machine-id was not created as an empty file which caused systemd to fail to do it's boot time update of it if the rootfs was ro at boot.
Finally instance creation from templates whilst present was no longer correctly exposed.
Changes in v2:
- use stateless-rootfs in IMAGE_FEATURES, not DISTRO_FEATURES
- restore support for enable command
- restore mask and preset targets, fix instance creation
Alex Kiernan (3):
systemd: Default to non-stateless images
systemd-systemctl: Restore support for enable command
systemd: Restore mask and preset targets, fix instance creation
meta/classes/image.bbclass | 9 +- meta/classes/systemd.bbclass | 37 +++- .../systemd/systemd-systemctl/systemctl | 186 +++++++++++------- 3 files changed, 142 insertions(+), 90 deletions(-)
Signature · 1 line
2.17.1
PATCH v2 1/3 systemd: Default to non-stateless images
When creating images, for anything other than the explicitly stateless case, touch /etc/machine-id so that the images can be booted without an initramfs and with ro set on the kernel command line, otherwise system refuses to start:
[ 7.222134] systemd[1]: No hostname configured. [ 7.227266] systemd[1]: Set hostname to <localhost>. [ 7.232622] systemd[1]: System cannot boot: Missing /etc/machine-id and /etc is mounted read-only. [ 7.241750] systemd[1]: Booting up is supported only when: [ 7.247362] systemd[1]: 1) /etc/machine-id exists and is populated. [ 7.253752] systemd[1]: 2) /etc/machine-id exists and is empty. [ 7.259757] systemd[1]: 3) /etc/machine-id is missing and /etc is writable.
If IMAGE_FEATURES includes stateless-rootfs then systemctl-native is not run on the image leaving the image for population at runtime by systemd.
---
Changes in v2:
- use stateless-rootfs in IMAGE_FEATURES, not DISTRO_FEATURES
meta/classes/image.bbclass | 9 ++++----- meta/recipes-core/systemd/systemd-systemctl/systemctl | 9 +++++++++ 2 files changed, 13 insertions(+), 5 deletions(-)
meta/classes/image.bbclass +4 -5
@@ -33,7 +33,7 @@ INHIBIT_DEFAULT_DEPS = "1" # IMAGE_FEATURES may contain any available package group IMAGE_FEATURES ?= "" IMAGE_FEATURES[type] = "list" -IMAGE_FEATURES[validitems] += "debug-tweaks read-only-rootfs empty-root-password allow-empty-password allow-root-login post-install-logging" +IMAGE_FEATURES[validitems] += "debug-tweaks read-only-rootfs stateless-rootfs empty-root-password allow-empty-password allow-root-login post-install-logging" # Generate companion debugfs? IMAGE_GEN_DEBUGFS ?= "0"
@@ -665,12 +665,11 @@ reproducible_final_image_task () { fi } -IMAGE_EXTRADEPENDS += "${@bb.utils.contains('DISTRO_FEATURES', 'systemd', 'systemd-systemctl-native', '', d)}" - systemd_preset_all () { - systemctl --root="${IMAGE_ROOTFS}" --preset-mode=enable-only preset-all + systemctl --root="${IMAGE_ROOTFS}" --preset-mode=enable-only preset-all } -IMAGE_PREPROCESS_COMMAND_append = " ${@bb.utils.contains('DISTRO_FEATURES', 'systemd', 'systemd_preset_all;', '', d)} reproducible_final_image_task; " +IMAGE_EXTRADEPENDS += "${@ 'systemd-systemctl-native' if bb.utils.contains('DISTRO_FEATURES', 'systemd', True, False, d) and not bb.utils.contains('IMAGE_FEATURES', 'stateless-rootfs', True, False, d) else ''}" +IMAGE_PREPROCESS_COMMAND_append = " ${@ 'systemd_preset_all;' if bb.utils.contains('DISTRO_FEATURES', 'systemd', True, False, d) and not bb.utils.contains('IMAGE_FEATURES', 'stateless-rootfs', True, False, d) else ''} reproducible_final_image_task; " CVE_PRODUCT = ""
meta/recipes-core/systemd/systemd-systemctl/systemctl +9 -0
@@ -241,6 +241,15 @@ def preset_all(root): if state == "enable" or state is None: enable(root, service, location, services) + # If we populate the systemd links we also create /etc/machine-id, which + # allows systemd to boot with the filesystem read-only before generating + # a real value and then committing it back. + # + # For the stateless configuration, where /etc is generated at runtime + # (for example on a tmpfs), this script shouldn't run at all and we + # allow systemd to completely populate /etc. + (root / SYSCONFDIR / "machine-id").touch() + def mask(root, *services): systemdir = root / SYSCONFDIR / "systemd" / "system"
Signature · 1 line
2.17.1
PATCH v2 2/3 systemd-systemctl: Restore support for enable command
In reply to Alex KiernanRefactor so that SystemdUnit is its own class, then add support for the enable command. This restores the ability of systemd.bbclass to create instances using syntax such as:
SYSTEMD_SERVICE_${PN} = "[email protected]"---
Changes in v2:
- restore support for enable command
.../systemd/systemd-systemctl/systemctl | 179 ++++++++++-------- 1 file changed, 102 insertions(+), 77 deletions(-)
meta/recipes-core/systemd/systemd-systemctl/systemctl +102 -77
@@ -20,6 +20,8 @@ SYSCONFDIR = Path("etc") BASE_LIBDIR = Path("lib") LIBDIR = Path("usr", "lib") +locations = list() + class SystemdFile(): """Class representing a single systemd configuration file"""
@@ -111,12 +113,6 @@ class Presets(): def _collect_presets(self, scope, root): """Collect list of preset files""" - locations = [SYSCONFDIR / "systemd"] - # Handle the usrmerge case by ignoring /lib when it's a symlink - if not BASE_LIBDIR.is_symlink(): - locations.append(BASE_LIBDIR / "systemd") - locations.append(LIBDIR / "systemd") - presets = dict() for location in locations: paths = (root / location / scope).glob("*.preset")
@@ -146,27 +142,6 @@ class Presets(): return None -def collect_services(root): - """Collect list of service files""" - locations = [SYSCONFDIR / "systemd"] - # Handle the usrmerge case by ignoring /lib when it's a symlink - if not BASE_LIBDIR.is_symlink(): - locations.append(BASE_LIBDIR / "systemd") - locations.append(LIBDIR / "systemd") - - services = dict() - for location in locations: - paths = (root / location / "system").glob("*") - for path in paths: - if path.is_dir(): - continue - # implement earlier names override later ones - if path.name not in services: - services[path.name] = path - - return services - - def add_link(path, target): try: path.parent.mkdir(parents=True)
@@ -177,69 +152,113 @@ def add_link(path, target): path.symlink_to(target) -def process_deps(root, config, service, location, prop, dirstem): - systemdir = SYSCONFDIR / "systemd" / "system" +class SystemdUnitNotFoundError(Exception): + pass - target = ROOT / location.relative_to(root) - try: - for dependent in config.get('Install', prop): - wants = root / systemdir / "{}.{}".format(dependent, dirstem) / service - add_link(wants, target) - except KeyError: - pass +class SystemdUnit(): + def __init__(self, root, unit): + self.root = root + self.unit = unit + self.config = None + + def _path_for_unit(self, unit): + for location in locations: + path = self.root / location / "system" / unit + if path.exists(): + return path + raise SystemdUnitNotFoundError(self.root, unit) -def enable(root, service, location, services): - if location.is_symlink(): - # ignore aliases - return + def _process_deps(self, config, service, location, prop, dirstem): + systemdir = self.root / SYSCONFDIR / "systemd" / "system" + + target = ROOT / location.relative_to(self.root) + try: + for dependent in config.get('Install', prop): + wants = systemdir / "{}.{}".format(dependent, dirstem) / service + add_link(wants, target) + + except KeyError: + pass + + def enable(self): + # if we're enabling an instance, first extract the actual instance + # then figure out what the template unit is + template = re.match(r"[^@]+@(?P<instance>[^\.]*)\.", self.unit) + if template: + instance = template.group('instance') + unit = re.sub(r"@[^\.]*\.", "@.", self.unit, 1) + else: + instance = None + unit = self.unit + + path = self._path_for_unit(unit) + + if path.is_symlink(): + # ignore aliases + return - config = SystemdFile(root, location) - template = re.match(r"[^@]+@(?P<instance>[^\.]*)\.", service) - if template: - instance = template.group('instance') - if not instance: + config = SystemdFile(self.root, path) + if instance == "": try: - instance = config.get('Install', 'DefaultInstance')[0] - service = service.replace("@.", "@{}.".format(instance)) + default_instance = config.get('Install', 'DefaultInstance')[0] except KeyError: - pass - if instance is None: - return - else: - instance = None + # no default instance, so nothing to enable + return - process_deps(root, config, service, location, 'WantedBy', 'wants') - process_deps(root, config, service, location, 'RequiredBy', 'requires') + service = self.unit.replace("@.", + "@{}.".format(default_instance)) + else: + service = self.unit - try: - for also in config.get('Install', 'Also'): - enable(root, also, services[also], services) + self._process_deps(config, service, path, 'WantedBy', 'wants') + self._process_deps(config, service, path, 'RequiredBy', 'requires') - except KeyError: - pass + try: + for also in config.get('Install', 'Also'): + SystemdUnit(self.root, also).enable() - systemdir = root / SYSCONFDIR / "systemd" / "system" - target = ROOT / location.relative_to(root) - try: - for dest in config.get('Install', 'Alias'): - alias = systemdir / dest - add_link(alias, target) + except KeyError: + pass - except KeyError: - pass + systemdir = self.root / SYSCONFDIR / "systemd" / "system" + target = ROOT / path.relative_to(self.root) + try: + for dest in config.get('Install', 'Alias'): + alias = systemdir / dest + add_link(alias, target) + + except KeyError: + pass + + def mask(self): + systemdir = self.root / SYSCONFDIR / "systemd" / "system" + add_link(systemdir / self.unit, "/dev/null") + + +def collect_services(root): + """Collect list of service files""" + services = set() + for location in locations: + paths = (root / location / "system").glob("*") + for path in paths: + if path.is_dir(): + continue + services.add(path.name) + + return services def preset_all(root): presets = Presets('system-preset', root) services = collect_services(root) - for service, location in services.items(): + for service in services: state = presets.state(service) if state == "enable" or state is None: - enable(root, service, location, services) + SystemdUnit(root, service).enable() # If we populate the systemd links we also create /etc/machine-id, which # allows systemd to boot with the filesystem read-only before generating
@@ -251,18 +270,13 @@ def preset_all(root): (root / SYSCONFDIR / "machine-id").touch() -def mask(root, *services): - systemdir = root / SYSCONFDIR / "systemd" / "system" - for service in services: - add_link(systemdir / service, "/dev/null") - - def main(): if sys.version_info < (3, 4, 0): sys.exit("Python 3.4 or greater is required") parser = argparse.ArgumentParser() - parser.add_argument('command', nargs=1, choices=['mask', 'preset-all']) + parser.add_argument('command', nargs=1, choices=['enable', 'mask', + 'preset-all']) parser.add_argument('service', nargs=argparse.REMAINDER) parser.add_argument('--root') parser.add_argument('--preset-mode',
@@ -272,9 +286,20 @@ def main(): args = parser.parse_args() root = Path(args.root) if args.root else ROOT + + locations.append(SYSCONFDIR / "systemd") + # Handle the usrmerge case by ignoring /lib when it's a symlink + if not (root / BASE_LIBDIR).is_symlink(): + locations.append(BASE_LIBDIR / "systemd") + locations.append(LIBDIR / "systemd") + command = args.command[0] if command == "mask": - mask(root, *args.service) + for service in args.service: + SystemdUnit(root, service).mask() + elif command == "enable": + for service in args.service: + SystemdUnit(root, service).enable() elif command == "preset-all": if len(args.service) != 0: sys.exit("Too many arguments.")
Signature · 1 line
2.17.1
PATCH v2 3/3 systemd: Restore mask and preset targets, fix instance creation
In reply to Alex KiernanEnsure that anyone who sets SYSTEMD_AUTO_ENABLE_${PN} = "mask" (or "preset") retains the previous behaviour. In addition fix "enable" so it can (again) create instance units out of templates.
Remove over-zealous quoting of SYSTEMD_SERVICE_ESCAPED as it's already safely escaped (and can be multiple arguments).
---
Changes in v2:
- restore mask and preset targets, fix instance creation
meta/classes/systemd.bbclass | 37 +++++++++++++++++++++++++++--------- 1 file changed, 28 insertions(+), 9 deletions(-)
meta/classes/systemd.bbclass +28 -9
@@ -16,18 +16,36 @@ python __anonymous() { # from doing any work so that pure-systemd images don't have redundant init # files. if bb.utils.contains('DISTRO_FEATURES', 'systemd', True, False, d): + d.appendVar("DEPENDS", " systemd-systemctl-native") + d.appendVar("PACKAGE_WRITE_DEPS", " systemd-systemctl-native") if not bb.utils.contains('DISTRO_FEATURES', 'sysvinit', True, False, d): d.setVar("INHIBIT_UPDATERCD_BBCLASS", "1") } systemd_postinst() { if type systemctl >/dev/null 2>/dev/null; then + OPTS="" + + if [ -n "$D" ]; then + OPTS="--root=$D" + fi + + if [ "${SYSTEMD_AUTO_ENABLE}" = "enable" ]; then + for service in ${SYSTEMD_SERVICE_ESCAPED}; do + case "${service}" in + *@*) + systemctl ${OPTS} enable "${service}" + ;; + esac + done + fi + if [ -z "$D" ]; then systemctl daemon-reload - systemctl preset "${SYSTEMD_SERVICE_ESCAPED}" + systemctl preset ${SYSTEMD_SERVICE_ESCAPED} if [ "${SYSTEMD_AUTO_ENABLE}" = "enable" ]; then - systemctl --no-block restart "${SYSTEMD_SERVICE_ESCAPED}" + systemctl --no-block restart ${SYSTEMD_SERVICE_ESCAPED} fi fi fi
@@ -36,9 +54,9 @@ fi systemd_prerm() { if type systemctl >/dev/null 2>/dev/null; then if [ -z "$D" ]; then - systemctl stop "${SYSTEMD_SERVICE_ESCAPED}" + systemctl stop ${SYSTEMD_SERVICE_ESCAPED} - systemctl disable "${SYSTEMD_SERVICE_ESCAPED}" + systemctl disable ${SYSTEMD_SERVICE_ESCAPED} fi fi }
@@ -162,10 +180,7 @@ python systemd_populate_packages() { else: bb.fatal("SYSTEMD_SERVICE_%s value %s does not exist" % (pkg_systemd, service)) - def systemd_create_presets(pkg): - action = get_package_var(d, 'SYSTEMD_AUTO_ENABLE', pkg) - if action not in ("enable", "disable"): - bb.fatal("SYSTEMD_AUTO_ENABLE_%s '%s' is not 'enable' or 'disable'" % (pkg, action)) + def systemd_create_presets(pkg, action): presetf = oe.path.join(d.getVar("PKGD"), d.getVar("systemd_unitdir"), "system-preset/98-%s.preset" % pkg) bb.utils.mkdirhier(os.path.dirname(presetf)) with open(presetf, 'a') as fd:
@@ -179,7 +194,11 @@ python systemd_populate_packages() { systemd_check_package(pkg) if d.getVar('SYSTEMD_SERVICE_' + pkg): systemd_generate_package_scripts(pkg) - systemd_create_presets(pkg) + action = get_package_var(d, 'SYSTEMD_AUTO_ENABLE', pkg) + if action in ("enable", "disable"): + systemd_create_presets(pkg, action) + elif action not in ("mask", "preset"): + bb.fatal("SYSTEMD_AUTO_ENABLE_%s '%s' is not 'enable', 'disable', 'mask' or 'preset'" % (pkg, action)) systemd_check_services() }
Signature · 1 line
2.17.1