Skip to contentSkip to conversation

systemd: Build the systemctl executable

PATCH Superseded [email protected] 7 participants 16 messages Feb 19, 2025 – Mar 4, 2025 1 message not in the archive

v6 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
#0 lore raw

PATCH v6 1/1 systemd: Build the systemctl executable

From: Vyacheslav Yurkov <[email protected]>

Instead of the python re-implementation build the actual systemctl from the systemd source tree. The python script was used when systemd didn't provide an option to build individual executables. It is possible in the meantime, so instead of always adapting the script when there's a new functionality, we simply use upstream implementation.

License-Update: Base recipe is used

---
 .../systemd/systemd-systemctl-native.bb       |  23 +-
 .../systemd/systemd-systemctl/systemctl       | 376 ------------------
 2 files changed, 11 insertions(+), 388 deletions(-)
 delete mode 100755 meta/recipes-core/systemd/systemd-systemctl/systemctl
meta/recipes-core/systemd/systemd-systemctl-native.bb +11 -12
index ffa024caef..73862b4e23 100644
@@ -1,17 +1,16 @@
-SUMMARY = "Wrapper for enabling systemd services"
+SUMMARY = "Systemctl executable from systemd"
 
-LICENSE = "MIT"
-LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
+require systemd.inc
 
+DEPENDS = "gperf-native libcap-native util-linux-native python3-jinja2-native"
 
-inherit native
+inherit pkgconfig meson native
 
-SRC_URI = "file://systemctl"
+MESON_TARGET = "systemctl:executable"
+MESON_INSTALL_TAGS = "systemctl"
+EXTRA_OEMESON:append = " -Dlink-systemctl-shared=false"
 
-S = "${WORKDIR}/sources"
-UNPACKDIR = "${S}"
-
-do_install() {
-	install -d ${D}${bindir}
-	install -m 0755 ${S}/systemctl ${D}${bindir}
-}
+# Systemctl is supposed to operate on target, but the target sysroot is not
+# determined at run-time, but rather set during configure
+# More details are here https://github.com/systemd/systemd/issues/35897#issuecomment-2665405887
+EXTRA_OEMESON:append = " --sysconfdir ${sysconfdir_native}"
meta/recipes-core/systemd/systemd-systemctl/systemctl deleted +0 -376

376 lines not on this page · raw message

#1 lore raw
Quoted 8 lines · Vyacheslav Yurkov via lists.openembedded.org

From: Vyacheslav Yurkov <[email protected]>

Instead of the python re-implementation build the actual systemctl from the systemd source tree. The python script was used when systemd didn't provide an option to build individual executables. It is possible in the meantime, so instead of always adapting the script when there's a new functionality, we simply use upstream implementation.

If I compare an image built with master and this, there’s one notable change in the file list:

Changes to images/qemuarm64/glibc/core-image-full-cmdline (files-in-image.txt):
/etc/machine-id was removed

The good news is that apart from that the final images are identical, but that change wasn’t expected.

Ross

#2 lore raw

Isn't is supposed to be created on first boot?

Slava

Quoted 16 lines · Ross Burton
Quoted 7 lines · Vyacheslav Yurkov via lists.openembedded.org

From: Vyacheslav Yurkov <[email protected]>

Instead of the python re-implementation build the actual systemctl from the systemd source tree. The python script was used when systemd didn't provide an option to build individual executables. It is possible in the meantime, so instead of always adapting the script when there's a new functionality, we simply use upstream implementation.

If I compare an image built with master and this, there’s one notable change in the file list:

Changes to images/qemuarm64/glibc/core-image-full-cmdline (files-in-image.txt):
/etc/machine-id was removed

The good news is that apart from that the final images are identical, but that change wasn’t expected.

Ross

A message that is not in the archive (sent off-list or not yet received): [email protected]

#4 lore raw

From meta/classes-recipe/rootfs-postcommands.bbclass:

    # 20:12 < mezcalero> koen: you have three options: a) run systemd-machine-id-setup at install time, b) have / read-only and an empty file there (for stateless) and c) boot with / writable touch ${IMAGE_ROOTFS}${sysconfdir}/machine-id
    fi

We could think of further improvements. One thing that comes to mind is changing systemd-systemctl-native to systemd-tools-native and enable the build of systemd-machine-id-setup.

Slava

Quoted 22 lines · Vyacheslav Yurkov via lists.openembedded.org

Isn't is supposed to be created on first boot?

Slava

Quoted 16 lines · Ross Burton
Quoted 7 lines · Vyacheslav Yurkov via lists.openembedded.org

From: Vyacheslav Yurkov <[email protected]>

Instead of the python re-implementation build the actual systemctl from the systemd source tree. The python script was used when systemd didn't provide an option to build individual executables. It is possible in the meantime, so instead of always adapting the script when there's a new functionality, we simply use upstream implementation.

If I compare an image built with master and this, there’s one notable change in the file list:

Changes to images/qemuarm64/glibc/core-image-full-cmdline (files-in-image.txt):
   /etc/machine-id was removed

The good news is that apart from that the final images are identical, but that change wasn’t expected.

Ross

#5 lore raw
Quoted 6 lines · Vyacheslav Yurkov via lists.openembedded.org

From meta/classes-recipe/rootfs-postcommands.bbclass:

# 20:12 < mezcalero> koen: you have three options: a) run
systemd-machine-id-setup at install time, b) have / read-only and an empty file there (for stateless) and c) boot with / writable

Quoted 9 lines
         touch ${IMAGE_ROOTFS}${sysconfdir}/machine-id
     fi

We could think of further improvements. One thing that comes to mind is changing systemd-systemctl-native to systemd-tools-native and enable the build of systemd-machine-id-setup.

Slava

I do know that it was a horrible thing to get right when I rewrote the shell script in python (which I'm very glad to see the back of):

# If we populate the systemd links we also create /etc/machine-id, which # allows systemd to boot with the filesystem read-only before generating # a real value and then committing it back.
#
# For the stateless configuration, where /etc is generated at runtime # (for example on a tmpfs), this script shouldn't run at all and we # allow systemd to completely populate /etc.

It may be that this behaviour was wrong in some cases, but at the same time, I remember spending a lot of time getting this right!

Alex

Quoted 39 lines · Vyacheslav Yurkov via lists.openembedded.org
Quoted 28 lines

Isn't is supposed to be created on first boot?

Slava

Quoted 22 lines · Ross Burton
Quoted 8 lines · Vyacheslav Yurkov via lists.openembedded.org

From: Vyacheslav Yurkov <[email protected]>

Instead of the python re-implementation build the actual systemctl from the systemd source tree. The python script was used when systemd didn't provide an option to build individual executables. It is possible in the
meantime, so instead of always adapting the script when there's a new functionality, we simply use upstream implementation.

If I compare an image built with master and this, there’s one notable change in the file list:

Changes to images/qemuarm64/glibc/core-image-full-cmdline (files-in-image.txt):
/etc/machine-id was removed

The good news is that apart from that the final images are identical, but that change wasn’t expected.

Ross

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#211766): https://lists.openembedded.org/g/openembedded-core/message/211766
Mute This Topic: https://lists.openembedded.org/mt/111267385/3618097
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

Signature · 1 line
Alex Kiernan
#6 lore raw
Quoted 30 lines · Alex Kiernan via lists.openembedded.org
Quoted 9 lines · Vyacheslav Yurkov via lists.openembedded.org

From meta/classes-recipe/rootfs-postcommands.bbclass:

     # 20:12 < mezcalero> koen: you have three options: a) run systemd-machine-id-setup at install time, b) have / read-only and an empty file there (for stateless) and c) boot with / writable touch ${IMAGE_ROOTFS}${sysconfdir}/machine-id
     fi

Quoted 6 lines

We could think of further improvements. One thing that comes to mind is changing systemd-systemctl-native to systemd-tools-native and enable the build of systemd-machine-id-setup.

Slava

I do know that it was a horrible thing to get right when I rewrote the shell script in python (which I'm very glad to see the back of):

    # If we populate the systemd links we also create /etc/machine-id, which

    # allows systemd to boot with the filesystem read-only before generating
    # a real value and then committing it back.
    #
    # For the stateless configuration, where /etc is generated at runtime
    # (for example on a tmpfs), this script shouldn't run at all and we
    # allow systemd to completely populate /etc.
Quoted 2 lines

It may be that this behaviour was wrong in some cases, but at the same time, I remember spending a lot of time getting this right!

I just wanted to add that we've been seeing an odd failure on meta-arm on the autobuilder:

https://autobuilder.yoctoproject.org/valkyrie/#/builders/75/builds/1032

and when I did into this, I see this in the logs:

[    3.700441] systemd[1]: Hostname set to <sbsa-ref>.
[    3.704040] systemd[1]: System cannot boot: Missing /etc/machine-id and /etc is mounted read-only.
[    3.704488] systemd[1]: Booting up is supported only when:
[    3.705023] systemd[1]: 1) /etc/machine-id exists and is populated.
[    3.705349] systemd[1]: 2) /etc/machine-id exists and is empty.
[    3.705651] systemd[1]: 3) /etc/machine-id is missing and /etc is writable.
[   10.206513] systemd[1]: Populated /etc with preset unit settings.
[   13.034155] systemd[1]: Queued start job for default target Graphical Interface.
[   13.149568] systemd[1]: Created slice Slice /system/getty.

We therefore need to fix the issue with machine-id before we can merge this patch. I will drop it from master-next in the meantime due to this issue.

Cheers,

Richard

#7 lore raw

Hi,

Quoted 50 lines · Richard Purdie via lists.openembedded.org
Quoted 30 lines · Alex Kiernan via lists.openembedded.org
Quoted 9 lines · Vyacheslav Yurkov via lists.openembedded.org

From meta/classes-recipe/rootfs-postcommands.bbclass:

     # 20:12 < mezcalero> koen: you have three options: a) run systemd-machine-id-setup at install time, b) have / read-only and an empty file there (for stateless) and c) boot with / writable touch ${IMAGE_ROOTFS}${sysconfdir}/machine-id
     fi

Quoted 6 lines

We could think of further improvements. One thing that comes to mind is changing systemd-systemctl-native to systemd-tools-native and enable the build of systemd-machine-id-setup.

Slava

I do know that it was a horrible thing to get right when I rewrote the shell script in python (which I'm very glad to see the back of):

    # If we populate the systemd links we also create /etc/machine-id, which

    # allows systemd to boot with the filesystem read-only before generating
    # a real value and then committing it back.
    #
    # For the stateless configuration, where /etc is generated at runtime
    # (for example on a tmpfs), this script shouldn't run at all and we
    # allow systemd to completely populate /etc.
Quoted 2 lines

It may be that this behaviour was wrong in some cases, but at the same time, I remember spending a lot of time getting this right!

I just wanted to add that we've been seeing an odd failure on meta-arm on the autobuilder:

https://autobuilder.yoctoproject.org/valkyrie/#/builders/75/builds/1032

and when I did into this, I see this in the logs:

[    3.700441] systemd[1]: Hostname set to <sbsa-ref>.
[    3.704040] systemd[1]: System cannot boot: Missing /etc/machine-id and /etc is mounted read-only.
[    3.704488] systemd[1]: Booting up is supported only when:
[    3.705023] systemd[1]: 1) /etc/machine-id exists and is populated.
[    3.705349] systemd[1]: 2) /etc/machine-id exists and is empty.
[    3.705651] systemd[1]: 3) /etc/machine-id is missing and /etc is writable.
[   10.206513] systemd[1]: Populated /etc with preset unit settings.
[   13.034155] systemd[1]: Queued start job for default target Graphical Interface.
[   13.149568] systemd[1]: Created slice Slice /system/getty.
Quoted 3 lines

We therefore need to fix the issue with machine-id before we can merge this patch. I will drop it from master-next in the meantime due to this issue.

I think option 2) should be simple. IMO this could also be done by image/rootfs classes or base-files when systemd is the init.

Where is build https://autobuilder.yoctoproject.org/valkyrie/#/builders/75/builds/1032
adding "read-only-rootfs" to IMAGE_FEATURES?

The failures log from step 22, full log in
https://autobuilder.yoctoproject.org/valkyrie/api/v2/logs/1586278/raw_inline
only shows issues with sshd server, which IMO is not installed to any of the built images. The builds are IMO missing IMAGE_FEATURES += "ssh-server-dropbear" which in meta-arm kas builds is set with ci/testimage.yml.
Even the core-image-minimal tests are passing without any communication with target qemu machine:

* ping.PingTest.test_ping with slirp pings localhost, not target machine
* parselogs.ParseLogsTest.test_get_context doesn't talk with booted qemu machine

How to get to the qemu boot log which shows the systemd errors above?

I don't see the connection between the systemd error and the build above. I could propose patches to the issues if I can figure out where :/

Cheers,

-Mikko

#8 lore raw
Quoted 57 lines · Mikko Rapeli

Hi,

Quoted 50 lines · Richard Purdie via lists.openembedded.org
Quoted 30 lines · Alex Kiernan via lists.openembedded.org
Quoted 9 lines · Vyacheslav Yurkov via lists.openembedded.org

From meta/classes-recipe/rootfs-postcommands.bbclass:

     # 20:12 < mezcalero> koen: you have three options: a) run systemd-machine-id-setup at install time, b) have / read-only and an empty file there (for stateless) and c) boot with / writable touch ${IMAGE_ROOTFS}${sysconfdir}/machine-id
     fi

Quoted 6 lines

We could think of further improvements. One thing that comes to mind is changing systemd-systemctl-native to systemd-tools-native and enable the build of systemd-machine-id-setup.

Slava

I do know that it was a horrible thing to get right when I rewrote the shell script in python (which I'm very glad to see the back of):

    # If we populate the systemd links we also create /etc/machine-id, which

    # allows systemd to boot with the filesystem read-only before generating
    # a real value and then committing it back.
    #
    # For the stateless configuration, where /etc is generated at runtime
    # (for example on a tmpfs), this script shouldn't run at all and we
    # allow systemd to completely populate /etc.
Quoted 2 lines

It may be that this behaviour was wrong in some cases, but at the same time, I remember spending a lot of time getting this right!

I just wanted to add that we've been seeing an odd failure on meta-arm on the autobuilder:

https://autobuilder.yoctoproject.org/valkyrie/#/builders/75/builds/1032

and when I did into this, I see this in the logs:

[    3.700441] systemd[1]: Hostname set to <sbsa-ref>.
[    3.704040] systemd[1]: System cannot boot: Missing /etc/machine-id and /etc is mounted read-only. [    3.704488] systemd[1]: Booting up is supported only when:
[    3.705023] systemd[1]: 1) /etc/machine-id exists and is populated.
[    3.705349] systemd[1]: 2) /etc/machine-id exists and is empty.
[    3.705651] systemd[1]: 3) /etc/machine-id is missing and /etc is writable.
[   10.206513] systemd[1]: Populated /etc with preset unit settings.
[   13.034155] systemd[1]: Queued start job for default target Graphical Interface.
[   13.149568] systemd[1]: Created slice Slice /system/getty.

Quoted 3 lines

We therefore need to fix the issue with machine-id before we can merge this patch. I will drop it from master-next in the meantime due to this issue.

Quoted 5 lines

I think option 2) should be simple. IMO this could also be done by image/rootfs classes or base-files when systemd is the init.

Where is build https://autobuilder.yoctoproject.org/valkyrie/#/builders/75/builds/1032
adding "read-only-rootfs" to IMAGE_FEATURES?

It isn't a read only image, just that it seems the filesystem is mounted ro at this point in the boot. Maybe timing related?

Quoted 5 lines

The failures log from step 22, full log in
https://autobuilder.yoctoproject.org/valkyrie/api/v2/logs/1586278/raw_inline
only shows issues with sshd server, which IMO is not installed to any of the built images. The builds are IMO missing IMAGE_FEATURES += "ssh-server-dropbear" which in meta-arm kas builds is set with ci/testimage.yml.

Those images do contain an ssh server, it just isn't taking connections for some reason.

Quoted 7 lines

Even the core-image-minimal tests are passing without any communication with target qemu machine:

 * ping.PingTest.test_ping with slirp pings localhost, not target machine
 * parselogs.ParseLogsTest.test_get_context doesn't talk with booted qemu machine

How to get to the qemu boot log which shows the systemd errors above?

I had to ssh into the autobuilder worker in question to obtain it unfortunately.

Quoted 2 lines

I don't see the connection between the systemd error and the build above. I could propose patches to the issues if I can figure out where :/

Ross was able to reproduce by including the patch in this thread, building and then running testimage against core-image-sato/core-image- weston. I'd note those tests are using slirp which may or may not be related. The issue is therefore reproducible to some degree at least.

Cheers,

Richard

#9 lore raw
In reply to Alex Kiernan
Quoted 1 line · Vyacheslav Yurkov

Isn't is supposed to be created on first boot?

Yes, ish, but our rootfs is read only at this point:

[    7.639766] systemd[1]: System cannot boot: Missing /etc/machine-id and /etc is mounted read-only.
[    7.641135] systemd[1]: Booting up is supported only when:
[    7.641844] systemd[1]: 1) /etc/machine-id exists and is populated.
[    7.642588] systemd[1]: 2) /etc/machine-id exists and is empty.
[    7.643279] systemd[1]: 3) /etc/machine-id is missing and /etc is writable.
Quoted 11 lines · Alex Kiernan

I do know that it was a horrible thing to get right when I rewrote the shell script in python (which I'm very glad to see the back of):

# If we populate the systemd links we also create /etc/machine-id, which

   # allows systemd to boot with the filesystem read-only before generating
   # a real value and then committing it back.
   #
   # For the stateless configuration, where /etc is generated at runtime
   # (for example on a tmpfs), this script shouldn't run at all and we
   # allow systemd to completely populate /etc.
Quoted 2 lines

It may be that this behaviour was wrong in some cases, but at the same time, I remember spending a lot of time getting this right!

It certainly looks like it’s “tricky” to get the right semantics here because the range of potential use-cases is too great.

I believe we should be writing /etc/machine-id at rootfs time if systemd is being used. Currently this is done in the systemctl class if it writes links, and then in the rootfs class if the rootfs is read-only.

I think we should consolidate those into the rootfs class, and give the user the choice of whether to treat the image as booted or not by writing either “” or “uninitialized” to the file as per https://www.freedesktop.org/software/systemd/man/latest/machine-id.html#First%20Boot%20Semantics.

tl;dr: if it contains “uninitialized” then this is the first boot, otherwise it’s not. By always touching the file we break the ability for anyone to use systemd-first-boot or ConditionFirstBoot in their own units, which seems wrong.

We’re about a week from feature freeze and getting this merged would be great, do you think you can finish this off Slava?

Thanks,
Ross

#10 lore raw
Quoted 29 lines · Ross Burton
Quoted 1 line · Vyacheslav Yurkov

Isn't is supposed to be created on first boot?

Yes, ish, but our rootfs is read only at this point:

[    7.639766] systemd[1]: System cannot boot: Missing /etc/machine-id and /etc is mounted read-only.
[    7.641135] systemd[1]: Booting up is supported only when:
[    7.641844] systemd[1]: 1) /etc/machine-id exists and is populated.
[    7.642588] systemd[1]: 2) /etc/machine-id exists and is empty.
[    7.643279] systemd[1]: 3) /etc/machine-id is missing and /etc is writable.
Quoted 11 lines · Alex Kiernan

I do know that it was a horrible thing to get right when I rewrote the shell script in python (which I'm very glad to see the back of):

# If we populate the systemd links we also create /etc/machine-id, which # allows systemd to boot with the filesystem read-only before generating # a real value and then committing it back.
#
# For the stateless configuration, where /etc is generated at runtime # (for example on a tmpfs), this script shouldn't run at all and we # allow systemd to completely populate /etc.

Quoted 2 lines

It may be that this behaviour was wrong in some cases, but at the same time, I remember spending a lot of time getting this right!

It certainly looks like it’s “tricky” to get the right semantics here because the range of potential use-cases is too great.

I believe we should be writing /etc/machine-id at rootfs time if systemd is being used. Currently this is done in the systemctl class if it writes links, and then in the rootfs class if the rootfs is read-only.

Quoted 8 lines

I think we should consolidate those into the rootfs class, and give the user the choice of whether to treat the image as booted or not by writing either “” or “uninitialized” to the file as per https://www.freedesktop.org/software/systemd/man/latest/machine-id.html#First%20Boot%20Semantics.

tl;dr: if it contains “uninitialized” then this is the first boot, otherwise it’s not. By always touching the file we break the ability for anyone to use systemd-first-boot or ConditionFirstBoot in their own units, which seems wrong.

We’re about a week from feature freeze and getting this merged would be great, do you think you can finish this off Slava?

Thanks,
Ross

I definitely gonna try. But I might be missing something, because I thought it sums up as follows:

RO rootfs -> empty machine-id
RW rootfs -> no machine-id

And it does work that way when I rebuild my image with or without RO rootfs.

Richard, could you please point me out which test actually failed? I looked at the logs, but most of them were "Skipped". Does the test explicitly remount rootfs as RO perhaps?

The other option to have a real machine ID will probably mean to build machine id and leave it up to a user to decide whether they want to have such image. But I don't think it's wise to add this option now so it ends up in the next release.

Slava

#11 lore raw
Quoted 40 lines · Vyacheslav Yurkov via lists.openembedded.org
Quoted 29 lines · Ross Burton
Quoted 1 line · Vyacheslav Yurkov

Isn't is supposed to be created on first boot?

Yes, ish, but our rootfs is read only at this point:

[    7.639766] systemd[1]: System cannot boot: Missing /etc/machine-id and /etc is mounted read-only.
[    7.641135] systemd[1]: Booting up is supported only when:
[    7.641844] systemd[1]: 1) /etc/machine-id exists and is populated.
[    7.642588] systemd[1]: 2) /etc/machine-id exists and is empty.
[    7.643279] systemd[1]: 3) /etc/machine-id is missing and /etc is writable.
Quoted 11 lines · Alex Kiernan

I do know that it was a horrible thing to get right when I rewrote the shell script in python (which I'm very glad to see the back of):

# If we populate the systemd links we also create /etc/machine-id, which # allows systemd to boot with the filesystem read-only before generating # a real value and then committing it back.
#
# For the stateless configuration, where /etc is generated at runtime # (for example on a tmpfs), this script shouldn't run at all and we # allow systemd to completely populate /etc.

Quoted 2 lines

It may be that this behaviour was wrong in some cases, but at the same time, I remember spending a lot of time getting this right!

It certainly looks like it’s “tricky” to get the right semantics here because the range of potential use-cases is too great.

I believe we should be writing /etc/machine-id at rootfs time if systemd is being used. Currently this is done in the systemctl class if it writes links, and then in the rootfs class if the rootfs is read-only.

Quoted 8 lines

I think we should consolidate those into the rootfs class, and give the user the choice of whether to treat the image as booted or not by writing either “” or “uninitialized” to the file as per https://www.freedesktop.org/software/systemd/man/latest/machine-id.html#First%20Boot%20Semantics.

tl;dr: if it contains “uninitialized” then this is the first boot, otherwise it’s not. By always touching the file we break the ability for anyone to use systemd-first-boot or ConditionFirstBoot in their own units, which seems wrong.

We’re about a week from feature freeze and getting this merged would be great, do you think you can finish this off Slava?

Thanks,
Ross

Quoted 7 lines

I definitely gonna try. But I might be missing something, because I thought it sums up as follows:

RO rootfs -> empty machine-id
RW rootfs -> no machine-id

And it does work that way when I rebuild my image with or without RO rootfs.

At the moment on master, as systemd-systemctl python script touch the file, we can end up with an empty file independently of the IMAGE_FEATURE read-only-rootfs. So we do break the ability of firstboot in any case.
Also, if you boot with your rootfs in ro (KERNEL_ARGS, wic, ...) without setting read-only-rootfs, with fstab that will remount your rootfs in rw (so you do not have a read-only rootfs), not having the /etc/machine-id file will make systemd fail (no not having /etc/machine-id seems dangerous te me).

So why not put "uninitialized" in /etc/machine-id all the time (probably in rootfs post commands, if the file does not exist yet, to allow user to put a predefine one), it allows firstboot and prevent the dangerous case of not having the file. The only strange case would be when you do have a read-only filesystem, as you will always have a first boot.
But if we do not want that strange behaviour by default, we can empty /etc/machine-id in case of read-only-rootfs (in rootfs post commands), or we let the decision to the user by letting him choose using the systemd PACKAGECONFIG to enable/disable the firstboot support.

Quoted 10 lines

Richard, could you please point me out which test actually failed? I looked at the logs, but most of them were "Skipped".
Does the test explicitly remount rootfs as RO perhaps?

The other option to have a real machine ID will probably mean to build machine id and leave it up to a user to decide whether they want to have such image. But I don't think it's wise to add this option now so it ends up in the next release.

Slava

Regards,

Signature · 2 lines
Alexis Cellier
Smile ECS Developer
#12 lore raw
Quoted 62 lines · Alexis CELLIER
Quoted 40 lines · Vyacheslav Yurkov via lists.openembedded.org
Quoted 29 lines · Ross Burton
Quoted 1 line · Vyacheslav Yurkov

Isn't is supposed to be created on first boot?

Yes, ish, but our rootfs is read only at this point:

[    7.639766] systemd[1]: System cannot boot: Missing /etc/machine-id and /etc is mounted read-only.
[    7.641135] systemd[1]: Booting up is supported only when:
[    7.641844] systemd[1]: 1) /etc/machine-id exists and is populated.
[    7.642588] systemd[1]: 2) /etc/machine-id exists and is empty.
[    7.643279] systemd[1]: 3) /etc/machine-id is missing and /etc is writable.
Quoted 11 lines · Alex Kiernan

I do know that it was a horrible thing to get right when I rewrote the shell script in python (which I'm very glad to see the back of):

# If we populate the systemd links we also create /etc/machine-id, which # allows systemd to boot with the filesystem read-only before generating # a real value and then committing it back.
#
# For the stateless configuration, where /etc is generated at runtime # (for example on a tmpfs), this script shouldn't run at all and we # allow systemd to completely populate /etc.

Quoted 2 lines

It may be that this behaviour was wrong in some cases, but at the same time, I remember spending a lot of time getting this right!

It certainly looks like it’s “tricky” to get the right semantics here because the range of potential use-cases is too great.

I believe we should be writing /etc/machine-id at rootfs time if systemd is being used. Currently this is done in the systemctl class if it writes links, and then in the rootfs class if the rootfs is read-only.

Quoted 8 lines

I think we should consolidate those into the rootfs class, and give the user the choice of whether to treat the image as booted or not by writing either “” or “uninitialized” to the file as per https://www.freedesktop.org/software/systemd/man/latest/machine-id.html#First%20Boot%20Semantics.

tl;dr: if it contains “uninitialized” then this is the first boot, otherwise it’s not. By always touching the file we break the ability for anyone to use systemd-first-boot or ConditionFirstBoot in their own units, which seems wrong.

We’re about a week from feature freeze and getting this merged would be great, do you think you can finish this off Slava?

Thanks,
Ross

Quoted 7 lines

I definitely gonna try. But I might be missing something, because I thought it sums up as follows:

RO rootfs -> empty machine-id
RW rootfs -> no machine-id

And it does work that way when I rebuild my image with or without RO rootfs.

At the moment on master, as systemd-systemctl python script touch the file, we can end up with an empty file independently of the IMAGE_FEATURE read-only-rootfs. So we do break the ability of firstboot in any case.
Also, if you boot with your rootfs in ro (KERNEL_ARGS, wic, ...) without setting read-only-rootfs, with fstab that will remount your rootfs in rw (so you do not have a read-only rootfs), not having the /etc/machine-id file will make systemd fail (no not having /etc/machine-id seems dangerous te me).

Quoted 11 lines

So why not put "uninitialized" in /etc/machine-id all the time (probably in rootfs post commands, if the file does not exist yet, to allow user to put a predefine one), it allows firstboot and prevent the dangerous case of not having the file. The only strange case would be when you do have a read-only filesystem, as you will always have a first boot.
But if we do not want that strange behaviour by default, we can empty /etc/machine-id in case of read-only-rootfs (in rootfs post commands), or we let the decision to the user by letting him choose using the systemd PACKAGECONFIG to enable/disable the firstboot support.

firstboot usage, really doesn't feel like a piece of package configuration to me, possibly distro config, but having a (potentially) interactive boot seems like an outlier in the OE world (though I'm not saying it shouldn't be supported).

Today we call the systemctl script as part of rootfs construction which may touch /etc/machine-id, removing it from there and having it obviously in one place as part of rootfs construction seems like it ought to be an improvement.

Quoted 16 lines
Quoted 10 lines

Richard, could you please point me out which test actually failed? I looked at the logs, but most of them were "Skipped".
Does the test explicitly remount rootfs as RO perhaps?

The other option to have a real machine ID will probably mean to build machine id and leave it up to a user to decide whether they want to have such image. But I don't think it's wise to add this option now so it ends up in the next release.

Slava

Regards,

--
Alexis Cellier
Smile ECS Developer

Signature · 1 line
Alex Kiernan
#13 lore raw
Quoted 76 lines · Alex Kiernan
Quoted 62 lines · Alexis CELLIER
Quoted 40 lines · Vyacheslav Yurkov via lists.openembedded.org
Quoted 29 lines · Ross Burton
Quoted 1 line · Vyacheslav Yurkov

Isn't is supposed to be created on first boot?

Yes, ish, but our rootfs is read only at this point:

[    7.639766] systemd[1]: System cannot boot: Missing /etc/machine-id and /etc is mounted read-only.
[    7.641135] systemd[1]: Booting up is supported only when:
[    7.641844] systemd[1]: 1) /etc/machine-id exists and is populated.
[    7.642588] systemd[1]: 2) /etc/machine-id exists and is empty.
[    7.643279] systemd[1]: 3) /etc/machine-id is missing and /etc is writable.
Quoted 11 lines · Alex Kiernan

I do know that it was a horrible thing to get right when I rewrote the shell script in python (which I'm very glad to see the back of):

# If we populate the systemd links we also create /etc/machine-id, which # allows systemd to boot with the filesystem read-only before generating # a real value and then committing it back.
#
# For the stateless configuration, where /etc is generated at runtime # (for example on a tmpfs), this script shouldn't run at all and we # allow systemd to completely populate /etc.

Quoted 2 lines

It may be that this behaviour was wrong in some cases, but at the same time, I remember spending a lot of time getting this right!

It certainly looks like it’s “tricky” to get the right semantics here because the range of potential use-cases is too great.

I believe we should be writing /etc/machine-id at rootfs time if systemd is being used. Currently this is done in the systemctl class if it writes links, and then in the rootfs class if the rootfs is read-only.

Quoted 8 lines

I think we should consolidate those into the rootfs class, and give the user the choice of whether to treat the image as booted or not by writing either “” or “uninitialized” to the file as per https://www.freedesktop.org/software/systemd/man/latest/machine-id.html#First%20Boot%20Semantics.

tl;dr: if it contains “uninitialized” then this is the first boot, otherwise it’s not. By always touching the file we break the ability for anyone to use systemd-first-boot or ConditionFirstBoot in their own units, which seems wrong.

We’re about a week from feature freeze and getting this merged would be great, do you think you can finish this off Slava?

Thanks,
Ross

Quoted 7 lines

I definitely gonna try. But I might be missing something, because I thought it sums up as follows:

RO rootfs -> empty machine-id
RW rootfs -> no machine-id

And it does work that way when I rebuild my image with or without RO rootfs.

At the moment on master, as systemd-systemctl python script touch the file, we can end up with an empty file independently of the IMAGE_FEATURE read-only-rootfs. So we do break the ability of firstboot in any case.
Also, if you boot with your rootfs in ro (KERNEL_ARGS, wic, ...) without setting read-only-rootfs, with fstab that will remount your rootfs in rw (so you do not have a read-only rootfs), not having the /etc/machine-id file will make systemd fail (no not having /etc/machine-id seems dangerous te me).

Quoted 11 lines

So why not put "uninitialized" in /etc/machine-id all the time (probably in rootfs post commands, if the file does not exist yet, to allow user to put a predefine one), it allows firstboot and prevent the dangerous case of not having the file. The only strange case would be when you do have a read-only filesystem, as you will always have a first boot.
But if we do not want that strange behaviour by default, we can empty /etc/machine-id in case of read-only-rootfs (in rootfs post commands), or we let the decision to the user by letting him choose using the systemd PACKAGECONFIG to enable/disable the firstboot support.

Quoted 4 lines

firstboot usage, really doesn't feel like a piece of package configuration to me, possibly distro config, but having a (potentially) interactive boot seems like an outlier in the OE world (though I'm not saying it shouldn't be supported).

It is already a PACKAGECONFIG of systemd, not enabled by default (not sure it was clear in my previous message).

Quoted 4 lines

Today we call the systemctl script as part of rootfs construction which may touch /etc/machine-id, removing it from there and having it obviously in one place as part of rootfs construction seems like it ought to be an improvement.

I do agree.

I do have a question, it is possible to not end up with the /etc/machine-id file, i.e. not having the python script called (on master)?
If it is not possible, it means there is now way to have the first boot mechanism working (without tweaks), and so there is no risk to create an empty file in all cases (if the file does not exist, to avoid breaking builds that were embedding it in some way) no? And then we could move the discussion about the first boot support (empty or "uninitalized") in another thread as it is not the point of this patch (and avoid blocking it)?
But if it is possible, no idea. Break the support but add documentation, a new image feature, or a distro feature (so systemd PACKAGECONFIG could adapt its default value)?

Quoted 13 lines
Quoted 13 lines
Quoted 10 lines

Richard, could you please point me out which test actually failed? I looked at the logs, but most of them were "Skipped".
Does the test explicitly remount rootfs as RO perhaps?

The other option to have a real machine ID will probably mean to build machine id and leave it up to a user to decide whether they want to have such image. But I don't think it's wise to add this option now so it ends up in the next release.

Slava

Regards,

Quoted 8 lines
Quoted 3 lines

--
Alexis Cellier
Smile ECS Developer

--
Alex Kiernan

Regards,

Signature · 2 lines
Alexis Cellier
Smile ECS Developer
#14 lore raw
Quoted 78 lines · Alexis CELLIER
Quoted 76 lines · Alex Kiernan
Quoted 73 lines · Alexis CELLIER
Quoted 40 lines · Vyacheslav Yurkov via lists.openembedded.org
Quoted 29 lines · Ross Burton
Quoted 1 line · Vyacheslav Yurkov

Isn't is supposed to be created on first boot?

Yes, ish, but our rootfs is read only at this point:

[    7.639766] systemd[1]: System cannot boot: Missing /etc/machine-id and /etc is mounted read-only.
[    7.641135] systemd[1]: Booting up is supported only when:
[    7.641844] systemd[1]: 1) /etc/machine-id exists and is populated.
[    7.642588] systemd[1]: 2) /etc/machine-id exists and is empty.
[    7.643279] systemd[1]: 3) /etc/machine-id is missing and /etc is writable.
Quoted 11 lines · Alex Kiernan

I do know that it was a horrible thing to get right when I rewrote the shell script in python (which I'm very glad to see the back of):

# If we populate the systemd links we also create /etc/machine-id, which # allows systemd to boot with the filesystem read-only before generating # a real value and then committing it back.
#
# For the stateless configuration, where /etc is generated at runtime # (for example on a tmpfs), this script shouldn't run at all and we # allow systemd to completely populate /etc.

Quoted 2 lines

It may be that this behaviour was wrong in some cases, but at the same time, I remember spending a lot of time getting this right!

It certainly looks like it’s “tricky” to get the right semantics here because the range of potential use-cases is too great.

I believe we should be writing /etc/machine-id at rootfs time if systemd is being used. Currently this is done in the systemctl class if it writes links, and then in the rootfs class if the rootfs is read-only.

Quoted 8 lines

I think we should consolidate those into the rootfs class, and give the user the choice of whether to treat the image as booted or not by writing either “” or “uninitialized” to the file as per https://www.freedesktop.org/software/systemd/man/latest/machine-id.html#First%20Boot%20Semantics.

tl;dr: if it contains “uninitialized” then this is the first boot, otherwise it’s not. By always touching the file we break the ability for anyone to use systemd-first-boot or ConditionFirstBoot in their own units, which seems wrong.

We’re about a week from feature freeze and getting this merged would be great, do you think you can finish this off Slava?

Thanks,
Ross

Quoted 7 lines

I definitely gonna try. But I might be missing something, because I thought it sums up as follows:

RO rootfs -> empty machine-id
RW rootfs -> no machine-id

And it does work that way when I rebuild my image with or without RO rootfs.

At the moment on master, as systemd-systemctl python script touch the file, we can end up with an empty file independently of the IMAGE_FEATURE read-only-rootfs. So we do break the ability of firstboot in any case.
Also, if you boot with your rootfs in ro (KERNEL_ARGS, wic, ...) without setting read-only-rootfs, with fstab that will remount your rootfs in rw (so you do not have a read-only rootfs), not having the /etc/machine-id file will make systemd fail (no not having /etc/machine-id seems dangerous te me).

So why not put "uninitialized" in /etc/machine-id all the time (probably in rootfs post commands, if the file does not exist yet, to allow user to put a predefine one), it allows firstboot and prevent the dangerous case of not having the file. The only strange case would be when you do have a read-only filesystem, as you will always have a first boot.
But if we do not want that strange behaviour by default, we can empty /etc/machine-id in case of read-only-rootfs (in rootfs post commands), or we let the decision to the user by letting him choose using the systemd PACKAGECONFIG to enable/disable the firstboot support.

Quoted 8 lines
Quoted 4 lines

firstboot usage, really doesn't feel like a piece of package configuration to me, possibly distro config, but having a (potentially) interactive boot seems like an outlier in the OE world (though I'm not saying it shouldn't be supported).

It is already a PACKAGECONFIG of systemd, not enabled by default (not sure it was clear in my previous message).

I didn't look... though that's not unreasonable - I was thinking of it as PACKAGECONFIG controlling whether firstboot happens.

Quoted 10 lines
Quoted 4 lines

Today we call the systemctl script as part of rootfs construction which may touch /etc/machine-id, removing it from there and having it obviously in one place as part of rootfs construction seems like it ought to be an improvement.

I do agree.

I do have a question, it is possible to not end up with the /etc/machine-id file, i.e. not having the python script called (on master)?

You can - if you add stateless-rootfs to IMAGE_FEATURES then systemctl isn't called at image construction time, hence this part of the comment in the script:

# For the stateless configuration, where /etc is generated at runtime # (for example on a tmpfs), this script shouldn't run at all and we # allow systemd to completely populate /etc.

Signature · 1 line
Alex Kiernan
#15 lore raw
In reply to Ross Burton
Quoted 28 lines

Op 25 feb 2025, om 15:48 heeft Ross Burton via lists.openembedded.org <[email protected]> het volgende geschreven:

Quoted 1 line · Vyacheslav Yurkov

Isn't is supposed to be created on first boot?

Yes, ish, but our rootfs is read only at this point:

[    7.639766] systemd[1]: System cannot boot: Missing /etc/machine-id and /etc is mounted read-only.
[    7.641135] systemd[1]: Booting up is supported only when:
[    7.641844] systemd[1]: 1) /etc/machine-id exists and is populated.
[    7.642588] systemd[1]: 2) /etc/machine-id exists and is empty.
[    7.643279] systemd[1]: 3) /etc/machine-id is missing and /etc is writable.
Quoted 11 lines · Alex Kiernan

I do know that it was a horrible thing to get right when I rewrote the shell script in python (which I'm very glad to see the back of):

# If we populate the systemd links we also create /etc/machine-id, which

  # allows systemd to boot with the filesystem read-only before generating
  # a real value and then committing it back.
  #
  # For the stateless configuration, where /etc is generated at runtime
  # (for example on a tmpfs), this script shouldn't run at all and we
  # allow systemd to completely populate /etc.
Quoted 8 lines
Quoted 2 lines

It may be that this behaviour was wrong in some cases, but at the same time, I remember spending a lot of time getting this right!

It certainly looks like it’s “tricky” to get the right semantics here because the range of potential use-cases is too great.

I believe we should be writing /etc/machine-id at rootfs time if systemd is being used. Currently this is done in the systemctl class if it writes links, and then in the rootfs class if the rootfs is read-only.

I think we should consolidate those into the rootfs class, and give the user the choice of whether to treat the image as booted or not by writing either “” or “uninitialized” to the file as per https://www.freedesktop.org/software/systemd/man/latest/machine-id.html#First%20Boot%20Semantics.

Generating the machine ID during the build will result in multiple devices sharing the same ID in the wild, which is something we should try to avoid as being the default. It is no fun to debug a cloud where gitlab-runner instances all generate the same token in separate VMs.

So when we a-priori know it will be a read-only FS, option 2) is the least bad one, but you'll get a different ID each boot.

regards,

Koen

#16 lore raw
In reply to Vyacheslav Yurkov
Quoted 8 lines · Vyacheslav Yurkov

From meta/classes-recipe/rootfs-postcommands.bbclass:

# 20:12 < mezcalero> koen: you have three options: a) run systemd-machine-id-setup at install time, b) have / read-only and an empty file there (for stateless) and c) boot with / writable
touch ${IMAGE_ROOTFS}${sysconfdir}/machine-id
fi

We could think of further improvements. One thing that comes to mind is changing systemd-systemctl-native to systemd-tools-native and enable the build of systemd-machine-id-setup.

As Alex said the old code just touched a file, so that seems overkill. :)

Ross