systemd: Build the systemctl executable
PATCH v6 1/1 systemd: Build the systemctl executable
From: Vyacheslav Yurkov <[email protected]>
Instead of the python re-implementation build the actual systemctl from the systemd source tree. The python script was used when systemd didn't provide an option to build individual executables. It is possible in the meantime, so instead of always adapting the script when there's a new functionality, we simply use upstream implementation.
License-Update: Base recipe is used
--- .../systemd/systemd-systemctl-native.bb | 23 +- .../systemd/systemd-systemctl/systemctl | 376 ------------------ 2 files changed, 11 insertions(+), 388 deletions(-) delete mode 100755 meta/recipes-core/systemd/systemd-systemctl/systemctl
meta/recipes-core/systemd/systemd-systemctl-native.bb +11 -12
@@ -1,17 +1,16 @@ -SUMMARY = "Wrapper for enabling systemd services" +SUMMARY = "Systemctl executable from systemd" -LICENSE = "MIT" -LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420" +require systemd.inc +DEPENDS = "gperf-native libcap-native util-linux-native python3-jinja2-native" -inherit native +inherit pkgconfig meson native -SRC_URI = "file://systemctl" +MESON_TARGET = "systemctl:executable" +MESON_INSTALL_TAGS = "systemctl" +EXTRA_OEMESON:append = " -Dlink-systemctl-shared=false" -S = "${WORKDIR}/sources" -UNPACKDIR = "${S}" - -do_install() { - install -d ${D}${bindir} - install -m 0755 ${S}/systemctl ${D}${bindir} -} +# Systemctl is supposed to operate on target, but the target sysroot is not +# determined at run-time, but rather set during configure +# More details are here https://github.com/systemd/systemd/issues/35897#issuecomment-2665405887 +EXTRA_OEMESON:append = " --sysconfdir ${sysconfdir_native}"
meta/recipes-core/systemd/systemd-systemctl/systemctl deleted +0 -376
376 lines not on this page · raw message
Quoted 8 lines · Vyacheslav Yurkov via lists.openembedded.org
From: Vyacheslav Yurkov <[email protected]>
Instead of the python re-implementation build the actual systemctl from the systemd source tree. The python script was used when systemd didn't provide an option to build individual executables. It is possible in the meantime, so instead of always adapting the script when there's a new functionality, we simply use upstream implementation.
If I compare an image built with master and this, there’s one notable change in the file list:
Changes to images/qemuarm64/glibc/core-image-full-cmdline (files-in-image.txt):
/etc/machine-id was removed
The good news is that apart from that the final images are identical, but that change wasn’t expected.
Ross
Isn't is supposed to be created on first boot?
Slava
Quoted 16 lines · Ross Burton
Quoted 7 lines · Vyacheslav Yurkov via lists.openembedded.org
From: Vyacheslav Yurkov <[email protected]>
Instead of the python re-implementation build the actual systemctl from the systemd source tree. The python script was used when systemd didn't provide an option to build individual executables. It is possible in the meantime, so instead of always adapting the script when there's a new functionality, we simply use upstream implementation.
If I compare an image built with master and this, there’s one notable change in the file list:
Changes to images/qemuarm64/glibc/core-image-full-cmdline (files-in-image.txt):
/etc/machine-idwas removedThe good news is that apart from that the final images are identical, but that change wasn’t expected.
Ross
A message that is not in the archive (sent off-list or not yet received): [email protected]
From meta/classes-recipe/rootfs-postcommands.bbclass:
# 20:12 < mezcalero> koen: you have three options: a) run systemd-machine-id-setup at install time, b) have / read-only and an empty file there (for stateless) and c) boot with / writable touch ${IMAGE_ROOTFS}${sysconfdir}/machine-id
fi
We could think of further improvements. One thing that comes to mind is changing systemd-systemctl-native to systemd-tools-native and enable the build of systemd-machine-id-setup.
Slava
Quoted 22 lines · Vyacheslav Yurkov via lists.openembedded.org
Isn't is supposed to be created on first boot?
Slava
Quoted 16 lines · Ross Burton
Quoted 7 lines · Vyacheslav Yurkov via lists.openembedded.org
From: Vyacheslav Yurkov <[email protected]>
Instead of the python re-implementation build the actual systemctl from the systemd source tree. The python script was used when systemd didn't provide an option to build individual executables. It is possible in the meantime, so instead of always adapting the script when there's a new functionality, we simply use upstream implementation.
If I compare an image built with master and this, there’s one notable change in the file list:
Changes to images/qemuarm64/glibc/core-image-full-cmdline (files-in-image.txt):
/etc/machine-idwas removedThe good news is that apart from that the final images are identical, but that change wasn’t expected.
Ross
Quoted 6 lines · Vyacheslav Yurkov via lists.openembedded.org
From meta/classes-recipe/rootfs-postcommands.bbclass:
# 20:12 < mezcalero> koen: you have three options: a) run
systemd-machine-id-setup at install time, b) have / read-only and an empty file there (for stateless) and c) boot with / writable
Quoted 9 lines
touch ${IMAGE_ROOTFS}${sysconfdir}/machine-id fiWe could think of further improvements. One thing that comes to mind is changing systemd-systemctl-native to systemd-tools-native and enable the build of systemd-machine-id-setup.
Slava
I do know that it was a horrible thing to get right when I rewrote the shell script in python (which I'm very glad to see the back of):
# If we populate the systemd links we also create /etc/machine-id, which # allows systemd to boot with the filesystem read-only before generating # a real value and then committing it back.
#
# For the stateless configuration, where /etc is generated at runtime # (for example on a tmpfs), this script shouldn't run at all and we # allow systemd to completely populate /etc.
It may be that this behaviour was wrong in some cases, but at the same time, I remember spending a lot of time getting this right!
Alex
Quoted 39 lines · Vyacheslav Yurkov via lists.openembedded.org
Quoted 28 lines
Isn't is supposed to be created on first boot?
Slava
Quoted 22 lines · Ross Burton
Quoted 8 lines · Vyacheslav Yurkov via lists.openembedded.org
From: Vyacheslav Yurkov <[email protected]>
Instead of the python re-implementation build the actual systemctl from the systemd source tree. The python script was used when systemd didn't provide an option to build individual executables. It is possible in the
meantime, so instead of always adapting the script when there's a new functionality, we simply use upstream implementation.If I compare an image built with master and this, there’s one notable change in the file list:
Changes to images/qemuarm64/glibc/core-image-full-cmdline (files-in-image.txt):
/etc/machine-idwas removedThe good news is that apart from that the final images are identical, but that change wasn’t expected.
Ross
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#211766): https://lists.openembedded.org/g/openembedded-core/message/211766
Mute This Topic: https://lists.openembedded.org/mt/111267385/3618097
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Signature · 1 line
Alex Kiernan
Quoted 30 lines · Alex Kiernan via lists.openembedded.org
Quoted 9 lines · Vyacheslav Yurkov via lists.openembedded.org
From meta/classes-recipe/rootfs-postcommands.bbclass:
# 20:12 < mezcalero> koen: you have three options: a) run systemd-machine-id-setup at install time, b) have / read-only and an empty file there (for stateless) and c) boot with / writable touch ${IMAGE_ROOTFS}${sysconfdir}/machine-id
fiQuoted 6 lines
We could think of further improvements. One thing that comes to mind is changing systemd-systemctl-native to systemd-tools-native and enable the build of systemd-machine-id-setup.
Slava
I do know that it was a horrible thing to get right when I rewrote the shell script in python (which I'm very glad to see the back of):
# If we populate the systemd links we also create
/etc/machine-id, which# allows systemd to boot with the filesystem read-only before generating # a real value and then committing it back. # # For the stateless configuration, where /etc is generated at runtime # (for example on a tmpfs), this script shouldn't run at all and we # allow systemd to completely populate /etc.
Quoted 2 lines
It may be that this behaviour was wrong in some cases, but at the same time, I remember spending a lot of time getting this right!
I just wanted to add that we've been seeing an odd failure on meta-arm on the autobuilder:
https://autobuilder.yoctoproject.org/valkyrie/#/builders/75/builds/1032
and when I did into this, I see this in the logs:
[ 3.700441] systemd[1]: Hostname set to <sbsa-ref>. [ 3.704040] systemd[1]: System cannot boot: Missing /etc/machine-id and /etc is mounted read-only. [ 3.704488] systemd[1]: Booting up is supported only when: [ 3.705023] systemd[1]: 1) /etc/machine-id exists and is populated. [ 3.705349] systemd[1]: 2) /etc/machine-id exists and is empty. [ 3.705651] systemd[1]: 3) /etc/machine-id is missing and /etc is writable. [ 10.206513] systemd[1]: Populated /etc with preset unit settings. [ 13.034155] systemd[1]: Queued start job for default target Graphical Interface. [ 13.149568] systemd[1]: Created slice Slice /system/getty.
We therefore need to fix the issue with machine-id before we can merge this patch. I will drop it from master-next in the meantime due to this issue.
Cheers,
Richard
Hi,
Quoted 50 lines · Richard Purdie via lists.openembedded.org
Quoted 30 lines · Alex Kiernan via lists.openembedded.org
Quoted 9 lines · Vyacheslav Yurkov via lists.openembedded.org
From meta/classes-recipe/rootfs-postcommands.bbclass:
# 20:12 < mezcalero> koen: you have three options: a) run systemd-machine-id-setup at install time, b) have / read-only and an empty file there (for stateless) and c) boot with / writable touch ${IMAGE_ROOTFS}${sysconfdir}/machine-id
fiQuoted 6 lines
We could think of further improvements. One thing that comes to mind is changing systemd-systemctl-native to systemd-tools-native and enable the build of systemd-machine-id-setup.
Slava
I do know that it was a horrible thing to get right when I rewrote the shell script in python (which I'm very glad to see the back of):
# If we populate the systemd links we also create
/etc/machine-id, which# allows systemd to boot with the filesystem read-only before generating # a real value and then committing it back. # # For the stateless configuration, where /etc is generated at runtime # (for example on a tmpfs), this script shouldn't run at all and we # allow systemd to completely populate /etc.Quoted 2 lines
It may be that this behaviour was wrong in some cases, but at the same time, I remember spending a lot of time getting this right!
I just wanted to add that we've been seeing an odd failure on meta-arm on the autobuilder:
https://autobuilder.yoctoproject.org/valkyrie/#/builders/75/builds/1032
and when I did into this, I see this in the logs:
[ 3.700441] systemd[1]: Hostname set to <sbsa-ref>. [ 3.704040] systemd[1]: System cannot boot: Missing /etc/machine-id and /etc is mounted read-only. [ 3.704488] systemd[1]: Booting up is supported only when: [ 3.705023] systemd[1]: 1) /etc/machine-id exists and is populated. [ 3.705349] systemd[1]: 2) /etc/machine-id exists and is empty. [ 3.705651] systemd[1]: 3) /etc/machine-id is missing and /etc is writable. [ 10.206513] systemd[1]: Populated /etc with preset unit settings. [ 13.034155] systemd[1]: Queued start job for default target Graphical Interface. [ 13.149568] systemd[1]: Created slice Slice /system/getty.
Quoted 3 lines
We therefore need to fix the issue with machine-id before we can merge this patch. I will drop it from master-next in the meantime due to this issue.
I think option 2) should be simple. IMO this could also be done by image/rootfs classes or base-files when systemd is the init.
Where is build https://autobuilder.yoctoproject.org/valkyrie/#/builders/75/builds/1032
adding "read-only-rootfs" to IMAGE_FEATURES?
The failures log from step 22, full log in
https://autobuilder.yoctoproject.org/valkyrie/api/v2/logs/1586278/raw_inline
only shows issues with sshd server, which IMO is not installed to any of the built images. The builds are IMO missing IMAGE_FEATURES += "ssh-server-dropbear" which in meta-arm kas builds is set with ci/testimage.yml.
Even the core-image-minimal tests are passing without any communication with target qemu machine:
* ping.PingTest.test_ping with slirp pings localhost, not target machine
* parselogs.ParseLogsTest.test_get_context doesn't talk with booted qemu machine
How to get to the qemu boot log which shows the systemd errors above?
I don't see the connection between the systemd error and the build above. I could propose patches to the issues if I can figure out where :/
Cheers,
-Mikko
Quoted 57 lines · Mikko Rapeli
Hi,
Quoted 50 lines · Richard Purdie via lists.openembedded.org
Quoted 30 lines · Alex Kiernan via lists.openembedded.org
Quoted 9 lines · Vyacheslav Yurkov via lists.openembedded.org
From meta/classes-recipe/rootfs-postcommands.bbclass:
# 20:12 < mezcalero> koen: you have three options: a) run systemd-machine-id-setup at install time, b) have / read-only and an empty file there (for stateless) and c) boot with / writable touch ${IMAGE_ROOTFS}${sysconfdir}/machine-id
fiQuoted 6 lines
We could think of further improvements. One thing that comes to mind is changing systemd-systemctl-native to systemd-tools-native and enable the build of systemd-machine-id-setup.
Slava
I do know that it was a horrible thing to get right when I rewrote the shell script in python (which I'm very glad to see the back of):
# If we populate the systemd links we also create
/etc/machine-id, which# allows systemd to boot with the filesystem read-only before generating # a real value and then committing it back. # # For the stateless configuration, where /etc is generated at runtime # (for example on a tmpfs), this script shouldn't run at all and we # allow systemd to completely populate /etc.Quoted 2 lines
It may be that this behaviour was wrong in some cases, but at the same time, I remember spending a lot of time getting this right!
I just wanted to add that we've been seeing an odd failure on meta-arm on the autobuilder:
https://autobuilder.yoctoproject.org/valkyrie/#/builders/75/builds/1032
and when I did into this, I see this in the logs:
[ 3.700441] systemd[1]: Hostname set to <sbsa-ref>.
[ 3.704040] systemd[1]: System cannot boot: Missing/etc/machine-idand /etc is mounted read-only. [ 3.704488] systemd[1]: Booting up is supported only when:
[ 3.705023] systemd[1]: 1)/etc/machine-idexists and is populated.
[ 3.705349] systemd[1]: 2)/etc/machine-idexists and is empty.
[ 3.705651] systemd[1]: 3)/etc/machine-idis missing and /etc is writable.
[ 10.206513] systemd[1]: Populated /etc with preset unit settings.
[ 13.034155] systemd[1]: Queued start job for default target Graphical Interface.
[ 13.149568] systemd[1]: Created slice Slice /system/getty.Quoted 3 lines
We therefore need to fix the issue with machine-id before we can merge this patch. I will drop it from master-next in the meantime due to this issue.
Quoted 5 lines
I think option 2) should be simple. IMO this could also be done by image/rootfs classes or base-files when systemd is the init.
Where is build https://autobuilder.yoctoproject.org/valkyrie/#/builders/75/builds/1032
adding "read-only-rootfs" toIMAGE_FEATURES?
It isn't a read only image, just that it seems the filesystem is mounted ro at this point in the boot. Maybe timing related?
Quoted 5 lines
The failures log from step 22, full log in
https://autobuilder.yoctoproject.org/valkyrie/api/v2/logs/1586278/raw_inline
only shows issues with sshd server, which IMO is not installed to any of the built images. The builds are IMO missingIMAGE_FEATURES+= "ssh-server-dropbear" which in meta-arm kas builds is set withci/testimage.yml.
Those images do contain an ssh server, it just isn't taking connections for some reason.
Quoted 7 lines
Even the core-image-minimal tests are passing without any communication with target qemu machine:
* ping.PingTest.test_ping with slirp pings localhost, not target machine
* parselogs.ParseLogsTest.test_get_context doesn't talk with booted qemu machineHow to get to the qemu boot log which shows the systemd errors above?
I had to ssh into the autobuilder worker in question to obtain it unfortunately.
Quoted 2 lines
I don't see the connection between the systemd error and the build above. I could propose patches to the issues if I can figure out where :/
Ross was able to reproduce by including the patch in this thread, building and then running testimage against core-image-sato/core-image- weston. I'd note those tests are using slirp which may or may not be related. The issue is therefore reproducible to some degree at least.
Cheers,
Richard
Quoted 1 line · Vyacheslav Yurkov
Isn't is supposed to be created on first boot?
Yes, ish, but our rootfs is read only at this point:
[ 7.639766] systemd[1]: System cannot boot: Missing /etc/machine-id and /etc is mounted read-only. [ 7.641135] systemd[1]: Booting up is supported only when: [ 7.641844] systemd[1]: 1) /etc/machine-id exists and is populated. [ 7.642588] systemd[1]: 2) /etc/machine-id exists and is empty. [ 7.643279] systemd[1]: 3) /etc/machine-id is missing and /etc is writable.
Quoted 11 lines · Alex Kiernan
I do know that it was a horrible thing to get right when I rewrote the shell script in python (which I'm very glad to see the back of):
# If we populate the systemd links we also create
/etc/machine-id, which# allows systemd to boot with the filesystem read-only before generating # a real value and then committing it back. # # For the stateless configuration, where /etc is generated at runtime # (for example on a tmpfs), this script shouldn't run at all and we # allow systemd to completely populate /etc.
Quoted 2 lines
It may be that this behaviour was wrong in some cases, but at the same time, I remember spending a lot of time getting this right!
It certainly looks like it’s “tricky” to get the right semantics here because the range of potential use-cases is too great.
I believe we should be writing /etc/machine-id at rootfs time if systemd is being used. Currently this is done in the systemctl class if it writes links, and then in the rootfs class if the rootfs is read-only.
I think we should consolidate those into the rootfs class, and give the user the choice of whether to treat the image as booted or not by writing either “” or “uninitialized” to the file as per https://www.freedesktop.org/software/systemd/man/latest/machine-id.html#First%20Boot%20Semantics.
tl;dr: if it contains “uninitialized” then this is the first boot, otherwise it’s not. By always touching the file we break the ability for anyone to use systemd-first-boot or ConditionFirstBoot in their own units, which seems wrong.
We’re about a week from feature freeze and getting this merged would be great, do you think you can finish this off Slava?
Thanks,
Ross
Quoted 29 lines · Ross Burton
Quoted 1 line · Vyacheslav Yurkov
Isn't is supposed to be created on first boot?
Yes, ish, but our rootfs is read only at this point:
[ 7.639766] systemd[1]: System cannot boot: Missing /etc/machine-id and /etc is mounted read-only. [ 7.641135] systemd[1]: Booting up is supported only when: [ 7.641844] systemd[1]: 1) /etc/machine-id exists and is populated. [ 7.642588] systemd[1]: 2) /etc/machine-id exists and is empty. [ 7.643279] systemd[1]: 3) /etc/machine-id is missing and /etc is writable.Quoted 11 lines · Alex Kiernan
I do know that it was a horrible thing to get right when I rewrote the shell script in python (which I'm very glad to see the back of):
# If we populate the systemd links we also create
/etc/machine-id, which # allows systemd to boot with the filesystem read-only before generating # a real value and then committing it back.
#
# For the stateless configuration, where /etc is generated at runtime # (for example on a tmpfs), this script shouldn't run at all and we # allow systemd to completely populate /etc.Quoted 2 lines
It may be that this behaviour was wrong in some cases, but at the same time, I remember spending a lot of time getting this right!
It certainly looks like it’s “tricky” to get the right semantics here because the range of potential use-cases is too great.
I believe we should be writing
/etc/machine-idat rootfs time if systemd is being used. Currently this is done in the systemctl class if it writes links, and then in the rootfs class if the rootfs is read-only.
Quoted 8 lines
I think we should consolidate those into the rootfs class, and give the user the choice of whether to treat the image as booted or not by writing either “” or “uninitialized” to the file as per https://www.freedesktop.org/software/systemd/man/latest/machine-id.html#First%20Boot%20Semantics.
tl;dr: if it contains “uninitialized” then this is the first boot, otherwise it’s not. By always touching the file we break the ability for anyone to use systemd-first-boot or ConditionFirstBoot in their own units, which seems wrong.
We’re about a week from feature freeze and getting this merged would be great, do you think you can finish this off Slava?
Thanks,
Ross
I definitely gonna try. But I might be missing something, because I thought it sums up as follows:
RO rootfs -> empty machine-id
RW rootfs -> no machine-id
And it does work that way when I rebuild my image with or without RO rootfs.
Richard, could you please point me out which test actually failed? I looked at the logs, but most of them were "Skipped". Does the test explicitly remount rootfs as RO perhaps?
The other option to have a real machine ID will probably mean to build machine id and leave it up to a user to decide whether they want to have such image. But I don't think it's wise to add this option now so it ends up in the next release.
Slava
Quoted 40 lines · Vyacheslav Yurkov via lists.openembedded.org
Quoted 29 lines · Ross Burton
Quoted 1 line · Vyacheslav Yurkov
Isn't is supposed to be created on first boot?
Yes, ish, but our rootfs is read only at this point:
[ 7.639766] systemd[1]: System cannot boot: Missing /etc/machine-id and /etc is mounted read-only. [ 7.641135] systemd[1]: Booting up is supported only when: [ 7.641844] systemd[1]: 1) /etc/machine-id exists and is populated. [ 7.642588] systemd[1]: 2) /etc/machine-id exists and is empty. [ 7.643279] systemd[1]: 3) /etc/machine-id is missing and /etc is writable.Quoted 11 lines · Alex Kiernan
I do know that it was a horrible thing to get right when I rewrote the shell script in python (which I'm very glad to see the back of):
# If we populate the systemd links we also create
/etc/machine-id, which # allows systemd to boot with the filesystem read-only before generating # a real value and then committing it back.
#
# For the stateless configuration, where /etc is generated at runtime # (for example on a tmpfs), this script shouldn't run at all and we # allow systemd to completely populate /etc.Quoted 2 lines
It may be that this behaviour was wrong in some cases, but at the same time, I remember spending a lot of time getting this right!
It certainly looks like it’s “tricky” to get the right semantics here because the range of potential use-cases is too great.
I believe we should be writing
/etc/machine-idat rootfs time if systemd is being used. Currently this is done in the systemctl class if it writes links, and then in the rootfs class if the rootfs is read-only.Quoted 8 lines
I think we should consolidate those into the rootfs class, and give the user the choice of whether to treat the image as booted or not by writing either “” or “uninitialized” to the file as per https://www.freedesktop.org/software/systemd/man/latest/machine-id.html#First%20Boot%20Semantics.
tl;dr: if it contains “uninitialized” then this is the first boot, otherwise it’s not. By always touching the file we break the ability for anyone to use systemd-first-boot or ConditionFirstBoot in their own units, which seems wrong.
We’re about a week from feature freeze and getting this merged would be great, do you think you can finish this off Slava?
Thanks,
Ross
Quoted 7 lines
I definitely gonna try. But I might be missing something, because I thought it sums up as follows:
RO rootfs -> empty machine-id
RW rootfs -> no machine-idAnd it does work that way when I rebuild my image with or without RO rootfs.
At the moment on master, as systemd-systemctl python script touch the file, we can end up with an empty file independently of the IMAGE_FEATURE read-only-rootfs. So we do break the ability of firstboot in any case.
Also, if you boot with your rootfs in ro (KERNEL_ARGS, wic, ...) without setting read-only-rootfs, with fstab that will remount your rootfs in rw (so you do not have a read-only rootfs), not having the /etc/machine-id file will make systemd fail (no not having /etc/machine-id seems dangerous te me).
So why not put "uninitialized" in /etc/machine-id all the time (probably in rootfs post commands, if the file does not exist yet, to allow user to put a predefine one), it allows firstboot and prevent the dangerous case of not having the file. The only strange case would be when you do have a read-only filesystem, as you will always have a first boot.
But if we do not want that strange behaviour by default, we can empty /etc/machine-id in case of read-only-rootfs (in rootfs post commands), or we let the decision to the user by letting him choose using the systemd PACKAGECONFIG to enable/disable the firstboot support.
Quoted 10 lines
Richard, could you please point me out which test actually failed? I looked at the logs, but most of them were "Skipped".
Does the test explicitly remount rootfs as RO perhaps?The other option to have a real machine ID will probably mean to build machine id and leave it up to a user to decide whether they want to have such image. But I don't think it's wise to add this option now so it ends up in the next release.
Slava
Regards,
Signature · 2 lines
Alexis Cellier Smile ECS Developer
Quoted 62 lines · Alexis CELLIER
Quoted 40 lines · Vyacheslav Yurkov via lists.openembedded.org
Quoted 29 lines · Ross Burton
Quoted 1 line · Vyacheslav Yurkov
Isn't is supposed to be created on first boot?
Yes, ish, but our rootfs is read only at this point:
[ 7.639766] systemd[1]: System cannot boot: Missing /etc/machine-id and /etc is mounted read-only. [ 7.641135] systemd[1]: Booting up is supported only when: [ 7.641844] systemd[1]: 1) /etc/machine-id exists and is populated. [ 7.642588] systemd[1]: 2) /etc/machine-id exists and is empty. [ 7.643279] systemd[1]: 3) /etc/machine-id is missing and /etc is writable.Quoted 11 lines · Alex Kiernan
I do know that it was a horrible thing to get right when I rewrote the shell script in python (which I'm very glad to see the back of):
# If we populate the systemd links we also create
/etc/machine-id, which # allows systemd to boot with the filesystem read-only before generating # a real value and then committing it back.
#
# For the stateless configuration, where /etc is generated at runtime # (for example on a tmpfs), this script shouldn't run at all and we # allow systemd to completely populate /etc.Quoted 2 lines
It may be that this behaviour was wrong in some cases, but at the same time, I remember spending a lot of time getting this right!
It certainly looks like it’s “tricky” to get the right semantics here because the range of potential use-cases is too great.
I believe we should be writing
/etc/machine-idat rootfs time if systemd is being used. Currently this is done in the systemctl class if it writes links, and then in the rootfs class if the rootfs is read-only.Quoted 8 lines
I think we should consolidate those into the rootfs class, and give the user the choice of whether to treat the image as booted or not by writing either “” or “uninitialized” to the file as per https://www.freedesktop.org/software/systemd/man/latest/machine-id.html#First%20Boot%20Semantics.
tl;dr: if it contains “uninitialized” then this is the first boot, otherwise it’s not. By always touching the file we break the ability for anyone to use systemd-first-boot or ConditionFirstBoot in their own units, which seems wrong.
We’re about a week from feature freeze and getting this merged would be great, do you think you can finish this off Slava?
Thanks,
RossQuoted 7 lines
I definitely gonna try. But I might be missing something, because I thought it sums up as follows:
RO rootfs -> empty machine-id
RW rootfs -> no machine-idAnd it does work that way when I rebuild my image with or without RO rootfs.
At the moment on master, as systemd-systemctl python script touch the file, we can end up with an empty file independently of the
IMAGE_FEATUREread-only-rootfs. So we do break the ability of firstboot in any case.
Also, if you boot with your rootfs in ro (KERNEL_ARGS, wic, ...) without setting read-only-rootfs, with fstab that will remount your rootfs in rw (so you do not have a read-only rootfs), not having the/etc/machine-idfile will make systemd fail (no not having/etc/machine-idseems dangerous te me).
Quoted 11 lines
So why not put "uninitialized" in
/etc/machine-idall the time (probably in rootfs post commands, if the file does not exist yet, to allow user to put a predefine one), it allows firstboot and prevent the dangerous case of not having the file. The only strange case would be when you do have a read-only filesystem, as you will always have a first boot.
But if we do not want that strange behaviour by default, we can empty/etc/machine-idin case of read-only-rootfs (in rootfs post commands), or we let the decision to the user by letting him choose using the systemd PACKAGECONFIG to enable/disable the firstboot support.
firstboot usage, really doesn't feel like a piece of package configuration to me, possibly distro config, but having a (potentially) interactive boot seems like an outlier in the OE world (though I'm not saying it shouldn't be supported).
Today we call the systemctl script as part of rootfs construction which may touch /etc/machine-id, removing it from there and having it obviously in one place as part of rootfs construction seems like it ought to be an improvement.
Quoted 16 lines
Quoted 10 lines
Richard, could you please point me out which test actually failed? I looked at the logs, but most of them were "Skipped".
Does the test explicitly remount rootfs as RO perhaps?The other option to have a real machine ID will probably mean to build machine id and leave it up to a user to decide whether they want to have such image. But I don't think it's wise to add this option now so it ends up in the next release.
Slava
Regards,
--
Alexis Cellier
Smile ECS Developer
Signature · 1 line
Alex Kiernan
Quoted 76 lines · Alex Kiernan
Quoted 62 lines · Alexis CELLIER
Quoted 40 lines · Vyacheslav Yurkov via lists.openembedded.org
Quoted 29 lines · Ross Burton
Quoted 1 line · Vyacheslav Yurkov
Isn't is supposed to be created on first boot?
Yes, ish, but our rootfs is read only at this point:
[ 7.639766] systemd[1]: System cannot boot: Missing /etc/machine-id and /etc is mounted read-only. [ 7.641135] systemd[1]: Booting up is supported only when: [ 7.641844] systemd[1]: 1) /etc/machine-id exists and is populated. [ 7.642588] systemd[1]: 2) /etc/machine-id exists and is empty. [ 7.643279] systemd[1]: 3) /etc/machine-id is missing and /etc is writable.Quoted 11 lines · Alex Kiernan
I do know that it was a horrible thing to get right when I rewrote the shell script in python (which I'm very glad to see the back of):
# If we populate the systemd links we also create
/etc/machine-id, which # allows systemd to boot with the filesystem read-only before generating # a real value and then committing it back.
#
# For the stateless configuration, where /etc is generated at runtime # (for example on a tmpfs), this script shouldn't run at all and we # allow systemd to completely populate /etc.Quoted 2 lines
It may be that this behaviour was wrong in some cases, but at the same time, I remember spending a lot of time getting this right!
It certainly looks like it’s “tricky” to get the right semantics here because the range of potential use-cases is too great.
I believe we should be writing
/etc/machine-idat rootfs time if systemd is being used. Currently this is done in the systemctl class if it writes links, and then in the rootfs class if the rootfs is read-only.Quoted 8 lines
I think we should consolidate those into the rootfs class, and give the user the choice of whether to treat the image as booted or not by writing either “” or “uninitialized” to the file as per https://www.freedesktop.org/software/systemd/man/latest/machine-id.html#First%20Boot%20Semantics.
tl;dr: if it contains “uninitialized” then this is the first boot, otherwise it’s not. By always touching the file we break the ability for anyone to use systemd-first-boot or ConditionFirstBoot in their own units, which seems wrong.
We’re about a week from feature freeze and getting this merged would be great, do you think you can finish this off Slava?
Thanks,
RossQuoted 7 lines
I definitely gonna try. But I might be missing something, because I thought it sums up as follows:
RO rootfs -> empty machine-id
RW rootfs -> no machine-idAnd it does work that way when I rebuild my image with or without RO rootfs.
At the moment on master, as systemd-systemctl python script touch the file, we can end up with an empty file independently of the
IMAGE_FEATUREread-only-rootfs. So we do break the ability of firstboot in any case.
Also, if you boot with your rootfs in ro (KERNEL_ARGS, wic, ...) without setting read-only-rootfs, with fstab that will remount your rootfs in rw (so you do not have a read-only rootfs), not having the/etc/machine-idfile will make systemd fail (no not having/etc/machine-idseems dangerous te me).Quoted 11 lines
So why not put "uninitialized" in
/etc/machine-idall the time (probably in rootfs post commands, if the file does not exist yet, to allow user to put a predefine one), it allows firstboot and prevent the dangerous case of not having the file. The only strange case would be when you do have a read-only filesystem, as you will always have a first boot.
But if we do not want that strange behaviour by default, we can empty/etc/machine-idin case of read-only-rootfs (in rootfs post commands), or we let the decision to the user by letting him choose using the systemd PACKAGECONFIG to enable/disable the firstboot support.
Quoted 4 lines
firstboot usage, really doesn't feel like a piece of package configuration to me, possibly distro config, but having a (potentially) interactive boot seems like an outlier in the OE world (though I'm not saying it shouldn't be supported).
It is already a PACKAGECONFIG of systemd, not enabled by default (not sure it was clear in my previous message).
Quoted 4 lines
Today we call the systemctl script as part of rootfs construction which may touch
/etc/machine-id, removing it from there and having it obviously in one place as part of rootfs construction seems like it ought to be an improvement.
I do agree.
I do have a question, it is possible to not end up with the /etc/machine-id file, i.e. not having the python script called (on master)?
If it is not possible, it means there is now way to have the first boot mechanism working (without tweaks), and so there is no risk to create an empty file in all cases (if the file does not exist, to avoid breaking builds that were embedding it in some way) no? And then we could move the discussion about the first boot support (empty or "uninitalized") in another thread as it is not the point of this patch (and avoid blocking it)?
But if it is possible, no idea. Break the support but add documentation, a new image feature, or a distro feature (so systemd PACKAGECONFIG could adapt its default value)?
Quoted 13 lines
Quoted 13 lines
Quoted 10 lines
Richard, could you please point me out which test actually failed? I looked at the logs, but most of them were "Skipped".
Does the test explicitly remount rootfs as RO perhaps?The other option to have a real machine ID will probably mean to build machine id and leave it up to a user to decide whether they want to have such image. But I don't think it's wise to add this option now so it ends up in the next release.
Slava
Regards,
Quoted 8 lines
Quoted 3 lines
--
Alexis Cellier
Smile ECS Developer--
Alex Kiernan
Regards,
Signature · 2 lines
Alexis Cellier Smile ECS Developer
Quoted 78 lines · Alexis CELLIER
Quoted 76 lines · Alex Kiernan
Quoted 73 lines · Alexis CELLIER
Quoted 40 lines · Vyacheslav Yurkov via lists.openembedded.org
Quoted 29 lines · Ross Burton
Quoted 1 line · Vyacheslav Yurkov
Isn't is supposed to be created on first boot?
Yes, ish, but our rootfs is read only at this point:
[ 7.639766] systemd[1]: System cannot boot: Missing /etc/machine-id and /etc is mounted read-only. [ 7.641135] systemd[1]: Booting up is supported only when: [ 7.641844] systemd[1]: 1) /etc/machine-id exists and is populated. [ 7.642588] systemd[1]: 2) /etc/machine-id exists and is empty. [ 7.643279] systemd[1]: 3) /etc/machine-id is missing and /etc is writable.Quoted 11 lines · Alex Kiernan
I do know that it was a horrible thing to get right when I rewrote the shell script in python (which I'm very glad to see the back of):
# If we populate the systemd links we also create
/etc/machine-id, which # allows systemd to boot with the filesystem read-only before generating # a real value and then committing it back.
#
# For the stateless configuration, where /etc is generated at runtime # (for example on a tmpfs), this script shouldn't run at all and we # allow systemd to completely populate /etc.Quoted 2 lines
It may be that this behaviour was wrong in some cases, but at the same time, I remember spending a lot of time getting this right!
It certainly looks like it’s “tricky” to get the right semantics here because the range of potential use-cases is too great.
I believe we should be writing
/etc/machine-idat rootfs time if systemd is being used. Currently this is done in the systemctl class if it writes links, and then in the rootfs class if the rootfs is read-only.Quoted 8 lines
I think we should consolidate those into the rootfs class, and give the user the choice of whether to treat the image as booted or not by writing either “” or “uninitialized” to the file as per https://www.freedesktop.org/software/systemd/man/latest/machine-id.html#First%20Boot%20Semantics.
tl;dr: if it contains “uninitialized” then this is the first boot, otherwise it’s not. By always touching the file we break the ability for anyone to use systemd-first-boot or ConditionFirstBoot in their own units, which seems wrong.
We’re about a week from feature freeze and getting this merged would be great, do you think you can finish this off Slava?
Thanks,
RossQuoted 7 lines
I definitely gonna try. But I might be missing something, because I thought it sums up as follows:
RO rootfs -> empty machine-id
RW rootfs -> no machine-idAnd it does work that way when I rebuild my image with or without RO rootfs.
At the moment on master, as systemd-systemctl python script touch the file, we can end up with an empty file independently of the
IMAGE_FEATUREread-only-rootfs. So we do break the ability of firstboot in any case.
Also, if you boot with your rootfs in ro (KERNEL_ARGS, wic, ...) without setting read-only-rootfs, with fstab that will remount your rootfs in rw (so you do not have a read-only rootfs), not having the/etc/machine-idfile will make systemd fail (no not having/etc/machine-idseems dangerous te me).So why not put "uninitialized" in
/etc/machine-idall the time (probably in rootfs post commands, if the file does not exist yet, to allow user to put a predefine one), it allows firstboot and prevent the dangerous case of not having the file. The only strange case would be when you do have a read-only filesystem, as you will always have a first boot.
But if we do not want that strange behaviour by default, we can empty/etc/machine-idin case of read-only-rootfs (in rootfs post commands), or we let the decision to the user by letting him choose using the systemd PACKAGECONFIG to enable/disable the firstboot support.
Quoted 8 lines
Quoted 4 lines
firstboot usage, really doesn't feel like a piece of package configuration to me, possibly distro config, but having a (potentially) interactive boot seems like an outlier in the OE world (though I'm not saying it shouldn't be supported).
It is already a PACKAGECONFIG of systemd, not enabled by default (not sure it was clear in my previous message).
I didn't look... though that's not unreasonable - I was thinking of it as PACKAGECONFIG controlling whether firstboot happens.
Quoted 10 lines
Quoted 4 lines
Today we call the systemctl script as part of rootfs construction which may touch
/etc/machine-id, removing it from there and having it obviously in one place as part of rootfs construction seems like it ought to be an improvement.I do agree.
I do have a question, it is possible to not end up with the
/etc/machine-idfile, i.e. not having the python script called (on master)?
You can - if you add stateless-rootfs to IMAGE_FEATURES then systemctl isn't called at image construction time, hence this part of the comment in the script:
# For the stateless configuration, where /etc is generated at runtime # (for example on a tmpfs), this script shouldn't run at all and we # allow systemd to completely populate /etc.
Signature · 1 line
Alex Kiernan
Quoted 28 lines
Op 25 feb 2025, om 15:48 heeft Ross Burton via lists.openembedded.org <[email protected]> het volgende geschreven:
Quoted 1 line · Vyacheslav Yurkov
Isn't is supposed to be created on first boot?
Yes, ish, but our rootfs is read only at this point:
[ 7.639766] systemd[1]: System cannot boot: Missing /etc/machine-id and /etc is mounted read-only. [ 7.641135] systemd[1]: Booting up is supported only when: [ 7.641844] systemd[1]: 1) /etc/machine-id exists and is populated. [ 7.642588] systemd[1]: 2) /etc/machine-id exists and is empty. [ 7.643279] systemd[1]: 3) /etc/machine-id is missing and /etc is writable.Quoted 11 lines · Alex Kiernan
I do know that it was a horrible thing to get right when I rewrote the shell script in python (which I'm very glad to see the back of):
# If we populate the systemd links we also create
/etc/machine-id, which# allows systemd to boot with the filesystem read-only before generating # a real value and then committing it back. # # For the stateless configuration, where /etc is generated at runtime # (for example on a tmpfs), this script shouldn't run at all and we # allow systemd to completely populate /etc.
Quoted 8 lines
Quoted 2 lines
It may be that this behaviour was wrong in some cases, but at the same time, I remember spending a lot of time getting this right!
It certainly looks like it’s “tricky” to get the right semantics here because the range of potential use-cases is too great.
I believe we should be writing
/etc/machine-idat rootfs time if systemd is being used. Currently this is done in the systemctl class if it writes links, and then in the rootfs class if the rootfs is read-only.I think we should consolidate those into the rootfs class, and give the user the choice of whether to treat the image as booted or not by writing either “” or “uninitialized” to the file as per https://www.freedesktop.org/software/systemd/man/latest/machine-id.html#First%20Boot%20Semantics.
Generating the machine ID during the build will result in multiple devices sharing the same ID in the wild, which is something we should try to avoid as being the default. It is no fun to debug a cloud where gitlab-runner instances all generate the same token in separate VMs.
So when we a-priori know it will be a read-only FS, option 2) is the least bad one, but you'll get a different ID each boot.
regards,
Koen
Quoted 8 lines · Vyacheslav Yurkov
From meta/classes-recipe/rootfs-postcommands.bbclass:
# 20:12 < mezcalero> koen: you have three options: a) run systemd-machine-id-setup at install time, b) have / read-only and an empty file there (for stateless) and c) boot with / writable
touch ${IMAGE_ROOTFS}${sysconfdir}/machine-id
fiWe could think of further improvements. One thing that comes to mind is changing systemd-systemctl-native to systemd-tools-native and enable the build of systemd-machine-id-setup.
As Alex said the old code just touched a file, so that seems overkill. :)
Ross