Skip to contentSkip to conversation

systemd: sort out the conflict between var factory and tmpfiles (branch systemdify-var)

PATCH In review Yann E. MORIN 3 participants 12 messages Oct 15, 2022 – Oct 18, 2022

lore
Share & tools
lore
#0 lore raw

cover systemd: sort out the conflict between var factory and tmpfiles (branch systemdify-var)

Hello All!

This small four-patch series touches on the delicate intricacies of the systemd package and how we integrate it in Buildroot, especially with regard to read-only filesystems.

The underlying issue is that systemd wants a writable /var filesystem (it will store a bunch of things at runtime in there). A lot of packages also expect a writable /var. After all, var stands for variadic, so it is not supposed to be set in stone and read-only.

The solution we implemented in Buildroot is to generate a factory for /var, and register that as tmpfiles, at build time, and mount a tmpfs on the then-empty /var at runtime, so that systemd-tmpfiles populates /var on boot. Having a tmpfs means /var is repopulated from a clean state at each boot, and people who want a persistent /var have to provide their own mechanism to mount their actual filesystem instead.

However, for some people, this does not seem to work as expected, so we introduced a call to systemd-tmpfiles at build time, to pre-populate the rootfs from all the tmpfiles, of which our var factory, at build time.

These two mechanism clash with each others, and are mostly incompatible, and causes files to be duplicated.

To solve the issue, this series introduces new options, so the user can elect to use either, none, or both mechanisms, as they see fit.

The first steps are borrowed from Yann@work, where we make it easier to change the location where /var is nounted from, by making it a systemd mount unit, rather than use the legacy fstab, and move the factory to /usr/share as it is a system factory (/etc is for the administartor to provide overrides). Note that those two are not strictly required for the following, but since that makes it easier to change the /var filesystem in a systemd-compliant manner, they are a nice addition to the series.

Then we move together the two mechanisms under the auspices of a common package, to make it easier to later make them conditional.

Finally, we introduce those two new options, that drive each mechanism.

Eventally, we want to highlight that, by default, the behaviour has not changed, since the two new options default to y, so as to explicitly keep the behaviour from nefore this patchset.

Regards,
Yann E. MORIN.

----------------------------------------------------------------
Yann E. MORIN (4):
package/skeleton-systemd: move /var factory out of /etc package/skeleton-systemd: systemd-ify mounting /var tmpfs with ro rootfs package/skeleton-systemd: host the tmpfiles preparation script
system: add options for var factory and tmpfiles pre-seed

 .../fakeroot_tmpfiles.sh                           |  0
 .../skeleton-init-systemd/skeleton-init-systemd.mk | 18 +++++++---
 package/skeleton-init-systemd/var.mount            | 18 ++++++++++
 package/systemd/systemd.mk                         |  6 ----
 system/Config.in                                   | 38 +++++++++++++++++++++-
 5 files changed, 69 insertions(+), 11 deletions(-)
 rename package/{systemd => skeleton-init-systemd}/fakeroot_tmpfiles.sh (100%)
 create mode 100644 package/skeleton-init-systemd/var.mount
Signature · 10 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#1 lore raw

PATCH 1/4 package/skeleton-systemd: move /var factory out of /etc

From: "Yann E. MORIN" <[email protected]>

When the rootfs is not remounted read-write (thus assuming a read-only rootfs like squashfs), we create a tmpfiles.d factory for /var.

However, we register those in /etc/tmpfiles.d/, but /etc could also be a tmpfs (for full state-less systems, or easy factory-reset, see [0]).

So, we move our var factory to /usr/lib/tmpfiles.d/, which is also the location where systemd itself places its own tmpfiles, and where we already put all our other tmpfiles (see audit, avahi, cryptsetup, dhcp, lighttpd, nfs-utils, quagga, samba4, swupdate) and our handling of systemd's catalog files too. We also rename the file to a better name, so that it is obvious it is generated by us (systemd already installs a var.conf of its own, so we want to avoid name clashing).

Last little detail: there is no need or reason to create .../tmpfiles.d/ at install time; it is only needed in the rootfs-pre-cmd hook, so we only create it just before we need it.

[0] http://0pointer.de/blog/projects/stateless.html

Cc Norbert Lange <[email protected]>
Cc Romain Naour <[email protected]>
Cc Jérémy Rosen <[email protected]>

[[email protected]:
- split original patch in two
- this one only moves out of /etc and into /usr/lib
- adapt commit log accordingly
]

---
 package/skeleton-init-systemd/skeleton-init-systemd.mk | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
package/skeleton-init-systemd/skeleton-init-systemd.mk +2 -2
index 795a171809..7b66732ef4 100644
@@ -29,7 +29,6 @@ else
 # a real (but empty) directory, and the "factory files" will be copied
 # back there by the tmpfiles.d mechanism.
 define SKELETON_INIT_SYSTEMD_ROOT_RO_OR_RW
-	mkdir -p $(TARGET_DIR)/etc/systemd/tmpfiles.d
 	echo "/dev/root / auto ro 0 1" >$(TARGET_DIR)/etc/fstab
 	echo "tmpfs /var tmpfs mode=1777 0 0" >>$(TARGET_DIR)/etc/fstab
 endef
@@ -38,6 +37,7 @@ define SKELETON_INIT_SYSTEMD_PRE_ROOTFS_VAR
 	rm -rf $(TARGET_DIR)/usr/share/factory/var
 	mv $(TARGET_DIR)/var $(TARGET_DIR)/usr/share/factory/var
 	mkdir -p $(TARGET_DIR)/var
+	mkdir -p $(TARGET_DIR)/usr/lib/tmpfiles.d
 	for i in $(TARGET_DIR)/usr/share/factory/var/* \
 		 $(TARGET_DIR)/usr/share/factory/var/lib/* \
 		 $(TARGET_DIR)/usr/share/factory/var/lib/systemd/*; do \
@@ -51,7 +51,7 @@ define SKELETON_INIT_SYSTEMD_PRE_ROOTFS_VAR
 			printf "C! %s - - - -\n" "$${j}" \
 			|| exit 1; \
 		fi; \
-	done >$(TARGET_DIR)/etc/tmpfiles.d/var-factory.conf
+	done >$(TARGET_DIR)/usr/lib/tmpfiles.d/buildroot-factory.conf
 endef
 SKELETON_INIT_SYSTEMD_ROOTFS_PRE_CMD_HOOKS += SKELETON_INIT_SYSTEMD_PRE_ROOTFS_VAR
 
Signature · 6 lines
2.25.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#2 lore raw

PATCH 2/4 package/skeleton-systemd: systemd-ify mounting /var tmpfs with ro rootfs

In reply to Yann E. MORIN

To mount our /var tmpfs when the rootfs is mounted read-only (really, not remounted reqd-write), we use an entry in fstab.

However, /etc could also be a tmpfs (for full state-less systems, or easy factory-reset, see [0]).

So, we register /var as a systemd mount unit, so that we can also have the /var factory populated and functional even when /etc is empty. The var.mount unit is heavily modelled after systemd's own tmp.mount one, so we carry the same license for that file (in case that may apply). We add an explicit reverse dependency to systemd-tmpfiles-setup.service, to ensure /var is mounted before we try to populate it.

[0] http://0pointer.de/blog/projects/stateless.html

Cc Norbert Lange <[email protected]>
Cc Romain Naour <[email protected]>
Cc Jérémy Rosen <[email protected]>

[[email protected]:
- split original patch in two
- this one only handles converting /var mounting into a systemd unit
- adapt commit log accordingly
]

---
 .../skeleton-init-systemd.mk                   |  3 ++-
 package/skeleton-init-systemd/var.mount        | 18 ++++++++++++++++++
 2 files changed, 20 insertions(+), 1 deletion(-)
 create mode 100644 package/skeleton-init-systemd/var.mount
package/skeleton-init-systemd/skeleton-init-systemd.mk +2 -1
index 7b66732ef4..970951d553 100644
@@ -30,7 +30,6 @@ else
 # back there by the tmpfiles.d mechanism.
 define SKELETON_INIT_SYSTEMD_ROOT_RO_OR_RW
 	echo "/dev/root / auto ro 0 1" >$(TARGET_DIR)/etc/fstab
-	echo "tmpfs /var tmpfs mode=1777 0 0" >>$(TARGET_DIR)/etc/fstab
 endef
 
 define SKELETON_INIT_SYSTEMD_PRE_ROOTFS_VAR
@@ -52,6 +51,8 @@ define SKELETON_INIT_SYSTEMD_PRE_ROOTFS_VAR
 			|| exit 1; \
 		fi; \
 	done >$(TARGET_DIR)/usr/lib/tmpfiles.d/buildroot-factory.conf
+	$(INSTALL) -D -m 0644 $(SKELETON_INIT_SYSTEMD_PKGDIR)/var.mount \
+		$(TARGET_DIR)/usr/lib/systemd/system/var.mount
 endef
 SKELETON_INIT_SYSTEMD_ROOTFS_PRE_CMD_HOOKS += SKELETON_INIT_SYSTEMD_PRE_ROOTFS_VAR
 
package/skeleton-init-systemd/var.mount new file +18 -0
new file mode 100644
index 0000000000..6b165dff6d
@@ -0,0 +1,18 @@
+# SPDX-License-Identifier: LGPL-2.1-or-later
+# Modelled after systemd's tmp.mount
+
+[Unit]
+Description=Buildroot /var tmpfs
+DefaultDependencies=no
+Conflicts=umount.target
+Before=basic.target local-fs.target umount.target systemd-tmpfiles-setup.service
+After=swap.target
+
+[Mount]
+What=tmpfs
+Where=/var
+Type=tmpfs
+Options=mode=1777,strictatime,nosuid,nodev,size=50%%,nr_inodes=1m
+
+[Install]
+WantedBy=basic.target
Signature · 6 lines
2.25.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#3 lore raw

PATCH 3/4 package/skeleton-systemd: host the tmpfiles preparation script

In reply to Yann E. MORIN

Commit 0d9b84b7a83f (package/systemd: invoke systemd-tmpfilesd on final image) forcefully introduced a call to systemd-tmpfiles as a per-rootfs hook, on the premise that would help with read-only rootfs.

However, that did not account for the then-pre-existing handling of /var as a factory when the user opted not to remount / read-write (by not setting BR2_TARGET_GENERIC_REMOUNT_ROOTFS_RW).

This means that, for users who want to use a factory for /var, the generated filesystem contains the factory files twice: once as stored in the factory, and once as populated by systemd-tmpefilesd.

In the hope to reconcile the two solutions, we move the handling of calling systemd-tmpfilesd to the skeleton-init-systemd package, where we already handle the /var factory. Having the two in the same package will make it easier, in the future, to provide the user with a choice whether to use one of the other.

Note that it is very important to keep the order of the hooks as they are.

Indeed, skeleton-init-systemd sorts before systemd, so its hooks were registered before systemd's hooks; now that we move the CREATE_TMPFILES hook, we must ensure it is called after the PRE_ROOTFS_VAR one, so that the behaviour of acting on the var factory remains.

As a final note: we chose the move this way, rather than move the var factory into the systemd package, because it is more related to the system integration on the Buildroot side, rather than the integration of the systemd package in Buildroot.

Similarly, the other four rootfs hooks, SYSTEMD_LOCALE_PURGE_CATALOGS, SYSTEMD_UPDATE_CATALOGS, SYSTEMD_RM_CATALOG_UPDATE_SERVICE, and specially SYSTEMD_PRESET_ALL, should also be moved out of the systemd package, because they too are more related to the Buildroot system, rather than to the systemd package itself; but the frontier is very porous is either way, for such a package as special as systemd.

Cc Norbert Lange <[email protected]>
Cc Arnout Vandecappelle (Essensium/Mind) <[email protected]>
Cc Romain Naour <[email protected]>
Cc Jérémy Rosen <[email protected]>
Cc Yann E. MORIN <[email protected]>
---
 .../{systemd => skeleton-init-systemd}/fakeroot_tmpfiles.sh | 0
 package/skeleton-init-systemd/skeleton-init-systemd.mk      | 6 ++++++
 package/systemd/systemd.mk                                  | 6 ------
 3 files changed, 6 insertions(+), 6 deletions(-)
 rename package/{systemd => skeleton-init-systemd}/fakeroot_tmpfiles.sh (100%)
package/skeleton-init-systemd/fakeroot_tmpfiles.sh renamed from package/systemd/fakeroot_tmpfiles.sh +0 -0
similarity index 100%
rename from package/systemd/fakeroot_tmpfiles.sh
rename to package/skeleton-init-systemd/fakeroot_tmpfiles.sh
package/skeleton-init-systemd/skeleton-init-systemd.mk +6 -0
index 970951d553..89a28d1780 100644
@@ -58,6 +58,12 @@ SKELETON_INIT_SYSTEMD_ROOTFS_PRE_CMD_HOOKS += SKELETON_INIT_SYSTEMD_PRE_ROOTFS_V
 
 endif
 
+define SKELETON_INIT_SYSTEMD_CREATE_TMPFILES_HOOK
+	HOST_SYSTEMD_TMPFILES=$(HOST_DIR)/bin/systemd-tmpfiles \
+		$(SKELETON_INIT_SYSTEMD_PKGDIR)/fakeroot_tmpfiles.sh $(TARGET_DIR)
+endef
+SKELETON_INIT_SYSTEMD_ROOTFS_PRE_CMD_HOOKS += SKELETON_INIT_SYSTEMD_CREATE_TMPFILES_HOOK
+
 define SKELETON_INIT_SYSTEMD_INSTALL_TARGET_CMDS
 	mkdir -p $(TARGET_DIR)/home
 	mkdir -p $(TARGET_DIR)/srv
package/systemd/systemd.mk +0 -6
index 1d7452de19..e4a8114221 100644
@@ -739,12 +739,6 @@ define SYSTEMD_RM_CATALOG_UPDATE_SERVICE
 endef
 SYSTEMD_ROOTFS_PRE_CMD_HOOKS += SYSTEMD_RM_CATALOG_UPDATE_SERVICE
 
-define SYSTEMD_CREATE_TMPFILES_HOOK
-	HOST_SYSTEMD_TMPFILES=$(HOST_DIR)/bin/systemd-tmpfiles \
-		$(SYSTEMD_PKGDIR)/fakeroot_tmpfiles.sh $(TARGET_DIR)
-endef
-SYSTEMD_ROOTFS_PRE_CMD_HOOKS += SYSTEMD_CREATE_TMPFILES_HOOK
-
 define SYSTEMD_PRESET_ALL
 	$(HOST_DIR)/bin/systemctl --root=$(TARGET_DIR) preset-all
 endef
Signature · 6 lines
2.25.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#4 lore raw

PATCH 4/4 system: add options for var factory and tmpfiles pre-seed

In reply to Yann E. MORIN

Currently, when one does not enable remounting the rootfs read-write, i.e. keep it read-only, for example because the filesystem is actualyl read-only by design, like squashfs, then two things happen:

- we create a factory from the content of /var at build time, register tmpfiles entries for it, and mount a tmpfs on /var at runtime, so that systemd-tmpfiles does populate /var from the factory; this is only done when the rootfs is not remounted r/w;

- we trigger systemd-tmpfiles at build time, which uses the tmpfiles db, of which our /var entries, to pre-populate the filesystem; this is always done, whether the rootfs is remounted r/w or not.

Note that Buildroot mounts a tmpfs on /var, and leaves to the integrator to care for providing an actual filesystem, as there are too many variants and is very specific to each use-case.

These two mechanisms are conflicting, semantically, bit also
technically: the files from the factory will be duplicated, but that may help in some situations when the actual /var filesystem is not mountable.

In some cases, it might be preferable to have none, either, or both mechanisms enabled; it highly depends on the ultimate integration scheme chosen for a device.

For example, some people will be very happy with a /var that is actually on a tmpfs and that it gets reseeded form scratch at every boot, while others may want to ensure that their system continue to work even when they can't mount something that makes /var writable.

YMMV, as they used to say back in the day...

So, we introduce two new options, in the system sub-menu, each to drive each mechanism. We default those options to y, to keep the previous behaviour by default, except the var factory is only available when the rootfs is not remounted r/w, as it were so far.

We still hint in the help text that there might be some conflict between the two mechanisms, bt since it has been that way for some time, it does not look too broken for most people.

Since that introduces more options related to systemd being chosen as an init system, we gather those two options and the existing one inside a if-endif block, rather than adding more 'depends on' on each options.

Cc Norbert Lange <[email protected]>
Cc Arnout Vandecappelle (Essensium/Mind) <[email protected]>
Cc Romain Naour <[email protected]>
Cc Jérémy Rosen <[email protected]>
Cc Yann E. MORIN <[email protected]>
---
 .../skeleton-init-systemd.mk                  |  7 +++-
 system/Config.in                              | 38 ++++++++++++++++++-
 2 files changed, 42 insertions(+), 3 deletions(-)
package/skeleton-init-systemd/skeleton-init-systemd.mk +5 -2
index 89a28d1780..69991265a5 100644
@@ -32,6 +32,7 @@ define SKELETON_INIT_SYSTEMD_ROOT_RO_OR_RW
 	echo "/dev/root / auto ro 0 1" >$(TARGET_DIR)/etc/fstab
 endef
 
+ifeq ($(BR2_INIT_SYSTEMD_VAR_FACTORY),y)
 define SKELETON_INIT_SYSTEMD_PRE_ROOTFS_VAR
 	rm -rf $(TARGET_DIR)/usr/share/factory/var
 	mv $(TARGET_DIR)/var $(TARGET_DIR)/usr/share/factory/var
@@ -55,14 +56,16 @@ define SKELETON_INIT_SYSTEMD_PRE_ROOTFS_VAR
 		$(TARGET_DIR)/usr/lib/systemd/system/var.mount
 endef
 SKELETON_INIT_SYSTEMD_ROOTFS_PRE_CMD_HOOKS += SKELETON_INIT_SYSTEMD_PRE_ROOTFS_VAR
+endif  # BR2_INIT_SYSTEMD_VAR_FACTORY
+endif  # BR2_TARGET_GENERIC_REMOUNT_ROOTFS_RW
 
-endif
-
+ifeq ($(BR2_INIT_SYSTEMD_POPULATE_TMPFILES),y)
 define SKELETON_INIT_SYSTEMD_CREATE_TMPFILES_HOOK
 	HOST_SYSTEMD_TMPFILES=$(HOST_DIR)/bin/systemd-tmpfiles \
 		$(SKELETON_INIT_SYSTEMD_PKGDIR)/fakeroot_tmpfiles.sh $(TARGET_DIR)
 endef
 SKELETON_INIT_SYSTEMD_ROOTFS_PRE_CMD_HOOKS += SKELETON_INIT_SYSTEMD_CREATE_TMPFILES_HOOK
+endif  # BR2_INIT_SYSTEMD_POPULATE_TMPFILES
 
 define SKELETON_INIT_SYSTEMD_INSTALL_TARGET_CMDS
 	mkdir -p $(TARGET_DIR)/home
system/Config.in +37 -1
index 888c24ce81..84c1b048cd 100644
@@ -154,10 +154,44 @@ source "$BR2_BASE_DIR/.br2-external.in.init"
 
 endchoice
 
+if BR2_INIT_SYSTEMD
+
+config BR2_INIT_SYSTEMD_VAR_FACTORY
+	bool "build a factory to populate a tmpfs on /var"
+	default y  # legacy
+	depends on !BR2_TARGET_GENERIC_REMOUNT_ROOTFS_RW
+	help
+	  Build a factory of the content of /var as installed by
+	  packages, mount a tmpfs on /var at runtime, so that
+	  systemd-tmpfiles can populate it from the factory.
+
+	  This may help on a read-only rootfs.
+
+	  It probably does not play very well with triggering a call
+	  to systemd-tmpfiles at build time (below).
+
+	  Note: Buildroot mounts a tmpfs on /var to at least make the
+	  system bootable out of the box; mounting a filesystem from
+	  actual storage is left to the integration, as it is too
+	  specific and may need preparatory work like partitionning a
+	  device and/or formatting a filesystem first, so that falls
+	  out of the scope of Buildroot.
+
+config BR2_INIT_SYSTEMD_POPULATE_TMPFILES
+	bool "trigger systemd-tmpfiles during build"
+	default y  # legacy
+	help
+	  Act on the systemd-tmpfiles.d database at build time, when
+	  assembling the root filesystems.
+
+	  This may help on a read-only filesystem.
+
+	  It probably does not play very well with the /var factory
+	  (above).
+
 config BR2_PACKAGE_SYSTEMD_DEFAULT_TARGET
 	string "The default unit systemd starts at bootup"
 	default "multi-user.target"
-	depends on BR2_INIT_SYSTEMD
 	help
 	  Specify the name of the unit configuration file to be started
 	  at bootup by systemd. Should end in ".target".
@@ -165,6 +199,8 @@ config BR2_PACKAGE_SYSTEMD_DEFAULT_TARGET
 
 	  https://www.freedesktop.org/software/systemd/man/systemd.special.html#default.target
 
+endif # BR2_INIT_SYSTEMD
+
 choice
 	prompt "/dev management" if !BR2_INIT_SYSTEMD
 	default BR2_ROOTFS_DEVICE_CREATION_DYNAMIC_DEVTMPFS
Signature · 6 lines
2.25.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#5 lore raw
In reply to Yann E. MORIN

Hi,

Am Sa., 15. Okt. 2022 um 23:23 Uhr schrieb Yann E. MORIN <[email protected]>:

Quoted 11 lines

Hello All!

This small four-patch series touches on the delicate intricacies of the systemd package and how we integrate it in Buildroot, especially with regard to read-only filesystems.

The underlying issue is that systemd wants a writable /var filesystem (it will store a bunch of things at runtime in there). A lot of packages also expect a writable /var. After all, var stands for variadic, so it is not supposed to be set in stone and read-only.

Nobody says var should be read-only, the argument is about doing work up-front. Which is enough to get alot things running even *if* its read-only.

Quoted 6 lines

The solution we implemented in Buildroot is to generate a factory for /var, and register that as tmpfiles, at build time, and mount a tmpfs on the then-empty /var at runtime, so that systemd-tmpfiles populates /var on boot. Having a tmpfs means /var is repopulated from a clean state at each boot, and people who want a persistent /var have to provide their own mechanism to mount their actual filesystem instead.

Here the issue is, that the logic will never be complete, I will have to dig up old threads, but it fails with various combinations of symlinks.

Its an inherently fragile approach, and I would question the solution as whole.

Quoted 3 lines

However, for some people, this does not seem to work as expected, so we introduced a call to systemd-tmpfiles at build time, to pre-populate the rootfs from all the tmpfiles, of which our var factory, at build time.

There are several usecases where this is atleast non-optimal, and for a default to mess with the filesystem in such a way is not a good foundation.

I am not sure if I am up-to-date, but the original call during the fakeroot stage you just do the work that wouldve been done on first boot.

Quoted 3 lines

These two mechanism clash with each others, and are mostly incompatible, and causes files to be duplicated.

One of them doesn't do anything than a regular system start, the other is likely to cause trouble, trying to fix packages that wont work with an empty var in an automatic way.

Quoted 13 lines

To solve the issue, this series introduces new options, so the user can elect to use either, none, or both mechanisms, as they see fit.

The first steps are borrowed from Yann@work, where we make it easier to change the location where /var is nounted from, by making it a systemd mount unit, rather than use the legacy fstab, and move the factory to /usr/share as it is a system factory (/etc is for the administartor to provide overrides). Note that those two are not strictly required for the following, but since that makes it easier to change the /var filesystem in a systemd-compliant manner, they are a nice addition to the series.

Quoted 8 lines

Then we move together the two mechanisms under the auspices of a common package, to make it easier to later make them conditional.

Finally, we introduce those two new options, that drive each mechanism.

Eventally, we want to highlight that, by default, the behaviour has not changed, since the two new options default to y, so as to explicitly keep the behaviour from nefore this patchset.

I would prefer a robust solution that provides a filesystem image that is compatible with further tinkering, particularly mounting /var in the initramfs.

I attacked a drop-in solution, that has none of the drawbacks, correctly works with mounting stuff in /var,
and only requires overlayfs to be available in the kernel.

Funny detail: together with the upfront creation
of some directories in /var via the systemd-tmpfiles build-time call, a basic usable system is available even if the overlay fails.

The recurring attempts to discuss this ended in [1].

Short on time nowadays so I can hardly pull together all relevant parts again.

regards, Norbert

Quoted 21 lines

Regards,
Yann E. MORIN.

----------------------------------------------------------------
Yann E. MORIN (4):
package/skeleton-systemd: move /var factory out of /etc package/skeleton-systemd: systemd-ify mounting /var tmpfs with ro rootfs package/skeleton-systemd: host the tmpfiles preparation script
system: add options for var factory and tmpfiles pre-seed

 .../fakeroot_tmpfiles.sh                           |  0
 .../skeleton-init-systemd/skeleton-init-systemd.mk | 18 +++++++---
 package/skeleton-init-systemd/var.mount            | 18 ++++++++++
 package/systemd/systemd.mk                         |  6 ----
 system/Config.in                                   | 38 +++++++++++++++++++++-
 5 files changed, 69 insertions(+), 11 deletions(-)
 rename package/{systemd => skeleton-init-systemd}/fakeroot_tmpfiles.sh (100%)
 create mode 100644 package/skeleton-init-systemd/var.mount
Quoted 7 lines

--
.-----------------.--------------------.------------------.--------------------.

|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |

'------------------------------^-------^------------------^--------------------'

[1] - https://lists.buildroot.org/pipermail/buildroot/2022-July/645680.html

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#6 lore raw

Norbert, All,

On 2022-10-17 14:12 +0200, Norbert Lange spake thusly:

Quoted 2 lines

Am Sa., 15. Okt. 2022 um 23:23 Uhr schrieb Yann E. MORIN <[email protected]>:

[--SNIP--]

Quoted 10 lines
Quoted 6 lines

The solution we implemented in Buildroot is to generate a factory for /var, and register that as tmpfiles, at build time, and mount a tmpfs on the then-empty /var at runtime, so that systemd-tmpfiles populates /var on boot. Having a tmpfs means /var is repopulated from a clean state at each boot, and people who want a persistent /var have to provide their own mechanism to mount their actual filesystem instead.

Here the issue is, that the logic will never be complete, I will have to dig up old threads, but it fails with various combinations of symlinks.

Then we should fix it up and make the procedure more robust, when you can point to actual issues so that we know how it is broken and how to fix it. I (see the post-scriptum) would be happy to help look into that.

Quoted 1 line

Its an inherently fragile approach, and I would question the solution as whole.

The thing is, out of the box, Buildroot needs a working situation.

Providing an overlay (see below) requires a setup by the user, setup That is highly dependent on the actual target: does it need to be partitioned and formatted on first boot? If so, how? Should it be mounted on the whole of / or just /var?

Those are not something that falls in the scope of Buildroot. Providing a var factory and a tmpfs on /var is the only solution that can be (rather easily) implemented.

Quoted 13 lines
Quoted 3 lines

However, for some people, this does not seem to work as expected, so we introduced a call to systemd-tmpfiles at build time, to pre-populate the rootfs from all the tmpfiles, of which our var factory, at build time.

There are several usecases where this is atleast non-optimal, and for a default to mess with the filesystem in such a way is not a good foundation.

I am not sure if I am up-to-date, but the original call during the fakeroot stage you just do the work that wouldve been done on first boot.

Quoted 1 line
Quoted 6 lines
Quoted 2 lines

These two mechanism clash with each others, and are mostly incompatible, and causes files to be duplicated.

One of them doesn't do anything than a regular system start, the other is likely to cause trouble, trying to fix packages that wont work with an empty var in an automatic way.

Look, I do understand that you do not want to be using the var factory, and I do appreciate your arguments.

Now, please understand that some people are very happy with using a /var factory, and some are even happy that /var is being mounted as a tmpfs too.

All this series does, is make the two mechanisms optional, so that people with different expectations can still have a functional systm in either case, without an overshoot in the filesystem size due to the duplication of files caused by unconditionally calling systemd-tmpfiles at build time.

It even keeps the curent behaviour by default.

Quoted 2 lines
Quoted 2 lines

To solve the issue, this series introduces new options, so the user can elect to use either, none, or both mechanisms, as they see fit.

[--SNIP--]

Quoted 3 lines

I would prefer a robust solution that provides a filesystem image that is compatible with further tinkering, particularly mounting /var in the initramfs.

That is assuming that one will use an initramfs. For example, in my systems, I do not use one, and I do not want to use one.

Quoted 3 lines

I attacked a drop-in solution, that has none of the drawbacks, correctly works with mounting stuff in /var,
and only requires overlayfs to be available in the kernel.

That is assuming two things:
- that an initramfs be used
- that an overlay be mounted

However, in my case I expressely do not want to use an initramfs: my rootfs is a squashfs mounted on an initrd. I also have a tmpfs mounted on /var, as I am very happy that /var gets factory value at boot (my resilient data is accessed in some other ways).

Quoted 3 lines

Funny detail: together with the upfront creation
of some directories in /var via the systemd-tmpfiles build-time call, a basic usable system is available even if the overlay fails.

That is not true. Anything that expects /var to be writable will fail to start.

Quoted 1 line

The recurring attempts to discuss this ended in [1].

And this series addresses the fact that, for some people, the var factory is not a good solution, so that they can disable it.

It also addresses the case of those people for whom the var factory actually works, and want to keep using it, can disable the systemd-tmpfiles buildtime call.

And finally, I believe it may even address more complex situations, where people will provide their own fakeroot-script to do the preparatory works.

Regards,
Yann E. MORIN.

PS. Please note that there are two different Yann in this thread: me at work using Buildroot, and Yann at home, one of the maintainers. Look at the email to be sure who's talking (not that I-at-work talk in the name of my employer either).
I try to segregate the two identities as much as I can, but sometime there is some overlap, like here.
YEM.

Quoted 40 lines

Short on time nowadays so I can hardly pull together all relevant parts again.

regards, Norbert

Quoted 21 lines

Regards,
Yann E. MORIN.

----------------------------------------------------------------
Yann E. MORIN (4):
package/skeleton-systemd: move /var factory out of /etc package/skeleton-systemd: systemd-ify mounting /var tmpfs with ro rootfs package/skeleton-systemd: host the tmpfiles preparation script
system: add options for var factory and tmpfiles pre-seed

 .../fakeroot_tmpfiles.sh                           |  0
 .../skeleton-init-systemd/skeleton-init-systemd.mk | 18 +++++++---
 package/skeleton-init-systemd/var.mount            | 18 ++++++++++
 package/systemd/systemd.mk                         |  6 ----
 system/Config.in                                   | 38 +++++++++++++++++++++-
 5 files changed, 69 insertions(+), 11 deletions(-)
 rename package/{systemd => skeleton-init-systemd}/fakeroot_tmpfiles.sh (100%)
 create mode 100644 package/skeleton-init-systemd/var.mount
Quoted 7 lines

--
.-----------------.--------------------.------------------.--------------------.

|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |

'------------------------------^-------^------------------^--------------------'

[1] - https://lists.buildroot.org/pipermail/buildroot/2022-July/645680.html

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

Signature · 26 lines
                                        ____________
.-----------------.--------------------:       _    :------------------.
|  Yann E. MORIN  | Real-Time Embedded |    __/ )   | /"\ ASCII RIBBON |
|                 | Software  Designer |  _/ - /'   | \ / CAMPAIGN     |
| +33 638.411.245 '--------------------: (_    `--, |  X  AGAINST      |
| yann.morin (at) orange.com           |_="    ,--' | / \ HTML MAIL    |
'--------------------------------------:______/_____:------------------'


_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#7 lore raw

Am Mo., 17. Okt. 2022 um 16:11 Uhr schrieb <[email protected]>:

Quoted 18 lines

Norbert, All,

On 2022-10-17 14:12 +0200, Norbert Lange spake thusly:

Quoted 2 lines

Am Sa., 15. Okt. 2022 um 23:23 Uhr schrieb Yann E. MORIN <[email protected]>:

[--SNIP--]

Quoted 10 lines
Quoted 6 lines

The solution we implemented in Buildroot is to generate a factory for /var, and register that as tmpfiles, at build time, and mount a tmpfs on the then-empty /var at runtime, so that systemd-tmpfiles populates /var on boot. Having a tmpfs means /var is repopulated from a clean state at each boot, and people who want a persistent /var have to provide their own mechanism to mount their actual filesystem instead.

Here the issue is, that the logic will never be complete, I will have to dig up old threads, but it fails with various combinations of symlinks.

Quoted 7 lines

Then we should fix it up and make the procedure more robust, when you can point to actual issues so that we know how it is broken and how to fix it. I (see the post-scriptum) would be happy to help look into that.

Quoted 1 line

Its an inherently fragile approach, and I would question the solution as whole.

The thing is, out of the box, Buildroot needs a working situation.

Yes, my argument would be a technical better solution.

Quoted 9 lines

Providing an overlay (see below) requires a setup by the user, setup That is highly dependent on the actual target: does it need to be partitioned and formatted on first boot? If so, how? Should it be mounted on the whole of / or just /var?

Those are not something that falls in the scope of Buildroot. Providing a var factory and a tmpfs on /var is the only solution that can be (rather easily) implemented.

Not sure you understand my appoach, more below.

Quoted 21 lines
Quoted 14 lines
Quoted 4 lines

However, for some people, this does not seem to work as expected, so we introduced a call to systemd-tmpfiles at build time, to pre-populate the rootfs from all the tmpfiles, of which our var factory, at build time.

There are several usecases where this is atleast non-optimal, and for a default to mess with the filesystem in such a way is not a good foundation.

I am not sure if I am up-to-date, but the original call during the fakeroot stage you just do the work that wouldve been done on first boot.

Quoted 1 line
Quoted 6 lines
Quoted 2 lines

These two mechanism clash with each others, and are mostly incompatible, and causes files to be duplicated.

One of them doesn't do anything than a regular system start, the other is likely to cause trouble, trying to fix packages that wont work with an empty var in an automatic way.

Quoted 6 lines

Look, I do understand that you do not want to be using the var factory, and I do appreciate your arguments.

Now, please understand that some people are very happy with using a /var factory, and some are even happy that /var is being mounted as a tmpfs too.

Would they be unhappy if everything works as it did before?

Quoted 8 lines

All this series does, is make the two mechanisms optional, so that people with different expectations can still have a functional systm in either case, without an overshoot in the filesystem size due to the duplication of files caused by unconditionally calling systemd-tmpfiles at build time.

It even keeps the curent behaviour by default.

I understand that, yes.

Quoted 10 lines
Quoted 3 lines
Quoted 3 lines

To solve the issue, this series introduces new options, so the user can elect to use either, none, or both mechanisms, as they see fit.

[--SNIP--]

Quoted 3 lines

I would prefer a robust solution that provides a filesystem image that is compatible with further tinkering, particularly mounting /var in the initramfs.

That is assuming that one will use an initramfs. For example, in my systems, I do not use one, and I do not want to use one.

No, initramfs is optional

Quoted 5 lines
Quoted 4 lines

I attacked a drop-in solution, that has none of the drawbacks, correctly works with mounting stuff in /var,
and only requires overlayfs to be available in the kernel.

Quoted 8 lines

That is assuming two things:
- that an initramfs be used
- that an overlay be mounted

However, in my case I expressely do not want to use an initramfs: my rootfs is a squashfs mounted on an initrd. I also have a tmpfs mounted on /var, as I am very happy that /var gets factory value at boot (my resilient data is accessed in some other ways).

I am not sure if initrd/initramfs means something different here.

Quoted 7 lines
Quoted 4 lines

Funny detail: together with the upfront creation
of some directories in /var via the systemd-tmpfiles build-time call, a basic usable system is available even if the overlay fails.

That is not true. Anything that expects /var to be writable will fail to start.

That's why I said basic usable system, more concrete:
the services with systemd will work, and stuff like networking + openssh.

Quoted 6 lines
Quoted 2 lines

The recurring attempts to discuss this ended in [1].

And this series addresses the fact that, for some people, the var factory is not a good solution, so that they can disable it.

Quoted 7 lines

It also addresses the case of those people for whom the var factory actually works, and want to keep using it, can disable the systemd-tmpfiles buildtime call.

And finally, I believe it may even address more complex situations, where people will provide their own fakeroot-script to do the preparatory works.

My own usecase is that use the buildroot as pristine rootfs and then slice it up as
necessary. But just to get this out of the way.

More importantly, I would propose a simpler, yet more robust solution than the var factory.

For this, the files in the attached archive have to end up in /lib/systemd/system.

What this does:
Before the time /var is supposed to be accessed, an tmpfs will be mounted in /run,
and the /var folder will be shifted around to end up in an overlayfs *below* this tmpfs.
The bind-mount is done with a systemd mount unit, means if you have mounts *in /var or below* those will be correctly ordered.

- No initramfs necessary
- No duplicate files in the var factory and tmpfs (unless they are
modified of course). So your suashed files stay compressed.
- No tricky autogenerated tmpfiles
- Would be possible to do something similar in other init systems

The only downside would be the reliance on overlayfs - technically you could also
adopt that solution to copy stuff into the tmpfs.

Besides my obvious biases there are technical arguments to be had.

Quoted 11 lines

Regards,
Yann E. MORIN.

PS. Please note that there are two different Yann in this thread: me at work using Buildroot, and Yann at home, one of the maintainers. Look at the email to be sure who's talking (not that I-at-work talk in the name of my employer either).
I try to segregate the two identities as much as I can, but sometime there is some overlap, like here.
YEM.

Not making things much clearer ;)

regards, Norbert
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#8 lore raw

Norert, All,

On 2022-10-17 16:50 +0200, Norbert Lange spake thusly:

Quoted 11 lines

Am Mo., 17. Okt. 2022 um 16:11 Uhr schrieb <[email protected]>:

Quoted 10 lines

On 2022-10-17 14:12 +0200, Norbert Lange spake thusly:

Quoted 2 lines

Am Sa., 15. Okt. 2022 um 23:23 Uhr schrieb Yann E. MORIN <[email protected]>:

[--SNIP--]

Quoted 6 lines
Quoted 6 lines

The solution we implemented in Buildroot is to generate a factory for /var, and register that as tmpfiles, at build time, and mount a tmpfs on the then-empty /var at runtime, so that systemd-tmpfiles populates /var on boot. Having a tmpfs means /var is repopulated from a clean state at each boot, and people who want a persistent /var have to provide their own mechanism to mount their actual filesystem instead.

Quoted 5 lines
Quoted 4 lines
Quoted 3 lines

Here the issue is, that the logic will never be complete, I will have to dig up old threads, but it fails with various combinations of symlinks.

The thing is, out of the box, Buildroot needs a working situation.

Yes, my argument would be a technical better solution.

But so far, we do not have that technically better solution, so we need to fix things that are arctually broken.

[--SNIP--]
[--SNIP--]

Quoted 4 lines
Quoted 3 lines

Now, please understand that some people are very happy with using a /var factory, and some are even happy that /var is being mounted as a tmpfs too.

Would they be unhappy if everything works as it did before?

By "works as it did before", do you mean "using a var factory that was not broken for them"? Yes, I am being teasing.

We'd have to evaluate the alternative proposal when we see it, if that can replace the existing solution.

Quoted 3 lines
Quoted 2 lines

That is assuming that one will use an initramfs. For example, in my systems, I do not use one, and I do not want to use one.

No, initramfs is optional

Ah, but you had so far only talked about using an initramfs to prepare an overlay, so I assumed your solution was tied to using an initramfs. If it's not, then let's see it.

[--SNIP--]

Quoted 5 lines
Quoted 4 lines

However, in my case I expressely do not want to use an initramfs: my rootfs is a squashfs mounted on an initrd. I also have a tmpfs mounted on /var, as I am very happy that /var gets factory value at boot (my resilient data is accessed in some other ways).

I am not sure if initrd/initramfs means something different here.

initrd is a blockdevice in memory. The filesystem one puts in there can be any filesystem that could be otherwise stored on another blockdevice.

initramfs is a filesystem in memory, like a tmpfs. The initramfs is read-write.

Quoted 1 line
Quoted 1 line
Quoted 1 line

The recurring attempts to discuss this ended in [1].

[--SNIP--]

Quoted 8 lines
Quoted 3 lines

And finally, I believe it may even address more complex situations, where people will provide their own fakeroot-script to do the preparatory works.

My own usecase is that use the buildroot as pristine rootfs and then slice it up as
necessary. But just to get this out of the way.

Quoted 5 lines

More importantly, I would propose a simpler, yet more robust solution than the var factory.

For this, the files in the attached archive have to end up in /lib/systemd/system.

Did you forget to attach something, by chance?

Quoted 3 lines

What this does:
Before the time /var is supposed to be accessed, an tmpfs will be mounted in /run,

We already have that: /run is automatically mounted by systemd, no?

Quoted 2 lines

and the /var folder will be shifted around to end up in an overlayfs *below* this tmpfs.

What do you mean by "shifted"?

Quoted 2 lines

The bind-mount is done with a systemd mount unit, means if you have mounts *in /var or below* those will be correctly ordered.

Yeah, switching away from the fstab is good.

Quoted 11 lines
(quoted diff) +0 -2
-   No initramfs necessary
-   No duplicate files in the var factory and tmpfs (unless they are

modified of course). So your suashed files stay compressed.

(quoted diff) +0 -2
-   No tricky autogenerated tmpfiles
-   Would be possible to do something similar in other init systems

The only downside would be the reliance on overlayfs - technically you could also
adopt that solution to copy stuff into the tmpfs.

Besides my obvious biases there are technical arguments to be had.

We all have biases, me ot the last. :-)

My technical arguments for this series are:
- it is currently kinda-broken
- var factory works until proven otherwise
- so we need to fix them

I'll be sure to do as unbiased a review as possible for your proposal when I see it. ;-)

Quoted 9 lines
Quoted 7 lines

PS. Please note that there are two different Yann in this thread: me at work using Buildroot, and Yann at home, one of the maintainers. Look at the email to be sure who's talking (not that I-at-work talk in the name of my employer either).
I try to segregate the two identities as much as I can, but sometime there is some overlap, like here.
YEM.

Not making things much clearer ;)

If it can comfort you, it is not much clearer in my own head either! ;-)

Regards,
Yann E. MORIN.

Signature · 26 lines
                                        ____________
.-----------------.--------------------:       _    :------------------.
|  Yann E. MORIN  | Real-Time Embedded |    __/ )   | /"\ ASCII RIBBON |
|                 | Software  Designer |  _/ - /'   | \ / CAMPAIGN     |
| +33 638.411.245 '--------------------: (_    `--, |  X  AGAINST      |
| yann.morin (at) orange.com           |_="    ,--' | / \ HTML MAIL    |
'--------------------------------------:______/_____:------------------'


_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#9 lore raw

Am Mo., 17. Okt. 2022 um 18:00 Uhr schrieb <[email protected]>:

Quoted 15 lines

Norert, All,

On 2022-10-17 16:50 +0200, Norbert Lange spake thusly:

Quoted 11 lines

Am Mo., 17. Okt. 2022 um 16:11 Uhr schrieb <[email protected]>:

Quoted 10 lines

On 2022-10-17 14:12 +0200, Norbert Lange spake thusly:

Quoted 2 lines

Am Sa., 15. Okt. 2022 um 23:23 Uhr schrieb Yann E. MORIN <[email protected]>:

[--SNIP--]

Quoted 6 lines
Quoted 6 lines

The solution we implemented in Buildroot is to generate a factory for /var, and register that as tmpfiles, at build time, and mount a tmpfs on the then-empty /var at runtime, so that systemd-tmpfiles populates /var on boot. Having a tmpfs means /var is repopulated from a clean state at each boot, and people who want a persistent /var have to provide their own mechanism to mount their actual filesystem instead.

Quoted 8 lines
Quoted 5 lines
Quoted 4 lines
Quoted 3 lines

Here the issue is, that the logic will never be complete, I will have to dig up old threads, but it fails with various combinations of symlinks.

The thing is, out of the box, Buildroot needs a working situation.

Yes, my argument would be a technical better solution.

But so far, we do not have that technically better solution, so we need to fix things that are arctually broken.

I tried to bring that in for ages. Probably just have cooked up a patch instead.

Quoted 14 lines

[--SNIP--]
[--SNIP--]

Quoted 4 lines
Quoted 3 lines

Now, please understand that some people are very happy with using a /var factory, and some are even happy that /var is being mounted as a tmpfs too.

Would they be unhappy if everything works as it did before?

By "works as it did before", do you mean "using a var factory that was not broken for them"? Yes, I am being teasing.

We'd have to evaluate the alternative proposal when we see it, if that can replace the existing solution.

Quoted 7 lines
Quoted 3 lines
Quoted 2 lines

That is assuming that one will use an initramfs. For example, in my systems, I do not use one, and I do not want to use one.

No, initramfs is optional

Ah, but you had so far only talked about using an initramfs to prepare an overlay, so I assumed your solution was tied to using an initramfs. If it's not, then let's see it.

Nah, it was always in respect of not messing around to much with the filesystem itself.

I consider managing patch-series time consuming, so I'd like to discuss the way this should be structured before.

This could be one completely separate package for example, allowing different solutions.

Quoted 6 lines

[--SNIP--]

Quoted 4 lines
Quoted 4 lines

However, in my case I expressely do not want to use an initramfs: my rootfs is a squashfs mounted on an initrd. I also have a tmpfs mounted on /var, as I am very happy that /var gets factory value at boot (my resilient data is accessed in some other ways).

Quoted 7 lines
Quoted 1 line

I am not sure if initrd/initramfs means something different here.

initrd is a blockdevice in memory. The filesystem one puts in there can be any filesystem that could be otherwise stored on another blockdevice.

initramfs is a filesystem in memory, like a tmpfs. The initramfs is read-write.

Ok, I am using those pretty much interchangeable,
just some block you tell the bootloader to load, either cpio or filesystem.

Quoted 11 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

The recurring attempts to discuss this ended in [1].

[--SNIP--]

Quoted 8 lines
Quoted 3 lines

And finally, I believe it may even address more complex situations, where people will provide their own fakeroot-script to do the preparatory works.

My own usecase is that use the buildroot as pristine rootfs and then slice it up as
necessary. But just to get this out of the way.

Quoted 7 lines
Quoted 5 lines

More importantly, I would propose a simpler, yet more robust solution than the var factory.

For this, the files in the attached archive have to end up in /lib/systemd/system.

Did you forget to attach something, by chance?

Was attached in my post before - now 2 mails back. gonna add it again for convienience.

Quoted 6 lines
Quoted 4 lines

What this does:
Before the time /var is supposed to be accessed, an tmpfs will be mounted in /run,

We already have that: /run is automatically mounted by systemd, no?

mounts a tmpfs in /run/varoverlay, own directory.
/run is guaranteed to be writable and available before any systemd unit is started.

Quoted 5 lines
Quoted 3 lines

and the /var folder will be shifted around to end up in an overlayfs *below* this tmpfs.

What do you mean by "shifted"?

you have to bind-mount /var to another directory, so you can use it as lower fs. /run/varoverlay/lower <- /var

then.. you mount an overlayfs *at* /var where you need to reference the original folder with /run/varoverlay/lower

/var <- overlay of /run/varoverlay/upper (tmpfs) and /run/varoverlay/lower( orignal /var )

Quoted 5 lines
Quoted 3 lines

The bind-mount is done with a systemd mount unit, means if you have mounts *in /var or below* those will be correctly ordered.

Yeah, switching away from the fstab is good.

fstab wouldnt allow for the needed dependencies.

Quoted 15 lines
Quoted 7 lines
(quoted diff) +0 -2
-   No initramfs necessary
-   No duplicate files in the var factory and tmpfs (unless they are

modified of course). So your suashed files stay compressed.

(quoted diff) +0 -2
-   No tricky autogenerated tmpfiles
-   Would be possible to do something similar in other init systems
Quoted 5 lines

The only downside would be the reliance on overlayfs - technically you could also
adopt that solution to copy stuff into the tmpfs.

Besides my obvious biases there are technical arguments to be had.

We all have biases, me ot the last. :-)

Quoted 7 lines

My technical arguments for this series are:
- it is currently kinda-broken
- var factory works until proven otherwise
- so we need to fix them

I'll be sure to do as unbiased a review as possible for your proposal when I see it. ;-)

Gonna take me few days. should be easy to test tough with the provided files, just throw them into the buildroot filesystem overlay.

Helps if you make some assessment where they should fit. the systemd package is already bloated enough.

Quoted 9 lines
Quoted 9 lines
Quoted 8 lines

PS. Please note that there are two different Yann in this thread: me at work using Buildroot, and Yann at home, one of the maintainers. Look at the email to be sure who's talking (not that I-at-work talk in the name of my employer either).
I try to segregate the two identities as much as I can, but sometime there is some overlap, like here.
YEM.

Quoted 3 lines
Quoted 1 line

Not making things much clearer ;)

If it can comfort you, it is not much clearer in my own head either! ;-)

Good to know, so we can discuss on even terms.

regards, Norbert

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#10 lore raw

Norbert, All,

On 2022-10-17 18:49 +0200, Norbert Lange spake thusly:

Quoted 4 lines

Am Mo., 17. Okt. 2022 um 18:00 Uhr schrieb <[email protected]>:

Quoted 2 lines

But so far, we do not have that technically better solution, so we need to fix things that are arctually broken.

I tried to bring that in for ages. Probably just have cooked up a patch instead.

Indeed, with a patch to look at, it is easier to discuss. I've seen the files now, and indeed I remember I already saw something like that some time ago...

I'll test it in our use-case @work tomorrow.

Ultimately, I wonder if your solution could be an alternative to the var factory, rather than replace it.

Quoted 4 lines

I consider managing patch-series time consuming, so I'd like to discuss the way this should be structured before.
This could be one completely separate package for example, allowing different solutions.

I am not sure separate packages in the way to go.

If we were to do those various solutions in different pacakges, then we'd have to make it easy for users to use a package from their br2-external trees, which is not so nice.

Also, see below: we already have a package where it is interesting to implement the systemd integration in Buildroot: skeleton-init-systemd.

Quoted 6 lines
Quoted 4 lines

initrd is a blockdevice in memory. The filesystem one puts in there can be any filesystem that could be otherwise stored on another blockdevice. initramfs is a filesystem in memory, like a tmpfs. The initramfs is read-write.

Ok, I am using those pretty much interchangeable,
just some block you tell the bootloader to load, either cpio or filesystem.

They are not the same thing at all, and they are not interchangeable; the underlying code is totally different.

[--SNIP--]

Quoted 6 lines
Quoted 2 lines

I'll be sure to do as unbiased a review as possible for your proposal when I see it. ;-)

Gonna take me few days. should be easy to test tough with the provided files, just throw them into the buildroot filesystem overlay.
Helps if you make some assessment where they should fit. the systemd package is already bloated enough.

Yes, our systemd.mk is a bit bloated.

What I'd like to do, is that the systemd integration at the package level is done in package/systemd/, while the choices we do in Buildroot, like using a var factory, calling systemd-tmpfiles, or using an overlayfs, etc... be done somewhere else.

We have the merged-usr at the init level, and that is handled in the skeleton-init-systemd package, and the var factory is also handled there, so I started moving more stuff in there, like the systemd-tmpfiles call, because I don't see a better place.

So, if you don't have a better idea, skeleton-init-systemd would be where we handle that.

Regards,
Yann E. MORIN.

Signature · 10 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#11 lore raw

Norbert, All,

On 2022-10-17 22:14 +0200, Yann E. MORIN spake thusly:

Quoted 6 lines

On 2022-10-17 18:49 +0200, Norbert Lange spake thusly:

Quoted 4 lines

Am Mo., 17. Okt. 2022 um 18:00 Uhr schrieb <[email protected]>:

Quoted 2 lines

But so far, we do not have that technically better solution, so we need to fix things that are arctually broken.

I tried to bring that in for ages. Probably just have cooked up a patch instead.

I'll test it in our use-case @work tomorrow.

I did preliminray tsts, and it works, that's great! :-)

I'll be cleaning up my series and repost something real quick I hope.

Regards,
Yann E. MORIN.

Signature · 26 lines
                                        ____________
.-----------------.--------------------:       _    :------------------.
|  Yann E. MORIN  | Real-Time Embedded |    __/ )   | /"\ ASCII RIBBON |
|                 | Software  Designer |  _/ - /'   | \ / CAMPAIGN     |
| +33 638.411.245 '--------------------: (_    `--, |  X  AGAINST      |
| yann.morin (at) orange.com           |_="    ,--' | / \ HTML MAIL    |
'--------------------------------------:______/_____:------------------'


_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot