Skip to contentSkip to conversation

systemd: Ensure /run and /var/run are the same on R/O rootfs

PATCH New Christian Stewart 2 participants 3 messages Feb 14, 2018 – Feb 21, 2018 1 message not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

Christian Stewart

author Applied
#1 lore raw

PATCH 1/3 systemd: Ensure /run and /var/run are the same on R/O rootfs

Hi Jan, all,

Quoted 29 lines

From: Jan Kundr?t <[email protected]>
I'm leaving that one to someone who is more familiar with systemd and buildroot conventions. My box now boots again, so I'm happy :).

[1] https://bugs.freedesktop.org/show_bug.cgi?id=101628

---
 package/skeleton-init-systemd/skeleton-init-systemd.mk | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
package/skeleton-init-systemd/skeleton-init-systemd.mk +3 -1
index a2d4e8c4b3..95142904f5 100644
@@ -42,7 +42,9 @@ define SKELETON_INIT_SYSTEMD_PRE_ROOTFS_VAR
 	mkdir $(TARGET_DIR)/var
 	for i in $(TARGET_DIR)/usr/share/factory/var/*; do \
 		j="$${i#$(TARGET_DIR)/usr/share/factory}"; \
-		if [ -L "$${i}" ]; then \
+		if [ "$${j}" = "/var/run" ]; then \
+			echo "# $${j} is being handled by systemd"
+		elif [ -L "$${i}" ]; then \
 			printf "L+! %s - - - - %s\n" \
 				"$${j}" "../usr/share/factory/$${j}" \
 			|| exit 1; \
Signature · 1 line
2.14.3

I applied this patch, thanks. Unfortunately, it seems the problem is still occurring. Maybe it's a different problem?

[   16.650294] systemd[1]: System is tainted: var-run-bad
[   16.680724] systemd[1]: Starting Network Connectivity...
[   16.710421] systemd[1]: Started D-Bus System Message Bus.
[   16.740678] systemd[1]: Failed to connect to system bus: No such file or directory
[   16.770425] systemd[1]: Failed to initialize D-Bus connection: No such file or directory
[   16.800386] systemd[1]: Starting Network Manager...
[1518648890.7497] hostname: hostname: hostnamed not used as proxy creation failed
[   39.020736] dbus-daemon[314]: [system] Failed to activate service 'org.freedesktop.systemd1': timed out
[   39.100451] dbus-daemon[314]: [system] Activating systemd to hand-off: service name='org.freedesktop.nm
_dispatcher' unit='dbus-org.freedesktop.nm-dispatcher.service' requested by ':1.0' (uid=0 pid=315 comm="/u
sr/sbin/NetworkManager --no-daemon ")                                                                    
[   39.150403] NetworkManager[315]: <info>  [1518648890.7540] manager[0x20ac030]: rfkill: WWAN hardware ra

It seems that systemd still cannot talk to the system bus.

I can also see this happening if I try to execute systemd-hostnamed:

# ./systemd-hostnamed 
Failed to get system bus connection: No such file or directory
# ls /var/run/
NetworkManager  dhcpcd.pid          docker       ifstate   utmp
dbus            dhcpcd.sock         docker.pid   sepermit
dhcpcd          dhcpcd.unpriv.sock  docker.sock  sshd.pid
# ls /var/run/dbus
system_bus_socket
# ls /run/
blkid   lock  mount     systemd  user  xtables.lock
docker  log   ntpd.pid  udev     utmp

If I link dbus like so:

# ln -s /var/run/dbus/ /run/dbus

It seems then that everything works fine.

~ Christian

#2 lore raw

PATCH 1/3 systemd: Ensure /run and /var/run are the same on R/O rootfs

Quoted 6 lines · Christian Stewart
Quoted 6 lines

From: Jan Kundr?t <[email protected]>
I'm leaving that one to someone who is more familiar with systemd and buildroot conventions. My box now boots again, so I'm happy :).

[1] https://bugs.freedesktop.org/show_bug.cgi?id=101628

Quoted 7 lines
Quoted 1 line

I applied this patch, thanks. Unfortunately, it seems the problem is still occurring. Maybe it's a different problem?

[   16.650294] systemd[1]: System is tainted: var-run-bad

Systemd reports that your /var/run is "bad". From what you wrote later, that's indeed the root cause which should be fixed.

Now, my patch fixed it only for a read-only rootfs. I assumed that everything worked well on a regular, r/w rootfs.

Can you please:

- state whether you're using a read-only or read-write rootfs (BR2_TARGET_GENERIC_REMOUNT_ROOTFS_RW config option),
- confirm that you completely nuked your build dir after applying this patch,
- check which packages actually provide your /var/run and /run, and how your setup looks like, i.e.:
- grep ,./run build/packages-file-list.txt
- grep ,./var build/packages-file-list.txt
- grep dbus build/packages-file-list.txt | grep run
- how is your /etc/fstab?

Quoted 1 line
# ls /var/run/

These are not sufficient as they do not show the type of the individual items. ls -al should be better.

Quoted 5 lines

If I link dbus like so:

# ln -s /var/run/dbus/ /run/dbus

It seems then that everything works fine.

Yeah, that's it. Based on how I read the code, though, this should already happen on a regular installation without a r/o rootfs...

Cheers,
Jan

#3 lore raw

PATCH 1/3 systemd: Ensure /run and /var/run are the same on R/O rootfs

Hi Jan,

To preface this note, it seems OK against the new release RC.

Quoted 4 lines · Jan Kundr?t
Quoted 1 line
[   16.650294] systemd[1]: System is tainted: var-run-bad

Systemd reports that your /var/run is "bad". From what you wrote later, that's indeed the root cause which should be fixed.

From what I've seen a lot of things can cause this taint, but I actually have not yet been able to find a single place where it's listed what this error actually means. The directory tree is there, and I haven't really modified it.

Quoted 2 lines

Now, my patch fixed it only for a read-only rootfs. I assumed that everything worked well on a regular, r/w rootfs.

The system I'm running is a bit non-standard, I basically execute an initramfs and then mount a ramfs to make the rootfs read/write.

Quoted 4 lines

Can you please:

- state whether you're using a read-only or read-write rootfs (BR2_TARGET_GENERIC_REMOUNT_ROOTFS_RW config option),

BR2_TARGET_GENERIC_REMOUNT_ROOTFS_RW=y

Quoted 2 lines

- confirm that you completely nuked your build dir after applying this patch,

Yes.

Quoted 6 lines

- check which packages actually provide your /var/run and /run, and how your setup looks like, i.e.:
- grep ,./run build/packages-file-list.txt
- grep ,./var build/packages-file-list.txt
-
- how is your /etc/fstab?

% grep ,./run build/packages-file-list.txt
skeleton-init-common,./run

% grep ,./var build/packages-file-list.txt
skeleton-init-systemd,./var
linux-pam,./var/run
linux-pam,./var/run/sepermit
systemd,./var/log/README
systemd,./var/lib
systemd,./var/lib/systemd
systemd,./var/log
systemd,./var/log/journal
dbus,./var/lib/dbus/machine-id
dbus,./var/lib/dbus
dbus,./var/run/dbus
dhcpcd,./var/db
network-manager,./var/lib/NetworkManager
openssh,./var/empty

% grep dbus build/packages-file-list.txt | grep run dbus,./usr/bin/dbus-run-session dbus,./var/run/dbus

% cat target/etc/fstab

# <file system> <mount pt>      <type>  <options>       <dump>  <pass>
proc            /proc           proc    defaults        0       0
devpts          /dev/pts        devpts  defaults,gid=5,mode=620 0       0
tmpfs           /dev/shm        tmpfs   mode=0777       0       0
tmpfs           /tmp            tmpfs   mode=1777       0       0
tmpfs           /run            tmpfs   mode=0755,nosuid,nodev  0       0
sysfs           /sys            sysfs   defaults        0       0
Quoted 4 lines
Quoted 1 line
# ls /var/run/

These are not sufficient as they do not show the type of the individual items. ls -al should be better.

my mistake, I usually use -lah.

# ls -lah /var/run/
total 20K
drwxr-xr-x  7 root root     0 Feb 21 22:18 .
drwxr-xr-x 11 root root     0 Feb 21 22:18 ..
drwxr-xr-x  3 root root     0 Feb 21 22:18 NetworkManager
drwxr-xr-x  2 dbus dbus     0 Feb 21 22:18 dbus
drwxr-xr-x  3 root root     0 Feb 21 22:18 dhcpcd
-rw-r--r--  1 root root     4 Feb 21 22:18 dhcpcd.pid
srw-rw----  1 root root     0 Feb 21 22:18 dhcpcd.sock
srw-rw-rw-  1 root root     0 Feb 21 22:18 dhcpcd.unpriv.sock
drwx------  5 root root     0 Feb 21 22:18 docker
-rw-r--r--  1 root root     3 Feb 21 22:18 docker.pid
srw-rw----  1 root docker   0 Feb 21 22:18 docker.sock
-rw-r--r--  1 root root     6 Feb 21 22:18 ifstate
drwxr-xr-x  2 root root     0 Feb 16 11:48 sepermit
-rw-r--r--  1 root root     4 Feb 21 22:18 sshd.pid
-rw-r--r--  1 root root   384 Feb 21 22:24 utmp
Quoted 8 lines
Quoted 5 lines

If I link dbus like so:

# ln -s /var/run/dbus/ /run/dbus

It seems then that everything works fine.

Yeah, that's it. Based on how I read the code, though, this should already happen on a regular installation without a r/o rootfs...

I upgraded Docker to the new RC and it seems OK now. I do have your patch applied:

4aaada448b * docker-engine: bump to v17.12.1-ce-rc1 4ff748f908 * docker-containerd: bump to v1.0.1
35e27a785d * runc: bump to 9f9c96235cc
cdf17f32a2 * docker-engine: add dependency on docker-proxy aa01038df2 * docker-proxy: new package
f642b99974 * docker-engine: select static for tini and dumb-init 011a49093a * docker-engine: add support for init processes 4bbf8495d1 * dumb-init: new package
da2b6103ec * tini: new package
72024b7e5a * generate-ipv6-address: new package
b00d9c9711 * rtl8192cu: new package
8f6e68358e * rtl8821au: bump to latest
7c4f49d4d9 * go: bump to 1.10
15e767672f * systemd: Ensure /run and /var/run are the same on R/O rootfs 5f6e1c0d61 * support/dependencies: recognize C.UTF-8 as a utf8 locale 44cec6431c * 2018.02-rc2 Update for 2018.02-rc2
5e66d15646 * libcpprestsdk: needs host-pkgconf

Best,
Christian