systemd: /run directory not created

When I attempt to boot a systemd enabled image I get an error stating that the run directory could not be mounted as it doesn't exist. I checked the target rootfs and there isn't a /run directory. What is responsible for creating this and why is it not doing it's job?

[ 2.631549] systemd[1]: Failed to mount /run: No such file or directory

Possibly of interest is that I am running a custom distro with manually specified DISTRO_FEATURES.

Possibly because /run was a link created by the initscripts which are now disabled when systemd is the init manager. I'll look into this and update 00-create-volatile.conf so that systemd will handle the link creation in stead.

Quoted 4 lines · Radu Moisan

Possibly because /run was a link created by the initscripts which are now disabled when systemd is the init manager. I'll look into this and update 00-create-volatile.conf so that systemd will handle the link creation in stead.

I wonder how my images booted... initscripts is sysvinit specific, the systemd startup should handle this somehow.

(insert standard argument that we should adopt /run across all init systems)

This is exactly what I was looking for. /run shouldn't be there, yet somehow it is. I was looking into how/where systemd is creating /run but no lead for now.

Quoted 9 lines

Quoted 4 lines

Possibly because /run was a link created by the initscripts which are now disabled when systemd is the init manager. I'll look into this and update 00-create-volatile.conf so that systemd will handle the link creation in stead.

I wonder how my images booted... initscripts is sysvinit specific, the systemd startup should handle this somehow.

(insert standard argument that we should adopt /run across all init systems)

This is exactly what I was looking for. /run shouldn't be there, yet somehow it is. I was looking into how/where systemd is creating /run but no lead for now.

Radu Moisan <[email protected]>

Quoted 9 lines · writes:

Quoted 9 lines

Quoted 4 lines

Possibly because /run was a link created by the initscripts which are now disabled when systemd is the init manager. I'll look into this and update 00-create-volatile.conf so that systemd will handle the link creation in stead.

I wonder how my images booted... initscripts is sysvinit specific, the systemd startup should handle this somehow.

(insert standard argument that we should adopt /run across all init systems)

yes; that's the best way. Make /var/run a symlink pointing to /run.

Quoted 3 lines

This is exactly what I was looking for. /run shouldn't be there, yet somehow it is. I was looking into how/where systemd is creating /run but no lead for now.

/run is part of systemd's api and mounted very early internally by systemd. This mountpoint must exist in the filesystem and it will probably not work when it is symlink into /var/volatile which gets mounted later.

Quoted 4 lines

/run is part of systemd's api and mounted very early internally by systemd. This mountpoint must exist in the filesystem and it will probably not work when it is symlink into /var/volatile which gets mounted later.

Isn't systemd looking into fstab.systemd for mount points?

Radu Moisan <[email protected]>

Quoted 4 lines · writes:

Quoted 4 lines

/run is part of systemd's api and mounted very early internally by systemd. This mountpoint must exist in the filesystem and it will probably not work when it is symlink into /var/volatile which gets mounted later.

Quoted 1 line

Isn't systemd looking into fstab.systemd for mount points?

not for /run or /proc or so. In src/core/mount-setup.c there is a 'mount_table[]' list which is evaluated before fstab is parsed. You can override this by setting $SYSTEMD_SKIP_API_MOUNTS but this will create other problems.

ok, thanks Enrico, this clears the picture for me.
Well then it looks like Jack's problems is coming from some other place.

Quoted 2 lines · Radu Moisan

ok, thanks Enrico, this clears the picture for me.
Well then it looks like Jack's problems is coming from some other place.

No - /run needs to exist for the mount to happen, and there isn't a /run in the filesystem. Whatever creates the root filesystem structure should also create /run I guess.

Quoted 3 lines · Ross

Quoted 2 lines · Radu Moisan

ok, thanks Enrico, this clears the picture for me.
Well then it looks like Jack's problems is coming from some other place.

Quoted 3 lines

No - /run needs to exist for the mount to happen, and there isn't a /run in the filesystem. Whatever creates the root filesystem structure should also create /run I guess.

I agree, but what I meant was that my initial assumptions were wrong with concern to Jack's problem. I run it on qemu and /run is there. His problem related to /run creation while I was thinking about mounting issue.

Radu

Did this ever go anywhere? I have just tried again today with exactly the same result, all I did was change the DISTRO_FEATURES_INITMAN to systemd.

Quoted 2 lines · Jack Mitchell

Did this ever go anywhere? I have just tried again today with exactly the same result, all I did was change the DISTRO_FEATURES_INITMAN to systemd.

Odd, as I just built and booted a systemd image (core-image-sato, in poky master), and it worked fine.

Ross

Quoted 3 lines · Ross

Quoted 2 lines · Jack Mitchell

Did this ever go anywhere? I have just tried again today with exactly the same result, all I did was change the DISTRO_FEATURES_INITMAN to systemd.

Quoted 4 lines

Odd, as I just built and booted a systemd image (core-image-sato, in poky master), and it worked fine.

Ross

I have a custom distro definition that cuts a lot of features out. Can you find out which package is supposed to create run and I can check if it is pulled in properly on my setup?

I think something must be assumed somewhere and I don't have the features enabled to trigger it.

Quoted 7 lines · Jack Mitchell

Quoted 6 lines · Ross

Quoted 2 lines · Jack Mitchell

Did this ever go anywhere? I have just tried again today with exactly the same result, all I did was change the DISTRO_FEATURES_INITMAN to systemd.

Odd, as I just built and booted a systemd image (core-image-sato, in poky master), and it worked fine.

Quoted 6 lines

Quoted 1 line

Ross

I have a custom distro definition that cuts a lot of features out. Can you find out which package is supposed to create run and I can check if it is pulled in properly on my setup?

I think something must be assumed somewhere and I don't have the features enabled to trigger it.

From a quick look it seems as though volatiles in initscripts-1.0 should create the run directory if it's not found. However the initscripts package doesn't get pulled in on my image at all, and I can't see an equivalent in the systemd package

Quoted 6 lines · Ross

Quoted 2 lines · Jack Mitchell

Did this ever go anywhere? I have just tried again today with exactly the same result, all I did was change the DISTRO_FEATURES_INITMAN to systemd.

Odd, as I just built and booted a systemd image (core-image-sato, in poky master), and it worked fine.

Quoted 1 line

Ross

I have a custom distro definition that cuts a lot of features out. Can you find out which package is supposed to create run and I can check if it is pulled in properly on my setup?

I think something must be assumed somewhere and I don't have the features enabled to trigger it.

Quoted 12 lines · Jack Mitchell

Quoted 7 lines · Ross

Quoted 2 lines · Jack Mitchell

Did this ever go anywhere? I have just tried again today with exactly the same result, all I did was change the DISTRO_FEATURES_INITMAN to systemd.

Odd, as I just built and booted a systemd image (core-image-sato, in poky master), and it worked fine.

Ross

I have a custom distro definition that cuts a lot of features out. Can you find out which package is supposed to create run and I can check if it is pulled in properly on my setup?

Quoted 2 lines

I think something must be assumed somewhere and I don't have the features enabled to trigger it.

its created by base-files package. If you need hint how it works with systemd take a look at bbappend that angstrom has for base-files in meta-angstrom

Quoted 13 lines · Khem Raj

Quoted 12 lines · Jack Mitchell

Quoted 7 lines · Ross

Quoted 2 lines · Jack Mitchell

Did this ever go anywhere? I have just tried again today with exactly the same result, all I did was change the DISTRO_FEATURES_INITMAN to systemd.

Odd, as I just built and booted a systemd image (core-image-sato, in poky master), and it worked fine.

Ross

I have a custom distro definition that cuts a lot of features out. Can you find out which package is supposed to create run and I can check if it is pulled in properly on my setup?

Quoted 5 lines

Quoted 2 lines

I think something must be assumed somewhere and I don't have the features enabled to trigger it.

its created by base-files package. If you need hint how it works with systemd take a look at bbappend that angstrom has for base-files in meta-angstrom

Ah, I see. Thanks Khem.

So, the next question is; how come Angstrom requires these files to be appended to base-files in order to get systemd going, when oe-core doesn't?

I notice there is:

${localstatedir}/volatile/run \

In the base-files oe-core package, while in Angstrom we have:

${localstatedir}/run \
/run \

I understand oe-core having it as a volatile; but I don't understand why it doesn't create the /run directory. I see some kind of logic regarding it here:

volatiles = "run log lock tmp"

for d in ${volatiles}; do
ln -sf volatile/$d ${D}${localstatedir}/$d
done

However, if it doesn't create the directory on my target filesystem then something must be wrong...

Quoted 34 lines · Jack Mitchell

Quoted 13 lines · Khem Raj

Quoted 12 lines · Jack Mitchell

Quoted 7 lines · Ross

Quoted 2 lines · Jack Mitchell

Did this ever go anywhere? I have just tried again today with exactly the same result, all I did was change the DISTRO_FEATURES_INITMAN to systemd.

Odd, as I just built and booted a systemd image (core-image-sato, in poky master), and it worked fine.

Ross

I have a custom distro definition that cuts a lot of features out. Can you find out which package is supposed to create run and I can check if it is pulled in properly on my setup?

Quoted 5 lines

Quoted 2 lines

I think something must be assumed somewhere and I don't have the features enabled to trigger it.

its created by base-files package. If you need hint how it works with systemd take a look at bbappend that angstrom has for base-files in meta-angstrom

Ah, I see. Thanks Khem.

So, the next question is; how come Angstrom requires these files to be appended to base-files in order to get systemd going, when oe-core doesn't?

I notice there is:

${localstatedir}/volatile/run \

In the base-files oe-core package, while in Angstrom we have:

${localstatedir}/run \
/run \

I understand oe-core having it as a volatile; but I don't understand why it doesn't create the /run directory. I see some kind of logic regarding it here:

Quoted 9 lines

volatiles = "run log lock tmp"

for d in ${volatiles}; do
ln -sf volatile/$d ${D}${localstatedir}/$d
done

However, if it doesn't create the directory on my target filesystem then something must be wrong...

Ok, some oddities and questions.

I have slightly altered the base-files volatiles create loop from:

for d in ${volatiles}; do
ln -sf volatile/$d ${D}${localstatedir}/$d
done

to

  for d in ${volatiles}; do
    ln -sfv /volatile/$d ${D}/$d
  done

Which gives the output:
DEBUG: Executing shell function do_install '/mnt/SSD/oe-r0005/r0005/tmp/work/beaglebone-oecore-linux-gnueabi/base-files/3.0.14-r73/image/run' -> '/volatile/run' '/mnt/SSD/oe-r0005/r0005/tmp/work/beaglebone-oecore-linux-gnueabi/base-files/3.0.14-r73/image/log' -> '/volatile/log' '/mnt/SSD/oe-r0005/r0005/tmp/work/beaglebone-oecore-linux-gnueabi/base-files/3.0.14-r73/image/lock' -> '/volatile/lock' '/mnt/SSD/oe-r0005/r0005/tmp/work/beaglebone-oecore-linux-gnueabi/base-files/3.0.14-r73/image/tmp/tmp' -> '/volatile/tmp'
DEBUG: Shell function do_install finished

The oddity is the volatiles variable looks like this:

volatiles = "run log lock tmp"

So why do we get:

'/mnt/SSD/oe-r0005/r0005/tmp/work/beaglebone-oecore-linux-gnueabi/base-files/3.0.14-r73/image/tmp/tmp' -> '/volatile/tmp'

with /tmp/tmp?

I have also removed the ${localstatedir} otherwise I was getting this:

'/mnt/SSD/oe-r0005/r0005/tmp/work/beaglebone-oecore-linux-gnueabi/base-files/3.0.14-r73/image/var/run' -> '/volatile/run'

With the extra var/ in. Which would make sense for why I wasn't getting directories made in the root folder. I suppose the issue is, why was everyone else?

Can anyone shed any light on this?

Ok, I have just come back to trying out the new systemd implementation, and this bug still exists.

Exactly the same error as before:

[    2.390730] systemd[1]: Failed to mount /run: No such file or directory

Can we shed any light on why this directory is failing to be created and what can be done to fix it. Having looked in my final target filesystem there are no links to the volatiles in /var/voltatile/* to /*, should there be? The base-files packages looks like it should do it but obviously something is going wrong.

   for d in ${volatiles}; do
     ln -sf volatile/$d ${D}${localstatedir}/$d
   done

Quoted 13 lines · Jack Mitchell

Ok, I have just come back to trying out the new systemd implementation, and this bug still exists.

Exactly the same error as before:

[    2.390730] systemd[1]: Failed to mount /run: No such file or directory

Can we shed any light on why this directory is failing to be created and what can be done to fix it. Having looked in my final target filesystem there are no links to the volatiles in /var/voltatile/* to /*, should there be? The base-files packages looks like it should do it but obviously something is going wrong.

Quoted 3 lines

    for d in ${volatiles}; do
      ln -sf volatile/$d ${D}${localstatedir}/$d
    done

We're using base-files bbappend to create it and also replacing few symlinks to volatile with real directories (not mandatory for systemd iirc, but useful in some cases).

https://github.com/shr-distribution/meta-smartphone/commit/24161eed3c185867b2fd91aab6532f370d327cdd

Quoted 21 lines · Martin Jansa

Quoted 13 lines · Jack Mitchell

Ok, I have just come back to trying out the new systemd implementation, and this bug still exists.

Exactly the same error as before:

[    2.390730] systemd[1]: Failed to mount /run: No such file or directory

Can we shed any light on why this directory is failing to be created and what can be done to fix it. Having looked in my final target filesystem there are no links to the volatiles in /var/voltatile/* to /*, should there be? The base-files packages looks like it should do it but obviously something is going wrong.

Quoted 3 lines

    for d in ${volatiles}; do
      ln -sf volatile/$d ${D}${localstatedir}/$d
    done

We're using base-files bbappend to create it and also replacing few symlinks to volatile with real directories (not mandatory for systemd iirc, but useful in some cases).

Quoted 3 lines

https://github.com/shr-distribution/meta-smartphone/commit/24161eed3c185867b2fd91aab6532f370d327cdd

Ok, manually adding /run to 775dirs in a base-files append fixed this for me. I still don't understand why this should be needed as an addition though...

Quoted 83 lines · Jack Mitchell

Quoted 34 lines · Jack Mitchell

Quoted 13 lines · Khem Raj

Quoted 12 lines · Jack Mitchell

Quoted 7 lines · Ross

Quoted 2 lines · Jack Mitchell

Did this ever go anywhere? I have just tried again today with exactly the same result, all I did was change the DISTRO_FEATURES_INITMAN to systemd.

Odd, as I just built and booted a systemd image (core-image-sato, in poky master), and it worked fine.

Ross

I have a custom distro definition that cuts a lot of features out. Can you find out which package is supposed to create run and I can check if it is pulled in properly on my setup?

Quoted 5 lines

Quoted 2 lines

I think something must be assumed somewhere and I don't have the features enabled to trigger it.

its created by base-files package. If you need hint how it works with systemd take a look at bbappend that angstrom has for base-files in meta-angstrom

Ah, I see. Thanks Khem.

So, the next question is; how come Angstrom requires these files to be appended to base-files in order to get systemd going, when oe-core doesn't?

I notice there is:

${localstatedir}/volatile/run \

In the base-files oe-core package, while in Angstrom we have:

${localstatedir}/run \
/run \

I understand oe-core having it as a volatile; but I don't understand why it doesn't create the /run directory. I see some kind of logic regarding it here:

Quoted 9 lines

volatiles = "run log lock tmp"

for d in ${volatiles}; do
ln -sf volatile/$d ${D}${localstatedir}/$d
done

However, if it doesn't create the directory on my target filesystem then something must be wrong...

Ok, some oddities and questions.

I have slightly altered the base-files volatiles create loop from:

for d in ${volatiles}; do
ln -sf volatile/$d ${D}${localstatedir}/$d
done

to

  for d in ${volatiles}; do
    ln -sfv /volatile/$d ${D}/$d
  done

Which gives the output:
DEBUG: Executing shell function do_install '/mnt/SSD/oe-r0005/r0005/tmp/work/beaglebone-oecore-linux-gnueabi/base-files/3.0.14-r73/image/run' -> '/volatile/run' '/mnt/SSD/oe-r0005/r0005/tmp/work/beaglebone-oecore-linux-gnueabi/base-files/3.0.14-r73/image/log' -> '/volatile/log' '/mnt/SSD/oe-r0005/r0005/tmp/work/beaglebone-oecore-linux-gnueabi/base-files/3.0.14-r73/image/lock' -> '/volatile/lock' '/mnt/SSD/oe-r0005/r0005/tmp/work/beaglebone-oecore-linux-gnueabi/base-files/3.0.14-r73/image/tmp/tmp' -> '/volatile/tmp'
DEBUG: Shell function do_install finished

The oddity is the volatiles variable looks like this:

volatiles = "run log lock tmp"

So why do we get:

'/mnt/SSD/oe-r0005/r0005/tmp/work/beaglebone-oecore-linux-gnueabi/base-files/3.0.14-r73/image/tmp/tmp' -> '/volatile/tmp'

with /tmp/tmp?

Quoted 9 lines

I have also removed the ${localstatedir} otherwise I was getting this:

'/mnt/SSD/oe-r0005/r0005/tmp/work/beaglebone-oecore-linux-gnueabi/base-files/3.0.14-r73/image/var/run' -> '/volatile/run'

With the extra var/ in. Which would make sense for why I wasn't getting directories made in the root folder. I suppose the issue is, why was everyone else?

Can anyone shed any light on this?

Right,

I think it should be this (minus the debug):

  for d in run log tmp lock; do
    echo ${D}
    echo $d
    ln -sfv ${localstatedir}/volatile/$d ${D}/$d
  done

Which gives:

DEBUG: Executing shell function do_install /mnt/SSD/oe-r0005/r0005/tmp/work/beaglebone-oecore-linux-gnueabi/base-files/3.0.14-r73/image

run '/mnt/SSD/oe-r0005/r0005/tmp/work/beaglebone-oecore-linux-gnueabi/base-files/3.0.14-r73/image/run' -> '/var/volatile/run' /mnt/SSD/oe-r0005/r0005/tmp/work/beaglebone-oecore-linux-gnueabi/base-files/3.0.14-r73/image

log '/mnt/SSD/oe-r0005/r0005/tmp/work/beaglebone-oecore-linux-gnueabi/base-files/3.0.14-r73/image/log' -> '/var/volatile/log' /mnt/SSD/oe-r0005/r0005/tmp/work/beaglebone-oecore-linux-gnueabi/base-files/3.0.14-r73/image

tmp '/mnt/SSD/oe-r0005/r0005/tmp/work/beaglebone-oecore-linux-gnueabi/base-files/3.0.14-r73/image/tmp/tmp' -> '/var/volatile/tmp' /mnt/SSD/oe-r0005/r0005/tmp/work/beaglebone-oecore-linux-gnueabi/base-files/3.0.14-r73/image

lock '/mnt/SSD/oe-r0005/r0005/tmp/work/beaglebone-oecore-linux-gnueabi/base-files/3.0.14-r73/image/lock' -> '/var/volatile/lock'
DEBUG: Shell function do_install finished
~

The only issue I am having is still the double tmp/tmp thing; anyone know why it is doing it? It seems like a weird substitution or something.....