Skip to contentSkip to conversation

Issue: /var/tmp does not persist due to /var/volatile being tmpfs in fstab #scarthgap

DISCUSSION Vish 5 participants 9 messages Mar 18, 2025 – Mar 19, 2025

lore
Share & tools
lore

Vish

author
#0 lore raw

The current Yocto basefiles fstab mounts /var/volatile as tmpfs, which results in the loss of all files stored in /var/tmp upon reboot.

fstab in base-files:

https://git.yoctoproject.org/poky/tree/meta/recipes-core/base-files/base-files/fstab?h=scarthgap&id=3cc8c3520cedb66500757f843f757646da69ab30#n7

Since /var/volatile is a tmpfs, its contents are lost on reboot.

This commit might have introduced the issue - s ince /var/volatile/tmp is inside a tmpfs mount, it gets wiped on reboot, meaning /var/tmp is also cleared.
Commit - https://git.yoctoproject.org/poky/commit/meta/recipes-core/systemd/systemd/00-create-volatile.conf?h=scarthgap&id=3cc8c3520cedb66500757f843f757646da69ab30 ( https://git.yoctoproject.org/poky/commit/meta/recipes-core/systemd/systemd/00-create-volatile.conf?h=scarthgap&id=3cc8c3520cedb66500757f843f757646da69ab30 )

This commit links /var/tmp to /var/volatile/tmp
*Expected Behavior*

* /var/tmp should retain files across reboots, as per standard Linux behavior.
* Using a symlink to a volatile tmpfs directory contradicts this expectation.

*Impact*

* Any application expecting persistent temporary storage in /var/tmp will lose data after a reboot.
* This behavior may break software relying on the standard /var/tmp persistence.

Should /var/tmp continue following standard Linux behavior by *not being linked to /var/volatile/tmp* , or is this change intentional for specific use cases? If persistence is desired, this approach seems problematic.

Would appreciate input from the maintainers on whether this is a bug or intended behavior.

#1 lore raw

This is configurable.

meta/conf/bitbake.conf:FILESYSTEM_PERMS_TABLES ?= "files/fs-perms.txt \
meta/conf/bitbake.conf- files/fs-perms-volatile-log.txt \
meta/conf/bitbake.conf- files/fs-perms-volatile-tmp.txt"

You can remove files/fs-perms-volatile-tmp.txt from FILESYSTEM_PERMS_TABLES, and /var/tmp will be persistent.

Regards,
Qi

From: [email protected] <[email protected]> On Behalf Of Vish via lists.openembedded.org
Sent: Tuesday, March 18, 2025 11:17 AM
To: [email protected]
Subject: [OE-core] Issue: /var/tmp does not persist due to /var/volatile being tmpfs in fstab #scarthgap

The current Yocto basefiles fstab mounts /var/volatile as tmpfs, which results in the loss of all files stored in /var/tmp upon reboot.

fstab in base-files:

https://git.yoctoproject.org/poky/tree/meta/recipes-core/base-files/base-files/fstab?h=scarthgap&id=3cc8c3520cedb66500757f843f757646da69ab30#n7

Since /var/volatile is a tmpfs, its contents are lost on reboot.

This commit might have introduced the issue - since /var/volatile/tmp is inside a tmpfs mount, it gets wiped on reboot, meaning /var/tmp is also cleared.
Commit - https://git.yoctoproject.org/poky/commit/meta/recipes-core/systemd/systemd/00-create-volatile.conf?h=scarthgap&id=3cc8c3520cedb66500757f843f757646da69ab30 <https://git.yoctoproject.org/poky/commit/meta/recipes-core/systemd/systemd/00-create-volatile.conf?h=scarthgap&id=3cc8c3520cedb66500757f843f757646da69ab30>

This commit links /var/tmp to /var/volatile/tmp

Expected Behavior

* /var/tmp should retain files across reboots, as per standard Linux behavior.
* Using a symlink to a volatile tmpfs directory contradicts this expectation.

Impact

* Any application expecting persistent temporary storage in /var/tmp will lose data after a reboot.
* This behavior may break software relying on the standard /var/tmp persistence.

Should /var/tmp continue following standard Linux behavior by not being linked to /var/volatile/tmp, or is this change intentional for specific use cases? If persistence is desired, this approach seems problematic.

Would appreciate input from the maintainers on whether this is a bug or intended behavior.

author
#2 lore raw

Thank you for the reply.

In Scarthgap LTS implementation is different, but I got the idea.
https://git.yoctoproject.org/poky/tree/meta/conf/bitbake.conf?h=scarthgap#n387

I believe default behavior should be same as before to persist /var/temp over reboot and these changes should only be required for any customization. Also somewhat difficult for us to upgrade CI/CD with this config.

Can we know the reasons for original changes here?
https://git.yoctoproject.org/poky/commit/meta/recipes-core/systemd/systemd/00-create-volatile.conf?h=scarthgap&id=3cc8c3520cedb66500757f843f757646da69ab30

I reviewed commit message but not giving much details. Considering default base-files and as persistence is required, this newer approach seems be an issue.

We recently migrated to Scarthgap and noticed the issues with our system. This can be issue for other developers as well whoever is planning to migrate to Scarthgap and embedded devices in the field. This commit requires all build systems to adopt changes bitbake.conf which is not preferable .

What is the usecases targeted with the changes? I believe the commit should be reviewed again and potentially reverted considering default behavior with base-files and 00-create-volatile.conf.

Regards,
VS.

#3 lore raw
Quoted 18 lines · Vish via lists.openembedded.org

Thank you for the reply.

In Scarthgap LTS implementation is different, but I got the idea.
https://git.yoctoproject.org/poky/tree/meta/conf/bitbake.conf?h=scarthgap#n387

I believe default behavior should be same as before to persist /var/temp over reboot and these changes should only be required for any customization. Also somewhat difficult for us to upgrade CI/CD with this config.

Can we know the reasons for original changes here?
https://git.yoctoproject.org/poky/commit/meta/recipes-core/systemd/systemd/00-create-volatile.conf?h=scarthgap&id=3cc8c3520cedb66500757f843f757646da69ab30

I reviewed commit message but not giving much details. Considering default base-files and as persistence is required, this newer approach seems be an issue.

We recently migrated to Scarthgap and noticed the issues with our

Quoted 8 lines

system. This can be issue for other developers as well whoever is planning to migrate to Scarthgap and embedded devices in the field. This commit requires all build systems to adopt changes bitbake.conf which is not preferable .

What is the usecases targeted with the changes? I believe the commit should be reviewed again and potentially reverted considering default behavior with base-files and 00-create-volatile.conf.

scarthgap has been released for a while now so making a change like this well into the stable release cycle would be a problem in its own right. I therefore don't think this can be reverted now regardless of whether it should have happened or not.

Cheers,

Richard

author
#4 lore raw

Thanks for the reply.

Do maintainers recognize this as an issue with having this as default option? There are chances that similar to us other devs/orgs are planning migration in phases to Scarthgap considering Kirkstone getting updates, so this could potentially become issue for others as well. We do have sizable amount of edge systems in production with various CI, and this change broke an important feature and took some time for us to find root cause.

I believe correcting is still a proper thing to do and can be planned during any releases.
Additional reference -
https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch05s15.html ( https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch05s15.html )

Also, it would be good to know what scenarios were considered with this change, probably we are missing some context.

Regards,
VS.

#5 lore raw

You can probably propose a patch to change the default in master, but this wouldn't be eligible for a backport to scarthgap. Once they're out, stable releases simply can't change behaviour like this.

Also it would be nice if you introduce yourself and your company.

Alex

Quoted 24 lines · Vish via lists.openembedded.org

Thanks for the reply.

Do maintainers recognize this as an issue with having this as default option? There are chances that similar to us other devs/orgs are planning migration in phases to Scarthgap considering Kirkstone getting updates, so this could potentially become issue for others as well. We do have sizable amount of edge systems in production with various CI, and this change broke an important feature and took some time for us to find root cause.

I believe correcting is still a proper thing to do and can be planned during any releases. Additional reference -
https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch05s15.html

Also, it would be good to know what scenarios were considered with this change, probably we are missing some context.

Regards,
VS.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#213259): https://lists.openembedded.org/g/openembedded-core/message/213259
Mute This Topic: https://lists.openembedded.org/mt/111764066/1686489
Mute #scarthgap:https://lists.openembedded.org/g/openembedded-core/mutehashtag/scarthgap Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#6 lore raw
In reply to Vish
Quoted 7 lines · Vish via lists.openembedded.org

Thanks for the reply.

Do maintainers recognize this as an issue with having this as default option? There are chances that similar to us other devs/orgs are planning migration in phases to Scarthgap considering Kirkstone getting updates, so this could potentially become issue for others as well.

I recognise there is an issue when behaviours change. We can't change this in that release now but I would happily see patches/updates to the migration guide and release notes making the issue clear.

Quoted 3 lines

We do have sizable amount of edge systems in production with various CI, and this change broke an important feature and took some time for us to find root cause.

Sorry it was a pain. If you are able to check the state of master periodically, it may help head some of these kinds of issues off.

Quoted 10 lines

I believe correcting is still a proper thing to do and can be planned during any releases.

Additional reference -

https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch05s15.html

Also, it would be good to know what scenarios were considered with this change, probably we are missing some context.

Traditionally, OpenEmbedded was used in resource constrained embedded systems and all "tmp" space was transient, not persistent. This configuration is in keeping with the project's history.

Our approach isn't entirely against FHS, it is just not quite what other distros do in our default config. There are other areas we do differ from FHS too.

On this specific one, we support two init systems and I think the configuration was mismatched between the two and the configuration options were hard to get right and not working in all cases. There were various patches over the last few years trying to make the docs, options and init systems match up and be configurable. I suspect somewhere on the way things changed in a way that wasn't realised.

The project is crying out for maintainer and review help. We do the best we can with what we have, like most open source projects. An implication of this is that where patches are submitted and nobody objects and I don't see/spot any problem with them, they can be merged even if there is an issue we didn't realise. We do the best we can.

How we react when things are merged depends on the circumstances. Scarthgap has been out for nearly a year so a major behaviour change is not really an option now. Even for master, I think the situation is relatively well understood and more consistent now so any change would need to demonstrate that isn't the case and that a change would improve things rather than further confuse people.

I don't know who you work for but if you are able to help support the project through patch review, recipe maintenance or anything else that be nice. Alternatively, project membership does give us funding which allows us to help in a different way too if that is an option. If you don't do any of these things...

Hope that helps give some of the context.

Cheers,

Richard

author
#7 lore raw

Thank you everyone for responses. We have proceeded as Niko suggested with appending config for now. (Also, will definitely introduce myself/company outside of this forum.)

Thank you very much Richard for the detailed context and history. And I absolutely admire all the efforts that you and community putting in maintaining this massive project.

Appreciating all valuable inputs here.

Regards,
VS

#8 lore raw
In reply to Vish
Quoted 3 lines · Vish via lists.openembedded.org

Can we know the reasons for original changes here?
https://git.yoctoproject.org/poky/commit/meta/recipes-core/systemd/ systemd/00-create-volatile.conf? h=scarthgap&id=3cc8c3520cedb66500757f843f757646da69ab30 <https:// git.yoctoproject.org/poky/commit/meta/recipes-core/systemd/systemd/00- create-volatile.conf? h=scarthgap&id=3cc8c3520cedb66500757f843f757646da69ab30> I reviewed commit message but not giving much details. Considering default base-files and as persistence is required, this newer approach seems be an issue.

Hi Vish, that commit should only have the effect to avoid seeing an associated error line in systemd journal ("/var/tmp" already exists and is not a directory). The error line was stemming from the fact that in Yocto Scarthgap base-files creates /var/tmp as symlink to /var/volatile/tmp (https://git.yoctoproject.org/poky/tree/meta/recipes-core/base-files/base-files_3.0.14.bb?h=scarthgap#n87) which contradicts systemd's default runtime expectations.

The base-files provided /var/tmp -> /var/volatile/tmp symlink has been prevalent convention in Yocto for a very long time. (If you want to have /var/tmp as directory instead in Scarthgap I believe you can e.g. append base-files do_install() task to remove the symlink and replace it with directory.)

BR, Niko