Skip to contentSkip to conversation

Remove /var/cache from volatiles directory

PATCH In review Laurentiu Palcu 8 participants 16 messages Feb 4, 2013 – Feb 5, 2013

lore
Share & tools
lore
#0 lore raw

cover Remove /var/cache from volatiles directory

Hi,

Re-generating applications' cache every time the system is rebooted is not a very efficient process for an embedded device. Usually, the cache directory is used by applications to store data resulting from time consmuming I/O or calculation. So, this patchset will take out /var/cache from tmpfs.

Thanks,
Laurentiu

The following changes since commit 75f470cd18d693a9a96d9849291c2c8de4dcbeb8:

qt4: Add space for _appends (2013-02-01 22:49:47 +0000)

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib lpalcu/remove_cache_from_volatiles
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/remove_cache_from_volatiles

Laurentiu Palcu (5):
base-files: remove /var/cache from volatiles
fs-perms: remove /var/cache from volatiles
initscripts: remove /var/cache from volatiles
systemd: remove /var/cache from volatiles
rpm: remove /var/volatiles/cache/rpm from the FILEs list

 meta/files/fs-perms.txt                            |    1 -
 meta/recipes-core/base-files/base-files_3.0.14.bb  |    4 ++--
 .../initscripts/initscripts-1.0/volatiles          |    2 --
 .../systemd/systemd/00-create-volatile.conf        |    1 -
 meta/recipes-devtools/rpm/rpm_5.4.9.bb             |    1 -
 5 files changed, 2 insertions(+), 7 deletions(-)
Signature · 1 line
1.7.9.5
#1 lore raw

PATCH 1/5 base-files: remove /var/cache from volatiles

Having the cache in tmpfs implies cache regeneration after every reboot. For an embedded device this might not be very efficient. So, it is better for the cache to be persistent between reboots.

---
 meta/recipes-core/base-files/base-files_3.0.14.bb |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
meta/recipes-core/base-files/base-files_3.0.14.bb +2 -2
index a90e995..6a3db50 100644
@@ -42,7 +42,7 @@ dirs755 = "/bin /boot /dev ${sysconfdir} ${sysconfdir}/default \
            ${mandir} ${datadir}/misc ${localstatedir} \
            ${localstatedir}/backups ${localstatedir}/lib \
            /sys ${localstatedir}/lib/misc ${localstatedir}/spool \
-           ${localstatedir}/volatile ${localstatedir}/volatile/cache \
+           ${localstatedir}/volatile \
            ${localstatedir}/volatile/lock/subsys \
            ${localstatedir}/volatile/log \
            ${localstatedir}/volatile/run \
@@ -55,7 +55,7 @@ dirs3755 = "/srv  \
             ${prefix}/local/share ${prefix}/local/src"
 dirs4775 = "/var/mail"
 
-volatiles = "cache run log lock tmp"
+volatiles = "run log lock tmp"
 conffiles = "${sysconfdir}/debian_version ${sysconfdir}/host.conf \
              ${sysconfdir}/inputrc ${sysconfdir}/issue /${sysconfdir}/issue.net \
              ${sysconfdir}/nsswitch.conf ${sysconfdir}/profile \
Signature · 1 line
1.7.9.5
#2 lore raw

PATCH 2/5 fs-perms: remove /var/cache from volatiles

In reply to Laurentiu Palcu
---
 meta/files/fs-perms.txt |    1 -
 1 file changed, 1 deletion(-)
meta/files/fs-perms.txt +0 -1
index f5a2b69..ef44e26 100644
@@ -45,7 +45,6 @@ ${oldincludedir}	0755	root	root	true	0644	root	root
 
 # Items from base-files
 # Links
-${localstatedir}/cache	link	volatile/cache
 ${localstatedir}/run	link	volatile/run
 ${localstatedir}/log	link	volatile/log
 ${localstatedir}/lock	link	volatile/lock
Signature · 1 line
1.7.9.5
#3 lore raw

PATCH 3/5 initscripts: remove /var/cache from volatiles

In reply to Laurentiu Palcu
---
 .../initscripts/initscripts-1.0/volatiles          |    2 --
 1 file changed, 2 deletions(-)
meta/recipes-core/initscripts/initscripts-1.0/volatiles +0 -2
index e0741aa..087ab0a 100644
@@ -21,12 +21,10 @@
 # This explains the order of "link before object" as in the example above, where
 # a link will be created at /var/test pointing to /tmp/testfile and due to this
 # link the file defined as /var/test will actually be created as /tmp/testfile.
-d root root 0755 /var/volatile/cache none
 d root root 1777 /var/volatile/lock none
 d root root 0755 /var/volatile/log none
 d root root 0755 /var/volatile/run none
 d root root 1777 /var/volatile/tmp none
-l root root 0755 /var/cache /var/volatile/cache
 l root root 1777 /var/lock /var/volatile/lock
 l root root 0755 /var/log /var/volatile/log
 l root root 0755 /var/run /var/volatile/run
Signature · 1 line
1.7.9.5
#4 lore raw

PATCH 4/5 systemd: remove /var/cache from volatiles

In reply to Laurentiu Palcu
---
 .../systemd/systemd/00-create-volatile.conf        |    1 -
 1 file changed, 1 deletion(-)
meta/recipes-core/systemd/systemd/00-create-volatile.conf +0 -1
index d160a9b..6163a9c 100644
@@ -3,7 +3,6 @@
 # inside /var/log.
 
 
-d		/var/volatile/cache	-	-	-	-
 d		/var/volatile/log		-	-	-	-
 d		/var/volatile/lock		-	-	-	-
 d		/var/volatile/run		-	-	-	-
Signature · 1 line
1.7.9.5
#5 lore raw

PATCH 5/5 rpm: remove /var/volatiles/cache/rpm from the FILEs list

In reply to Laurentiu Palcu

Since /var/cache is not in volatiles anymore, this entry has to go.

---
 meta/recipes-devtools/rpm/rpm_5.4.9.bb |    1 -
 1 file changed, 1 deletion(-)
meta/recipes-devtools/rpm/rpm_5.4.9.bb +0 -1
index 881b960..39b0481 100644
@@ -223,7 +223,6 @@ FILES_${PN} =  "${bindir}/rpm \
 		${libdir}/rpm/bin/wget \
 		/var/lib/rpm \
 		/var/cache/rpm \
-		/var/volatile/cache/rpm \
 		"
 
 FILES_${PN}-dbg += "${libdir}/rpm/.debug \
Signature · 1 line
1.7.9.5
#6 lore raw
In reply to Laurentiu Palcu
Quoted 6 lines · Laurentiu Palcu

Hi,

Re-generating applications' cache every time the system is rebooted is not a very efficient process for an embedded device. Usually, the cache directory is used by applications to store data resulting from time consmuming I/O or calculation. So, this patchset will take out /var/cache from tmpfs.

What if you have a READONLY rootfs? This patch will "break" those devices I think.

Quoted 27 lines

Thanks,
Laurentiu

The following changes since commit 75f470cd18d693a9a96d9849291c2c8de4dcbeb8:

   qt4: Add space for _appends (2013-02-01 22:49:47 +0000)

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib lpalcu/remove_cache_from_volatiles
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/remove_cache_from_volatiles

Laurentiu Palcu (5):
base-files: remove /var/cache from volatiles
fs-perms: remove /var/cache from volatiles
initscripts: remove /var/cache from volatiles
systemd: remove /var/cache from volatiles
rpm: remove /var/volatiles/cache/rpm from the FILEs list

  meta/files/fs-perms.txt                            |    1 -
  meta/recipes-core/base-files/base-files_3.0.14.bb  |    4 ++--
  .../initscripts/initscripts-1.0/volatiles          |    2 --
  .../systemd/systemd/00-create-volatile.conf        |    1 -
  meta/recipes-devtools/rpm/rpm_5.4.9.bb             |    1 -
  5 files changed, 2 insertions(+), 7 deletions(-)
#7 lore raw
Quoted 11 lines · Mike Looijmans
Quoted 6 lines · Laurentiu Palcu
Quoted 6 lines

Hi,

Re-generating applications' cache every time the system is rebooted is not a very efficient process for an embedded device. Usually, the cache directory is used by applications to store data resulting from time consmuming I/O or calculation. So, this patchset will take out /var/cache from tmpfs.

What if you have a READONLY rootfs? This patch will "break" those devices I think.

Not necessarily. We are trying to have all postinstalls run on host, so we don't have to do any postinstall activity on target. Any cache generated during postinstall, will be generated at do_rootfs time. Then, we can deploy the image with the cache already in place.

Also, on a RO rootfs, there is really no need to update the cache since the cached files do not really change. Hence, the cache created on host should, in theory, be valid.

If the above logic is missing anything, feel free to correct me.

Thanks,
Laurentiu

#8 lore raw
Quoted 19 lines · Laurentiu Palcu
Quoted 11 lines · Mike Looijmans
Quoted 6 lines · Laurentiu Palcu
Quoted 6 lines

Hi,

Re-generating applications' cache every time the system is rebooted is not a very efficient process for an embedded device. Usually, the cache directory is used by applications to store data resulting from time consmuming I/O or calculation. So, this patchset will take out /var/cache from tmpfs.

What if you have a READONLY rootfs? This patch will "break" those devices I think.

Not necessarily. We are trying to have all postinstalls run on host, so we don't have to do any postinstall activity on target. Any cache generated during postinstall, will be generated at do_rootfs time. Then, we can deploy the image with the cache already in place.

Quoted 5 lines

Also, on a RO rootfs, there is really no need to update the cache since the cached files do not really change. Hence, the cache created on host should, in theory, be valid.

If the above logic is missing anything, feel free to correct me.

If this is an issue and something does need to touch the cache directory, we can mount a unionfs over /var/cache like we're doing for /var/lib in the readonly rootfs patch series.

Ultimately, I think we're better served with that approach all round.

Cheers,

Richard

#9 lore raw
In reply to Laurentiu Palcu
Quoted 8 lines · Laurentiu Palcu

Not necessarily. We are trying to have all postinstalls run on host, so we don't have to do any postinstall activity on target. Any cache generated during postinstall, will be generated at do_rootfs time. Then, we can deploy the image with the cache already in place.

Also, on a RO rootfs, there is really no need to update the cache since the cached files do not really change. Hence, the cache created on host should, in theory, be valid.

What sort of files in /var/cache are you trying to address with this change? The impression I have is that most of the things in there are created dynamically by applications running on the target system and not by postinst scripts that could be run on the host.

For example, an (admittedly somewhat unscientific) survey on my desktop machine suggests that fontconfig, samba and cups, at least, are creating files in /var/cache at run time with contents that you wouldn't be able to predict in advance. (Apt also uses /var/cache as its download area, but I guess if you have a read-only roofs then you aren't going to be running apt very much so that one probably isn't an issue.) I wasn't able to immediately identify any files in there that would be obvious candidates for creating in a postinst and then leaving untouched afterwards.

p.

#10 lore raw
Quoted 4 lines · Phil Blundell

For example, an (admittedly somewhat unscientific) survey on my desktop machine suggests that fontconfig, samba and cups, at least, are creating files in /var/cache at run time with contents that you wouldn't be able to predict in advance

The fontconfig cache is perfectly capable of generation at image creation time, and that's actually what prompted removal of /var/cache from volatiles.

If samba and cups are using /var/cache as something that isn't a persistent cache, then the should use something like /var/run/ instead.

Ross

#11 lore raw

"Burton, Ross" <[email protected]>

Quoted 2 lines · writes:

The fontconfig cache is perfectly capable of generation at image creation time,

really? The cache files are architecture dependent and I am not aware of a cross-fc-cache tool.

Enrico

#13 lore raw
In reply to Laurentiu Palcu
Quoted 26 lines · Laurentiu Palcu

Hi,

Re-generating applications' cache every time the system is rebooted is not a very efficient process for an embedded device. Usually, the cache directory is used by applications to store data resulting from time consmuming I/O or calculation. So, this patchset will take out /var/cache from tmpfs.

Thanks,
Laurentiu

The following changes since commit 75f470cd18d693a9a96d9849291c2c8de4dcbeb8:

   qt4: Add space for _appends (2013-02-01 22:49:47 +0000)

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib lpalcu/remove_cache_from_volatiles
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/remove_cache_from_volatiles

Laurentiu Palcu (5):
base-files: remove /var/cache from volatiles
fs-perms: remove /var/cache from volatiles
initscripts: remove /var/cache from volatiles
systemd: remove /var/cache from volatiles
rpm: remove /var/volatiles/cache/rpm from the FILEs list

Quoted 7 lines
  meta/files/fs-perms.txt                            |    1 -
  meta/recipes-core/base-files/base-files_3.0.14.bb  |    4 ++--
  .../initscripts/initscripts-1.0/volatiles          |    2 --
  .../systemd/systemd/00-create-volatile.conf        |    1 -
  meta/recipes-devtools/rpm/rpm_5.4.9.bb             |    1 -
  5 files changed, 2 insertions(+), 7 deletions(-)

what about /var/log?

https://bugzilla.yoctoproject.org/show_bug.cgi?id=3397

Acked-by on the v1 series
#15 lore raw
In reply to Laurentiu Palcu
Quoted 7 lines · Laurentiu Palcu

Hi,

Re-generating applications' cache every time the system is rebooted is not a very efficient process for an embedded device. Usually, the cache directory is used by applications to store data resulting from time consmuming I/O or calculation. So, this patchset will take out /var/cache from tmpfs.

I did a basic test with this change along with the "read-only-rootfs" IMAGE_FEATURE and things are looking pretty good. The only thing I noticed is that we leave some /etc/*- files around, I know it's not this patch.

Thanks
Sau!

Quoted 26 lines

Thanks,
Laurentiu

The following changes since commit 75f470cd18d693a9a96d9849291c2c8de4dcbeb8:

   qt4: Add space for _appends (2013-02-01 22:49:47 +0000)

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib lpalcu/remove_cache_from_volatiles
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=lpalcu/remove_cache_from_volatiles

Laurentiu Palcu (5):
base-files: remove /var/cache from volatiles
fs-perms: remove /var/cache from volatiles
initscripts: remove /var/cache from volatiles
systemd: remove /var/cache from volatiles
rpm: remove /var/volatiles/cache/rpm from the FILEs list

  meta/files/fs-perms.txt                            |    1 -
  meta/recipes-core/base-files/base-files_3.0.14.bb  |    4 ++--
  .../initscripts/initscripts-1.0/volatiles          |    2 --
  .../systemd/systemd/00-create-volatile.conf        |    1 -
  meta/recipes-devtools/rpm/rpm_5.4.9.bb             |    1 -
  5 files changed, 2 insertions(+), 7 deletions(-)