dev-manual: disk-space: mention faster "find" command to trim sstate cache
A message that is not in the archive (sent off-list or not yet received): [email protected]
PATCH v2 1/1 dev-manual: disk-space: mention faster "find" command to trim sstate cache
From: Michael Opdenacker <[email protected]>
[YOCTO #15182]
---
- Changes in V2:
- Using a more universal "-mtime" option to find files to remove as suggested by Richard Purdie. V1 was using "-atime" which won't work with systems mounted with the "noatime" option used to increase disk performance.
- Add more details about sstate-cache-management.sh provided by Richard Purdie, perhaps not recommended but useful to document as long as it is shipped in OE-core.
--- documentation/dev-manual/disk-space.rst | 31 ++++++++++++++++++------- 1 file changed, 22 insertions(+), 9 deletions(-)
documentation/dev-manual/disk-space.rst +22 -9
@@ -27,19 +27,32 @@ Purging Duplicate Shared State Cache Files ========================================== After multiple build iterations, the Shared State (sstate) cache can contain -duplicate cache files for a given package, while only the most recent one -is likely to be reusable. The following command purges all but the -newest sstate cache file for each package:: +duplicate cache files for a given package, consuming a substantial amount of +disk space. However, only the most recent cache files are likeky to be reusable. - sstate-cache-management.sh --remove-duplicated --cache-dir=build/sstate-cache +The following command is a quick way to purge all the cache files which +haven't been used for a least a specified number of days:: -This command will ask you to confirm the deletions it identifies. + find build/sstate-cache -type f -mtime +$DAYS -delete -.. note:: +The above command relies on the fact that BitBake touches the sstate cache +files as it accesses them, when it has write access to the cache. + +For more advanced needs, OpenEmbedded-Core also offers a more elaborate +command. It has the ability to purge all but the newest cache files on each +architecture, and also to remove files that it considers unreachable by +exploring a set of build configurations. However, this command +requires a full build environment to be available and doesn't work well +covering multiple releases. It won't work either on limited environments +such as BSD based NAS:: - The duplicated sstate cache files of one package must have the same - architecture, which means that sstate cache files with multiple - architectures are not considered as duplicate. + sstate-cache-management.sh --remove-duplicated --cache-dir=build/sstate-cache +This command will ask you to confirm the deletions it identifies. Run ``sstate-cache-management.sh`` for more details about this script. +.. note:: + + As this command is much more cautious and selective, removing only cache files, + it will execute much slower than the simple ``find`` command described above. + Therefore, it may not be your best option to trim huge cache directories.
Signature · 1 line
2.34.1
Quoted 37 lines · Michael Opdenacker via lists.yoctoproject.org
From: Michael Opdenacker <[email protected]>
[YOCTO #15182]
Signed-off-by Michael Opdenacker <[email protected]>Reported-by Yoann CONGAL <[email protected]>Reported-by Randy MacLeod <[email protected]>Reported-by Josef Holzmayr <[email protected]>---
- Changes in V2:
- Using a more universal "-mtime" option to find files to remove as suggested by Richard Purdie. V1 was using "-atime" which won't work with systems mounted with the "noatime" option used to increase disk performance.
- Add more details aboutsstate-cache-management.shprovided by Richard Purdie, perhaps not recommended but useful to document as long as it is shipped in OE-core.--- documentation/dev-manual/disk-space.rst | 31 ++++++++++++++++++------- 1 file changed, 22 insertions(+), 9 deletions(-)documentation/dev-manual/disk-space.rst +2
-3@@ -27,19 +27,32 @@ Purging Duplicate Shared State Cache Files ========================================== After multiple build iterations, the Shared State (sstate) cache can contain -duplicate cache files for a given package, while only the most recent one -is likely to be reusable. The following command purges all but the -newest sstate cache file for each package:: +duplicate cache files for a given package, consuming a substantial amount of +disk space. However, only the most recent cache files are likeky to be reusable.
documentation/dev-manual/disk-space.rst Purging Duplicate Shared State Cache Files · line 39
Quoted 10 lines
documentation/dev-manual/disk-space.rst +5
-3- sstate-cache-management.sh --remove-duplicated --cache-dir=build/sstate-cache +The following command is a quick way to purge all the cache files which +haven't been used for a least a specified number of days:: -This command will ask you to confirm the deletions it identifies. + find build/sstate-cache -type f -mtime +$DAYS -delete -.. note:: +The above command relies on the fact that BitBake touches the sstate cache +files as it accesses them, when it has write access to the cache.
Add "You could use -atime instead if the partition isn't mounted with the noatime option for a read only cache."
?
Otherwise looks good thanks.
Cheers,
Richard
PATCH v3 1/1 dev-manual: disk-space: mention faster "find" command to trim sstate cache
From: Michael Opdenacker <[email protected]>
[YOCTO #15182]
---
- Changes in V3:
- Mention the use of "-atime" if we don't have the cache on storage mounted with the "-noatime" option, as suggested by Richard Purdie.
- Changes in V2:
- Using a more universal "-mtime" option to find files to remove as suggested by Richard Purdie. V1 was using "-atime" which won't work with systems mounted with the "noatime" option used to increase disk performance.
- Add more details about sstate-cache-management.sh provided by Richard Purdie, perhaps not recommended but useful to document as long as it is shipped in OE-core.
--- documentation/dev-manual/disk-space.rst | 34 ++++++++++++++++++------- 1 file changed, 25 insertions(+), 9 deletions(-)
documentation/dev-manual/disk-space.rst +25 -9
@@ -27,19 +27,35 @@ Purging Duplicate Shared State Cache Files ========================================== After multiple build iterations, the Shared State (sstate) cache can contain -duplicate cache files for a given package, while only the most recent one -is likely to be reusable. The following command purges all but the -newest sstate cache file for each package:: +duplicate cache files for a given package, consuming a substantial amount of +disk space. However, only the most recent cache files are likeky to be reusable. - sstate-cache-management.sh --remove-duplicated --cache-dir=build/sstate-cache +The following command is a quick way to purge all the cache files which +haven't been used for a least a specified number of days:: -This command will ask you to confirm the deletions it identifies. + find build/sstate-cache -type f -mtime +$DAYS -delete -.. note:: +The above command relies on the fact that BitBake touches the sstate cache +files as it accesses them, when it has write access to the cache. - The duplicated sstate cache files of one package must have the same - architecture, which means that sstate cache files with multiple - architectures are not considered as duplicate. +You could use ``-atime`` instead of ``-mtime`` if the partition isn't mounted +with the ``noatime`` option for a read only cache. +For more advanced needs, OpenEmbedded-Core also offers a more elaborate +command. It has the ability to purge all but the newest cache files on each +architecture, and also to remove files that it considers unreachable by +exploring a set of build configurations. However, this command +requires a full build environment to be available and doesn't work well +covering multiple releases. It won't work either on limited environments +such as BSD based NAS:: + + sstate-cache-management.sh --remove-duplicated --cache-dir=build/sstate-cache + +This command will ask you to confirm the deletions it identifies. Run ``sstate-cache-management.sh`` for more details about this script. +.. note:: + + As this command is much more cautious and selective, removing only cache files, + it will execute much slower than the simple ``find`` command described above. + Therefore, it may not be your best option to trim huge cache directories.
Signature · 1 line
2.34.1