dev-manual: disk-space: mention faster "find" command to trim sstate cache
PATCH 1/1 dev-manual: disk-space: mention faster "find" command to trim sstate cache
From: Michael Opdenacker <[email protected]>
[YOCTO #15182]
--- documentation/dev-manual/disk-space.rst | 24 +++++++++++++++--------- 1 file changed, 15 insertions(+), 9 deletions(-)
documentation/dev-manual/disk-space.rst +15 -9
@@ -27,19 +27,25 @@ Purging Duplicate Shared State Cache Files ========================================== After multiple build iterations, the Shared State (sstate) cache can contain -duplicate cache files for a given package, while only the most recent one -is likely to be reusable. The following command purges all but the -newest sstate cache file for each package:: +duplicate cache files for a given package, consuming a substantial amount of +disk space. However, only the most recent cache files are likeky to be reusable. - sstate-cache-management.sh --remove-duplicated --cache-dir=build/sstate-cache +The following command is a quick way to purge all the cache files which +are older than a specified number of days:: -This command will ask you to confirm the deletions it identifies. + find build/sstate-cache -type f -atime +$DAYS -delete -.. note:: +OpenEmbedded-Core also offers a command which can be used to look for +cache files only for enabled architectures, and purge all but the newest +ones on each architecture:: - The duplicated sstate cache files of one package must have the same - architecture, which means that sstate cache files with multiple - architectures are not considered as duplicate. + sstate-cache-management.sh --remove-duplicated --cache-dir=build/sstate-cache +This command will ask you to confirm the deletions it identifies. Run ``sstate-cache-management.sh`` for more details about this script. +.. note:: + + As this command is much more cautious and selective, removing only cache files, + it will execute much slower than the simple ``find`` command described above. + Therefore, it may not be your best option to trim huge cache directories.
Signature · 1 line
2.34.1
Quoted 35 lines · Michael Opdenacker via lists.yoctoproject.org
From: Michael Opdenacker <[email protected]>
[YOCTO #15182]
Signed-off-by Michael Opdenacker <[email protected]>Reported-by Yoann CONGAL <[email protected]>Reported-by Randy MacLeod <[email protected]>Reported-by Josef Holzmayr <[email protected]>--- documentation/dev-manual/disk-space.rst | 24 +++++++++++++++--------- 1 file changed, 15 insertions(+), 9 deletions(-)documentation/dev-manual/disk-space.rst +6
-6@@ -27,19 +27,25 @@ Purging Duplicate Shared State Cache Files ========================================== After multiple build iterations, the Shared State (sstate) cache can contain -duplicate cache files for a given package, while only the most recent one -is likely to be reusable. The following command purges all but the -newest sstate cache file for each package:: +duplicate cache files for a given package, consuming a substantial amount of +disk space. However, only the most recent cache files are likeky to be reusable. - sstate-cache-management.sh --remove-duplicated --cache-dir=build/sstate-cache +The following command is a quick way to purge all the cache files which +are older than a specified number of days:: -This command will ask you to confirm the deletions it identifies. + find build/sstate-cache -type f -atime +$DAYS -delete -.. note:: +OpenEmbedded-Core also offers a command which can be used to look for
documentation/dev-manual/disk-space.rst Purging Duplicate Shared State Cache Files · line 45
Quoted 10 lines
documentation/dev-manual/disk-space.rst +4
-3+cache files only for enabled architectures, and purge all but the newest +ones on each architecture:: - The duplicated sstate cache files of one package must have the same - architecture, which means that sstate cache files with multiple - architectures are not considered as duplicate. + sstate-cache-management.sh --remove-duplicated --cache-dir=build/sstate-cache +This command will ask you to confirm the deletions it identifies. Run ``sstate-cache-management.sh`` for more details about this script.
As long as bitbake builds have writable access to the sstate cache, files are touched as they are accessed so it is easy to know which ones are being used and which ones are not. As such you can use mtime in the above example even for a partition mounted as "noatime" for performance reasons.
documentation/dev-manual/disk-space.rst Purging Duplicate Shared State Cache Files · line 51
Quoted 6 lines
documentation/dev-manual/disk-space.rst +5
-0+.. note:: + + As this command is much more cautious and selective, removing only cache files, + it will execute much slower than the simple ``find`` command described above. + Therefore, it may not be your best option to trim huge cache directories.
It doesn't remove only cache files, it removes files that it considers to be unreachable by exploring a set of build configurations. It requires full build environment to be available and doesn't work well covering multiple releases. It also doesn't work in a limited environment like a BSD based NAS where the above find command would still likely work easily.
Can we get rid of sstate-cache-management.sh? :)
Cheers,
Richard
Hi Richard,
Quoted 45 lines · Richard Purdie
Quoted 43 lines · Michael Opdenacker via lists.yoctoproject.org
From: Michael Opdenacker <[email protected]>
[YOCTO #15182]
Signed-off-by Michael Opdenacker <[email protected]>Reported-by Yoann CONGAL <[email protected]>Reported-by Randy MacLeod <[email protected]>Reported-by Josef Holzmayr <[email protected]>--- documentation/dev-manual/disk-space.rst | 24 +++++++++++++++--------- 1 file changed, 15 insertions(+), 9 deletions(-)documentation/dev-manual/disk-space.rst +9
-9@@ -27,19 +27,25 @@ Purging Duplicate Shared State Cache Files ========================================== After multiple build iterations, the Shared State (sstate) cache can contain -duplicate cache files for a given package, while only the most recent one -is likely to be reusable. The following command purges all but the -newest sstate cache file for each package:: +duplicate cache files for a given package, consuming a substantial amount of +disk space. However, only the most recent cache files are likeky to be reusable. - sstate-cache-management.sh --remove-duplicated --cache-dir=build/sstate-cache +The following command is a quick way to purge all the cache files which +are older than a specified number of days:: -This command will ask you to confirm the deletions it identifies. + find build/sstate-cache -type f -atime +$DAYS -delete -.. note:: +OpenEmbedded-Core also offers a command which can be used to look for +cache files only for enabled architectures, and purge all but the newest +ones on each architecture:: - The duplicated sstate cache files of one package must have the same - architecture, which means that sstate cache files with multiple - architectures are not considered as duplicate. + sstate-cache-management.sh --remove-duplicated --cache-dir=build/sstate-cache
Quoted 7 lines
documentation/dev-manual/disk-space.rst Purging Duplicate Shared State Cache Files · line 45
Quoted 2 lines
documentation/dev-manual/disk-space.rst +1
-0+This command will ask you to confirm the deletions it identifies. Run ``sstate-cache-management.sh`` for more details about this script.As long as bitbake builds have writable access to the sstate cache, files are touched as they are accessed so it is easy to know which ones are being used and which ones are not. As such you can use mtime in the above example even for a partition mounted as "noatime" for performance reasons.
Good idea, this way the solution is a little broader as "noatime" is frequently used.
Quoted 7 lines
documentation/dev-manual/disk-space.rst Purging Duplicate Shared State Cache Files · line 51
Quoted 7 lines
documentation/dev-manual/disk-space.rst +5
-0+.. note:: + + As this command is much more cautious and selective, removing only cache files, + it will execute much slower than the simple ``find`` command described above. + Therefore, it may not be your best option to trim huge cache directories.
Quoted 8 lines
It doesn't remove only cache files, it removes files that it considers to be unreachable by exploring a set of build configurations. It requires full build environment to be available and doesn't work well covering multiple releases. It also doesn't work in a limited environment like a BSD based NAS where the above find command would still likely work easily.
Can we get rid of
sstate-cache-management.sh? :)
Thanks for these useful details about this script!
Are you suggesting we could at least remove sstate-cache-management.sh from the docs? Thanks again,
Michael.
Signature · 3 lines
Michael Opdenacker, Bootlin Embedded Linux and Kernel engineering https://bootlin.com