Robert Yang : A script to clean obsolete sstate cache files

oe-commits Martin Jansa 2 participants 8 messages Mar 27, 2012 – Apr 1, 2012 1 message not in the archive

lore

Share & tools

lore

Martin Jansa

author

2012-03-27 11:15 UTC #1 lore raw

In reply to a message not in the archive

Quoted 11 lines · On Fri, Feb 24, 2012 at 12:01:37AM +0000, [email protected] wrote:

Module: openembedded-core.git
Branch: master
Commit: 7c99ef6d2173b14e1109a540ee5ae47b56d707e7
URL:    http://git.openembedded.org/?p=openembedded-core.git&a=commit;h=7c99ef6d2173b14e1109a540ee5ae47b56d707e7

Author: Robert Yang <[email protected]>
Date:   Wed Feb 22 20:38:30 2012 +0800

A script to clean obsolete sstate cache files

Review

Quoted 5 lines

(quoted diff) +5 -0

+# Use the "_" to substitute "-", e.g., x86-64 to x86_64
+all_archs=`grep -r DEFAULTTUNE $tunedir | \
+    sed -e 's/.*\"\(.*\)\"/\1/' -e 's/-/_/g' | sort -u`
+# Add the qemu archs
+all_archs="$all_archs qemuarm qemux86 qemumips qemuppc"

This doesn't list any thumb variants, current list is:

arm1136jfs arm920t arm926ejs arm9tdmi armv4 armv5 armv6 armv7 armv7a core2 cortexa8_neon cortexa9_neon cortexm1 cortexm3 cortexr4 ep9312 i586 iwmmxt mips mips32 powerpc powerpc64 ppc603e ppce300c2 ppce500 ppce500mc ppce500v2 ppce5500 strongarm x86 x86_64 xscale

so thumb versions (e.g. armv4t) are not found in

Review

Quoted 4 lines

(quoted diff) +4 -0

+for arch in $all_archs; do
+    ls | grep -q -w $arch
+    [ $? -eq 0 ] && ava_archs="$ava_archs $arch"
+done

Cheers,

Signature · 1 line

Martin 'JaMa' Jansa     jabber: [email protected]

Martin Jansa

author

2012-03-27 11:42 UTC #2 lore raw

Quoted 25 lines · Martin Jansa

Quoted 11 lines · On Fri, Feb 24, 2012 at 12:01:37AM +0000, [email protected] wrote:
Module: openembedded-core.git
Branch: master
Commit: 7c99ef6d2173b14e1109a540ee5ae47b56d707e7
URL:    http://git.openembedded.org/?p=openembedded-core.git&a=commit;h=7c99ef6d2173b14e1109a540ee5ae47b56d707e7
Author: Robert Yang <[email protected]>
Date:   Wed Feb 22 20:38:30 2012 +0800
A script to clean obsolete sstate cache files
Review
Quoted 5 lines
(quoted diff) +5 -0
+# Use the "_" to substitute "-", e.g., x86-64 to x86_64
+all_archs=`grep -r DEFAULTTUNE $tunedir | \
+    sed -e 's/.*\"$.*$\"/\1/' -e 's/-/_/g' | sort -u`
+# Add the qemu archs
+all_archs="$all_archs qemuarm qemux86 qemumips qemuppc"
This doesn't list any thumb variants, current list is:
arm1136jfs arm920t arm926ejs arm9tdmi armv4 armv5 armv6 armv7 armv7a core2 cortexa8_neon cortexa9_neon cortexm1 cortexm3 cortexr4 ep9312 i586 iwmmxt mips mips32 powerpc powerpc64 ppc603e ppce300c2 ppce500 ppce500mc ppce500v2 ppce5500 strongarm x86 x86_64 xscale

Quoted 8 lines

so thumb versions (e.g. armv4t) are not found in
Review
Quoted 4 lines
(quoted diff) +4 -0
+for arch in $all_archs; do
+    ls | grep -q -w $arch
+    [ $? -eq 0 ] && ava_archs="$ava_archs $arch"
+done
Cheers,

And after extending list of machines in all_archs with armv4t and my machines I got only one file in sstate-cache left :/

First run without armv4t found:
OE crespo@shr ~/shr-core $ du -hs sstate-cache/; openembedded-core/scripts/sstate-cache-management.sh --cache-dir=sstate-cache --remove-duplicated; du -hs sstate-cache/
14G sstate-cache/
Figuring out the archs in the sstate cache dir ...
The following archs have been found in the sstate cache dir:
armv7a x86_64
Removing the sstate-xxx_deploy-rpm.tgz ... (0 files)
Removing the sstate-xxx_deploy-ipk.tgz ... (58 files)
Removing the sstate-xxx_deploy-deb.tgz ... (0 files)
Removing the sstate-xxx_deploy.tgz ... (0 files)
Removing the sstate-xxx_package.tgz ... (62 files)
Removing the sstate-xxx_populate-lic.tgz ... (46 files)
Removing the sstate-xxx_populate-sysroot.tgz ... (102 files)
268 files have been removed
13G sstate-cache/

Second run with armv4t manually added
-all_archs="$all_archs qemuarm qemux86 qemumips qemuppc"
+all_archs="$all_archs qemuarm qemux86 qemumips qemuppc armv4t"
OE crespo@shr ~/openembedded-core $ du -hs ../shr-core/sstate-cache/; scripts/sstate-cache-management.sh --cache-dir=../shr-core/sstate-cache --remove-duplicated; du -hs ../shr-core/sstate-cache/
13G ../shr-core/sstate-cache/
/OE/openembedded-core/meta/conf/machine/include
Figuring out the archs in the sstate cache dir ...
The following archs have been found in the sstate cache dir:
armv4t armv7a x86_64
Removing the sstate-xxx_deploy-rpm.tgz ... (0 files)
Removing the sstate-xxx_deploy-ipk.tgz ... (460 files)
Removing the sstate-xxx_deploy-deb.tgz ... (0 files)
Removing the sstate-xxx_deploy.tgz ... (0 files)
Removing the sstate-xxx_package.tgz ... (454 files)
Removing the sstate-xxx_populate-lic.tgz ... (456 files)
Removing the sstate-xxx_populate-sysroot.tgz ... (468 files)
1838 files have been removed
6.2G ../shr-core/sstate-cache/

Third run with machines I'm building manually added
-all_archs="$all_archs qemuarm qemux86 qemumips qemuppc armv4t"
+all_archs="$all_archs qemuarm qemux86 qemumips qemuppc armv4t om_gta01 om_gta02 om_gta04 nokia900 palmpre palmpre2 crespo spitz"
OE crespo@shr ~/openembedded-core $ du -hs ../shr-core/sstate-cache/; scripts/sstate-cache-management.sh --cache-dir=../shr-core/sstate-cache --remove-duplicated; du -hs ../shr-core/sstate-cache/
6.2G ../shr-core/sstate-cache/
/OE/openembedded-core/meta/conf/machine/include
Figuring out the archs in the sstate cache dir ...
The following archs have been found in the sstate cache dir:
armv4t armv7a x86_64 nokia900 palmpre palmpre2 crespo
Removing the sstate-xxx_deploy-rpm.tgz ... (0 files)
Removing the sstate-xxx_deploy-ipk.tgz ... (7024 files)
Removing the sstate-xxx_deploy-deb.tgz ... (0 files)
Removing the sstate-xxx_deploy.tgz ... (0 files)
Removing the sstate-xxx_package.tgz ... (0 files)
Removing the sstate-xxx_populate-lic.tgz ... (0 files)
Removing the sstate-xxx_populate-sysroot.tgz ... (0 files)
7024 files have been removed
3.1M ../shr-core/sstate-cache/

OE crespo@shr ~/openembedded-core $ ls ../shr-core/sstate-cache/ sstate-shr-version-crespo-oe-linux-gnueabi-1.020120327-r0-crespo-2-fcb5bb2c0ac03d75285d77e7c30af869_populate-sysroot.tgz

Original count of sstate packages was something like:
Sum 3043
all-oe-linux 661
gnueabi 94
nativesdk 953 (i686 + x86_64)
x86_64-nativesdk 480
gnueabi 37
i686-nativesdk 473
gnueabi 36

x86_64-linux            309
nokia900                229
    gnueabi             222
crespo                  215
    gnueabi             211
palmpre                 475 (palmpre + palmpre2)
  palmpre               244
    gnueabi             236
  palmpre2              231
    gnueabi             223
om_gta04                201
    gnueabi             193

Cheers,

Signature · 1 line

Martin 'JaMa' Jansa     jabber: [email protected]

Robert Yang

2012-03-27 13:37 UTC #3 lore raw

Hi Martin,

Thanks for reporting this, and I'm very sorry for the inconvenience, please see my comment inline ...

Quoted 74 lines · Martin Jansa

Quoted 25 lines · Martin Jansa
Quoted 11 lines · On Fri, Feb 24, 2012 at 12:01:37AM +0000, [email protected] wrote:
Module: openembedded-core.git
Branch: master
Commit: 7c99ef6d2173b14e1109a540ee5ae47b56d707e7
URL:    http://git.openembedded.org/?p=openembedded-core.git&a=commit;h=7c99ef6d2173b14e1109a540ee5ae47b56d707e7
Author: Robert Yang<[email protected]>
Date:   Wed Feb 22 20:38:30 2012 +0800
A script to clean obsolete sstate cache files
Review
Quoted 5 lines
(quoted diff) +5 -0
+# Use the "_" to substitute "-", e.g., x86-64 to x86_64
+all_archs=`grep -r DEFAULTTUNE $tunedir | \
+    sed -e 's/.*\"$.*$\"/\1/' -e 's/-/_/g' | sort -u`
+# Add the qemu archs
+all_archs="$all_archs qemuarm qemux86 qemumips qemuppc"
This doesn't list any thumb variants, current list is:
arm1136jfs arm920t arm926ejs arm9tdmi armv4 armv5 armv6 armv7 armv7a core2 cortexa8_neon cortexa9_neon cortexm1 cortexm3 cortexr4 ep9312 i586 iwmmxt mips mips32 powerpc powerpc64 ppc603e ppce300c2 ppce500 ppce500mc ppce500v2 ppce5500 strongarm x86 x86_64 xscale
Quoted 8 lines
so thumb versions (e.g. armv4t) are not found in
Review
Quoted 4 lines
(quoted diff) +4 -0
+for arch in $all_archs; do
+    ls | grep -q -w $arch
+    [ $? -eq 0 ]&&  ava_archs="$ava_archs $arch"
+done
Cheers,
And after extending list of machines in all_archs with armv4t and my machines I got only one file in sstate-cache left :/
First run without armv4t found:
OE crespo@shr ~/shr-core $ du -hs sstate-cache/; openembedded-core/scripts/sstate-cache-management.sh --cache-dir=sstate-cache --remove-duplicated; du -hs sstate-cache/
14G sstate-cache/
Figuring out the archs in the sstate cache dir ...
The following archs have been found in the sstate cache dir:
armv7a x86_64
Removing the sstate-xxx_deploy-rpm.tgz ... (0 files)
Removing the sstate-xxx_deploy-ipk.tgz ... (58 files)
Removing the sstate-xxx_deploy-deb.tgz ... (0 files)
Removing the sstate-xxx_deploy.tgz ... (0 files)
Removing the sstate-xxx_package.tgz ... (62 files)
Removing the sstate-xxx_populate-lic.tgz ... (46 files)
Removing the sstate-xxx_populate-sysroot.tgz ... (102 files)
268 files have been removed
13G sstate-cache/
Second run with armv4t manually added
(quoted diff) +1 -1
-all_archs="$all_archs qemuarm qemux86 qemumips qemuppc"
+all_archs="$all_archs qemuarm qemux86 qemumips qemuppc armv4t"
OE crespo@shr ~/openembedded-core $ du -hs ../shr-core/sstate-cache/; scripts/sstate-cache-management.sh --cache-dir=../shr-core/sstate-cache --remove-duplicated; du -hs ../shr-core/sstate-cache/
13G ../shr-core/sstate-cache/
/OE/openembedded-core/meta/conf/machine/include
Figuring out the archs in the sstate cache dir ...
The following archs have been found in the sstate cache dir:
armv4t armv7a x86_64
Removing the sstate-xxx_deploy-rpm.tgz ... (0 files)
Removing the sstate-xxx_deploy-ipk.tgz ... (460 files)
Removing the sstate-xxx_deploy-deb.tgz ... (0 files)
Removing the sstate-xxx_deploy.tgz ... (0 files)
Removing the sstate-xxx_package.tgz ... (454 files)
Removing the sstate-xxx_populate-lic.tgz ... (456 files)
Removing the sstate-xxx_populate-sysroot.tgz ... (468 files)
1838 files have been removed
6.2G ../shr-core/sstate-cache/
Third run with machines I'm building manually added

Review

Quoted 2 lines

(quoted diff) +1 -1

-all_archs="$all_archs qemuarm qemux86 qemumips qemuppc armv4t"
+all_archs="$all_archs qemuarm qemux86 qemumips qemuppc armv4t om_gta01 om_gta02 om_gta04 nokia900 palmpre palmpre2 crespo spitz"

The script figures the archs according to the 'meta/conf/machine/include', this is incorrect since it only searches the oe-core layer, I will fix it.

And where can I get the "om_gta01 om_gta02 om_gta04 ..." archs? I mean which layer are they in? So that I can test them.

Quoted 7 lines

OE crespo@shr ~/openembedded-core $ du -hs ../shr-core/sstate-cache/; scripts/sstate-cache-management.sh --cache-dir=../shr-core/sstate-cache --remove-duplicated; du -hs ../shr-core/sstate-cache/
6.2G ../shr-core/sstate-cache/
/OE/openembedded-core/meta/conf/machine/include
Figuring out the archs in the sstate cache dir ...
The following archs have been found in the sstate cache dir:
armv4t armv7a x86_64 nokia900 palmpre palmpre2 crespo
Removing the sstate-xxx_deploy-rpm.tgz ... (0 files)

Quoted 8 lines

Removing the sstate-xxx_deploy-ipk.tgz ... (7024 files) Removing the sstate-xxx_deploy-deb.tgz ... (0 files) Removing the sstate-xxx_deploy.tgz ... (0 files) Removing the sstate-xxx_package.tgz ... (0 files) Removing the sstate-xxx_populate-lic.tgz ... (0 files) Removing the sstate-xxx_populate-sysroot.tgz ... (0 files) 7024 files have been removed
3.1M ../shr-core/sstate-cache/

This seems terrible, I don't know what's wrong currently, I will try to fix it tomorrow.

// Robert

Quoted 28 lines

OE crespo@shr ~/openembedded-core $ ls ../shr-core/sstate-cache/ sstate-shr-version-crespo-oe-linux-gnueabi-1.020120327-r0-crespo-2-fcb5bb2c0ac03d75285d77e7c30af869_populate-sysroot.tgz
Original count of sstate packages was something like:
Sum 3043
all-oe-linux 661
gnueabi 94
nativesdk 953 (i686 + x86_64)
x86_64-nativesdk 480
gnueabi 37
i686-nativesdk 473
gnueabi 36
x86_64-linux 309
nokia900 229
gnueabi 222
crespo 215
gnueabi 211
palmpre 475 (palmpre + palmpre2)
   palmpre               244
     gnueabi             236
   palmpre2              231
     gnueabi             223
om_gta04 201
gnueabi 193
Cheers,

Martin Jansa

author

2012-03-27 13:44 UTC #4 lore raw

Quoted 80 lines · Robert Yang

Hi Martin,
Thanks for reporting this, and I'm very sorry for the inconvenience, please see my comment inline ...
Quoted 73 lines · Martin Jansa
Quoted 25 lines · Martin Jansa
Quoted 11 lines · On Fri, Feb 24, 2012 at 12:01:37AM +0000, [email protected] wrote:
Module: openembedded-core.git
Branch: master
Commit: 7c99ef6d2173b14e1109a540ee5ae47b56d707e7
URL:    http://git.openembedded.org/?p=openembedded-core.git&a=commit;h=7c99ef6d2173b14e1109a540ee5ae47b56d707e7
Author: Robert Yang<[email protected]>
Date:   Wed Feb 22 20:38:30 2012 +0800
A script to clean obsolete sstate cache files
Review
Quoted 5 lines
(quoted diff) +5 -0
+# Use the "_" to substitute "-", e.g., x86-64 to x86_64
+all_archs=`grep -r DEFAULTTUNE $tunedir | \
+    sed -e 's/.*\"$.*$\"/\1/' -e 's/-/_/g' | sort -u`
+# Add the qemu archs
+all_archs="$all_archs qemuarm qemux86 qemumips qemuppc"
This doesn't list any thumb variants, current list is:
arm1136jfs arm920t arm926ejs arm9tdmi armv4 armv5 armv6 armv7 armv7a core2 cortexa8_neon cortexa9_neon cortexm1 cortexm3 cortexr4 ep9312 i586 iwmmxt mips mips32 powerpc powerpc64 ppc603e ppce300c2 ppce500 ppce500mc ppce500v2 ppce5500 strongarm x86 x86_64 xscale
Quoted 8 lines
so thumb versions (e.g. armv4t) are not found in
Review
Quoted 4 lines
(quoted diff) +4 -0
+for arch in $all_archs; do
+    ls | grep -q -w $arch
+    [ $? -eq 0 ]&&  ava_archs="$ava_archs $arch"
+done
Cheers,
And after extending list of machines in all_archs with armv4t and my machines I got only one file in sstate-cache left :/
First run without armv4t found:
OE crespo@shr ~/shr-core $ du -hs sstate-cache/; openembedded-core/scripts/sstate-cache-management.sh --cache-dir=sstate-cache --remove-duplicated; du -hs sstate-cache/
14G sstate-cache/
Figuring out the archs in the sstate cache dir ...
The following archs have been found in the sstate cache dir:
armv7a x86_64
Removing the sstate-xxx_deploy-rpm.tgz ... (0 files)
Removing the sstate-xxx_deploy-ipk.tgz ... (58 files)
Removing the sstate-xxx_deploy-deb.tgz ... (0 files)
Removing the sstate-xxx_deploy.tgz ... (0 files)
Removing the sstate-xxx_package.tgz ... (62 files)
Removing the sstate-xxx_populate-lic.tgz ... (46 files)
Removing the sstate-xxx_populate-sysroot.tgz ... (102 files)
268 files have been removed
13G sstate-cache/
Second run with armv4t manually added
(quoted diff) +1 -1
-all_archs="$all_archs qemuarm qemux86 qemumips qemuppc"
+all_archs="$all_archs qemuarm qemux86 qemumips qemuppc armv4t"
OE crespo@shr ~/openembedded-core $ du -hs ../shr-core/sstate-cache/; scripts/sstate-cache-management.sh --cache-dir=../shr-core/sstate-cache --remove-duplicated; du -hs ../shr-core/sstate-cache/
13G ../shr-core/sstate-cache/
/OE/openembedded-core/meta/conf/machine/include
Figuring out the archs in the sstate cache dir ...
The following archs have been found in the sstate cache dir:
armv4t armv7a x86_64
Removing the sstate-xxx_deploy-rpm.tgz ... (0 files)
Removing the sstate-xxx_deploy-ipk.tgz ... (460 files)
Removing the sstate-xxx_deploy-deb.tgz ... (0 files)
Removing the sstate-xxx_deploy.tgz ... (0 files)
Removing the sstate-xxx_package.tgz ... (454 files)
Removing the sstate-xxx_populate-lic.tgz ... (456 files)
Removing the sstate-xxx_populate-sysroot.tgz ... (468 files)
1838 files have been removed
6.2G ../shr-core/sstate-cache/

Quoted 6 lines

Review
Quoted 3 lines
Third run with machines I'm building manually added
(quoted diff) +1 -1
-all_archs="$all_archs qemuarm qemux86 qemumips qemuppc armv4t"
+all_archs="$all_archs qemuarm qemux86 qemumips qemuppc armv4t om_gta01 om_gta02 om_gta04 nokia900 palmpre palmpre2 crespo spitz"
The script figures the archs according to the 'meta/conf/machine/include', this is incorrect since it only searches the oe-core layer, I will fix it.

The first issue is that armv4t and other thumb variants are also in oe-core layer.

Quoted 2 lines

And where can I get the "om_gta01 om_gta02 om_gta04 ..." archs? I mean which layer are they in? So that I can test them.

Those are machines from meta-smartphone sublayers
http://git.shr-project.org/git/?p=meta-smartphone.git;a=summary

Maybe add some param to add extra archs/machines or just list all used xxx variants from 'sstate-xxx_'.

Cheers,

Signature · 1 line

Martin 'JaMa' Jansa     jabber: [email protected]

Robert Yang

2012-03-27 13:58 UTC #5 lore raw

Quoted 91 lines · Martin Jansa

Quoted 80 lines · Robert Yang
Hi Martin,
Thanks for reporting this, and I'm very sorry for the inconvenience, please see my comment inline ...
Quoted 73 lines · Martin Jansa
Quoted 25 lines · Martin Jansa
Quoted 11 lines · On Fri, Feb 24, 2012 at 12:01:37AM +0000, [email protected] wrote:
Module: openembedded-core.git
Branch: master
Commit: 7c99ef6d2173b14e1109a540ee5ae47b56d707e7
URL:    http://git.openembedded.org/?p=openembedded-core.git&a=commit;h=7c99ef6d2173b14e1109a540ee5ae47b56d707e7
Author: Robert Yang<[email protected]>
Date:   Wed Feb 22 20:38:30 2012 +0800
A script to clean obsolete sstate cache files
Review
Quoted 5 lines
(quoted diff) +5 -0
+# Use the "_" to substitute "-", e.g., x86-64 to x86_64
+all_archs=`grep -r DEFAULTTUNE $tunedir | \
+    sed -e 's/.*\"$.*$\"/\1/' -e 's/-/_/g' | sort -u`
+# Add the qemu archs
+all_archs="$all_archs qemuarm qemux86 qemumips qemuppc"
This doesn't list any thumb variants, current list is:
arm1136jfs arm920t arm926ejs arm9tdmi armv4 armv5 armv6 armv7 armv7a core2 cortexa8_neon cortexa9_neon cortexm1 cortexm3 cortexr4 ep9312 i586 iwmmxt mips mips32 powerpc powerpc64 ppc603e ppce300c2 ppce500 ppce500mc ppce500v2 ppce5500 strongarm x86 x86_64 xscale
Quoted 8 lines
so thumb versions (e.g. armv4t) are not found in
Review
Quoted 4 lines
(quoted diff) +4 -0
+for arch in $all_archs; do
+    ls | grep -q -w $arch
+    [ $? -eq 0 ]&&   ava_archs="$ava_archs $arch"
+done
Cheers,
And after extending list of machines in all_archs with armv4t and my machines I got only one file in sstate-cache left :/
First run without armv4t found:
OE crespo@shr ~/shr-core $ du -hs sstate-cache/; openembedded-core/scripts/sstate-cache-management.sh --cache-dir=sstate-cache --remove-duplicated; du -hs sstate-cache/
14G sstate-cache/
Figuring out the archs in the sstate cache dir ...
The following archs have been found in the sstate cache dir:
armv7a x86_64
Removing the sstate-xxx_deploy-rpm.tgz ... (0 files)
Removing the sstate-xxx_deploy-ipk.tgz ... (58 files)
Removing the sstate-xxx_deploy-deb.tgz ... (0 files)
Removing the sstate-xxx_deploy.tgz ... (0 files)
Removing the sstate-xxx_package.tgz ... (62 files)
Removing the sstate-xxx_populate-lic.tgz ... (46 files)
Removing the sstate-xxx_populate-sysroot.tgz ... (102 files)
268 files have been removed
13G sstate-cache/
Second run with armv4t manually added
(quoted diff) +1 -1
-all_archs="$all_archs qemuarm qemux86 qemumips qemuppc"
+all_archs="$all_archs qemuarm qemux86 qemumips qemuppc armv4t"
OE crespo@shr ~/openembedded-core $ du -hs ../shr-core/sstate-cache/; scripts/sstate-cache-management.sh --cache-dir=../shr-core/sstate-cache --remove-duplicated; du -hs ../shr-core/sstate-cache/
13G ../shr-core/sstate-cache/
/OE/openembedded-core/meta/conf/machine/include
Figuring out the archs in the sstate cache dir ...
The following archs have been found in the sstate cache dir:
armv4t armv7a x86_64
Removing the sstate-xxx_deploy-rpm.tgz ... (0 files)
Removing the sstate-xxx_deploy-ipk.tgz ... (460 files)
Removing the sstate-xxx_deploy-deb.tgz ... (0 files)
Removing the sstate-xxx_deploy.tgz ... (0 files)
Removing the sstate-xxx_package.tgz ... (454 files)
Removing the sstate-xxx_populate-lic.tgz ... (456 files)
Removing the sstate-xxx_populate-sysroot.tgz ... (468 files)
1838 files have been removed
6.2G ../shr-core/sstate-cache/
Quoted 6 lines
Review
Quoted 3 lines
Third run with machines I'm building manually added
(quoted diff) +1 -1
-all_archs="$all_archs qemuarm qemux86 qemumips qemuppc armv4t"
+all_archs="$all_archs qemuarm qemux86 qemumips qemuppc armv4t om_gta01 om_gta02 om_gta04 nokia900 palmpre palmpre2 crespo spitz"
The script figures the archs according to the 'meta/conf/machine/include', this is incorrect since it only searches the oe-core layer, I will fix it.
The first issue is that armv4t and other thumb variants are also in oe-core layer.

Quoted 9 lines

Quoted 2 lines
And where can I get the "om_gta01 om_gta02 om_gta04 ..." archs? I mean which layer are they in? So that I can test them.
Those are machines from meta-smartphone sublayers
http://git.shr-project.org/git/?p=meta-smartphone.git;a=summary
Maybe add some param to add extra archs/machines or just list all used xxx variants from 'sstate-xxx_'.

Thanks, add some param is good choice. Figure out the used variants from sstate-xxx is the best way, but it seems not easy since it uses the "-" as the separator, and I had counted the number of field, it is not always the same.

// Robert

Quoted 2 lines

Cheers,

Robert Yang

2012-03-27 13:58 UTC #8 lore raw

In reply to Martin Jansa

Quoted 91 lines · Martin Jansa

Quoted 80 lines · Robert Yang
Hi Martin,
Thanks for reporting this, and I'm very sorry for the inconvenience, please see my comment inline ...
Quoted 73 lines · Martin Jansa
Quoted 25 lines · Martin Jansa
Quoted 11 lines · On Fri, Feb 24, 2012 at 12:01:37AM +0000, [email protected] wrote:
Module: openembedded-core.git
Branch: master
Commit: 7c99ef6d2173b14e1109a540ee5ae47b56d707e7
URL:    http://git.openembedded.org/?p=openembedded-core.git&a=commit;h=7c99ef6d2173b14e1109a540ee5ae47b56d707e7
Author: Robert Yang<[email protected]>
Date:   Wed Feb 22 20:38:30 2012 +0800
A script to clean obsolete sstate cache files
Review
Quoted 5 lines
(quoted diff) +5 -0
+# Use the "_" to substitute "-", e.g., x86-64 to x86_64
+all_archs=`grep -r DEFAULTTUNE $tunedir | \
+    sed -e 's/.*\"$.*$\"/\1/' -e 's/-/_/g' | sort -u`
+# Add the qemu archs
+all_archs="$all_archs qemuarm qemux86 qemumips qemuppc"
This doesn't list any thumb variants, current list is:
arm1136jfs arm920t arm926ejs arm9tdmi armv4 armv5 armv6 armv7 armv7a core2 cortexa8_neon cortexa9_neon cortexm1 cortexm3 cortexr4 ep9312 i586 iwmmxt mips mips32 powerpc powerpc64 ppc603e ppce300c2 ppce500 ppce500mc ppce500v2 ppce5500 strongarm x86 x86_64 xscale
Quoted 8 lines
so thumb versions (e.g. armv4t) are not found in
Review
Quoted 4 lines
(quoted diff) +4 -0
+for arch in $all_archs; do
+    ls | grep -q -w $arch
+    [ $? -eq 0 ]&&   ava_archs="$ava_archs $arch"
+done
Cheers,
And after extending list of machines in all_archs with armv4t and my machines I got only one file in sstate-cache left :/
First run without armv4t found:
OE crespo@shr ~/shr-core $ du -hs sstate-cache/; openembedded-core/scripts/sstate-cache-management.sh --cache-dir=sstate-cache --remove-duplicated; du -hs sstate-cache/
14G sstate-cache/
Figuring out the archs in the sstate cache dir ...
The following archs have been found in the sstate cache dir:
armv7a x86_64
Removing the sstate-xxx_deploy-rpm.tgz ... (0 files)
Removing the sstate-xxx_deploy-ipk.tgz ... (58 files)
Removing the sstate-xxx_deploy-deb.tgz ... (0 files)
Removing the sstate-xxx_deploy.tgz ... (0 files)
Removing the sstate-xxx_package.tgz ... (62 files)
Removing the sstate-xxx_populate-lic.tgz ... (46 files)
Removing the sstate-xxx_populate-sysroot.tgz ... (102 files)
268 files have been removed
13G sstate-cache/
Second run with armv4t manually added
(quoted diff) +1 -1
-all_archs="$all_archs qemuarm qemux86 qemumips qemuppc"
+all_archs="$all_archs qemuarm qemux86 qemumips qemuppc armv4t"
OE crespo@shr ~/openembedded-core $ du -hs ../shr-core/sstate-cache/; scripts/sstate-cache-management.sh --cache-dir=../shr-core/sstate-cache --remove-duplicated; du -hs ../shr-core/sstate-cache/
13G ../shr-core/sstate-cache/
/OE/openembedded-core/meta/conf/machine/include
Figuring out the archs in the sstate cache dir ...
The following archs have been found in the sstate cache dir:
armv4t armv7a x86_64
Removing the sstate-xxx_deploy-rpm.tgz ... (0 files)
Removing the sstate-xxx_deploy-ipk.tgz ... (460 files)
Removing the sstate-xxx_deploy-deb.tgz ... (0 files)
Removing the sstate-xxx_deploy.tgz ... (0 files)
Removing the sstate-xxx_package.tgz ... (454 files)
Removing the sstate-xxx_populate-lic.tgz ... (456 files)
Removing the sstate-xxx_populate-sysroot.tgz ... (468 files)
1838 files have been removed
6.2G ../shr-core/sstate-cache/
Quoted 6 lines
Review
Quoted 3 lines
Third run with machines I'm building manually added
(quoted diff) +1 -1
-all_archs="$all_archs qemuarm qemux86 qemumips qemuppc armv4t"
+all_archs="$all_archs qemuarm qemux86 qemumips qemuppc armv4t om_gta01 om_gta02 om_gta04 nokia900 palmpre palmpre2 crespo spitz"
The script figures the archs according to the 'meta/conf/machine/include', this is incorrect since it only searches the oe-core layer, I will fix it.
The first issue is that armv4t and other thumb variants are also in oe-core layer.

Quoted 9 lines

Quoted 2 lines
And where can I get the "om_gta01 om_gta02 om_gta04 ..." archs? I mean which layer are they in? So that I can test them.
Those are machines from meta-smartphone sublayers
http://git.shr-project.org/git/?p=meta-smartphone.git;a=summary
Maybe add some param to add extra archs/machines or just list all used xxx variants from 'sstate-xxx_'.

Thanks, add some param is a good choice. Figure out the used variants from sstate-xxx is the best way, but it seems not easy since it uses the "-" as the separator, and I had counted the number of field, it is not always the same.

// Robert

Quoted 2 lines

Cheers,

Martin Jansa

author

2012-03-27 15:28 UTC #6 lore raw

In reply to Robert Yang

Quoted 100 lines · Robert Yang

Quoted 97 lines · Martin Jansa
Quoted 80 lines · Robert Yang
Hi Martin,
Thanks for reporting this, and I'm very sorry for the inconvenience, please see my comment inline ...
Quoted 73 lines · Martin Jansa
Quoted 25 lines · Martin Jansa
Quoted 11 lines · On Fri, Feb 24, 2012 at 12:01:37AM +0000, [email protected] wrote:
Module: openembedded-core.git
Branch: master
Commit: 7c99ef6d2173b14e1109a540ee5ae47b56d707e7
URL:    http://git.openembedded.org/?p=openembedded-core.git&a=commit;h=7c99ef6d2173b14e1109a540ee5ae47b56d707e7
Author: Robert Yang<[email protected]>
Date:   Wed Feb 22 20:38:30 2012 +0800
A script to clean obsolete sstate cache files
Review
Quoted 5 lines
(quoted diff) +5 -0
+# Use the "_" to substitute "-", e.g., x86-64 to x86_64
+all_archs=`grep -r DEFAULTTUNE $tunedir | \
+    sed -e 's/.*\"$.*$\"/\1/' -e 's/-/_/g' | sort -u`
+# Add the qemu archs
+all_archs="$all_archs qemuarm qemux86 qemumips qemuppc"
This doesn't list any thumb variants, current list is:
arm1136jfs arm920t arm926ejs arm9tdmi armv4 armv5 armv6 armv7 armv7a core2 cortexa8_neon cortexa9_neon cortexm1 cortexm3 cortexr4 ep9312 i586 iwmmxt mips mips32 powerpc powerpc64 ppc603e ppce300c2 ppce500 ppce500mc ppce500v2 ppce5500 strongarm x86 x86_64 xscale
Quoted 8 lines
so thumb versions (e.g. armv4t) are not found in
Review
Quoted 4 lines
(quoted diff) +4 -0
+for arch in $all_archs; do
+    ls | grep -q -w $arch
+    [ $? -eq 0 ]&&   ava_archs="$ava_archs $arch"
+done
Cheers,
And after extending list of machines in all_archs with armv4t and my machines I got only one file in sstate-cache left :/
First run without armv4t found:
OE crespo@shr ~/shr-core $ du -hs sstate-cache/; openembedded-core/scripts/sstate-cache-management.sh --cache-dir=sstate-cache --remove-duplicated; du -hs sstate-cache/
14G sstate-cache/
Figuring out the archs in the sstate cache dir ...
The following archs have been found in the sstate cache dir:
armv7a x86_64
Removing the sstate-xxx_deploy-rpm.tgz ... (0 files)
Removing the sstate-xxx_deploy-ipk.tgz ... (58 files)
Removing the sstate-xxx_deploy-deb.tgz ... (0 files)
Removing the sstate-xxx_deploy.tgz ... (0 files)
Removing the sstate-xxx_package.tgz ... (62 files)
Removing the sstate-xxx_populate-lic.tgz ... (46 files)
Removing the sstate-xxx_populate-sysroot.tgz ... (102 files)
268 files have been removed
13G sstate-cache/
Second run with armv4t manually added
(quoted diff) +1 -1
-all_archs="$all_archs qemuarm qemux86 qemumips qemuppc"
+all_archs="$all_archs qemuarm qemux86 qemumips qemuppc armv4t"
OE crespo@shr ~/openembedded-core $ du -hs ../shr-core/sstate-cache/; scripts/sstate-cache-management.sh --cache-dir=../shr-core/sstate-cache --remove-duplicated; du -hs ../shr-core/sstate-cache/
13G ../shr-core/sstate-cache/
/OE/openembedded-core/meta/conf/machine/include
Figuring out the archs in the sstate cache dir ...
The following archs have been found in the sstate cache dir:
armv4t armv7a x86_64
Removing the sstate-xxx_deploy-rpm.tgz ... (0 files)
Removing the sstate-xxx_deploy-ipk.tgz ... (460 files)
Removing the sstate-xxx_deploy-deb.tgz ... (0 files)
Removing the sstate-xxx_deploy.tgz ... (0 files)
Removing the sstate-xxx_package.tgz ... (454 files)
Removing the sstate-xxx_populate-lic.tgz ... (456 files)
Removing the sstate-xxx_populate-sysroot.tgz ... (468 files)
1838 files have been removed
6.2G ../shr-core/sstate-cache/
Quoted 6 lines
Review
Quoted 3 lines
Third run with machines I'm building manually added
(quoted diff) +1 -1
-all_archs="$all_archs qemuarm qemux86 qemumips qemuppc armv4t"
+all_archs="$all_archs qemuarm qemux86 qemumips qemuppc armv4t om_gta01 om_gta02 om_gta04 nokia900 palmpre palmpre2 crespo spitz"
The script figures the archs according to the 'meta/conf/machine/include', this is incorrect since it only searches the oe-core layer, I will fix it.
The first issue is that armv4t and other thumb variants are also in oe-core layer.
Quoted 2 lines
And where can I get the "om_gta01 om_gta02 om_gta04 ..." archs? I mean which layer are they in? So that I can test them.
Those are machines from meta-smartphone sublayers
http://git.shr-project.org/git/?p=meta-smartphone.git;a=summary

Quoted 7 lines

Quoted 3 lines
Maybe add some param to add extra archs/machines or just list all used xxx variants from 'sstate-xxx_'.
Thanks, add some param is good choice. Figure out the used variants from sstate-xxx is the best way, but it seems not easy since it uses the "-" as the separator, and I had counted the number of field, it is not always the same.

I see, maybe add also param to specify stamps dir, which can be used to find all arch ever built if it's the same host which populated sstate-cache dir.

That would include also all sdk and native variants etc.

Cheers,

Signature · 1 line

Martin 'JaMa' Jansa     jabber: [email protected]

Robert Yang

2012-04-01 02:50 UTC #7 lore raw

Hi Martin,

The fixes have been merged into oe-core, and it changed a lot.

// Robert

Quoted 117 lines · Martin Jansa

Quoted 100 lines · Robert Yang
Quoted 97 lines · Martin Jansa
Quoted 80 lines · Robert Yang
Hi Martin,
Thanks for reporting this, and I'm very sorry for the inconvenience, please see my comment inline ...
Quoted 73 lines · Martin Jansa
Quoted 25 lines · Martin Jansa
Quoted 11 lines · On Fri, Feb 24, 2012 at 12:01:37AM +0000, [email protected] wrote:
Module: openembedded-core.git
Branch: master
Commit: 7c99ef6d2173b14e1109a540ee5ae47b56d707e7
URL:    http://git.openembedded.org/?p=openembedded-core.git&a=commit;h=7c99ef6d2173b14e1109a540ee5ae47b56d707e7
Author: Robert Yang<[email protected]>
Date:   Wed Feb 22 20:38:30 2012 +0800
A script to clean obsolete sstate cache files
Review
Quoted 5 lines
(quoted diff) +5 -0
+# Use the "_" to substitute "-", e.g., x86-64 to x86_64
+all_archs=`grep -r DEFAULTTUNE $tunedir | \
+    sed -e 's/.*\"$.*$\"/\1/' -e 's/-/_/g' | sort -u`
+# Add the qemu archs
+all_archs="$all_archs qemuarm qemux86 qemumips qemuppc"
This doesn't list any thumb variants, current list is:
arm1136jfs arm920t arm926ejs arm9tdmi armv4 armv5 armv6 armv7 armv7a core2 cortexa8_neon cortexa9_neon cortexm1 cortexm3 cortexr4 ep9312 i586 iwmmxt mips mips32 powerpc powerpc64 ppc603e ppce300c2 ppce500 ppce500mc ppce500v2 ppce5500 strongarm x86 x86_64 xscale
Quoted 8 lines
so thumb versions (e.g. armv4t) are not found in
Review
Quoted 4 lines
(quoted diff) +4 -0
+for arch in $all_archs; do
+    ls | grep -q -w $arch
+    [ $? -eq 0 ]&&    ava_archs="$ava_archs $arch"
+done
Cheers,
And after extending list of machines in all_archs with armv4t and my machines I got only one file in sstate-cache left :/
First run without armv4t found:
OE crespo@shr ~/shr-core $ du -hs sstate-cache/; openembedded-core/scripts/sstate-cache-management.sh --cache-dir=sstate-cache --remove-duplicated; du -hs sstate-cache/
14G sstate-cache/
Figuring out the archs in the sstate cache dir ...
The following archs have been found in the sstate cache dir:
armv7a x86_64
Removing the sstate-xxx_deploy-rpm.tgz ... (0 files)
Removing the sstate-xxx_deploy-ipk.tgz ... (58 files)
Removing the sstate-xxx_deploy-deb.tgz ... (0 files)
Removing the sstate-xxx_deploy.tgz ... (0 files)
Removing the sstate-xxx_package.tgz ... (62 files)
Removing the sstate-xxx_populate-lic.tgz ... (46 files)
Removing the sstate-xxx_populate-sysroot.tgz ... (102 files)
268 files have been removed
13G sstate-cache/
Second run with armv4t manually added
(quoted diff) +1 -1
-all_archs="$all_archs qemuarm qemux86 qemumips qemuppc"
+all_archs="$all_archs qemuarm qemux86 qemumips qemuppc armv4t"
OE crespo@shr ~/openembedded-core $ du -hs ../shr-core/sstate-cache/; scripts/sstate-cache-management.sh --cache-dir=../shr-core/sstate-cache --remove-duplicated; du -hs ../shr-core/sstate-cache/
13G ../shr-core/sstate-cache/
/OE/openembedded-core/meta/conf/machine/include
Figuring out the archs in the sstate cache dir ...
The following archs have been found in the sstate cache dir:
armv4t armv7a x86_64
Removing the sstate-xxx_deploy-rpm.tgz ... (0 files)
Removing the sstate-xxx_deploy-ipk.tgz ... (460 files)
Removing the sstate-xxx_deploy-deb.tgz ... (0 files)
Removing the sstate-xxx_deploy.tgz ... (0 files)
Removing the sstate-xxx_package.tgz ... (454 files)
Removing the sstate-xxx_populate-lic.tgz ... (456 files)
Removing the sstate-xxx_populate-sysroot.tgz ... (468 files)
1838 files have been removed
6.2G ../shr-core/sstate-cache/
Quoted 6 lines
Review
Quoted 3 lines
Third run with machines I'm building manually added
(quoted diff) +1 -1
-all_archs="$all_archs qemuarm qemux86 qemumips qemuppc armv4t"
+all_archs="$all_archs qemuarm qemux86 qemumips qemuppc armv4t om_gta01 om_gta02 om_gta04 nokia900 palmpre palmpre2 crespo spitz"
The script figures the archs according to the 'meta/conf/machine/include', this is incorrect since it only searches the oe-core layer, I will fix it.
The first issue is that armv4t and other thumb variants are also in oe-core layer.
Quoted 2 lines
And where can I get the "om_gta01 om_gta02 om_gta04 ..." archs? I mean which layer are they in? So that I can test them.
Those are machines from meta-smartphone sublayers
http://git.shr-project.org/git/?p=meta-smartphone.git;a=summary
Quoted 7 lines
Quoted 3 lines
Maybe add some param to add extra archs/machines or just list all used xxx variants from 'sstate-xxx_'.
Thanks, add some param is good choice. Figure out the used variants from sstate-xxx is the best way, but it seems not easy since it uses the "-" as the separator, and I had counted the number of field, it is not always the same.
I see, maybe add also param to specify stamps dir, which can be used to find all arch ever built if it's the same host which populated sstate-cache dir.
That would include also all sdk and native variants etc.
Cheers,