Skip to contentSkip to conversation

State of bitbake world, files not tracked by sstate

DISCUSSION Martin Jansa 2 participants 2 messages Oct 22, 2014

lore
Share & tools
lore
#0 lore raw

These are the files which are installed into sysroot without sstate knowledge, so if you reuse the components which created them from sstate these files will be missing.

cat << EOF > sstate-sysroot-cruft-whitelist.txt [^/]*/home/builder [^/]*/usr/src/kernel/patches [^/]*/usr/src/kernel/scripts/.* [^/]*/usr/lib/gdk-pixbuf-2.0/.*/loaders.cache [^/]*/etc/sgml/sgml-docbook.cat [^/]*/usr/src/kernel/patches [^/]*/etc/sgml/sgml-docbook.cat [^/]*/usr/lib/python3.3/__pycache__ [^/]*/usr/lib/python3.3/[^/]*/__pycache__ [^/]*/usr/lib/python3.3/[^/]*/[^/]*/__pycache__ [^/]*/usr/share/dbus [^/]*/usr/share/dbus/dbus-bus-introspect.xml [^/]*/usr/share/dbus/session.conf [^/]*/usr/bin/crossscripts/guile-config [^/]*/usr/lib/python2.7/config/libpython2.7.so [^/]*/var
[^/]*/usr/bin/i586-oe-linux-g77 [^/]*/usr/bin/arm-oe-linux-gnueabi-g77 [^/]*/usr/lib/php/\.channels.* [^/]*/usr/lib/php/\.registry.* [^/]*/usr/lib/php/\.depdb.* [^/]*/usr/lib/php/\.filemap [^/]*/usr/lib/php/\.lock [^/]*/usr/lib/gdk-pixbuf-2.0/.*/loaders.cache [^/]*/usr/include/ruby-1.9.1/i386-linux [^/]*/usr/include/ruby-1.9.1/i386-linux/ruby [^/]*/usr/include/ruby-1.9.1/i386-linux/ruby/config.h [^/]*/usr/include/ruby-1.9.1/ruby/win32.h [^/]*/usr/lib/ruby/i386-linux [^/]*/usr/lib/ruby/i386-linux/fake.rb [^/]*/usr/lib/ruby/i386-linux/libruby.so.1.9.1 [^/]*/usr/lib/ruby/i386-linux/libruby-static.a [^/]*/usr/lib/ruby/i386-linux/rbconfig.rb [^/]*/usr/lib/qt4/plugins/webkit [^/]*/usr/lib/qt5/plugins/webkit
EOF

You can easily verify it in your build (after build without sstate-cache) with:
openembedded-core/scripts/sstate-sysroot-cruft.sh --tmpdir=tmp-glibc --whitelist=sstate-sysroot-cruft-whitelist.txt 2>&1

RFC for g77 fix was sent couple months ago (no reply):
http://lists.openembedded.org/pipermail/openembedded-core/2014-April/091480.html

Signature · 1 line
Martin 'JaMa' Jansa     jabber: [email protected]
#1 lore raw
Quoted 32 lines · Martin Jansa

These are the files which are installed into sysroot without sstate knowledge, so if you reuse the components which created them from sstate these files will be missing.

cat << EOF > sstate-sysroot-cruft-whitelist.txt [^/]*/home/builder [^/]*/usr/src/kernel/patches [^/]*/usr/src/kernel/scripts/.* [^/]*/usr/lib/gdk-pixbuf-2.0/.*/loaders.cache [^/]*/etc/sgml/sgml-docbook.cat [^/]*/usr/src/kernel/patches [^/]*/etc/sgml/sgml-docbook.cat [^/]*/usr/lib/python3.3/__pycache__ [^/]*/usr/lib/python3.3/[^/]*/__pycache__ [^/]*/usr/lib/python3.3/[^/]*/[^/]*/__pycache__ [^/]*/usr/share/dbus [^/]*/usr/share/dbus/dbus-bus-introspect.xml [^/]*/usr/share/dbus/session.conf [^/]*/usr/bin/crossscripts/guile-config [^/]*/usr/lib/python2.7/config/libpython2.7.so [^/]*/var
[^/]*/usr/bin/i586-oe-linux-g77 [^/]*/usr/bin/arm-oe-linux-gnueabi-g77 [^/]*/usr/lib/php/\.channels.* [^/]*/usr/lib/php/\.registry.* [^/]*/usr/lib/php/\.depdb.* [^/]*/usr/lib/php/\.filemap [^/]*/usr/lib/php/\.lock [^/]*/usr/lib/gdk-pixbuf-2.0/.*/loaders.cache [^/]*/usr/include/ruby-1.9.1/i386-linux [^/]*/usr/include/ruby-1.9.1/i386-linux/ruby [^/]*/usr/include/ruby-1.9.1/i386-linux/ruby/config.h

Quoted 8 lines

[^/]*/usr/include/ruby-1.9.1/ruby/win32.h
[^/]*/usr/lib/ruby/i386-linux
[^/]*/usr/lib/ruby/i386-linux/fake.rb
[^/]*/usr/lib/ruby/i386-linux/libruby.so.1.9.1
[^/]*/usr/lib/ruby/i386-linux/libruby-static.a
[^/]*/usr/lib/ruby/i386-linux/rbconfig.rb
[^/]*/usr/lib/qt4/plugins/webkit
[^/]*/usr/lib/qt5/plugins/webkit

I just want to be clear that this list is not 100% accurate. Some of these files are not tracked by sstate however they are generated by sstate 'postinsts' so they will be rebuilt upon a rebuild. Examples that jump out are:

[^/]*/usr/lib/gdk-pixbuf-2.0/.*/loaders.cache
[^/]*/usr/bin/crossscripts/guile-config

Also,

[^/]*/usr/lib/python3.3/__pycache__
[^/]*/usr/lib/python3.3/[^/]*/__pycache__
[^/]*/usr/lib/python3.3/[^/]*/[^/]*/__pycache__

are automatically generated/updated by python.

[^/]*/usr/src/kernel/scripts/.*

is generated by the modules class amongst other things. There was a recent different bug report about these files not being cleaned and triggering file conflict issues but that is a different problem.

I don't know much about the others and there are also some 'real' issues here.

Cheers,

Richard