Skip to contentSkip to conversation

bitbake dependency cache problem

DISCUSSION Steffen Sledz 2 participants 4 messages Mar 1, 2012 – Mar 5, 2012

lore
Share & tools
lore
#0 lore raw

I'm working with oe-classic and BitBake Build Tool Core version 1.12.0, bitbake version 1.12.0.

Because of some special development requirements we like to generate the Package Version from the SVN Revision (not the Last Changed Rev!) of the bitbake recipe in the local workspace.

Therefor the recipe contains this:

---------------->snip<-----------------
PR = "r8"
PV = "svnr${@svn_revision(d)}"

inherit svn-helper ---------------->snip<-----------------

The svn-helper.bbclass contains a little helper function to determine the needed value:

---------------->snip<-----------------
def svn_revision(d):

     import subprocess
     bbpath = os.path.dirname(bb.data.getVar('FILE',d,1))
     return subprocess.check_output(["svn", "info", bbpath]).partition("Revision: ")[2].splitlines()[0]

---------------->snip<-----------------

After this changes i make a first build and everything works fine. Assuming the SVN Revsion is 42 a package called foo-svnr42-r8.ipk is generated.

Now i make an "svn update" inside the workspace and the SVN Revision increases to 66.

A call of "bitbake foo" now results in an "Tasks Summary: Attempted 1182 tasks of which 1182 didn't need to be rerun and 0 failed." an no new ipg is generated. :(

The log generated by "bitbake -DDDDDvvvvv foo" contains

---------------->snip<-----------------
DEBUG: providers for foo are: ['foo']
NOTE: checking PREFERRED_PROVIDER_foo
NOTE: checking PREFERRED_PROVIDER_foo-svnr42
NOTE: checking PREFERRED_PROVIDER_foo-svnr42-r8 ---------------->snip<-----------------

Why does bitbake not respects the new PV and generates a foo-svnr66-r8.ipk here???

Regards,
Steffen

Signature · 7 lines
DResearch Fahrzeugelektronik GmbH
Otto-Schmirgal-Str. 3, 10319 Berlin, Germany
Tel: +49 30 515932-237 mailto:[email protected]
Fax: +49 30 515932-299
Geschäftsführer: Dr. Michael Weber, Werner Mögle;
Amtsgericht Berlin Charlottenburg; HRB 130120 B;
Ust.-IDNr. DE273952058
#1 lore raw
Quoted 30 lines · Steffen Sledz

I'm working with oe-classic and BitBake Build Tool Core version 1.12.0, bitbake version 1.12.0.

Because of some special development requirements we like to generate the Package Version from the SVN Revision (not the Last Changed Rev!) of the bitbake recipe in the local workspace.

Therefor the recipe contains this:

---------------->snip<-----------------
PR = "r8"
PV = "svnr${@svn_revision(d)}"

inherit svn-helper
---------------->snip<-----------------

The svn-helper.bbclass contains a little helper function to determine the needed value:

---------------->snip<-----------------
def svn_revision(d):

     import subprocess
     bbpath = os.path.dirname(bb.data.getVar('FILE',d,1))
     return subprocess.check_output(["svn", "info", bbpath]).partition("Revision: ")[2].splitlines()[0]

---------------->snip<-----------------

After this changes i make a first build and everything works fine. Assuming the SVN Revsion is 42 a package called foo-svnr42-r8.ipk is generated.

Now i make an "svn update" inside the workspace and the SVN Revision increases to 66.

A call of "bitbake foo" now results in an "Tasks Summary: Attempted 1182 tasks of which 1182 didn't need to be rerun and 0 failed." an no new ipg is generated. :(

The log generated by "bitbake -DDDDDvvvvv foo" contains

Quoted 8 lines

---------------->snip<-----------------
DEBUG: providers for foo are: ['foo']
NOTE: checking PREFERRED_PROVIDER_foo
NOTE: checking PREFERRED_PROVIDER_foo-svnr42
NOTE: checking PREFERRED_PROVIDER_foo-svnr42-r8
---------------->snip<-----------------

Why does bitbake not respects the new PV and generates a foo-svnr66-r8.ipk here???

Ping!

Can anyone of the bitbake gurus lease give a feedback here?

Thx,
Steffen

Signature · 7 lines
DResearch Fahrzeugelektronik GmbH
Otto-Schmirgal-Str. 3, 10319 Berlin, Germany
Tel: +49 30 515932-237 mailto:[email protected]
Fax: +49 30 515932-299
Geschäftsführer: Dr. Michael Weber, Werner Mögle;
Amtsgericht Berlin Charlottenburg; HRB 130120 B;
Ust.-IDNr. DE273952058
#2 lore raw
In reply to Steffen Sledz
Quoted 46 lines · Steffen Sledz

I'm working with oe-classic and BitBake Build Tool Core version 1.12.0, bitbake version 1.12.0.

Because of some special development requirements we like to generate the Package Version from the SVN Revision (not the Last Changed Rev!) of the bitbake recipe in the local workspace.

Therefor the recipe contains this:

---------------->snip<-----------------
PR = "r8"
PV = "svnr${@svn_revision(d)}"

inherit svn-helper
---------------->snip<-----------------

The svn-helper.bbclass contains a little helper function to determine the needed value:

---------------->snip<-----------------
def svn_revision(d):

     import subprocess
     bbpath = os.path.dirname(bb.data.getVar('FILE',d,1))
     return subprocess.check_output(["svn", "info", bbpath]).partition("Revision: ")[2].splitlines()[0]

---------------->snip<-----------------

After this changes i make a first build and everything works fine. Assuming the SVN Revsion is 42 a package called foo-svnr42-r8.ipk is generated.

Now i make an "svn update" inside the workspace and the SVN Revision increases to 66.

A call of "bitbake foo" now results in an "Tasks Summary: Attempted 1182 tasks of which 1182 didn't need to be rerun and 0 failed." an no new ipg is generated. :(

The log generated by "bitbake -DDDDDvvvvv foo" contains

---------------->snip<-----------------
DEBUG: providers for foo are: ['foo']
NOTE: checking PREFERRED_PROVIDER_foo
NOTE: checking PREFERRED_PROVIDER_foo-svnr42
NOTE: checking PREFERRED_PROVIDER_foo-svnr42-r8
---------------->snip<-----------------

Quoted 2 lines

Why does bitbake not respects the new PV and generates a foo-svnr66-r8.ipk here???

Bitbake has no idea that the PV has changed and that it needs to ignore the cache value and reparse the metadata for that file. The similar example is when autorev is used with a source control system. You can can see the code for this in lib/bb/fetch2/__init__.py in get_autorev(). In summary, if you set __BB_DONT_CACHE = "1" in your recipe, I think it will do what you expect, at the cost that it will reparse the recipe every time (which it has to in case PV changes).

Cheers,

Richard

#3 lore raw
Quoted 50 lines · Richard Purdie
Quoted 46 lines · Steffen Sledz

I'm working with oe-classic and BitBake Build Tool Core version 1.12.0, bitbake version 1.12.0.

Because of some special development requirements we like to generate the Package Version from the SVN Revision (not the Last Changed Rev!) of the bitbake recipe in the local workspace.

Therefor the recipe contains this:

---------------->snip<-----------------
PR = "r8"
PV = "svnr${@svn_revision(d)}"

inherit svn-helper
---------------->snip<-----------------

The svn-helper.bbclass contains a little helper function to determine the needed value:

---------------->snip<-----------------
def svn_revision(d):

     import subprocess
     bbpath = os.path.dirname(bb.data.getVar('FILE',d,1))
     return subprocess.check_output(["svn", "info", bbpath]).partition("Revision: ")[2].splitlines()[0]

---------------->snip<-----------------

After this changes i make a first build and everything works fine. Assuming the SVN Revsion is 42 a package called foo-svnr42-r8.ipk is generated.

Now i make an "svn update" inside the workspace and the SVN Revision increases to 66.

A call of "bitbake foo" now results in an "Tasks Summary: Attempted 1182 tasks of which 1182 didn't need to be rerun and 0 failed." an no new ipg is generated. :(

The log generated by "bitbake -DDDDDvvvvv foo" contains

---------------->snip<-----------------
DEBUG: providers for foo are: ['foo']
NOTE: checking PREFERRED_PROVIDER_foo
NOTE: checking PREFERRED_PROVIDER_foo-svnr42
NOTE: checking PREFERRED_PROVIDER_foo-svnr42-r8
---------------->snip<-----------------

Quoted 2 lines

Why does bitbake not respects the new PV and generates a foo-svnr66-r8.ipk here???

Quoted 7 lines

Bitbake has no idea that the PV has changed and that it needs to ignore the cache value and reparse the metadata for that file. The similar example is when autorev is used with a source control system. You can can see the code for this in lib/bb/fetch2/__init__.py in get_autorev(). In summary, if you set __BB_DONT_CACHE = "1" in your recipe, I think it will do what you expect, at the cost that it will reparse the recipe every time (which it has to in case PV changes).

This solution seems to work. :)

Thx,
Steffen

Signature · 7 lines
DResearch Fahrzeugelektronik GmbH
Otto-Schmirgal-Str. 3, 10319 Berlin, Germany
Tel: +49 30 515932-237 mailto:[email protected]
Fax: +49 30 515932-299
Geschäftsführer: Dr. Michael Weber, Werner Mögle;
Amtsgericht Berlin Charlottenburg; HRB 130120 B;
Ust.-IDNr. DE273952058