Skip to contentSkip to conversation

bitbake.conf: define shared PERSISTENT_DIR

PATCH In review Martin Jansa 3 participants 9 messages Feb 24, 2011 – Mar 21, 2011

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 bitbake.conf: define shared PERSISTENT_DIR

* import from poky
commit 6a11cf7dfe930461a6660e3b783b546fa2634900
Author: Richard Purdie <[email protected]>
Date: Sun May 31 22:23:30 2009 +0100
* we had CACHE var pointing to machine specific dir since commit caf077679022f37ce55d758101f130e4e93bd7b5
Author: Richard Purdie <[email protected]>
Date: Mon Nov 20 17:40:20 2006 +0000

* current bitbake is looking for cache dir like this:
cachedir = (bb.data.getVar("PERSISTENT_DIR", d, True) or

              bb.data.getVar("CACHE", d, True))

which without PERSISTENT_DIR defined picks machine-specific CACHE dir so LOCALCOUNT numbers in SRCPV keeps rolling with every MACHINE switch in same builddir :/

---
 conf/bitbake.conf |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)
conf/bitbake.conf +2 -0
index dcc5c70..53bb7b7 100644
@@ -294,6 +294,8 @@ FILESDIR = "${@bb.which(d.getVar('FILESPATH', 1), '.')}"
 
 TMPDIR = "${TOPDIR}/tmp"
 CACHE = "${TMPDIR}/cache${@['', '/' + str(bb.data.getVar('MACHINE', d, 1))][bool(bb.data.getVar('MACHINE', d, 1))]}"
+# The persistent cache should be shared by all builds
+PERSISTENT_DIR = "${TMPDIR}/cache"
 CO_DIR = "${DL_DIR}"
 CVSDIR = "${CO_DIR}/cvs"
 SVNDIR = "${CO_DIR}/svn"
Signature · 1 line
1.7.4.1
#1 lore raw
Quoted 26 lines · Martin Jansa

* import from poky
commit 6a11cf7dfe930461a6660e3b783b546fa2634900
Author: Richard Purdie <[email protected]>
Date: Sun May 31 22:23:30 2009 +0100
* we had CACHE var pointing to machine specific dir since commit caf077679022f37ce55d758101f130e4e93bd7b5
Author: Richard Purdie <[email protected]>
Date: Mon Nov 20 17:40:20 2006 +0000

* current bitbake is looking for cache dir like this:
cachedir = (bb.data.getVar("PERSISTENT_DIR", d, True) or

              bb.data.getVar("CACHE", d, True))

which without PERSISTENT_DIR defined picks machine-specific CACHE dir so LOCALCOUNT numbers in SRCPV keeps rolling with every MACHINE switch in same builddir :/

---
 conf/bitbake.conf |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)
conf/bitbake.conf +0 -0
index dcc5c70..53bb7b7 100644

conf/bitbake.conf FILESDIR = "${@bb.which(d.getVar('FILESPATH', 1), '.')}" · line 298

Quoted 5 lines
conf/bitbake.conf +2 -0
@@ -294,6 +294,8 @@ FILESDIR = "${@bb.which(d.getVar('FILESPATH', 1), '.')}"
 
 TMPDIR = "${TOPDIR}/tmp"
 CACHE = "${TMPDIR}/cache${@['', '/' + str(bb.data.getVar('MACHINE', d, 1))][bool(bb.data.getVar('MACHINE', d, 1))]}"
+# The persistent cache should be shared by all builds
+PERSISTENT_DIR = "${TMPDIR}/cache"

Or even better we can move it outside ${TMPDIR} and add sanity check to warn user that he still has ${CACHE}/bb_persist_data.sqlite3 and should move it to ${PERSISTENT_DIR}/bb_persist_data.sqlite3. (I'll send such patch if requested).

But while thinking about it I found that there are probably other issues with that too.

Machine-specific persistent cache was rebuilding packages with FEED_ARCH, which works ok only as long as all machines started with same cache and all machines are always built (so _count are incremented consistently in all caches).

Shared persistent cache is much better, but still fails if some machine is overriding SRCREV_machine ie in u-boot_git it's quite common with same git SRC_URI and same branch (so key in persistent cache is the same, but _rev is different so _count is incremented twice every time this _machine is built. But overriding SRCREV is less common than multiple machines building _same_ package with FEED_ARCH or even "all".

It could be maybe improved if we add actual PACKAGE_ARCH to key, but I think that's not available while bitbake is parsing recipe (ie before calling utils.is_machine_specific and changing PACKAGE_ARCH to MACHINE_ARCH.

This improved cache would provide consistent _count, but will still fail on target. ie when SRCREV for FEED_ARCH was changed twice and for om-gta02 just once:

Package: frameworkd-config
Version: 1:0.9.5.9+gitr2+e31fa77906981a56be2271246693fd929c973abf-r12.6
Architecture: armv4t

will be preferred instead of

Package: frameworkd-config
Version: 1:0.9.5.9+gitr1+aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-r12.6
Architecture: om-gta02

even with:
SHR root@gjama / $ cat /etc/opkg/arch.conf
..
arch armv4t 26
arch om-gta02 31

because afaik opkg prefers newer version before higher arch priority.

I don't see how to fix this in all corner cases :/.
But shared PERSISTENT_DIR outside TMPDIR seems much better than machine specific in CACHE.

BTW: I'm quite frustrated lately to see that OE is great only for 99%, but last 1% of corner cases like this and ie substrings in packages with debian.bbclass are quite hard to find/debug/explain/fix :(.

Regards,

Signature · 1 line
Martin 'JaMa' Jansa     jabber: [email protected]
#2 lore raw

PATCH 1/1 bitbake.conf: define shared PERSISTENT_DIR and point it outside TMPDIR

* import from poky 6a11cf7dfe930461a6660e3b783b546fa2634900
* we had CACHE var pointing to machine specific dir since 2006 caf077679022f37ce55d758101f130e4e93bd7b5
* current bitbake is looking for cache dir like this:
cachedir = (bb.data.getVar("PERSISTENT_DIR", d, True) or

              bb.data.getVar("CACHE", d, True))

which without PERSISTENT_DIR defined picks machine-specific CACHE dir so LOCALCOUNT numbers in SRCPV keeps rolling with every MACHINE switch in same builddir :/

---
 classes/sanity.bbclass |   12 ++++++++++++
 conf/bitbake.conf      |    2 ++
 conf/sanity.conf       |    1 +
 3 files changed, 15 insertions(+), 0 deletions(-)
classes/sanity.bbclass +12 -0
index 1e6b170..ffe3a49 100644
@@ -189,6 +189,18 @@ def check_sanity(e):
 	if messages != "":
 		raise_sanity_error(messages)
 
+	#
+	# Check there isn't old persistent cache
+	#
+	cache = data.getVar('CACHE', e.data, True)
+	persistent_dir = data.getVar('PERSISTENT_DIR', e.data, True)
+	persistent_cache_filename = data.getVar('SANITY_PERSIST_DATA_FILE', e.data, True)
+	if os.path.exists(cache + '/' + persistent_cache_filename):
+		messages = messages + "Error, persistent cache file '%s' exists in old location '%s', please migrate it to new location '%s' and merge them together if you have one for each MACHINE.\n" % (persistent_cache_filename, cache, persistent_dir)
+
+	if messages != "":
+		raise_sanity_error(messages)
+
 python check_sanity_eventhandler() {
     if isinstance(e, bb.event.BuildStarted):
         check_sanity(e)
conf/bitbake.conf +2 -0
index 88ad2ba..0c11cdb 100644
@@ -294,6 +294,8 @@ FILESDIR = "${@bb.which(d.getVar('FILESPATH', 1), '.')}"
 
 TMPDIR = "${TOPDIR}/tmp"
 CACHE = "${TMPDIR}/cache${@['', '/' + str(bb.data.getVar('MACHINE', d, 1))][bool(bb.data.getVar('MACHINE', d, 1))]}"
+# The persistent cache should be shared by all builds
+PERSISTENT_DIR = "${TOPDIR}/conf/persistent"
 CO_DIR = "${DL_DIR}"
 CVSDIR = "${CO_DIR}/cvs"
 SVNDIR = "${CO_DIR}/svn"
conf/sanity.conf +1 -0
index e9902f6..c1ac332 100644
@@ -7,5 +7,6 @@ BB_MIN_VERSION = "1.10.2"
 
 SANITY_ABIFILE = "${TMPDIR}/abi_version"
 SANITY_PRFILE = "${TMPDIR}/distro_pr"
+SANITY_PERSIST_DATA_FILE = "bb_persist_data.sqlite3"
 
 INHERIT += "sanity"
Signature · 1 line
1.7.4.1
#3 lore raw

Re: [PATCH] bitbake.conf: define shared PERSISTENT_DIR and point it outside TMPDIR

Quoted 9 lines · Martin Jansa

* import from poky 6a11cf7dfe930461a6660e3b783b546fa2634900
* we had CACHE var pointing to machine specific dir since 2006 caf077679022f37ce55d758101f130e4e93bd7b5
* current bitbake is looking for cache dir like this:
cachedir = (bb.data.getVar("PERSISTENT_DIR", d, True) or

              bb.data.getVar("CACHE", d, True))

which without PERSISTENT_DIR defined picks machine-specific CACHE dir so LOCALCOUNT numbers in SRCPV keeps rolling with every MACHINE switch in same builddir :/

ping?

Quoted 58 lines
---
 classes/sanity.bbclass |   12 ++++++++++++
 conf/bitbake.conf      |    2 ++
 conf/sanity.conf       |    1 +
 3 files changed, 15 insertions(+), 0 deletions(-)
classes/sanity.bbclass +12 -0
index 1e6b170..ffe3a49 100644
@@ -189,6 +189,18 @@ def check_sanity(e):
 	if messages != "":
 		raise_sanity_error(messages)
 
+	#
+	# Check there isn't old persistent cache
+	#
+	cache = data.getVar('CACHE', e.data, True)
+	persistent_dir = data.getVar('PERSISTENT_DIR', e.data, True)
+	persistent_cache_filename = data.getVar('SANITY_PERSIST_DATA_FILE', e.data, True)
+	if os.path.exists(cache + '/' + persistent_cache_filename):
+		messages = messages + "Error, persistent cache file '%s' exists in old location '%s', please migrate it to new location '%s' and merge them together if you have one for each MACHINE.\n" % (persistent_cache_filename, cache, persistent_dir)
+
+	if messages != "":
+		raise_sanity_error(messages)
+
 python check_sanity_eventhandler() {
     if isinstance(e, bb.event.BuildStarted):
         check_sanity(e)
conf/bitbake.conf +2 -0
index 88ad2ba..0c11cdb 100644
@@ -294,6 +294,8 @@ FILESDIR = "${@bb.which(d.getVar('FILESPATH', 1), '.')}"
 
 TMPDIR = "${TOPDIR}/tmp"
 CACHE = "${TMPDIR}/cache${@['', '/' + str(bb.data.getVar('MACHINE', d, 1))][bool(bb.data.getVar('MACHINE', d, 1))]}"
+# The persistent cache should be shared by all builds
+PERSISTENT_DIR = "${TOPDIR}/conf/persistent"
 CO_DIR = "${DL_DIR}"
 CVSDIR = "${CO_DIR}/cvs"
 SVNDIR = "${CO_DIR}/svn"
conf/sanity.conf +1 -0
index e9902f6..c1ac332 100644
@@ -7,5 +7,6 @@ BB_MIN_VERSION = "1.10.2"
 
 SANITY_ABIFILE = "${TMPDIR}/abi_version"
 SANITY_PRFILE = "${TMPDIR}/distro_pr"
+SANITY_PERSIST_DATA_FILE = "bb_persist_data.sqlite3"
 
 INHERIT += "sanity"
Signature · 1 line
1.7.4.1
Signature · 1 line
Martin 'JaMa' Jansa     jabber: [email protected]
#4 lore raw

Re: [PATCH] bitbake.conf: define shared PERSISTENT_DIR and point it outside TMPDIR

Quoted 11 lines · Martin Jansa
Quoted 9 lines · Martin Jansa

* import from poky 6a11cf7dfe930461a6660e3b783b546fa2634900
* we had CACHE var pointing to machine specific dir since 2006 caf077679022f37ce55d758101f130e4e93bd7b5
* current bitbake is looking for cache dir like this:
cachedir = (bb.data.getVar("PERSISTENT_DIR", d, True) or

               bb.data.getVar("CACHE", d, True))

which without PERSISTENT_DIR defined picks machine-specific CACHE dir so LOCALCOUNT numbers in SRCPV keeps rolling with every MACHINE switch in same builddir :/

Quoted 2 lines

ping?

I'd like Richards comments here.

Quoted 65 lines
Quoted 56 lines
---
  classes/sanity.bbclass |   12 ++++++++++++
  conf/bitbake.conf      |    2 ++
  conf/sanity.conf       |    1 +
  3 files changed, 15 insertions(+), 0 deletions(-)
classes/sanity.bbclass +12 -0
index 1e6b170..ffe3a49 100644
@@ -189,6 +189,18 @@ def check_sanity(e):
  	if messages != "":
  		raise_sanity_error(messages)
 
+	#
+	# Check there isn't old persistent cache
+	#
+	cache = data.getVar('CACHE', e.data, True)
+	persistent_dir = data.getVar('PERSISTENT_DIR', e.data, True)
+	persistent_cache_filename = data.getVar('SANITY_PERSIST_DATA_FILE', e.data, True)
+	if os.path.exists(cache + '/' + persistent_cache_filename):
+		messages = messages + "Error, persistent cache file '%s' exists in old location '%s', please migrate it to new location '%s' and merge them together if you have one for each MACHINE.\n" % (persistent_cache_filename, cache, persistent_dir)
+
+	if messages != "":
+		raise_sanity_error(messages)
+
  python check_sanity_eventhandler() {
      if isinstance(e, bb.event.BuildStarted):
          check_sanity(e)
conf/bitbake.conf +2 -0
index 88ad2ba..0c11cdb 100644
@@ -294,6 +294,8 @@ FILESDIR = "${@bb.which(d.getVar('FILESPATH', 1), '.')}"
 
  TMPDIR = "${TOPDIR}/tmp"
  CACHE = "${TMPDIR}/cache${@['', '/' + str(bb.data.getVar('MACHINE', d, 1))][bool(bb.data.getVar('MACHINE', d, 1))]}"
+# The persistent cache should be shared by all builds
+PERSISTENT_DIR = "${TOPDIR}/conf/persistent"
  CO_DIR = "${DL_DIR}"
  CVSDIR = "${CO_DIR}/cvs"
  SVNDIR = "${CO_DIR}/svn"
conf/sanity.conf +1 -1
index e9902f6..c1ac332 100644
@@ -7,5 +7,6 @@ BB_MIN_VERSION = "1.10.2"
 
  SANITY_ABIFILE = "${TMPDIR}/abi_version"
  SANITY_PRFILE = "${TMPDIR}/distro_pr"
+SANITY_PERSIST_DATA_FILE = "bb_persist_data.sqlite3"
 
  INHERIT += "sanity"
--
Quoted 2 lines

1.7.4.1

_______________________________________________
Openembedded-devel mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel

Signature · 2 lines
Tom Rini
Mentor Graphics Corporation
#5 lore raw

Re: [PATCH] bitbake.conf: define shared PERSISTENT_DIR and point it outside TMPDIR

Quoted 12 lines · Tom Rini
Quoted 11 lines · Martin Jansa
Quoted 9 lines · Martin Jansa

* import from poky 6a11cf7dfe930461a6660e3b783b546fa2634900
* we had CACHE var pointing to machine specific dir since 2006 caf077679022f37ce55d758101f130e4e93bd7b5
* current bitbake is looking for cache dir like this:
cachedir = (bb.data.getVar("PERSISTENT_DIR", d, True) or

               bb.data.getVar("CACHE", d, True))

which without PERSISTENT_DIR defined picks machine-specific CACHE dir so LOCALCOUNT numbers in SRCPV keeps rolling with every MACHINE switch in same builddir :/

Quoted 4 lines
Quoted 2 lines

ping?

I'd like Richards comments here.

The intent was always to have CACHE be machine specific and PERSISTENT_DIR not be machine specific. This means all builds share the idea of revision numbers yet the main cache is machine specific so you don't hit reparses when switching machines unnecessarily.

So yes, I think the change is valid, I hadn't realised OE didn't have that change and I'm happy to ack it...

Cheers,

Richard

Quoted 67 lines
Quoted 65 lines
Quoted 56 lines
---
  classes/sanity.bbclass |   12 ++++++++++++
  conf/bitbake.conf      |    2 ++
  conf/sanity.conf       |    1 +
  3 files changed, 15 insertions(+), 0 deletions(-)
classes/sanity.bbclass +12 -0
index 1e6b170..ffe3a49 100644
@@ -189,6 +189,18 @@ def check_sanity(e):
  	if messages != "":
  		raise_sanity_error(messages)
 
+	#
+	# Check there isn't old persistent cache
+	#
+	cache = data.getVar('CACHE', e.data, True)
+	persistent_dir = data.getVar('PERSISTENT_DIR', e.data, True)
+	persistent_cache_filename = data.getVar('SANITY_PERSIST_DATA_FILE', e.data, True)
+	if os.path.exists(cache + '/' + persistent_cache_filename):
+		messages = messages + "Error, persistent cache file '%s' exists in old location '%s', please migrate it to new location '%s' and merge them together if you have one for each MACHINE.\n" % (persistent_cache_filename, cache, persistent_dir)
+
+	if messages != "":
+		raise_sanity_error(messages)
+
  python check_sanity_eventhandler() {
      if isinstance(e, bb.event.BuildStarted):
          check_sanity(e)
conf/bitbake.conf +2 -0
index 88ad2ba..0c11cdb 100644
@@ -294,6 +294,8 @@ FILESDIR = "${@bb.which(d.getVar('FILESPATH', 1), '.')}"
 
  TMPDIR = "${TOPDIR}/tmp"
  CACHE = "${TMPDIR}/cache${@['', '/' + str(bb.data.getVar('MACHINE', d, 1))][bool(bb.data.getVar('MACHINE', d, 1))]}"
+# The persistent cache should be shared by all builds
+PERSISTENT_DIR = "${TOPDIR}/conf/persistent"
  CO_DIR = "${DL_DIR}"
  CVSDIR = "${CO_DIR}/cvs"
  SVNDIR = "${CO_DIR}/svn"
conf/sanity.conf +1 -1
index e9902f6..c1ac332 100644
@@ -7,5 +7,6 @@ BB_MIN_VERSION = "1.10.2"
 
  SANITY_ABIFILE = "${TMPDIR}/abi_version"
  SANITY_PRFILE = "${TMPDIR}/distro_pr"
+SANITY_PERSIST_DATA_FILE = "bb_persist_data.sqlite3"
 
  INHERIT += "sanity"
--
Quoted 2 lines

1.7.4.1

_______________________________________________
Openembedded-devel mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel

#6 lore raw

Re: [PATCH] bitbake.conf: define shared PERSISTENT_DIR and point it outside TMPDIR

Quoted 16 lines · Richard Purdie
Quoted 15 lines · Tom Rini
Quoted 13 lines · Martin Jansa
Quoted 9 lines · Martin Jansa

* import from poky 6a11cf7dfe930461a6660e3b783b546fa2634900
* we had CACHE var pointing to machine specific dir since 2006 caf077679022f37ce55d758101f130e4e93bd7b5
* current bitbake is looking for cache dir like this:
cachedir = (bb.data.getVar("PERSISTENT_DIR", d, True) or

               bb.data.getVar("CACHE", d, True))

which without PERSISTENT_DIR defined picks machine-specific CACHE dir so LOCALCOUNT numbers in SRCPV keeps rolling with every MACHINE switch in same builddir :/

ping?

Quoted 7 lines
Quoted 1 line

I'd like Richards comments here.

The intent was always to have CACHE be machine specific and PERSISTENT_DIR not be machine specific. This means all builds share the

idea of revision numbers yet the main cache is machine specific so you don't hit reparses when switching machines unnecessarily.

BTW: with bitbake master and shared CACHE on one system I noticed that if I set MACHINE/DISTRO only in shell environment (with correct BB_ENV_EXTRAWHITE), then cache is not invalidated and metadata reparsed, resulting in broken builds where parts were respecting ${MACHINE} from shell and some parts had different machine from previous build.

IIRC I noticed it on different machine listed in build summary and then another one in filename of machine-specific ipk. Removing cache after switching MACHINE in shell env worked fine.

Regards,

Quoted 82 lines

So yes, I think the change is valid, I hadn't realised OE didn't have that change and I'm happy to ack it...

Cheers,

Richard

Quoted 54 lines
Quoted 54 lines
Quoted 54 lines
---
  classes/sanity.bbclass |   12 ++++++++++++
  conf/bitbake.conf      |    2 ++
  conf/sanity.conf       |    1 +
  3 files changed, 15 insertions(+), 0 deletions(-)
classes/sanity.bbclass +12 -0
index 1e6b170..ffe3a49 100644
@@ -189,6 +189,18 @@ def check_sanity(e):
  	if messages != "":
  		raise_sanity_error(messages)
 
+	#
+	# Check there isn't old persistent cache
+	#
+	cache = data.getVar('CACHE', e.data, True)
+	persistent_dir = data.getVar('PERSISTENT_DIR', e.data, True)
+	persistent_cache_filename = data.getVar('SANITY_PERSIST_DATA_FILE', e.data, True)
+	if os.path.exists(cache + '/' + persistent_cache_filename):
+		messages = messages + "Error, persistent cache file '%s' exists in old location '%s', please migrate it to new location '%s' and merge them together if you have one for each MACHINE.\n" % (persistent_cache_filename, cache, persistent_dir)
+
+	if messages != "":
+		raise_sanity_error(messages)
+
  python check_sanity_eventhandler() {
      if isinstance(e, bb.event.BuildStarted):
          check_sanity(e)
conf/bitbake.conf +2 -0
index 88ad2ba..0c11cdb 100644
@@ -294,6 +294,8 @@ FILESDIR = "${@bb.which(d.getVar('FILESPATH', 1), '.')}"
 
  TMPDIR = "${TOPDIR}/tmp"
  CACHE = "${TMPDIR}/cache${@['', '/' + str(bb.data.getVar('MACHINE', d, 1))][bool(bb.data.getVar('MACHINE', d, 1))]}"
+# The persistent cache should be shared by all builds
+PERSISTENT_DIR = "${TOPDIR}/conf/persistent"
  CO_DIR = "${DL_DIR}"
  CVSDIR = "${CO_DIR}/cvs"
  SVNDIR = "${CO_DIR}/svn"
conf/sanity.conf +1 -0
index e9902f6..c1ac332 100644
@@ -7,5 +7,6 @@ BB_MIN_VERSION = "1.10.2"
 
  SANITY_ABIFILE = "${TMPDIR}/abi_version"
  SANITY_PRFILE = "${TMPDIR}/distro_pr"
+SANITY_PERSIST_DATA_FILE = "bb_persist_data.sqlite3"
 
Quoted 13 lines
Quoted 11 lines
Quoted 4 lines
conf/sanity.conf +0 -1
  INHERIT += "sanity"
--

1.7.4.1

_______________________________________________
Openembedded-devel mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel

_______________________________________________
Openembedded-devel mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel

Signature · 1 line
Martin 'JaMa' Jansa     jabber: [email protected]
#7 lore raw

Re: [PATCH] bitbake.conf: define shared PERSISTENT_DIR and point it outside TMPDIR

In reply to Richard Purdie
Quoted 18 lines · Richard Purdie
Quoted 12 lines · Tom Rini
Quoted 11 lines · Martin Jansa
Quoted 9 lines · Martin Jansa

* import from poky 6a11cf7dfe930461a6660e3b783b546fa2634900
* we had CACHE var pointing to machine specific dir since 2006 caf077679022f37ce55d758101f130e4e93bd7b5
* current bitbake is looking for cache dir like this:
cachedir = (bb.data.getVar("PERSISTENT_DIR", d, True) or

               bb.data.getVar("CACHE", d, True))

which without PERSISTENT_DIR defined picks machine-specific CACHE dir so LOCALCOUNT numbers in SRCPV keeps rolling with every MACHINE switch in same builddir :/

Quoted 4 lines
Quoted 2 lines

ping?

I'd like Richards comments here.

Quoted 7 lines

The intent was always to have CACHE be machine specific and PERSISTENT_DIR not be machine specific. This means all builds share the idea of revision numbers yet the main cache is machine specific so you don't hit reparses when switching machines unnecessarily.

So yes, I think the change is valid, I hadn't realised OE didn't have that change and I'm happy to ack it...

Can I get at least one more ACK?

And if we all agree on new location "${TOPDIR}/conf/persistent"
then I would send similar patch to oe-core and move it too

Regards,

Signature · 1 line
Martin 'JaMa' Jansa     jabber: [email protected]
#8 lore raw
In reply to Martin Jansa
Quoted 9 lines · Martin Jansa

* import from poky
commit 6a11cf7dfe930461a6660e3b783b546fa2634900
Author: Richard Purdie <[email protected]>
Date: Sun May 31 22:23:30 2009 +0100
* we had CACHE var pointing to machine specific dir since commit caf077679022f37ce55d758101f130e4e93bd7b5
Author: Richard Purdie <[email protected]>
Date: Mon Nov 20 17:40:20 2006 +0000

Quoted 8 lines

* current bitbake is looking for cache dir like this:
cachedir = (bb.data.getVar("PERSISTENT_DIR", d, True) or

              bb.data.getVar("CACHE", d, True))

which without PERSISTENT_DIR defined picks machine-specific CACHE dir so LOCALCOUNT numbers in SRCPV keeps rolling with every MACHINE switch in same builddir :/

ping?

2nd version which moved it outside TMPDIR
[oe] [PATCH] bitbake.conf: define shared PERSISTENT_DIR and point it outside TMPDIR was sort of rejected:
http://lists.linuxtogo.org/pipermail/openembedded-core/2011-March/000298.html

but this part without moving it outside TMPDIR is still valid and I'll push it after month of review if nobody objects :).

Quoted 20 lines
---
 conf/bitbake.conf |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)
conf/bitbake.conf +2 -0
index dcc5c70..53bb7b7 100644
@@ -294,6 +294,8 @@ FILESDIR = "${@bb.which(d.getVar('FILESPATH', 1), '.')}"
 
 TMPDIR = "${TOPDIR}/tmp"
 CACHE = "${TMPDIR}/cache${@['', '/' + str(bb.data.getVar('MACHINE', d, 1))][bool(bb.data.getVar('MACHINE', d, 1))]}"
+# The persistent cache should be shared by all builds
+PERSISTENT_DIR = "${TMPDIR}/cache"
 CO_DIR = "${DL_DIR}"
 CVSDIR = "${CO_DIR}/cvs"
 SVNDIR = "${CO_DIR}/svn"
Signature · 1 line
1.7.4.1
Signature · 1 line
Martin 'JaMa' Jansa     jabber: [email protected]