Skip to contentSkip to conversation

Support symbolic links in paths in PSEUDO_IGNORE_PATHS

PATCH Superseded Peter Kjellerstedt 2 participants 12 messages Nov 25, 2020 – Dec 1, 2020

v2 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

v2 cover Support symbolic links in paths in PSEUDO_IGNORE_PATHS

The changes in this patch series aim to rectify a problem with pseudo and its support for ignoring paths. When pseudo compares a path to the paths specified in PSEUDO_IGNORE_PATHS, it uses a path that has been canonicalized. However, it does not canonicalize the paths in PSEUDO_IGNORE_PATHS itself and unless they are canonicalized, they will not match as intended. Thus the paths in PSEUDO_IGNORE_PATHS needs to be canonicalized. These patches do that by adding a new PSEUDO_IGNORE_REAL_PATHS variable, which contains the paths from PSEUDO_IGNORE_PATHS after having run them through os.path.realpath().

There is also one patch that adds two patches to pseudo to clean up pseudo_client_ignore_path_chroot(), and they also plug a memory leak. The patches were brought about as I initially intended to do the canonicalization in pseudo itself in this function.

I have not tested the change to wic as we do not use it, though I do not expect it to be problematic.

PATCHv2: Removed some unrelated changes in the second patch.

//Peter

The following changes since commit 3ecf5d9692fec97b37af6a4e6c747a4e2c2ca292:

uninative: Don't use single sstate for pseudo-native (2020-11-24 15:53:07 +0000)

are available in the Git repository at:

git://push.yoctoproject.org/poky-contrib pkj/pseudo-ignore

Peter Kjellerstedt (4):
pseudo: Simplify pseudo_client_ignore_path_chroot() bitbake.conf: Add all layers (from BBLAYERS) to PSEUDO_IGNORE_PATHS bitbake.conf: Canonicalize paths in PSEUDO_IGNORE_PATHS
wic: Pass canonicalized paths in PSEUDO_IGNORE_PATHS

 meta/classes/image_types_wic.bbclass          |  2 +-
 meta/conf/bitbake.conf                        | 10 +--
 ...ssen-indentation-of-pseudo_client_ig.patch | 69 +++++++++++++++++++
 ...mplify-pseudo_client_ignore_path_chr.patch | 50 ++++++++++++++
 meta/recipes-devtools/pseudo/pseudo_git.bb    |  2 +
 scripts/lib/wic/partition.py                  |  2 +-
 6 files changed, 129 insertions(+), 6 deletions(-)
 create mode 100644 meta/recipes-devtools/pseudo/files/0002-pseudo_client-Lessen-indentation-of-pseudo_client_ig.patch
 create mode 100644 meta/recipes-devtools/pseudo/files/0003-pseudo_client-Simplify-pseudo_client_ignore_path_chr.patch
#1 lore raw

PATCH v2 1/4 pseudo: Simplify pseudo_client_ignore_path_chroot()

This also plugs a memory leak in pseudo_client_ignore_path_chroot().

---
 ...ssen-indentation-of-pseudo_client_ig.patch | 69 +++++++++++++++++++
 ...mplify-pseudo_client_ignore_path_chr.patch | 50 ++++++++++++++
 meta/recipes-devtools/pseudo/pseudo_git.bb    |  2 +
 3 files changed, 121 insertions(+)
 create mode 100644 meta/recipes-devtools/pseudo/files/0002-pseudo_client-Lessen-indentation-of-pseudo_client_ig.patch
 create mode 100644 meta/recipes-devtools/pseudo/files/0003-pseudo_client-Simplify-pseudo_client_ignore_path_chr.patch
meta/recipes-devtools/pseudo/files/0002-pseudo_client-Lessen-indentation-of-pseudo_client_ig.patch new file +69 -0
new file mode 100644
index 0000000000..e4a5356f5c
@@ -0,0 +1,69 @@
+From 28c760542eecd7c5b35ea88aa2b14d62afbda961 Mon Sep 17 00:00:00 2001
+From: Peter Kjellerstedt <[email protected]>
+Date: Sat, 21 Nov 2020 17:22:38 +0100
+Subject: [PATCH] pseudo_client: Lessen indentation of
+ pseudo_client_ignore_path_chroot()
+
+Change-Id: I739b18efb7a95ce2d2d907d0faf7df539ab9af1c
+---
+ pseudo_client.c | 45 +++++++++++++++++++++++++--------------------
+ 1 file changed, 25 insertions(+), 20 deletions(-)
+
+diff --git a/pseudo_client.c b/pseudo_client.c
+index 116d926..a8bc3dc 100644
+--- a/pseudo_client.c
++++ b/pseudo_client.c
+@@ -1527,28 +1527,33 @@ int pseudo_client_ignore_fd(int fd) {
+ 
+ int pseudo_client_ignore_path_chroot(const char *path, int ignore_chroot) {
+ 	char *env;
+-	if (path) {
+-		if (ignore_chroot && pseudo_chroot && strncmp(path, pseudo_chroot, pseudo_chroot_len) == 0)
+-			return 0;
+-		env = pseudo_get_value("PSEUDO_IGNORE_PATHS");
+-		if (env) {
+-			char *p = env;
+-        	        while (*p) {
+-				char *next = strchr(p, ',');
+-				if (!next)
+-				    next = strchr(p, '\0');
+-				if ((next - p) && !strncmp(path, p, next - p)) {
+-		 			pseudo_debug(PDBGF_PATH | PDBGF_VERBOSE, "ignoring path: '%s'\n", path);
+-					return 1;
+-				}
+-				if (next && *next != '\0')
+-					p = next+1;
+-				else
+-					break;
+-			}
+-			free(env);
++
++	if (!path)
++		return 0;
++
++	if (ignore_chroot && pseudo_chroot && strncmp(path, pseudo_chroot, pseudo_chroot_len) == 0)
++		return 0;
++
++	env = pseudo_get_value("PSEUDO_IGNORE_PATHS");
++	if (!env)
++		return 0;
++
++	char *p = env;
++	while (*p) {
++		char *next = strchr(p, ',');
++		if (!next)
++			next = strchr(p, '\0');
++		if ((next - p) && !strncmp(path, p, next - p)) {
++ 			pseudo_debug(PDBGF_PATH | PDBGF_VERBOSE, "ignoring path: '%s'\n", path);
++			return 1;
+ 		}
++		if (next && *next != '\0')
++			p = next+1;
++		else
++			break;
+ 	}
++	free(env);
++
+ 	return 0;
+ }
+ 
meta/recipes-devtools/pseudo/files/0003-pseudo_client-Simplify-pseudo_client_ignore_path_chr.patch new file +50 -0

50 lines not on this page · raw message

meta/recipes-devtools/pseudo/pseudo_git.bb +2 -0
index 2e13fec540..50933c4bc1 100644
@@ -4,6 +4,8 @@ SRC_URI = "git://git.yoctoproject.org/pseudo;branch=oe-core \
            file://0001-configure-Prune-PIE-flags.patch \
            file://fallback-passwd \
            file://fallback-group \
+           file://0002-pseudo_client-Lessen-indentation-of-pseudo_client_ig.patch \
+           file://0003-pseudo_client-Simplify-pseudo_client_ignore_path_chr.patch \
            "
 
 SRCREV = "cca0d7f15b7197095cd587420d31b187620c3093"
#2 lore raw

Re: [OE-core] [PATCHv2 1/4] pseudo: Simplify pseudo_client_ignore_path_chroot()

Hi Peter,

Quoted 3 lines · Peter Kjellerstedt

This also plugs a memory leak in pseudo_client_ignore_path_chroot().

I make no claim to understand what happened but I tried patches 1 and 2 in a build over the weekend and with the first two present:

https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/1602
https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/1584
https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/1586
https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/1610

I then removed this one leaving only patch 2 and the build was fine.

So there is something odd in this patch but I've not looked into what or why only selftest is affected.

Cheers,

Richard

#3 lore raw

Re: [OE-core] [PATCHv2 1/4] pseudo: Simplify pseudo_client_ignore_path_chroot()

Quoted 23 lines

-----Original Message-----
From: Richard Purdie <[email protected]>
Sent: den 30 november 2020 12:47
To: Peter Kjellerstedt <[email protected]>; openembedded- [email protected]
Subject: Re: [OE-core] [PATCHv2 1/4] pseudo: Simplify pseudo_client_ignore_path_chroot()

Hi Peter,

Quoted 3 lines · Peter Kjellerstedt

This also plugs a memory leak in pseudo_client_ignore_path_chroot().

I make no claim to understand what happened but I tried patches 1 and 2 in a build over the weekend and with the first two present:

https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/1602
https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/1584
https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/1586
https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/1610

Quoted 8 lines

I then removed this one leaving only patch 2 and the build was fine.

So there is something odd in this patch but I've not looked into what or why only selftest is affected.

Cheers,

Richard

I believe I have found the problem. I had missed the edge case where two commas are specified immediately after each other, or there is a leading/trailing comma. This would effectively cause pseudo to ignore all paths. Thankfully the fix is trivial.

I have verified that I could repeat the oe-selftest problems for the wic.Wic2 tests, and that they are back to working with my updated code.

I will wait with sending an updated patch series until I have a response regarding how you want me to handle the third patch in the series (see my other mail from earlier today).

//Peter

#4 lore raw

PATCH v2 2/4 bitbake.conf: Add all layers (from BBLAYERS) to PSEUDO_IGNORE_PATHS

In reply to Peter Kjellerstedt

Instead of ignoring ${COREBASE}/meta in PSEUDO_IGNORE_PATHS (which may or may not ignore all layers depending on how they are named and placed under ${COREBASE}), ignore all layers.

---
 meta/conf/bitbake.conf | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/conf/bitbake.conf +1 -1
index 0d38eac094..9742fe4fe2 100644
@@ -685,7 +685,7 @@ SRC_URI = ""
 PSEUDO_LOCALSTATEDIR ?= "${WORKDIR}/pseudo/"
 PSEUDO_PASSWD ?= "${STAGING_DIR_TARGET}:${PSEUDO_SYSROOT}"
 PSEUDO_SYSROOT = "${COMPONENTS_DIR}/${BUILD_ARCH}/pseudo-native"
-PSEUDO_IGNORE_PATHS = "/usr/,/etc/,/lib,/dev/,${T},${WORKDIR}/recipe-sysroot,${SSTATE_DIR},${STAMPS_DIR},${WORKDIR}/pkgdata-sysroot,${TMPDIR}/sstate-control,${DEPLOY_DIR},${WORKDIR}/deploy-,${TMPDIR}/buildstats,${WORKDIR}/sstate-build-package_,${WORKDIR}/sstate-install-package_,${WORKDIR}/sstate-build-image_complete,${TMPDIR}/sysroots-components,${BUILDHISTORY_DIR},${TMPDIR}/pkgdata,${TOPDIR}/cache,${COREBASE}/scripts,${COREBASE}/meta,${CCACHE_DIR}"
+PSEUDO_IGNORE_PATHS = "/usr/,/etc/,/lib,/dev/,${T},${WORKDIR}/recipe-sysroot,${SSTATE_DIR},${STAMPS_DIR},${WORKDIR}/pkgdata-sysroot,${TMPDIR}/sstate-control,${DEPLOY_DIR},${WORKDIR}/deploy-,${TMPDIR}/buildstats,${WORKDIR}/sstate-build-package_,${WORKDIR}/sstate-install-package_,${WORKDIR}/sstate-build-image_complete,${TMPDIR}/sysroots-components,${BUILDHISTORY_DIR},${TMPDIR}/pkgdata,${TOPDIR}/cache,${COREBASE}/scripts,${@','.join(d.getVar('BBLAYERS').split())},${CCACHE_DIR}"
 
 export PSEUDO_DISABLED = "1"
 #export PSEUDO_PREFIX = "${STAGING_DIR_NATIVE}${prefix_native}"
#5 lore raw

PATCH v2 3/4 bitbake.conf: Canonicalize paths in PSEUDO_IGNORE_PATHS

In reply to Peter Kjellerstedt

Introduce PSEUDO_IGNORE_REAL_PATHS and make it contain the canonicalized paths from PSEUDO_IGNORE_PATHS, obtained by passing the latter to oe.path.to_real_paths(). This is needed since pseudo's pseudo_client_ignore_path_chroot() will compare the ignored paths to paths that have been canonicalized.

---
 meta/conf/bitbake.conf | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
meta/conf/bitbake.conf +5 -3
index 9742fe4fe2..4862095a1b 100644
@@ -685,15 +685,16 @@ SRC_URI = ""
 PSEUDO_LOCALSTATEDIR ?= "${WORKDIR}/pseudo/"
 PSEUDO_PASSWD ?= "${STAGING_DIR_TARGET}:${PSEUDO_SYSROOT}"
 PSEUDO_SYSROOT = "${COMPONENTS_DIR}/${BUILD_ARCH}/pseudo-native"
+PSEUDO_IGNORE_REAL_PATHS = "${@','.join(os.path.realpath(path) for path in (d.getVar('PSEUDO_IGNORE_PATHS') or '').split(','))}"
 PSEUDO_IGNORE_PATHS = "/usr/,/etc/,/lib,/dev/,${T},${WORKDIR}/recipe-sysroot,${SSTATE_DIR},${STAMPS_DIR},${WORKDIR}/pkgdata-sysroot,${TMPDIR}/sstate-control,${DEPLOY_DIR},${WORKDIR}/deploy-,${TMPDIR}/buildstats,${WORKDIR}/sstate-build-package_,${WORKDIR}/sstate-install-package_,${WORKDIR}/sstate-build-image_complete,${TMPDIR}/sysroots-components,${BUILDHISTORY_DIR},${TMPDIR}/pkgdata,${TOPDIR}/cache,${COREBASE}/scripts,${@','.join(d.getVar('BBLAYERS').split())},${CCACHE_DIR}"
 
 export PSEUDO_DISABLED = "1"
 #export PSEUDO_PREFIX = "${STAGING_DIR_NATIVE}${prefix_native}"
 #export PSEUDO_BINDIR = "${STAGING_DIR_NATIVE}${bindir_native}"
 #export PSEUDO_LIBDIR = "${STAGING_DIR_NATIVE}$PSEUDOBINDIR/../lib/pseudo/lib
-FAKEROOTBASEENV = "PSEUDO_BINDIR=${PSEUDO_SYSROOT}${bindir_native} PSEUDO_LIBDIR=${PSEUDO_SYSROOT}${prefix_native}/lib/pseudo/lib PSEUDO_PREFIX=${PSEUDO_SYSROOT}${prefix_native} PSEUDO_IGNORE_PATHS=${PSEUDO_IGNORE_PATHS} PSEUDO_DISABLED=1"
+FAKEROOTBASEENV = "PSEUDO_BINDIR=${PSEUDO_SYSROOT}${bindir_native} PSEUDO_LIBDIR=${PSEUDO_SYSROOT}${prefix_native}/lib/pseudo/lib PSEUDO_PREFIX=${PSEUDO_SYSROOT}${prefix_native} PSEUDO_IGNORE_PATHS=${PSEUDO_IGNORE_REAL_PATHS} PSEUDO_DISABLED=1"
 FAKEROOTCMD = "${PSEUDO_SYSROOT}${bindir_native}/pseudo"
-FAKEROOTENV = "PSEUDO_PREFIX=${PSEUDO_SYSROOT}${prefix_native} PSEUDO_LOCALSTATEDIR=${PSEUDO_LOCALSTATEDIR} PSEUDO_PASSWD=${PSEUDO_PASSWD} PSEUDO_NOSYMLINKEXP=1 PSEUDO_IGNORE_PATHS=${PSEUDO_IGNORE_PATHS} PSEUDO_DISABLED=0"
+FAKEROOTENV = "PSEUDO_PREFIX=${PSEUDO_SYSROOT}${prefix_native} PSEUDO_LOCALSTATEDIR=${PSEUDO_LOCALSTATEDIR} PSEUDO_PASSWD=${PSEUDO_PASSWD} PSEUDO_NOSYMLINKEXP=1 PSEUDO_IGNORE_PATHS=${PSEUDO_IGNORE_REAL_PATHS} PSEUDO_DISABLED=0"
 FAKEROOTNOENV = "PSEUDO_UNLOAD=1"
 FAKEROOTDIRS = "${PSEUDO_LOCALSTATEDIR}"
 PREFERRED_PROVIDER_virtual/fakeroot-native ?= "pseudo-native"
@@ -884,7 +885,8 @@ BB_HASHEXCLUDE_COMMON ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH BBSERVER DL_DI
     BB_WORKERCONTEXT BB_LIMITEDDEPS BB_UNIHASH extend_recipe_sysroot DEPLOY_DIR \
     SSTATE_HASHEQUIV_METHOD SSTATE_HASHEQUIV_REPORT_TASKDATA \
     SSTATE_HASHEQUIV_OWNER CCACHE_TOP_DIR BB_HASHSERVE GIT_CEILING_DIRECTORIES"
-BB_HASHBASE_WHITELIST ?= "${BB_HASHEXCLUDE_COMMON} PSEUDO_IGNORE_PATHS BUILDHISTORY_DIR SSTATE_DIR "
+BB_HASHBASE_WHITELIST ?= "${BB_HASHEXCLUDE_COMMON} PSEUDO_IGNORE_PATHS \
+    PSEUDO_IGNORE_REAL_PATHS BUILDHISTORY_DIR SSTATE_DIR "
 BB_HASHCONFIG_WHITELIST ?= "${BB_HASHEXCLUDE_COMMON} DATE TIME SSH_AGENT_PID \
     SSH_AUTH_SOCK PSEUDO_BUILD BB_ENV_EXTRAWHITE DISABLE_SANITY_CHECKS \
     PARALLEL_MAKE BB_NUMBER_THREADS BB_ORIGENV BB_INVALIDCONF BBINCLUDED \
#6 lore raw

Re: [OE-core] [PATCHv2 3/4] bitbake.conf: Canonicalize paths in PSEUDO_IGNORE_PATHS

Quoted 10 lines · Peter Kjellerstedt

Introduce PSEUDO_IGNORE_REAL_PATHS and make it contain the canonicalized paths from PSEUDO_IGNORE_PATHS, obtained by passing the latter to oe.path.to_real_paths(). This is needed since pseudo's pseudo_client_ignore_path_chroot() will compare the ignored paths to paths that have been canonicalized.

---
 meta/conf/bitbake.conf | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

This looks like a good way to fix this, except I have a strong dislike of "REAL" variables.

Quoted 11 lines
meta/conf/bitbake.conf +1 -0
index 9742fe4fe2..4862095a1b 100644
@@ -685,15 +685,16 @@ SRC_URI = ""
 PSEUDO_LOCALSTATEDIR ?= "${WORKDIR}/pseudo/"
 PSEUDO_PASSWD ?= "${STAGING_DIR_TARGET}:${PSEUDO_SYSROOT}"
 PSEUDO_SYSROOT = "${COMPONENTS_DIR}/${BUILD_ARCH}/pseudo-native"
+PSEUDO_IGNORE_REAL_PATHS = "${@','.join(os.path.realpath(path) for path in (d.getVar('PSEUDO_IGNORE_PATHS') or '').split(','))}"
 PSEUDO_IGNORE_PATHS = "/usr/,/etc/,/lib,/dev/,${T},${WORKDIR}/recipe-sysroot,${SSTATE_DIR},${STAMPS_DIR},${WORKDIR}/pkgdata-sysroot,${TMPDIR}/sstate-control,${DEPLOY_DIR},${WORKDIR}/deploy-,${TMPDIR}/buildstats,${WORKDIR}/sstate-build-package_,${WORKDIR}/sstate-install-package_,${WORKDIR}/sstate-build-image_complete,${TMPDIR}/sysroots-components,${BUILDHISTORY_DIR},${TMPDIR}/pkgdata,${TOPDIR}/cache,${COREBASE}/scripts,${@','.join(d.getVar('BBLAYERS').split())},${CCACHE_DIR}"
 

meta/conf/bitbake.conf SRC_URI = "" · line 698

Quoted 10 lines
meta/conf/bitbake.conf +2 -2
 export PSEUDO_DISABLED = "1"
 #export PSEUDO_PREFIX = "${STAGING_DIR_NATIVE}${prefix_native}"
 #export PSEUDO_BINDIR = "${STAGING_DIR_NATIVE}${bindir_native}"
 #export PSEUDO_LIBDIR = "${STAGING_DIR_NATIVE}$PSEUDOBINDIR/../lib/pseudo/lib
-FAKEROOTBASEENV = "PSEUDO_BINDIR=${PSEUDO_SYSROOT}${bindir_native} PSEUDO_LIBDIR=${PSEUDO_SYSROOT}${prefix_native}/lib/pseudo/lib PSEUDO_PREFIX=${PSEUDO_SYSROOT}${prefix_native} PSEUDO_IGNORE_PATHS=${PSEUDO_IGNORE_PATHS} PSEUDO_DISABLED=1"
+FAKEROOTBASEENV = "PSEUDO_BINDIR=${PSEUDO_SYSROOT}${bindir_native} PSEUDO_LIBDIR=${PSEUDO_SYSROOT}${prefix_native}/lib/pseudo/lib PSEUDO_PREFIX=${PSEUDO_SYSROOT}${prefix_native} PSEUDO_IGNORE_PATHS=${PSEUDO_IGNORE_REAL_PATHS} PSEUDO_DISABLED=1"
 FAKEROOTCMD = "${PSEUDO_SYSROOT}${bindir_native}/pseudo"
-FAKEROOTENV = "PSEUDO_PREFIX=${PSEUDO_SYSROOT}${prefix_native} PSEUDO_LOCALSTATEDIR=${PSEUDO_LOCALSTATEDIR} PSEUDO_PASSWD=${PSEUDO_PASSWD} PSEUDO_NOSYMLINKEXP=1 PSEUDO_IGNORE_PATHS=${PSEUDO_IGNORE_PATHS} PSEUDO_DISABLED=0"
+FAKEROOTENV = "PSEUDO_PREFIX=${PSEUDO_SYSROOT}${prefix_native} PSEUDO_LOCALSTATEDIR=${PSEUDO_LOCALSTATEDIR} PSEUDO_PASSWD=${PSEUDO_PASSWD} PSEUDO_NOSYMLINKEXP=1 PSEUDO_IGNORE_PATHS=${PSEUDO_IGNORE_REAL_PATHS} PSEUDO_DISABLED=0"
 

I'm wondering if we create a function in lib/oe/utils.py and then use something like:

PSEUDO_IGNORE_PATHS=${@oe.utils.realpath("PSEUDO_IGNORE_REAL_PATHS")}

?

Cheers,

Richard

#7 lore raw

Re: [OE-core] [PATCHv2 3/4] bitbake.conf: Canonicalize paths in PSEUDO_IGNORE_PATHS

Quoted 16 lines

-----Original Message-----
From: Richard Purdie <[email protected]>
Sent: den 26 november 2020 15:47
To: Peter Kjellerstedt <[email protected]>; openembedded- [email protected]
Subject: Re: [OE-core] [PATCHv2 3/4] bitbake.conf: Canonicalize paths in PSEUDO_IGNORE_PATHS

Quoted 1 line · Peter Kjellerstedt

Introduce PSEUDO_IGNORE_REAL_PATHS and make it contain the

canonicalized

Quoted 5 lines

paths from PSEUDO_IGNORE_PATHS, obtained by passing the latter to oe.path.to_real_paths(). This is needed since pseudo's pseudo_client_ignore_path_chroot() will compare the ignored paths to paths that have been canonicalized.

Quoted 7 lines
Quoted 4 lines
---
 meta/conf/bitbake.conf | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

This looks like a good way to fix this, except I have a strong dislike of "REAL" variables.

Well, I named it PSEUDO_IGNORE_REAL_PATHS to match os.path.realpath(). I can name it, e.g., PSEUDO_IGNORE_CANONICAL_PATHS if you prefer that?

Quoted 44 lines

meta/conf/bitbake.conf SRC_URI = "" · line 688

Quoted 10 lines
meta/conf/bitbake.conf +1 -0
index 9742fe4fe2..4862095a1b 100644
@@ -685,15 +685,16 @@ SRC_URI = ""
 PSEUDO_LOCALSTATEDIR ?= "${WORKDIR}/pseudo/"
 PSEUDO_PASSWD ?= "${STAGING_DIR_TARGET}:${PSEUDO_SYSROOT}"
 PSEUDO_SYSROOT = "${COMPONENTS_DIR}/${BUILD_ARCH}/pseudo-native"
+PSEUDO_IGNORE_REAL_PATHS = "${@','.join(os.path.realpath(path) for

path in (d.getVar('PSEUDO_IGNORE_PATHS') or '').split(','))}"

meta/conf/bitbake.conf SRC_URI = "" · line 689

Quoted 1 line
meta/conf/bitbake.conf +0 -0
 PSEUDO_IGNORE_PATHS = "/usr/,/etc/,/lib,/dev/,${T},${WORKDIR}/recipe-

sysroot,${SSTATE_DIR},${STAMPS_DIR},${WORKDIR}/pkgdata- sysroot,${TMPDIR}/sstate-control,${DEPLOY_DIR},${WORKDIR}/deploy- ,${TMPDIR}/buildstats,${WORKDIR}/sstate-build-package_,${WORKDIR}/sstate- install-package_,${WORKDIR}/sstate-build- image_complete,${TMPDIR}/sysroots- components,${BUILDHISTORY_DIR},${TMPDIR}/pkgdata,${TOPDIR}/cache,${COREBA SE}/scripts,${@','.join(d.getVar('BBLAYERS').split())},${CCACHE_DIR}"

meta/conf/bitbake.conf SRC_URI = ""

Quoted 5 lines
meta/conf/bitbake.conf +0 -0
 export PSEUDO_DISABLED = "1"
 #export PSEUDO_PREFIX = "${STAGING_DIR_NATIVE}${prefix_native}"
 #export PSEUDO_BINDIR = "${STAGING_DIR_NATIVE}${bindir_native}"
 #export PSEUDO_LIBDIR =

"${STAGING_DIR_NATIVE}$PSEUDOBINDIR/../lib/pseudo/lib

meta/conf/bitbake.conf SRC_URI = "" · line 693

Quoted 1 line
meta/conf/bitbake.conf +0 -1
-FAKEROOTBASEENV = "PSEUDO_BINDIR=${PSEUDO_SYSROOT}${bindir_native}

PSEUDO_LIBDIR=${PSEUDO_SYSROOT}${prefix_native}/lib/pseudo/lib PSEUDO_PREFIX=${PSEUDO_SYSROOT}${prefix_native} PSEUDO_IGNORE_PATHS=${PSEUDO_IGNORE_PATHS} PSEUDO_DISABLED=1"

meta/conf/bitbake.conf SRC_URI = "" · line 694

Quoted 1 line
meta/conf/bitbake.conf +1 -0
+FAKEROOTBASEENV = "PSEUDO_BINDIR=${PSEUDO_SYSROOT}${bindir_native}

PSEUDO_LIBDIR=${PSEUDO_SYSROOT}${prefix_native}/lib/pseudo/lib PSEUDO_PREFIX=${PSEUDO_SYSROOT}${prefix_native} PSEUDO_IGNORE_PATHS=${PSEUDO_IGNORE_REAL_PATHS} PSEUDO_DISABLED=1"

meta/conf/bitbake.conf SRC_URI = "" · line 695

Quoted 2 lines
meta/conf/bitbake.conf +0 -1
 FAKEROOTCMD = "${PSEUDO_SYSROOT}${bindir_native}/pseudo"
-FAKEROOTENV = "PSEUDO_PREFIX=${PSEUDO_SYSROOT}${prefix_native}

PSEUDO_LOCALSTATEDIR=${PSEUDO_LOCALSTATEDIR} PSEUDO_PASSWD=${PSEUDO_PASSWD} PSEUDO_NOSYMLINKEXP=1 PSEUDO_IGNORE_PATHS=${PSEUDO_IGNORE_PATHS} PSEUDO_DISABLED=0"

meta/conf/bitbake.conf SRC_URI = "" · line 696

Quoted 1 line
meta/conf/bitbake.conf +1 -0
+FAKEROOTENV = "PSEUDO_PREFIX=${PSEUDO_SYSROOT}${prefix_native}

PSEUDO_LOCALSTATEDIR=${PSEUDO_LOCALSTATEDIR} PSEUDO_PASSWD=${PSEUDO_PASSWD} PSEUDO_NOSYMLINKEXP=1 PSEUDO_IGNORE_PATHS=${PSEUDO_IGNORE_REAL_PATHS} PSEUDO_DISABLED=0"

Quoted 1 line
Quoted 4 lines

I'm wondering if we create a function in lib/oe/utils.py and then use something like:

PSEUDO_IGNORE_PATHS=${@oe.utils.realpath("PSEUDO_IGNORE_REAL_PATHS")}

There already is an oe.path.realpath(). However, I am not sure when it is expected to be used instead of os.path.realpath().

I initially created an oe.path.realpaths(d, var, separator=','), but after I had rewritten it a couple of times I was down to just the single list comprehension and then I felt as I could just as well use it directly since I don't see a great need for a common function to canonicalize a list of paths. But if you prefer that solution, I can restore it.

Quoted 6 lines

?

Cheers,

Richard

//Peter

#8 lore raw

Re: [OE-core] [PATCHv2 3/4] bitbake.conf: Canonicalize paths in PSEUDO_IGNORE_PATHS

Quoted 23 lines · Peter Kjellerstedt
Quoted 23 lines

-----Original Message-----
From: Richard Purdie <[email protected]>
Sent: den 26 november 2020 15:47
To: Peter Kjellerstedt <[email protected]>; openembedded- [email protected]
Subject: Re: [OE-core] [PATCHv2 3/4] bitbake.conf: Canonicalize paths in
PSEUDO_IGNORE_PATHS

Quoted 1 line · Peter Kjellerstedt

Introduce PSEUDO_IGNORE_REAL_PATHS and make it contain the

canonicalized

Quoted 10 lines

paths from PSEUDO_IGNORE_PATHS, obtained by passing the latter to oe.path.to_real_paths(). This is needed since pseudo's pseudo_client_ignore_path_chroot() will compare the ignored paths to
paths that have been canonicalized.

---
 meta/conf/bitbake.conf | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
Quoted 8 lines
Quoted 3 lines

This looks like a good way to fix this, except I have a strong dislike
of "REAL" variables.

Well, I named it PSEUDO_IGNORE_REAL_PATHS to match os.path.realpath().
I can name it, e.g., PSEUDO_IGNORE_CANONICAL_PATHS if you prefer that?

My point was less about the name and more that I'd prefer not to have an indirect variable which isn't useful.

Quoted 41 lines
Quoted 41 lines

meta/conf/bitbake.conf SRC_URI = "" · line 688

Quoted 9 lines
meta/conf/bitbake.conf +1 -0
index 9742fe4fe2..4862095a1b 100644
@@ -685,15 +685,16 @@ SRC_URI = ""
 PSEUDO_LOCALSTATEDIR ?= "${WORKDIR}/pseudo/"
 PSEUDO_PASSWD ?= "${STAGING_DIR_TARGET}:${PSEUDO_SYSROOT}"
 PSEUDO_SYSROOT = "${COMPONENTS_DIR}/${BUILD_ARCH}/pseudo-native"
+PSEUDO_IGNORE_REAL_PATHS = "${@','.join(os.path.realpath(path) for

path in (d.getVar('PSEUDO_IGNORE_PATHS') or '').split(','))}"

meta/conf/bitbake.conf SRC_URI = "" · line 689

Quoted 1 line
meta/conf/bitbake.conf +0 -0
 PSEUDO_IGNORE_PATHS = "/usr/,/etc/,/lib,/dev/,${T},${WORKDIR}/recipe-

sysroot,${SSTATE_DIR},${STAMPS_DIR},${WORKDIR}/pkgdata- sysroot,${TMPDIR}/sstate-control,${DEPLOY_DIR},${WORKDIR}/deploy- ,${TMPDIR}/buildstats,${WORKDIR}/sstate-build-package_,${WORKDIR}/sstate- install-package_,${WORKDIR}/sstate-build- image_complete,${TMPDIR}/sysroots- components,${BUILDHISTORY_DIR},${TMPDIR}/pkgdata,${TOPDIR}/cache,${COREBA SE}/scripts,${@','.join(d.getVar('BBLAYERS').split())},${CCACHE_DIR}"

meta/conf/bitbake.conf SRC_URI = ""

Quoted 4 lines
meta/conf/bitbake.conf +0 -0
 export PSEUDO_DISABLED = "1"
 #export PSEUDO_PREFIX = "${STAGING_DIR_NATIVE}${prefix_native}"
 #export PSEUDO_BINDIR = "${STAGING_DIR_NATIVE}${bindir_native}"
 #export PSEUDO_LIBDIR =

"${STAGING_DIR_NATIVE}$PSEUDOBINDIR/../lib/pseudo/lib

meta/conf/bitbake.conf SRC_URI = "" · line 693

Quoted 1 line
meta/conf/bitbake.conf +0 -1
-FAKEROOTBASEENV = "PSEUDO_BINDIR=${PSEUDO_SYSROOT}${bindir_native}

PSEUDO_LIBDIR=${PSEUDO_SYSROOT}${prefix_native}/lib/pseudo/lib PSEUDO_PREFIX=${PSEUDO_SYSROOT}${prefix_native} PSEUDO_IGNORE_PATHS=${PSEUDO_IGNORE_PATHS} PSEUDO_DISABLED=1"

meta/conf/bitbake.conf SRC_URI = "" · line 694

Quoted 1 line
meta/conf/bitbake.conf +1 -0
+FAKEROOTBASEENV = "PSEUDO_BINDIR=${PSEUDO_SYSROOT}${bindir_native}

PSEUDO_LIBDIR=${PSEUDO_SYSROOT}${prefix_native}/lib/pseudo/lib PSEUDO_PREFIX=${PSEUDO_SYSROOT}${prefix_native} PSEUDO_IGNORE_PATHS=${PSEUDO_IGNORE_REAL_PATHS} PSEUDO_DISABLED=1"

meta/conf/bitbake.conf SRC_URI = "" · line 695

Quoted 2 lines
meta/conf/bitbake.conf +0 -1
 FAKEROOTCMD = "${PSEUDO_SYSROOT}${bindir_native}/pseudo"
-FAKEROOTENV = "PSEUDO_PREFIX=${PSEUDO_SYSROOT}${prefix_native}

PSEUDO_LOCALSTATEDIR=${PSEUDO_LOCALSTATEDIR} PSEUDO_PASSWD=${PSEUDO_PASSWD} PSEUDO_NOSYMLINKEXP=1 PSEUDO_IGNORE_PATHS=${PSEUDO_IGNORE_PATHS} PSEUDO_DISABLED=0"

meta/conf/bitbake.conf SRC_URI = "" · line 696

Quoted 1 line
meta/conf/bitbake.conf +1 -0
+FAKEROOTENV = "PSEUDO_PREFIX=${PSEUDO_SYSROOT}${prefix_native}

PSEUDO_LOCALSTATEDIR=${PSEUDO_LOCALSTATEDIR} PSEUDO_PASSWD=${PSEUDO_PASSWD} PSEUDO_NOSYMLINKEXP=1 PSEUDO_IGNORE_PATHS=${PSEUDO_IGNORE_REAL_PATHS} PSEUDO_DISABLED=0"

Quoted 7 lines
Quoted 4 lines

I'm wondering if we create a function in lib/oe/utils.py and then use something like:

PSEUDO_IGNORE_PATHS=${@oe.utils.realpath("PSEUDO_IGNORE_REAL_PATHS")}

There already is an oe.path.realpath(). However, I am not sure when it is expected to be used instead of os.path.realpath().

That is a bad choice of name now I look at it! I was thinking more about the use of a function.

Quoted 6 lines

I initially created an oe.path.realpaths(d, var, separator=','), but after I had rewritten it a couple of times I was down to just the single list comprehension and then I felt as I could just as well use it directly since I don't see a great need for a common function to canonicalize a list of paths. But if you prefer that solution, I can restore it.

I don't really mind the list comprehension directly or a function, I'd just prefer not to have an indirect variable around to confuse people.

Cheers,

Richard

#9 lore raw

Re: [OE-core] [PATCHv2 3/4] bitbake.conf: Canonicalize paths in PSEUDO_IGNORE_PATHS

Quoted 107 lines

-----Original Message-----
From: [email protected] <openembedded- [email protected]> On Behalf Of Richard Purdie
Sent: den 27 november 2020 00:42
To: Peter Kjellerstedt <[email protected]>; openembedded- [email protected]
Subject: Re: [OE-core] [PATCHv2 3/4] bitbake.conf: Canonicalize paths in PSEUDO_IGNORE_PATHS

Quoted 23 lines · Peter Kjellerstedt
Quoted 23 lines

-----Original Message-----
From: Richard Purdie <[email protected]>
Sent: den 26 november 2020 15:47
To: Peter Kjellerstedt <[email protected]>; openembedded- [email protected]
Subject: Re: [OE-core] [PATCHv2 3/4] bitbake.conf: Canonicalize paths in
PSEUDO_IGNORE_PATHS

Quoted 1 line · Peter Kjellerstedt

Introduce PSEUDO_IGNORE_REAL_PATHS and make it contain the

canonicalized

Quoted 10 lines

paths from PSEUDO_IGNORE_PATHS, obtained by passing the latter to oe.path.to_real_paths(). This is needed since pseudo's pseudo_client_ignore_path_chroot() will compare the ignored paths to
paths that have been canonicalized.

---
 meta/conf/bitbake.conf | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
Quoted 8 lines
Quoted 3 lines

This looks like a good way to fix this, except I have a strong dislike
of "REAL" variables.

Well, I named it PSEUDO_IGNORE_REAL_PATHS to match os.path.realpath().
I can name it, e.g., PSEUDO_IGNORE_CANONICAL_PATHS if you prefer that?

My point was less about the name and more that I'd prefer not to have an indirect variable which isn't useful.

Quoted 11 lines
Quoted 11 lines

meta/conf/bitbake.conf SRC_URI = "" · line 688

Quoted 9 lines
meta/conf/bitbake.conf +1 -0
index 9742fe4fe2..4862095a1b 100644
@@ -685,15 +685,16 @@ SRC_URI = ""
 PSEUDO_LOCALSTATEDIR ?= "${WORKDIR}/pseudo/"
 PSEUDO_PASSWD ?= "${STAGING_DIR_TARGET}:${PSEUDO_SYSROOT}"
 PSEUDO_SYSROOT = "${COMPONENTS_DIR}/${BUILD_ARCH}/pseudo-native"
+PSEUDO_IGNORE_REAL_PATHS = "${@','.join(os.path.realpath(path) for

path in (d.getVar('PSEUDO_IGNORE_PATHS') or '').split(','))}"

Quoted 1 line
meta/conf/bitbake.conf +0 -0
 PSEUDO_IGNORE_PATHS =

"/usr/,/etc/,/lib,/dev/,${T},${WORKDIR}/recipe-

Quoted 3 lines
Quoted 3 lines

sysroot,${SSTATE_DIR},${STAMPS_DIR},${WORKDIR}/pkgdata- sysroot,${TMPDIR}/sstate-control,${DEPLOY_DIR},${WORKDIR}/deploy- ,${TMPDIR}/buildstats,${WORKDIR}/sstate-build-

package_,${WORKDIR}/sstate-

Quoted 3 lines
Quoted 3 lines

install-package_,${WORKDIR}/sstate-build-
image_complete,${TMPDIR}/sysroots-

components,${BUILDHISTORY_DIR},${TMPDIR}/pkgdata,${TOPDIR}/cache,${COREBA

Quoted 24 lines
Quoted 24 lines

SE}/scripts,${@','.join(d.getVar('BBLAYERS').split())},${CCACHE_DIR}"

meta/conf/bitbake.conf SRC_URI = ""

Quoted 4 lines
meta/conf/bitbake.conf +0 -0
 export PSEUDO_DISABLED = "1"
 #export PSEUDO_PREFIX = "${STAGING_DIR_NATIVE}${prefix_native}"
 #export PSEUDO_BINDIR = "${STAGING_DIR_NATIVE}${bindir_native}"
 #export PSEUDO_LIBDIR =

"${STAGING_DIR_NATIVE}$PSEUDOBINDIR/../lib/pseudo/lib

meta/conf/bitbake.conf SRC_URI = "" · line 693

Quoted 1 line
meta/conf/bitbake.conf +0 -1
-FAKEROOTBASEENV = "PSEUDO_BINDIR=${PSEUDO_SYSROOT}${bindir_native}

PSEUDO_LIBDIR=${PSEUDO_SYSROOT}${prefix_native}/lib/pseudo/lib PSEUDO_PREFIX=${PSEUDO_SYSROOT}${prefix_native} PSEUDO_IGNORE_PATHS=${PSEUDO_IGNORE_PATHS} PSEUDO_DISABLED=1"

meta/conf/bitbake.conf SRC_URI = "" · line 694

Quoted 1 line
meta/conf/bitbake.conf +1 -0
+FAKEROOTBASEENV = "PSEUDO_BINDIR=${PSEUDO_SYSROOT}${bindir_native}

PSEUDO_LIBDIR=${PSEUDO_SYSROOT}${prefix_native}/lib/pseudo/lib PSEUDO_PREFIX=${PSEUDO_SYSROOT}${prefix_native} PSEUDO_IGNORE_PATHS=${PSEUDO_IGNORE_REAL_PATHS} PSEUDO_DISABLED=1"

meta/conf/bitbake.conf SRC_URI = "" · line 695

Quoted 2 lines
meta/conf/bitbake.conf +0 -1
 FAKEROOTCMD = "${PSEUDO_SYSROOT}${bindir_native}/pseudo"
-FAKEROOTENV = "PSEUDO_PREFIX=${PSEUDO_SYSROOT}${prefix_native}

PSEUDO_LOCALSTATEDIR=${PSEUDO_LOCALSTATEDIR} PSEUDO_PASSWD=${PSEUDO_PASSWD} PSEUDO_NOSYMLINKEXP=1 PSEUDO_IGNORE_PATHS=${PSEUDO_IGNORE_PATHS} PSEUDO_DISABLED=0"

meta/conf/bitbake.conf SRC_URI = "" · line 696

Quoted 1 line
meta/conf/bitbake.conf +1 -0
+FAKEROOTENV = "PSEUDO_PREFIX=${PSEUDO_SYSROOT}${prefix_native}

PSEUDO_LOCALSTATEDIR=${PSEUDO_LOCALSTATEDIR} PSEUDO_PASSWD=${PSEUDO_PASSWD} PSEUDO_NOSYMLINKEXP=1 PSEUDO_IGNORE_PATHS=${PSEUDO_IGNORE_REAL_PATHS} PSEUDO_DISABLED=0"

Quoted 7 lines
Quoted 4 lines

I'm wondering if we create a function in lib/oe/utils.py and then use something like:

PSEUDO_IGNORE_PATHS=${@oe.utils.realpath("PSEUDO_IGNORE_REAL_PATHS")}

There already is an oe.path.realpath(). However, I am not sure when it is expected to be used instead of os.path.realpath().

That is a bad choice of name now I look at it! I was thinking more about the use of a function.

Quoted 6 lines

I initially created an oe.path.realpaths(d, var, separator=','), but after I had rewritten it a couple of times I was down to just the single list comprehension and then I felt as I could just as well use it directly since I don't see a great need for a common function to canonicalize a list of paths. But if you prefer that solution, I can restore it.

Quoted 2 lines

I don't really mind the list comprehension directly or a function, I'd just prefer not to have an indirect variable around to confuse people.

Ok, I take that to mean that you instead want me to, e.g., use a function to canonicalize the paths where ${PSEUDO_IGNORE_REAL_PATHS} is used now? I.e., something like this (line broken here for readability):

  FAKEROOTENV = "PSEUDO_PREFIX=${PSEUDO_SYSROOT}${prefix_native} \
                 PSEUDO_LOCALSTATEDIR=${PSEUDO_LOCALSTATEDIR} \
                 PSEUDO_PASSWD=${PSEUDO_PASSWD} \
                 PSEUDO_NOSYMLINKEXP=1 \
                 PSEUDO_IGNORE_PATHS=${@oe.paths.canonicalize(d, 'PSEUDO_IGNORE_PATHS')} \
                 PSEUDO_DISABLED=0"
  FAKEROOTENV[vardeps] += "PSEUDO_IGNORE_PATHS"

alternatively:

  FAKEROOTENV = "PSEUDO_PREFIX=${PSEUDO_SYSROOT}${prefix_native} \
                 PSEUDO_LOCALSTATEDIR=${PSEUDO_LOCALSTATEDIR} \
                 PSEUDO_PASSWD=${PSEUDO_PASSWD} \
                 PSEUDO_NOSYMLINKEXP=1 \
                 PSEUDO_IGNORE_PATHS=${@oe.paths.canonicalize(d.getVar('PSEUDO_IGNORE_PATHS'))} \
                 PSEUDO_DISABLED=0"

I am not sure whether there is a preference for utility functions to take d and the name of a bitbake variable as arguments, or to take the variable's value. The former is a little bit shorter, but on the other hand it requires the extra vardeps declaration.

There is a minor drawback with this solution, and that is that I cannot use that function in wic. So I will use the list comprehension directly, with a comment referring to the oe.path.canonicalize() function.

Quoted 3 lines

Cheers,

Richard

//Peter

#10 lore raw

Re: [OE-core] [PATCHv2 3/4] bitbake.conf: Canonicalize paths in PSEUDO_IGNORE_PATHS

Quoted 136 lines · Peter Kjellerstedt
Quoted 107 lines

-----Original Message-----
From: [email protected] <openembedded- [email protected]> On Behalf Of Richard Purdie
Sent: den 27 november 2020 00:42
To: Peter Kjellerstedt <[email protected]>; openembedded- [email protected]
Subject: Re: [OE-core] [PATCHv2 3/4] bitbake.conf: Canonicalize paths in PSEUDO_IGNORE_PATHS

Quoted 23 lines · Peter Kjellerstedt
Quoted 23 lines

-----Original Message-----
From: Richard Purdie <[email protected]>
Sent: den 26 november 2020 15:47
To: Peter Kjellerstedt <[email protected]>; openembedded- [email protected]
Subject: Re: [OE-core] [PATCHv2 3/4] bitbake.conf: Canonicalize paths in
PSEUDO_IGNORE_PATHS

Quoted 1 line · Peter Kjellerstedt

Introduce PSEUDO_IGNORE_REAL_PATHS and make it contain the

canonicalized

Quoted 10 lines

paths from PSEUDO_IGNORE_PATHS, obtained by passing the latter to oe.path.to_real_paths(). This is needed since pseudo's pseudo_client_ignore_path_chroot() will compare the ignored paths to
paths that have been canonicalized.

---
 meta/conf/bitbake.conf | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
Quoted 8 lines
Quoted 3 lines

This looks like a good way to fix this, except I have a strong dislike
of "REAL" variables.

Well, I named it PSEUDO_IGNORE_REAL_PATHS to match os.path.realpath().
I can name it, e.g., PSEUDO_IGNORE_CANONICAL_PATHS if you prefer that?

My point was less about the name and more that I'd prefer not to have an indirect variable which isn't useful.

Quoted 11 lines
Quoted 11 lines

meta/conf/bitbake.conf SRC_URI = "" · line 688

Quoted 9 lines
meta/conf/bitbake.conf +1 -0
index 9742fe4fe2..4862095a1b 100644
@@ -685,15 +685,16 @@ SRC_URI = ""
 PSEUDO_LOCALSTATEDIR ?= "${WORKDIR}/pseudo/"
 PSEUDO_PASSWD ?= "${STAGING_DIR_TARGET}:${PSEUDO_SYSROOT}"
 PSEUDO_SYSROOT = "${COMPONENTS_DIR}/${BUILD_ARCH}/pseudo-native"
+PSEUDO_IGNORE_REAL_PATHS = "${@','.join(os.path.realpath(path) for

path in (d.getVar('PSEUDO_IGNORE_PATHS') or '').split(','))}"

Quoted 1 line
meta/conf/bitbake.conf +0 -0
 PSEUDO_IGNORE_PATHS =

"/usr/,/etc/,/lib,/dev/,${T},${WORKDIR}/recipe-

Quoted 3 lines
Quoted 3 lines

sysroot,${SSTATE_DIR},${STAMPS_DIR},${WORKDIR}/pkgdata- sysroot,${TMPDIR}/sstate-control,${DEPLOY_DIR},${WORKDIR}/deploy- ,${TMPDIR}/buildstats,${WORKDIR}/sstate-build-

package_,${WORKDIR}/sstate-

Quoted 3 lines
Quoted 3 lines

install-package_,${WORKDIR}/sstate-build-
image_complete,${TMPDIR}/sysroots-

components,${BUILDHISTORY_DIR},${TMPDIR}/pkgdata,${TOPDIR}/cache,${COREBA

Quoted 24 lines
Quoted 24 lines

SE}/scripts,${@','.join(d.getVar('BBLAYERS').split())},${CCACHE_DIR}"

meta/conf/bitbake.conf SRC_URI = ""

Quoted 4 lines
meta/conf/bitbake.conf +0 -0
 export PSEUDO_DISABLED = "1"
 #export PSEUDO_PREFIX = "${STAGING_DIR_NATIVE}${prefix_native}"
 #export PSEUDO_BINDIR = "${STAGING_DIR_NATIVE}${bindir_native}"
 #export PSEUDO_LIBDIR =

"${STAGING_DIR_NATIVE}$PSEUDOBINDIR/../lib/pseudo/lib

meta/conf/bitbake.conf SRC_URI = "" · line 693

Quoted 1 line
meta/conf/bitbake.conf +0 -1
-FAKEROOTBASEENV = "PSEUDO_BINDIR=${PSEUDO_SYSROOT}${bindir_native}

PSEUDO_LIBDIR=${PSEUDO_SYSROOT}${prefix_native}/lib/pseudo/lib PSEUDO_PREFIX=${PSEUDO_SYSROOT}${prefix_native} PSEUDO_IGNORE_PATHS=${PSEUDO_IGNORE_PATHS} PSEUDO_DISABLED=1"

meta/conf/bitbake.conf SRC_URI = "" · line 694

Quoted 1 line
meta/conf/bitbake.conf +1 -0
+FAKEROOTBASEENV = "PSEUDO_BINDIR=${PSEUDO_SYSROOT}${bindir_native}

PSEUDO_LIBDIR=${PSEUDO_SYSROOT}${prefix_native}/lib/pseudo/lib PSEUDO_PREFIX=${PSEUDO_SYSROOT}${prefix_native} PSEUDO_IGNORE_PATHS=${PSEUDO_IGNORE_REAL_PATHS} PSEUDO_DISABLED=1"

meta/conf/bitbake.conf SRC_URI = "" · line 695

Quoted 2 lines
meta/conf/bitbake.conf +0 -1
 FAKEROOTCMD = "${PSEUDO_SYSROOT}${bindir_native}/pseudo"
-FAKEROOTENV = "PSEUDO_PREFIX=${PSEUDO_SYSROOT}${prefix_native}

PSEUDO_LOCALSTATEDIR=${PSEUDO_LOCALSTATEDIR} PSEUDO_PASSWD=${PSEUDO_PASSWD} PSEUDO_NOSYMLINKEXP=1 PSEUDO_IGNORE_PATHS=${PSEUDO_IGNORE_PATHS} PSEUDO_DISABLED=0"

meta/conf/bitbake.conf SRC_URI = "" · line 696

Quoted 1 line
meta/conf/bitbake.conf +1 -0
+FAKEROOTENV = "PSEUDO_PREFIX=${PSEUDO_SYSROOT}${prefix_native}

PSEUDO_LOCALSTATEDIR=${PSEUDO_LOCALSTATEDIR} PSEUDO_PASSWD=${PSEUDO_PASSWD} PSEUDO_NOSYMLINKEXP=1 PSEUDO_IGNORE_PATHS=${PSEUDO_IGNORE_REAL_PATHS} PSEUDO_DISABLED=0"

Quoted 7 lines
Quoted 4 lines

I'm wondering if we create a function in lib/oe/utils.py and then use something like:

PSEUDO_IGNORE_PATHS=${@oe.utils.realpath("PSEUDO_IGNORE_REAL_PATHS")}

There already is an oe.path.realpath(). However, I am not sure when it is expected to be used instead of os.path.realpath().

That is a bad choice of name now I look at it! I was thinking more about the use of a function.

Quoted 6 lines

I initially created an oe.path.realpaths(d, var, separator=','), but after I had rewritten it a couple of times I was down to just the single list comprehension and then I felt as I could just as well use it directly since I don't see a great need for a common function to canonicalize a list of paths. But if you prefer that solution, I can restore it.

Quoted 2 lines

I don't really mind the list comprehension directly or a function, I'd just prefer not to have an indirect variable around to confuse people.

Ok, I take that to mean that you instead want me to, e.g., use a function to canonicalize the paths where ${PSEUDO_IGNORE_REAL_PATHS} is used now? I.e., something like this (line broken here for readability):

  FAKEROOTENV = "PSEUDO_PREFIX=${PSEUDO_SYSROOT}${prefix_native} \
                 PSEUDO_LOCALSTATEDIR=${PSEUDO_LOCALSTATEDIR} \
                 PSEUDO_PASSWD=${PSEUDO_PASSWD} \
                 PSEUDO_NOSYMLINKEXP=1 \
                 PSEUDO_IGNORE_PATHS=${@oe.paths.canonicalize(d, 'PSEUDO_IGNORE_PATHS')} \
                 PSEUDO_DISABLED=0"
  FAKEROOTENV[vardeps] += "PSEUDO_IGNORE_PATHS"

alternatively:

  FAKEROOTENV = "PSEUDO_PREFIX=${PSEUDO_SYSROOT}${prefix_native} \
                 PSEUDO_LOCALSTATEDIR=${PSEUDO_LOCALSTATEDIR} \
                 PSEUDO_PASSWD=${PSEUDO_PASSWD} \
                 PSEUDO_NOSYMLINKEXP=1 \
                 PSEUDO_IGNORE_PATHS=${@oe.paths.canonicalize(d.getVar('PSEUDO_IGNORE_PATHS'))} \
                 PSEUDO_DISABLED=0"

I am not sure whether there is a preference for utility functions to take d and the name of a bitbake variable as arguments, or to take the variable's value. The former is a little bit shorter, but on the other hand it requires the extra vardeps declaration.

Quoted 4 lines

There is a minor drawback with this solution, and that is that I cannot use that function in wic. So I will use the list comprehension directly, with a comment referring to the oe.path.canonicalize() function.

The version which avoids the vardeps is probably slightly neater overall. I wasn't sure if using the list comprehension here would be too ugly or not but I'm happy just not to have another variable name used here, thanks!

The wic issue is unfortunate but hard to avoid.

Cheers,

Richard

#11 lore raw

PATCH v2 4/4 wic: Pass canonicalized paths in PSEUDO_IGNORE_PATHS

In reply to Peter Kjellerstedt
---
 meta/classes/image_types_wic.bbclass | 2 +-
 scripts/lib/wic/partition.py         | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
meta/classes/image_types_wic.bbclass +1 -1
index 286e0f5d54..9c1605c29c 100644
@@ -5,7 +5,7 @@ WICVARS ?= "\
            IMAGE_LINK_NAME IMAGE_ROOTFS INITRAMFS_FSTYPES INITRD INITRD_LIVE ISODIR RECIPE_SYSROOT_NATIVE \
            ROOTFS_SIZE STAGING_DATADIR STAGING_DIR STAGING_LIBDIR TARGET_SYS \
            KERNEL_IMAGETYPE MACHINE INITRAMFS_IMAGE INITRAMFS_IMAGE_BUNDLE INITRAMFS_LINK_NAME APPEND \
-           ASSUME_PROVIDED PSEUDO_IGNORE_PATHS"
+           ASSUME_PROVIDED PSEUDO_IGNORE_REAL_PATHS"
 
 inherit ${@bb.utils.contains('INITRAMFS_IMAGE_BUNDLE', '1', 'kernel-artifact-names', '', d)}
 
scripts/lib/wic/partition.py +1 -1
index ebe250b00d..e910fd3fa0 100644
@@ -211,7 +211,7 @@ class Partition():
             pseudo += "export PSEUDO_LOCALSTATEDIR=%s;" % pseudo_dir
             pseudo += "export PSEUDO_PASSWD=%s;" % rootfs_dir
             pseudo += "export PSEUDO_NOSYMLINKEXP=1;"
-            pseudo += "export PSEUDO_IGNORE_PATHS=%s;" % (rootfs + "," + (get_bitbake_var("PSEUDO_IGNORE_PATHS") or ""))
+            pseudo += "export PSEUDO_IGNORE_PATHS=%s;" % (os.path.realpath(rootfs) + "," + (get_bitbake_var("PSEUDO_IGNORE_REAL_PATHS") or ""))
             pseudo += "%s " % get_bitbake_var("FAKEROOTCMD")
         else:
             pseudo = None