glib-2.0: call os.path.normpath on THISDIR
PATCH 1/1 glib-2.0: call os.path.normpath on THISDIR
* some build environments have relative paths in THISDIR, e.g. from OEROOT set in:
https://github.com/96boards/oe-rpb-manifest/blob/1e3345c26c56f77f3a15a3978f412a25955d2606/conf/bblayers.conf#L4
and then the paths in filename normalized in:
filename = os.path.normpath(os.path.join(path, meson.cross.d, element)) don't match.
* COREBASE used here before didn't have this issue because the value is already normalized when set in:
meta/conf/layer.conf:COREBASE = '${@os.path.normpath("${LAYERDIR}/../")}'
--- meta/recipes-core/glib-2.0/glib-2.0_2.66.7.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-core/glib-2.0/glib-2.0_2.66.7.bb +1 -1
@@ -31,7 +31,7 @@ def find_meson_cross_files(d): if bb.data.inherits_class('native', d): return "" - thisdir = d.getVar("THISDIR") + thisdir = os.path.normpath(d.getVar("THISDIR")) import collections sitedata = siteinfo_data(d) # filename -> found
Signature · 1 line
2.30.0
Quoted 15 lines
-----Original Message-----
From: [email protected] <openembedded- [email protected]> On Behalf Of Martin Jansa
Sent: den 2 mars 2021 20:32
To: [email protected]
Cc: [email protected]; [email protected]; Martin Jansa <[email protected]>
Subject: [OE-core] [PATCH] glib-2.0: call os.path.normpath on THISDIR* some build environments have relative paths in THISDIR, e.g. from OEROOT set in:
https://github.com/96boards/oe-rpb-manifest/blob/1e3345c26c56f77f3a15a3978f412a25955d2606/conf/bblayers.conf#L4
and then the paths in filename normalized in:
filename =os.path.normpath(os.path.join(path, meson.cross.d, element))don't match.
Quoted 3 lines
* COREBASE used here before didn't have this issue because the value is already normalized when set in:
meta/conf/layer.conf:COREBASE = '${@os.path.normpath("${LAYERDIR}/../")}'
That is not the only problem with the change to use ${THISDIR}. We get a million errors because we have bbappend files for glib-2.0. This is because bitbake will now give an error for any potential cross-files from those layers, regardless if they exist or not (they don't).
I will send a patch that I believe solves this in a way that also avoids the need for the bb.error().
//Peter
Quoted 22 lines
Signed-off-by Martin Jansa <[email protected]>--- meta/recipes-core/glib-2.0/glib-2.0_2.66.7.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)meta/recipes- renamed from meta/recipes-core/glib-2.0/glib-2.0_2.66.7.bb +0
-0core/glib-2.0/glib-2.0_2.66.7.bb
index882a89da7a..a2fa5345b4100644meta/recipes-core/glib-2.0/glib-2.0_2.66.7.bb +1
-2@@ -31,7 +31,7 @@ def find_meson_cross_files(d): if bb.data.inherits_class('native', d): return "" - thisdir = d.getVar("THISDIR") + thisdir = os.path.normpath(d.getVar("THISDIR")) import collections sitedata = siteinfo_data(d) # filename -> found --2.30.0
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 30 lines
-----Original Message-----
From: [email protected] <[email protected]> On Behalf Of Peter Kjellerstedt
Sent: den 5 mars 2021 11:54
To: Martin Jansa <[email protected]>; [email protected]
Cc: [email protected]; [email protected]
Subject: Re: [OE-core] [PATCH] glib-2.0: call os.path.normpath on THISDIRQuoted 13 lines
-----Original Message-----
From: [email protected] <[email protected]> On Behalf Of Martin Jansa
Sent: den 2 mars 2021 20:32
To: [email protected]
Cc: [email protected]; [email protected]; Martin Jansa <[email protected]>
Subject: [OE-core] [PATCH] glib-2.0: call os.path.normpath on THISDIR* some build environments have relative paths in THISDIR, e.g. from OEROOT set in:
https://github.com/96boards/oe-rpb-manifest/blob/1e3345c26c56f77f3a15a3978f412a25955d2606/conf/bblayers.conf#L4
and then the paths in filename normalized in:
filename =os.path.normpath(os.path.join(path, meson.cross.d, element))don't match.Quoted 3 lines
* COREBASE used here before didn't have this issue because the value is already normalized when set in:
meta/conf/layer.conf:COREBASE ='${@os.path.normpath("${LAYERDIR}/../")}'
That is not the only problem with the change to use ${THISDIR}. We get a million errors because we have bbappend files for glib-2.0. This is because bitbake will now give an error for any potential cross-files from those layers, regardless if they exist or not (they don't).
Quoted 4 lines
I will send a patch that I believe solves this in a way that also avoids the need for the
bb.error().//Peter
Actually, thinking a bit more on this, I believe that with my patch applied, this patch should _not_ be applied. That is because it makes the paths relative to thisdir, but then uses ${THISDIR} when using them. Thus if thisdir is normalized, it may no longer match ${THISDIR} and the relative paths will not work as intended.
//Peter
Quoted 22 lines
Review
Quoted 6 lines
Signed-off-by Martin Jansa <[email protected]>--- meta/recipes-core/glib-2.0/glib-2.0_2.66.7.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)(quoted diff) +0
-0b/meta/recipes-
Quoted 15 lines
core/glib-2.0/glib-2.0_2.66.7.bb
index882a89da7a..a2fa5345b4100644meta/recipes-core/glib-2.0/glib-2.0_2.66.7.bb +1
-2@@ -31,7 +31,7 @@ def find_meson_cross_files(d): if bb.data.inherits_class('native', d): return "" - thisdir = d.getVar("THISDIR") + thisdir = os.path.normpath(d.getVar("THISDIR")) import collections sitedata = siteinfo_data(d) # filename -> found --2.30.0
Quoted 33 lines · Peter Kjellerstedt
Quoted 33 lines
-----Original Message-----
From: [email protected] <[email protected]> On Behalf Of Peter Kjellerstedt
Sent: den 5 mars 2021 11:54
To: Martin Jansa <[email protected]>; [email protected]
Cc: [email protected]; [email protected]
Subject: Re: [OE-core] [PATCH] glib-2.0: call os.path.normpath on THISDIRQuoted 13 lines
-----Original Message-----
From: [email protected] <[email protected]> On Behalf Of Martin Jansa
Sent: den 2 mars 2021 20:32
To: [email protected]
Cc: [email protected]; [email protected]; Martin Jansa <[email protected]>
Subject: [OE-core] [PATCH] glib-2.0: call os.path.normpath on THISDIR* some build environments have relative paths in THISDIR, e.g. from OEROOT set in:
https://github.com/96boards/oe-rpb-manifest/blob/1e3345c26c56f77f3a15a3978f412a25955d2606/conf/bblayers.conf#L4
and then the paths in filename normalized in:
filename =os.path.normpath(os.path.join(path, meson.cross.d, element))don't match.Quoted 3 lines
* COREBASE used here before didn't have this issue because the value is already normalized when set in:
meta/conf/layer.conf:COREBASE ='${@os.path.normpath("${LAYERDIR}/../")}'
That is not the only problem with the change to use ${THISDIR}. We get a million errors because we have bbappend files for glib-2.0. This is because bitbake will now give an error for any potential cross-files from those layers, regardless if they exist or not (they don't).
I will send a patch that I believe solves this in a way that also avoids the need for the
bb.error().
Quoted 7 lines
Quoted 1 line
//Peter
Actually, thinking a bit more on this, I believe that with my patch applied, this patch should _not_ be applied. That is because it makes the paths relative to thisdir, but then uses ${THISDIR} when using them. Thus if thisdir is normalized, it may no longer match ${THISDIR} and the relative paths will not work as intended.
I'm sorry I've tested and wrote that part of commit message when I had if os.path.exists(filename):
at the beginning of this whole section, which I've then re-wrote to just store only existing filenames in files as a list instead of a map (to simplify the rest of this function as well), but then I was wondering why Ross used the map in the initial implementation here:
https://git.openembedded.org/openembedded-core/commit/?id=5acd9cbc9d5c6355010775250fb25f043441c5cd
and assumed it was needed to get all possible filenames in do_configure[file-checksums] in case they are created later.
So I've removed all my cleanups and returned to smallest change to just replace COREBASE with THISDIR.
What about checking the existence before showing the error, that will show an error only when matching file exists while keeping all not-skipped paths in do_configure[file-checksums] even when they don't exist yet.
Will send a patch for this shortly.
Quoted 26 lines
//Peter
Quoted 22 lines
Review
Quoted 6 lines
Signed-off-by Martin Jansa <[email protected]>--- meta/recipes-core/glib-2.0/glib-2.0_2.66.7.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)(quoted diff) +0
-0b/meta/recipes-
Quoted 15 lines
core/glib-2.0/glib-2.0_2.66.7.bb
index882a89da7a..a2fa5345b4100644meta/recipes-core/glib-2.0/glib-2.0_2.66.7.bb +1
-2@@ -31,7 +31,7 @@ def find_meson_cross_files(d): if bb.data.inherits_class('native', d): return "" - thisdir = d.getVar("THISDIR") + thisdir = os.path.normpath(d.getVar("THISDIR")) import collections sitedata = siteinfo_data(d) # filename -> found --2.30.0
Quoted 75 lines
-----Original Message-----
From: Martin Jansa <[email protected]>
Sent: den 5 mars 2021 12:51
To: Peter Kjellerstedt <[email protected]>
Cc: [email protected]; [email protected]; [email protected]
Subject: Re: [OE-core] [PATCH] glib-2.0: call os.path.normpath on THISDIRQuoted 2 lines · Peter Kjellerstedt
Quoted 2 lines
-----Original Message-----
From: [email protected] <openembedded-[email protected]> On Behalf Of Peter Kjellerstedt
Quoted 2 lines
Quoted 2 lines
Sent: den 5 mars 2021 11:54
To: Martin Jansa <[email protected]>; openembedded-Quoted 2 lines
Quoted 2 lines
Cc: [email protected]; [email protected]
Subject: Re: [OE-core] [PATCH] glib-2.0: call os.path.normpath onTHISDIR
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
-----Original Message-----
From: [email protected] <openembedded-[email protected]> On Behalf Of Martin Jansa
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Sent: den 2 mars 2021 20:32
To: [email protected]
Cc: [email protected]; [email protected]; Martin JansaQuoted 1 line
Quoted 1 line
Quoted 1 line
Subject: [OE-core] [PATCH] glib-2.0: call os.path.normpath on
THISDIR
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
* some build environments have relative paths in THISDIR, e.g. from
OEROOT set in:
Quoted 1 line
Quoted 1 line
Quoted 1 line
manifest/blob/1e3345c26c56f77f3a15a3978f412a25955d2606/conf/bblayers.conf# L4
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
and then the paths in filename normalized in:
filename = os.path.normpath(os.path.join(path, meson.cross.d,element))
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
don't match.
* COREBASE used here before didn't have this issue because the value
is already
Quoted 12 lines
Quoted 12 lines
Quoted 2 lines
normalized when set in:
meta/conf/layer.conf:COREBASE ='${@os.path.normpath("${LAYERDIR}/../")}'
That is not the only problem with the change to use ${THISDIR}. We get a million errors because we have bbappend files for glib-2.0. This is because bitbake will now give an error for any potential cross-files from those layers, regardless if they exist or not (they don't).
I will send a patch that I believe solves this in a way that also avoids the need for the
bb.error().Quoted 7 lines
Quoted 1 line
//Peter
Actually, thinking a bit more on this, I believe that with my patch applied, this patch should _not_ be applied. That is because it makes the paths relative to thisdir, but then uses ${THISDIR} when using them. Thus if thisdir is normalized, it may no longer match ${THISDIR} and the relative paths will not work as intended.
I'm sorry I've tested and wrote that part of commit message when I had if
os.path.exists(filename):
at the beginning of this whole section, which I've then re-wrote to just store only existing filenames in files as a list instead of a map (to simplify the rest of this function as well), but then I was wondering why Ross used the map in the initial implementation here:
https://git.openembedded.org/openembedded-
core/commit/?id=5acd9cbc9d5c6355010775250fb25f043441c5cd
and assumed it was needed to get all possible filenames in do_configure[file-checksums] in case they are created later.So I've removed all my cleanups and returned to smallest change to just replace COREBASE with THISDIR.
Quoted 6 lines
What about checking the existence before showing the error, that will show an error only when matching file exists while keeping all not-skipped paths in do_configure[file-checksums] even when they don't exist yet.
Will send a patch for this shortly.
Any objections to my patch, where there is no need for the bb.error() in the first place?
//Peter
Quoted 31 lines · Peter Kjellerstedt
Quoted 31 lines
-----Original Message-----
From: Martin Jansa <[email protected]>
Sent: den 5 mars 2021 12:51
To: Peter Kjellerstedt <[email protected]>
Cc: [email protected]; [email protected];Quoted 6 lines
Actually, thinking a bit more on this, I believe that with my patch applied, this patch should _not_ be applied. That is because it makes the paths relative to thisdir, but then uses ${THISDIR} when using them. Thus if thisdir is normalized, it may no longer match ${THISDIR} and the relative paths will not work as intended.
I'm sorry I've tested and wrote that part of commit message when I had if
os.path.exists(filename):
at the beginning of this whole section, which I've then re-wrote to just store only existing filenames in files as a list instead of a map (to simplify the rest of this function as well), but then I was wondering why Ross used the map in the initial implementation here:
https://git.openembedded.org/openembedded-
core/commit/?id=5acd9cbc9d5c6355010775250fb25f043441c5cd
and assumed it was needed to get all possible filenames in do_configure[file-checksums] in case they are created later.So I've removed all my cleanups and returned to smallest change to just replace COREBASE with THISDIR.
What about checking the existence before showing the error, that will show an error only when matching file exists while keeping all not-skipped paths in do_configure[file-checksums] even when they don't exist yet.
Quoted 4 lines
Quoted 1 line
Will send a patch for this shortly.
Any objections to my patch, where there is no need for the
bb.error()in the first place?
Your patch encodes the paths of layers relative to each other into the sstate checksums. I'm not sure we want to encourage that and this code will end up copy and pasted around without people realising what they're doing :/.
Cheers,
Richard
Quoted 47 lines
-----Original Message-----
From: Richard Purdie <[email protected]>
Sent: den 5 mars 2021 18:54
To: Peter Kjellerstedt <[email protected]>; Martin Jansa <[email protected]>
Cc: [email protected]; [email protected]; [email protected]
Subject: Re: [OE-core] [PATCH] glib-2.0: call os.path.normpath on THISDIRQuoted 8 lines · Peter Kjellerstedt
Quoted 8 lines
-----Original Message-----
From: Martin Jansa <[email protected]>
Sent: den 5 mars 2021 12:51
To: Peter Kjellerstedt <[email protected]>
Cc: [email protected]; [email protected];Quoted 3 lines
Actually, thinking a bit more on this, I believe that with my patch applied, this patch should _not_ be applied. That is because it
makes
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
the paths relative to thisdir, but then uses ${THISDIR} when using them. Thus if thisdir is normalized, it may no longer match
${THISDIR}
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
and the relative paths will not work as intended.
I'm sorry I've tested and wrote that part of commit message when I had if
os.path.exists(filename):
at the beginning of this whole section, which I've then re-wrote tojust
Quoted 9 lines
Quoted 9 lines
store only existing filenames in files as a list instead of a map (to simplify the rest of this function as well), but then I was wondering why Ross used the map in the initial implementation here:
https://git.openembedded.org/openembedded-
core/commit/?id=5acd9cbc9d5c6355010775250fb25f043441c5cd and assumed it was needed to get all possible filenames in do_configure[file-checksums] in case they are created later.So I've removed all my cleanups and returned to smallest change to
just
Quoted 9 lines
Quoted 8 lines
replace COREBASE with THISDIR.
What about checking the existence before showing the error, that will show an error only when matching file exists while keeping all not-skipped paths in do_configure[file-checksums] even when they don't exist yet.
Will send a patch for this shortly.
Quoted 7 lines
Quoted 2 lines
Any objections to my patch, where there is no need for the
bb.error()in the first place?Your patch encodes the paths of layers relative to each other into the sstate checksums. I'm not sure we want to encourage that and this code will end up copy and pasted around without people realising what they're doing :/.
Hmm, I guess that may be considered bad. :/
Quoted 3 lines
Cheers,
Richard
Then we will have to decide on what to do with the cross-files. Currently the code tries to handle cross-files provided by any layer, but with the recent changes and without mine it really only supports specifying cross-files in the same layer where the recipe is. So we will either have to find a way to actually support cross-files from multiple layers, or we may as well rip out the current attempt at supporting it.
//Peter
Quoted 55 lines · Peter Kjellerstedt
Quoted 47 lines
-----Original Message-----
From: Richard Purdie <[email protected]>
Sent: den 5 mars 2021 18:54
To: Peter Kjellerstedt <[email protected]>; Martin Jansa <[email protected]>
Cc: [email protected]; [email protected]; [email protected]
Subject: Re: [OE-core] [PATCH] glib-2.0: call os.path.normpath on THISDIRQuoted 8 lines · Peter Kjellerstedt
Quoted 8 lines
-----Original Message-----
From: Martin Jansa <[email protected]>
Sent: den 5 mars 2021 12:51
To: Peter Kjellerstedt <[email protected]>
Cc: [email protected]; [email protected];Quoted 3 lines
Actually, thinking a bit more on this, I believe that with my patch applied, this patch should _not_ be applied. That is because it
makes
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
the paths relative to thisdir, but then uses ${THISDIR} when using them. Thus if thisdir is normalized, it may no longer match
${THISDIR}
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
and the relative paths will not work as intended.
I'm sorry I've tested and wrote that part of commit message when I had if
os.path.exists(filename):
at the beginning of this whole section, which I've then re-wrote tojust
Quoted 9 lines
Quoted 9 lines
store only existing filenames in files as a list instead of a map (to simplify the rest of this function as well), but then I was wondering why Ross used the map in the initial implementation here:
https://git.openembedded.org/openembedded-
core/commit/?id=5acd9cbc9d5c6355010775250fb25f043441c5cd and assumed it was needed to get all possible filenames in do_configure[file-checksums] in case they are created later.So I've removed all my cleanups and returned to smallest change to
just
Quoted 9 lines
Quoted 8 lines
replace COREBASE with THISDIR.
What about checking the existence before showing the error, that will show an error only when matching file exists while keeping all not-skipped paths in do_configure[file-checksums] even when they don't exist yet.
Will send a patch for this shortly.
Quoted 7 lines
Quoted 2 lines
Any objections to my patch, where there is no need for the
bb.error()in the first place?Your patch encodes the paths of layers relative to each other into the sstate checksums. I'm not sure we want to encourage that and this code will end up copy and pasted around without people realising what they're doing :/.
Quoted 8 lines
Hmm, I guess that may be considered bad. :/
Quoted 3 lines
Cheers,
Richard
Then we will have to decide on what to do with the cross-files. Currently the code tries to handle cross-files provided by any layer, but with the
I wouldn't say that by using FILESPATH it really tried to handle cross-files from other layers.
Quoted 4 lines
recent changes and without mine it really only supports specifying cross-files in the same layer where the recipe is. So we will either have to find a way to actually support cross-files from multiple layers, or we may as well rip out the current attempt at supporting it.
And replace it with what?
https://git.openembedded.org/openembedded-core/commit/?id=5acd9cbc9d5c6355010775250fb25f043441c5cd
wasn't added to support cross-files from other layers, so we still need this for the cross-files in oe-core, right?
Now it will just refuse to consider cross-files when they are outside oe-core while showing an error, instead of silently making signatures of glib-2.0 and everything depending specific to the build path where it was built.
And as small benefit of using THISDIR it will also work when you copy glib-2.0 and all its cross-files to some other layer (like somebody did @LGE when backporting newer glib-2.0).