archiver: avoid empty incfile in ar_recipe
PATCH 1/1 archiver: avoid empty incfile in ar_recipe
do_ar_recipe fails on perf recipe on line:
include ${@bb.utils.contains('PACKAGECONFIG', 'scripting', 'perf-perl.inc', '', d)}
1. "${...}" part expands into empty string
2. bb.utils.which() takes empty string and returns first directory name from bbpath
3. shutil.copy() fails on copying directory:
Exception: IsADirectoryError: [Errno 21] Is a directory: ......
Hence, check "incfile" variable on each step.
--- meta/classes/archiver.bbclass | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
meta/classes/archiver.bbclass +3 -2
@@ -441,9 +441,10 @@ python do_ar_recipe () { incfile = include_re.match(line).group(1) if incfile: incfile = d.expand(incfile) + if incfile: incfile = bb.utils.which(bbpath, incfile) - if incfile: - shutil.copy(incfile, outdir) + if incfile: + shutil.copy(incfile, outdir) create_tarball(d, outdir, 'recipe', d.getVar('ARCHIVER_OUTDIR')) bb.utils.remove(outdir, recurse=True)
Signature · 1 line
2.23.0
Hi,
Quoted 6 lines · grygorii tertychnyi via Openembedded-core
do_ar_recipefails on perf recipe on line:include ${@bb.utils.contains('PACKAGECONFIG', 'scripting', 'perf-perl.inc', '', d)}
1. "${...}" part expands into empty string
2.bb.utils.which()takes empty string and returns first directory name from bbpath
This doesn't sound sane. If the include directive has no argument, incfile should end up None. That's what the code "assumes" at this point. I would fix it either at the regex expression level or strip the match. I reckon former would make more sense. It sounds like the match expression should use * (none or more).
Quoted 29 lines
3.
shutil.copy()fails on copying directory:Exception: IsADirectoryError: [Errno 21] Is a directory: ......
Hence, check "incfile" variable on each step.
Signed-off-by grygorii tertychnyi <[email protected]>--- meta/classes/archiver.bbclass | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)meta/classes/archiver.bbclass +3
-2@@ -441,9 +441,10 @@ python do_ar_recipe () { incfile = include_re.match(line).group(1) if incfile: incfile = d.expand(incfile) + if incfile: incfile = bb.utils.which(bbpath, incfile) - if incfile: - shutil.copy(incfile, outdir) + if incfile: + shutil.copy(incfile, outdir) create_tarball(d, outdir, 'recipe', d.getVar('ARCHIVER_OUTDIR')) bb.utils.remove(outdir, recurse=True)
Signature · 1 line
Andrei Gherzan
Hi,
Quoted 6 lines · grygorii tertychnyi via Openembedded-core
do_ar_recipefails on perf recipe on line:include ${@bb.utils.contains('PACKAGECONFIG', 'scripting', 'perf-perl.inc', '', d)}
1. "${...}" part expands into empty string
2.bb.utils.which()takes empty string and returns first directory name from bbpath
This doesn't sound sane. If the include directive has no argument, incfile should end up None. That's what the code "assumes" at this point. I would fix it either at the regex expression level or stripping the matched string. I reckon the former makes more sense (.*).
--
Andrei Gherzan
Andrei,
From: Andrei Gherzan <[email protected]>
Sent: Friday, November 1, 2019 13:28
To: Grygorii Tertychnyi (gtertych); [email protected]
Cc: xe-linux-external(mailer list)
Subject: Re: [OE-core] [PATCH] archiver: avoid empty incfile in ar_recipe
Quoted 9 lines
Quoted 6 lines
do_ar_recipefails on perf recipe on line:include ${@bb.utils.contains('PACKAGECONFIG', 'scripting', 'perf-perl.inc', '', d)}
1. "${...}" part expands into empty string
2.bb.utils.which()takes empty string and returns first directory name from bbpathThis doesn't sound sane. If the include directive has no argument, incfile should end up None. That's what the code "assumes" at this
I agree.
Quoted 3 lines
point. I would fix it either at the regex expression level or stripping the matched string. I reckon the former makes more sense (.*).
Not sure I understand. Archiver class does not interpret "include" directive. It just parses text files. The regular expression looks correct:
These lines:
440 elif include_re.match(line):
441 incfile = include_re.match(line).group(1)
put "${...}" _string_ into "incfile" variable. So, "incfile" is not None at this stage. Then,
443 incfile = d.expand(incfile)
Now "incfile" is empty and nobody checks it.
444 incfile = bb.utils.which(bbpath, incfile)
Now "incfile" is set to first directory name in BBPATH (wrong behavour?)
445 if incfile:
446 shutil.copy(incfile, outdir)
Exception here: "incfile" is directory, not a file.
Signature · 1 line
Grygorii
ping
________________________________________
From: Grygorii Tertychnyi (gtertych) <[email protected]>
Sent: Monday, November 4, 2019 17:09
To: Andrei Gherzan; [email protected]
Cc: xe-linux-external(mailer list)
Subject: Re: [OE-core] [PATCH] archiver: avoid empty incfile in ar_recipe
Andrei,
From: Andrei Gherzan <[email protected]>
Sent: Friday, November 1, 2019 13:28
To: Grygorii Tertychnyi (gtertych); [email protected]
Cc: xe-linux-external(mailer list)
Subject: Re: [OE-core] [PATCH] archiver: avoid empty incfile in ar_recipe
Quoted 9 lines
Quoted 6 lines
do_ar_recipefails on perf recipe on line:include ${@bb.utils.contains('PACKAGECONFIG', 'scripting', 'perf-perl.inc', '', d)}
1. "${...}" part expands into empty string
2.bb.utils.which()takes empty string and returns first directory name from bbpathThis doesn't sound sane. If the include directive has no argument, incfile should end up None. That's what the code "assumes" at this
I agree.
Quoted 3 lines
point. I would fix it either at the regex expression level or stripping the matched string. I reckon the former makes more sense (.*).
Not sure I understand. Archiver class does not interpret "include" directive. It just parses text files. The regular expression looks correct:
These lines:
440 elif include_re.match(line):
441 incfile = include_re.match(line).group(1)
put "${...}" _string_ into "incfile" variable. So, "incfile" is not None at this stage. Then,
443 incfile = d.expand(incfile)
Now "incfile" is empty and nobody checks it.
444 incfile = bb.utils.which(bbpath, incfile)
Now "incfile" is set to first directory name in BBPATH (wrong behavour?)
445 if incfile:
446 shutil.copy(incfile, outdir)
Exception here: "incfile" is directory, not a file.
--
Grygorii
Hi Grygorii,
Quoted 27 lines · Grygorii Tertychnyi (gtertych)
Andrei,
From: Andrei Gherzan <[email protected]>
Sent: Friday, November 1, 2019 13:28
To: Grygorii Tertychnyi (gtertych); [email protected]
Cc: xe-linux-external(mailer list)
Subject: Re: [OE-core] [PATCH] archiver: avoid empty incfile inar_recipeQuoted 9 lines
Quoted 6 lines
do_ar_recipefails on perf recipe on line:include ${@bb.utils.contains('PACKAGECONFIG', 'scripting', 'perf-perl.inc', '', d)}
1. "${...}" part expands into empty string
2.bb.utils.which()takes empty string and returns first directory name from bbpathThis doesn't sound sane. If the include directive has no argument, incfile should end up None. That's what the code "assumes" at this
I agree.
Quoted 3 lines
point. I would fix it either at the regex expression level or stripping the matched string. I reckon the former makes more sense (.*).
Not sure I understand. Archiver class does not interpret "include" directive. It just parses text files. The regular expression looks correct:
Quoted 6 lines
These lines:
440 elif
include_re.match(line):
441 incfile = include_re.match(line).group(1)put "${...}" _string_ into "incfile" variable. So, "incfile" is not None at "this stage.
"${...}" it's already expanded to a white-space. So in that case it matches "include ".
Quoted 5 lines
Then,
443 incfile =
d.expand(incfile)Now "incfile" is empty and nobody checks it.
incfile is not empty. It's actually a string containing one white-space.
Quoted 10 lines
444 incfile =
bb.utils.which(bbpath, incfile)Now "incfile" is set to first directory name in BBPATH (wrong behavour?)
445 if incfile:
446shutil.copy(incfile, outdir)Exception here: "incfile" is directory, not a file.
The include regex is the following:
include_re = re.compile( r"include\s+(.+)" )
The issue is that when this is matched on a string suffixed with only spaces, it will match the last space as group(1). This is because ".+" forces to match the last white-space. Changing that to ".*" will make the group(1) be an empty string and later the if will evaluate False.
Signature · 1 line
Andrei Gherzan
Hi Andrei,
From: Andrei Gherzan <[email protected]>
Sent: Monday, November 11, 2019 13:18
Subject: Re: [OE-core] [PATCH] archiver: avoid empty incfile in ar_recipe
Quoted 5 lines
Quoted 5 lines
Not sure I understand. Archiver class does not interpret "include" directive. It just parses text files. The regular expression looks correct:
These lines:
Quoted 7 lines
Quoted 4 lines
440 elif
include_re.match(line):
441 incfile = include_re.match(line).group(1)put "${...}" _string_ into "incfile" variable. So, "incfile" is not None at "this stage.
"${...}" it's already expanded to a white-space. So in that case it matches "include ".
No. "incfile" is not expanded here, it actually contains "${...}", e.g. ${@bb.utils.contains('PACKAGECONFIG', 'scripting', 'perf-perl.inc', '', d)}
Quoted 7 lines
Quoted 5 lines
Then,
443 incfile =
d.expand(incfile)Now "incfile" is empty and nobody checks it.
incfile is not empty. It's actually a string containing one white-space.
It is actually empty, e.g. "".
Quoted 11 lines
Quoted 10 lines
444 incfile =
bb.utils.which(bbpath, incfile)Now "incfile" is set to first directory name in BBPATH (wrong behavour?)
445 if incfile:
446shutil.copy(incfile, outdir)Exception here: "incfile" is directory, not a file.
Quoted 8 lines
The include regex is the following:
include_re=re.compile( r"include\s+(.+)" )The issue is that when this is matched on a string suffixed with only spaces, it will match the last space as group(1). This is because ".+" forces to match the last white-space. Changing that to ".*" will make the group(1) be an empty string and later the if will evaluate False.
I don't see how it helps here.
Signature · 1 line
Grygorii
Hi,
Quoted 14 lines · Grygorii Tertychnyi (gtertych)
Hi Andrei,
From: Andrei Gherzan <[email protected]>
Sent: Monday, November 11, 2019 13:18
Subject: Re: [OE-core] [PATCH] archiver: avoid empty incfile inar_recipeQuoted 8 lines
Quoted 8 lines
Not sure I understand. Archiver class does not interpret "include" directive. It just parses text files. The regular expression looks correct:
These lines:
440 elif
include_re.match(line):
441 incfile = include_re.match(line).group(1)
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
put "${...}" _string_ into "incfile" variable. So, "incfile" is not None at "this stage.
"${...}" it's already expanded to a white-space. So in that case it matches "include ".
No. "incfile" is not expanded here, it actually contains "${...}", e.g. ${@bb.utils.contains('PACKAGECONFIG', 'scripting', 'perf-perl.inc', '', d)}
Give me a simple way to reproduce this.
Signature · 1 line
Andrei Gherzan
Hi Andrei,
Quoted 19 lines · Andrei Gherzan
Hi,
Quoted 16 lines · Grygorii Tertychnyi (gtertych)
Hi Andrei,
From: Andrei Gherzan <[email protected]>
Sent: Monday, November 11, 2019 13:18
Subject: Re: [OE-core] [PATCH] archiver: avoid empty incfile inar_recipeQuoted 9 lines
Quoted 9 lines
Not sure I understand. Archiver class does not interpret "include" directive. It just parses text files. The regular expression looks correct:
These lines:
440 elif
include_re.match(line):
441 incfile = include_re.match(line).group(1)put "${...}" _string_ into "incfile" variable. So, "incfile" is not None at "this stage.
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
"${...}" it's already expanded to a white-space. So in that case it matches "include ".
No. "incfile" is not expanded here, it actually contains "${...}", e.g. ${@bb.utils.contains('PACKAGECONFIG', 'scripting', 'perf-perl.inc', '', d)}
Give me a simple way to reproduce this.
1. Put two lines to local.conf
% cat <<CONF >> conf/local.conf
PACKAGECONFIG_remove_pn-perf = "scripting"
INHERIT += "archiver"
CONF
2. Run do_ar_recipe for perf
% bitbake -c ar_recipe perf
3. Observe the output error:
Exception: IsADirectoryError: [Errno 21] Is a directory
Signature · 1 line
Grygorii