stress-ng: Add bash to DEPENDS

Fixed:

$ bitbake stress-ng bash -ccleansstate && bitbake stress-ng
ERROR: stress-ng-0.10.00-r0 do_package_qa: QA Issue: /usr/share/bash-completion/completions/stress-ng contained in package stress-ng-bash-completion requires /bin/bash, but no providers found in RDEPENDS_stress-ng-bash-completion? [file-rdeps]

Signed-off-by Robert Yang <[email protected]>

---
 meta/recipes-extended/stress-ng/stress-ng_0.10.00.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

meta/recipes-extended/stress-ng/stress-ng_0.10.00.bb +0 -0

index f7faf97..904eb0c 100644

Quoted 17 lines · Robert Yang

Fixed:

$ bitbake stress-ng bash -ccleansstate && bitbake stress-ng
ERROR: stress-ng-0.10.00-r0 do_package_qa: QA Issue: /usr/share/bash-completion/completions/stress-ng contained in package stress-ng-bash-completion requires /bin/bash, but no providers found in RDEPENDS_stress-ng-bash-completion? [file-rdeps]

Signed-off-by Robert Yang <[email protected]>

---
  meta/recipes-extended/stress-ng/stress-ng_0.10.00.bb | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

meta/recipes-extended/stress-ng/stress-ng_0.10.00.bb +0 -0

index f7faf97..904eb0c 100644
@@ -11,7 +11,7 @@ SRC_URI = "https://kernel.ubuntu.com/~cking/tarballs/${BPN}/${BP}.tar.xz \
  SRC_URI[md5sum] = "46aa41d37690324ceab4febfcc549018"
  SRC_URI[sha256sum] = "d09dd2a1aea549e478995bf9be90b38906a4cdf33ea7b245ef9d46aa5213c074"
 

meta/recipes-extended/stress-ng/stress-ng_0.10.00.bb SRC_URI = "https://kernel.ubuntu.com/~cking/tarballs/${BPN}/${BP}.tar.xz \ · line 14

Quoted 2 lines

meta/recipes-extended/stress-ng/stress-ng_0.10.00.bb +1 -1

-DEPENDS = "coreutils-native"
+DEPENDS = "coreutils-native bash"

Technically you just want to make this an RDEPENDS of the appropriate package, which would give some small dependency improvement too.

Cheers,

Richard

Yes, exactly, I will send a V2 for it. What mis-leaded me was that stress-ng-bash-completion RDEPENDS on bash-completion, and bash-completion RDEPENDS on bash, so bash is on the dependencies chain, but stress-ng can't find it, and there is no qa issues if we build bash firstly:

$ bitbake bash
$ bitbake stress-ng
No qa issues.

So the reproducer is:

$ bitbake stress-ng bash -ccleansstate && bitbake stress-ng

This might be a bug?

On Tue, 6 Aug 2019 at 13:01, Robert Yang <[email protected] <mailto:[email protected]>> wrote:

Yes, exactly, I will send a V2 for it. What mis-leaded me was that stress-ng-bash-completion RDEPENDS on bash-completion, and bash-completion
RDEPENDS on bash, so bash is on the dependencies chain, but stress-ng can't
find it, and there is no qa issues if we build bash firstly:

$ bitbake bash
$ bitbake stress-ng
No qa issues.

So the reproducer is:
$ bitbake stress-ng bash -ccleansstate && bitbake stress-ng

This might be a bug?

Note that bash-completion files are not executable, and we do not observe this with other recipes. We might have a false positive here - can you look at the actual contents of the package please and see why _qa thinks it needs /bin/bash?

Quoted 4 lines

Note that bash-completion files are not executable, and we do not observe this with other recipes. We might have a false positive here - can you look at the actual contents of the package please and see why _qa thinks it needs /bin/bash?

Also I don't see this with master. Do you have some local patches or QA changes?

Quoted 8 lines · Ross Burton

Quoted 4 lines

Note that bash-completion files are not executable, and we do not observe this with other recipes. We might have a false positive here - can you look at the actual contents of the package please and see why _qa thinks it needs /bin/bash?

Also I don't see this with master. Do you have some local patches or QA changes?

Actually there is a determinism problem here. I have reproduced it with master, and the error happens if and only if bash's do_package task has not yet run. If it has already run, then the package_qa for stress-ng passes fine. I'll poke at it to see if it's a recent issue.

Alex
--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Maybe another case of
https://bugzilla.yoctoproject.org/show_bug.cgi?id=9217#c6 ?

Quoted 23 lines · Alexander Kanavin

Quoted 8 lines · Ross Burton

Quoted 4 lines

Note that bash-completion files are not executable, and we do not observe this with other recipes. We might have a false positive here - can you look at the actual contents of the package please and see why _qa thinks it needs /bin/bash?

Also I don't see this with master. Do you have some local patches or QA changes?

Actually there is a determinism problem here. I have reproduced it with master, and the error happens if and only if bash's do_package task has not yet run. If it has already run, then the package_qa for stress-ng passes fine. I'll poke at it to see if it's a recent issue.

Alex
--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Could be, yes. I will next try to untangle the file-rdeps and see why it fails if bash hasn't been packaged yet, and passes if it has. I have reproduced this on thud, so this is an old problem.

Alex

Quoted 24 lines · Martin Jansa

Maybe another case of
https://bugzilla.yoctoproject.org/show_bug.cgi?id=9217#c6 ?

Quoted 19 lines · Alexander Kanavin

Quoted 11 lines · Ross Burton

Quoted 1 line

Note that bash-completion files are not executable, and we do

not

Quoted 1 line

observe this with other recipes. We might have a false

positive here -

Quoted 1 line

can you look at the actual contents of the package please and

see why

Quoted 1 line

_qa thinks it needs /bin/bash?

Also I don't see this with master. Do you have some local patches or QA
changes?

Actually there is a determinism problem here. I have reproduced it with master, and the error happens if and only if bash's do_package task has not yet run. If it has already run, then the package_qa for stress-ng passes fine. I'll poke at it to see if it's a recent issue.

We now filter the pkgdata to the things a recipe can see in its dependencies after:

http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=99206cb7b41be7a124ea8339bc891077f02128a2

Does that help? If not, is it because some code isn't using the filtered copy? Or could we use this technique somewhere else?

I've not looked at the underlying issues enough to tell which code is actually triggering it but that kind of approach should help with this kind of problem and at least make it deterministic...

I think the issue might be actually in the QA check where it looks beyond the recipe's immediate RDEPENDS (and then fails or not depending on whether those indirect dependencies have been packaged). I need to better understand what happens there so will add some printouts first. The problem does happen with latest master.

Quoted 7 lines · Alexander Kanavin

I think the issue might be actually in the QA check where it looks beyond the recipe's immediate RDEPENDS (and then fails or not depending on whether those indirect dependencies have been packaged). I need to better understand what happens there so will add some printouts first. The problem does happen with latest master.

I can confirm that this bit in insane.bbclass is non-deterministic:

# Find all the rdepends on the dependency chain while next:
new = []
for rdep in next:
rdep_data =
oe.packagedata.read_subpkgdata(rdep, d)
sub_rdeps = rdep_data.get("RDEPENDS_" + rdep) if not sub_rdeps:
continue
for sub_rdep in
bb.utils.explode_deps(sub_rdeps):
if sub_rdep in done:
continue
if
oe.packagedata.has_subpkgdata(sub_rdep, d):
# It's a new rdep done.append(sub_rdep) new.append(sub_rdep)
next = new

It only fully expands the dependencies if all of them have been already packaged. As there is no mechanism to ensure that package_qa runs after packaging all of the recursive RDEPENDS, this is causing a race between packaging them, and running the package qa check.

I think the right way to address this is to actually drop the recursive RDEPENDS resolution, and only consider the immediate runtime dependencies. Alternatively, we may introduce additional task dependencies to fix the race. Thoughts?

I partly agree with this. There is:

insane.bbclass:do_package_qa[rdeptask] = "do_packagedata"

which does help the situation by forcing package_qa after all the runtime dependencies.

I suspect the real problem is that it can also see RDEPENDS for things it doesn't rdepend upon...

You might want package_prepare_pkgdata() from package_pkgdata called for package_qa's context on a separate directory, then to use that as the pkgdata to resolve against?

Note that package_prepare_pkgdata() called for do_package is different than package_prepare_pkgdata() called for do_package_qa as the tasks have different dependencies...

Quoted 10 lines · Richard Purdie

I partly agree with this. There is:

insane.bbclass:do_package_qa[rdeptask] = "do_packagedata"

which does help the situation by forcing package_qa after all the runtime dependencies.

I suspect the real problem is that it can also see RDEPENDS for things it doesn't rdepend upon...

Quoted 8 lines

You might want package_prepare_pkgdata() from package_pkgdata called for package_qa's context on a separate directory, then to use that as the pkgdata to resolve against?

Note that package_prepare_pkgdata() called for do_package is different than package_prepare_pkgdata() called for do_package_qa as the tasks have different dependencies...

I am actually inclined to remove the recursive resolver in insane.bbclass. It gives a QA pass to a situation when there is a bash script in A, and the bash runtime dependency is satisfied indirectly through a chain of arbitrary length: A -> bash-completion -> bash. It becomes hard to reason where the dependencies come from; I think any runtime dependencies should be explicitly listed. We may need to fix up a few recipes, but the outcome is a better definition of runtime dependencies.

Quoted 5 lines · Richard Purdie

We now filter the pkgdata to the things a recipe can see in its dependencies after:

http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=99206cb7b41be7a124ea8339bc891077f02128a2

Quoted 8 lines

Does that help? If not, is it because some code isn't using the filtered copy? Or could we use this technique somewhere else?

I've not looked at the underlying issues enough to tell which code is
actually triggering it but that kind of approach should help with this
kind of problem and at least make it deterministic...

I think the issue might be actually in the QA check where it looks beyond the recipe's immediate RDEPENDS (and then fails or not depending on whether those indirect dependencies have been packaged). I need to better understand what happens there so will add some printouts first. The problem does happen with latest master.