Skip to contentSkip to conversation

glibc: re-package for libnss-db

PATCH Superseded Chen Qi 3 participants 5 messages Aug 15, 2018 – Aug 16, 2018

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Chen Qi

author
#0 lore raw

cover glibc: re-package for libnss-db

*** BLURB HERE ***
The following changes since commit 63a4ff7cf5f7d1671ab85800bc2212dd9cd9748d:

bitbake: checksum: sanity check path when recursively checksumming (2018-08-14 16:32:48 +0100)

are available in the git repository at:

git://git.pokylinux.org/poky-contrib ChenQi/libnss-db
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=ChenQi/libnss-db

Jackie Huang (1):
glibc: re-package for libnss-db

 meta/recipes-core/glibc/glibc-package.inc | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
Signature · 1 line
1.9.1
author
#1 lore raw

PATCH 1/1 glibc: re-package for libnss-db

From: Jackie Huang <[email protected]>

On other distros like ubuntu/centos, libnss-db usually provides:
- The libraries
- The Makefile to create database
(in /var/db for centos, /var/lib/misc/ for ubuntu)
- The makedb command (it's in glibc-common for centos7)

What we had is:
- The libraries are in glibc-extra-nss
- The Makefile is removed
- The makedb command is in glibc-utils (lack of dependency)

So when glibc-extra-nss is installed but glibc-utils is not,
we see error like:
nscd[165]: 165 checking for monitored file /var/db/group.db': No such file or directory nscd[165]: 165 checking for monitored file /var/db/passwd.db': No such file or directory

And there is not an easy way to create these databases.

To fix the issue:
- Re-package the libraries into libnss-db
- Don't remove the Makefile and add it in libnss-db
- Add RDEPENDS for libnss-db on glibc-utils and make

Notes:
For external toolchain, an extra package 'libnss-db' need to be provided If replacing glibc from core.

Signed-off-by Jackie Huang <[email protected]>
Signed-off-by Chen Qi <[email protected]>
---
 meta/recipes-core/glibc/glibc-package.inc | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
meta/recipes-core/glibc/glibc-package.inc +3 -2
index 8ac01ce..305094f 100644
@@ -1,6 +1,6 @@
 INHIBIT_SYSROOT_STRIP = "1"
 
-PACKAGES = "${PN}-dbg catchsegv sln nscd ldd tzcode glibc-thread-db ${PN}-pic libcidn libmemusage libsegfault ${PN}-pcprofile libsotruss ${PN} ${PN}-utils glibc-extra-nss ${PN}-dev ${PN}-staticdev ${PN}-doc"
+PACKAGES = "${PN}-dbg catchsegv sln nscd ldd tzcode glibc-thread-db ${PN}-pic libcidn libmemusage libnss-db libsegfault ${PN}-pcprofile libsotruss ${PN} ${PN}-utils glibc-extra-nss ${PN}-dev ${PN}-staticdev ${PN}-doc"
 
 # The ld.so in this glibc supports the GNU_HASH
 RPROVIDES_${PN} = "eglibc rtld(GNU_HASH)"

@@ -23,6 +23,8 @@ FILES_ldd = "${bindir}/ldd"
 FILES_libsegfault = "${base_libdir}/libSegFault*"
 FILES_libcidn = "${base_libdir}/libcidn-*.so ${base_libdir}/libcidn.so.*"
 FILES_libmemusage = "${base_libdir}/libmemusage.so"
+FILES_libnss-db = "${base_libdir}/libnss_db.so.* ${base_libdir}/libnss_db-*.so ${localstatedir}/db/Makefile"
+RDEPENDS_libnss-db = "${PN}-utils make"
 FILES_glibc-extra-nss = "${base_libdir}/libnss_*-*.so ${base_libdir}/libnss_*.so.*"
 FILES_sln = "${base_sbindir}/sln"
 FILES_${PN}-pic = "${libdir}/*_pic.a ${libdir}/*_pic.map ${libdir}/libc_pic/*.o"

@@ -59,7 +61,6 @@ inherit libc-common multilib_header
 
 do_install_append () {
 	rm -f ${D}${sysconfdir}/localtime
-	rm -rf ${D}${localstatedir}
 
 	# remove empty glibc dir
 	if [ -d ${D}${libexecdir} ]; then
Signature · 1 line
1.9.1
#2 lore raw
Quoted 32 lines · Chen Qi

From: Jackie Huang <[email protected]>

On other distros like ubuntu/centos, libnss-db usually provides:
- The libraries
- The Makefile to create database
(in /var/db for centos, /var/lib/misc/ for ubuntu)
- The makedb command (it's in glibc-common for centos7)

What we had is:
- The libraries are in glibc-extra-nss
- The Makefile is removed
- The makedb command is in glibc-utils (lack of dependency)

So when glibc-extra-nss is installed but glibc-utils is not, we see error like:
nscd[165]: 165 checking for monitored file `/var/db/group.db': No such file or directory
nscd[165]: 165 checking for monitored file `/var/db/passwd.db': No such file or directory

And there is not an easy way to create these databases.

To fix the issue:
- Re-package the libraries into libnss-db
- Don't remove the Makefile and add it in libnss-db
- Add RDEPENDS for libnss-db on glibc-utils and make

Notes:
For external toolchain, an extra package 'libnss-db' need to be provided
If replacing glibc from core.

Quoted 5 lines
Signed-off-by Jackie Huang <[email protected]>
Signed-off-by Chen Qi <[email protected]>
---
 meta/recipes-core/glibc/glibc-package.inc | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

This breaks on the autobuilder in oe-selftest:

2018-08-15 13:09:15,551 - oe-selftest - INFO - ======================================================================
2018-08-15 13:09:15,551 - oe-selftest - INFO - FAIL: bbtests.BitbakeTests.test_non_gplv3 (subunit.RemotedTestCase)
2018-08-15 13:09:15,552 - oe-selftest - INFO - ----------------------------------------------------------------------
2018-08-15 13:09:15,552 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
  File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/core/decorator/__init__.py", line 32, in wrapped_f
    return func(*args, **kwargs)
  File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/cases/bbtests.py", line 246, in test_non_gplv3
    self.assertEqual(result.status, 0, "Bitbake failed, exit code %s, output %s" % (result.status, result.output))
  File "/usr/lib64/python3.6/unittest/case.py", line 829, in assertEqual
    assertion_func(first, second, msg=msg)
  File "/usr/lib64/python3.6/unittest/case.py", line 822, in _baseAssertEqual
    raise self.failureException(msg)

AssertionError: 1 != 0 : Bitbake failed, exit code 1, output Parsing recipes...done.
Parsing of 811 .bb files complete (0 cached, 811 parsed). 1277 targets, 99 skipped, 0 masked, 0 errors.
Removing 10 recipes from the core2-64 sysroot...done.
Removing 10 recipes from the qemux86_64 sysroot...done.
NOTE: Resolving any missing task queue dependencies
ERROR: Nothing RPROVIDES 'make' (but /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/recipes-core/glibc/glibc_2.28.bb RDEPENDS on or otherwise requires it) make was skipped: it has an incompatible license: GPLv3 & LGPLv2
NOTE: Runtime target 'make' is unbuildable, removing...
Missing or unbuildable dependency chain was: ['make']
ERROR: Required build target 'selftest-ed' has no buildable providers.
Missing or unbuildable dependency chain was: ['selftest-ed', 'virtual/libc', 'make']

Summary: There were 2 ERROR messages shown, returning a non-zero exit code.

https://autobuilder.yocto.io/builders/nightly-oe-selftest/builds/1281/steps/Running%20oe-selftest/logs/stdio

We really don't want a make dependency in glibc :(

Cheers,

Richard

#3 lore raw
Quoted 42 lines · Richard Purdie
Quoted 32 lines · Chen Qi

From: Jackie Huang <[email protected]>

On other distros like ubuntu/centos, libnss-db usually provides:
- The libraries
- The Makefile to create database
(in /var/db for centos, /var/lib/misc/ for ubuntu)
- The makedb command (it's in glibc-common for centos7)

What we had is:
- The libraries are in glibc-extra-nss
- The Makefile is removed
- The makedb command is in glibc-utils (lack of dependency)

So when glibc-extra-nss is installed but glibc-utils is not, we see error like:
nscd[165]: 165 checking for monitored file `/var/db/group.db': No such file or directory
nscd[165]: 165 checking for monitored file `/var/db/passwd.db': No such file or directory

And there is not an easy way to create these databases.

To fix the issue:
- Re-package the libraries into libnss-db
- Don't remove the Makefile and add it in libnss-db
- Add RDEPENDS for libnss-db on glibc-utils and make

Notes:
For external toolchain, an extra package 'libnss-db' need to be provided
If replacing glibc from core.

Quoted 5 lines
Signed-off-by Jackie Huang <[email protected]>
Signed-off-by Chen Qi <[email protected]>
---
  meta/recipes-core/glibc/glibc-package.inc | 5 +++--
  1 file changed, 3 insertions(+), 2 deletions(-)

This breaks on the autobuilder in oe-selftest:

2018-08-15 13:09:15,551 - oe-selftest - INFO - ======================================================================

...

Quoted 8 lines

NOTE: Resolving any missing task queue dependencies
ERROR: Nothing RPROVIDES 'make' (but /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/recipes-core/glibc/glibc_2.28.bb RDEPENDS on or otherwise requires it) make was skipped: it has an incompatible license: GPLv3 & LGPLv2
NOTE: Runtime target 'make' is unbuildable, removing...
Missing or unbuildable dependency chain was: ['make']
ERROR: Required build target 'selftest-ed' has no buildable providers.
Missing or unbuildable dependency chain was: ['selftest-ed', 'virtual/libc', 'make']

Quoted 5 lines

Summary: There were 2 ERROR messages shown, returning a non-zero exit code.

https://autobuilder.yocto.io/builders/nightly-oe-selftest/builds/1281/steps/Running%20oe-selftest/logs/stdio

We really don't want a make dependency in glibc :(

Switching from RDEPENDS to RSUGGESTS seems like a reasonable solution. The GPLv2 build would work but libnss-db would fail unless meta-gplv2 were used to get an old version of make.

One way to avoid needing make is to demand that the bdb files are generated during the build. That would be nice but it shouldn't be a requirement since it's not the typical workflow and since the db will usually be dynamic.

History and more details below.

../Randy

To support on target generation of the bdb files that libnss-db uses then we need to:
- separate the libnss-db files into a libnss-db pkg
- *suggest* a 'make' as described below:

Back in January, Jackie had replied to:
[OE-core] [PATCH 0/2 v2] glibc: fixes for nscd and libnss-db

with:

Quoted 6 lines

Sorry, I didn't noticed that make is gplv3. I change the dependency to RSUGGESTS and the test passed:

$ oe-selftest -r bbtests.BitbakeTests.test_non_gplv3
...
2018-01-16 21:46:22,540 - oe-selftest - INFO - oe-selftest - OK - All required tests passed

but of course that papers over the failure by allowing the build to pass even though runtime needs to include make to be able to rebuild the db(s). And of course libnss-db isn't even being tested yet so once we add that to oeqa, the test will fail for GPLv2-only images where meta-gplv2 isn't added to the list of layers. I'm fine with such oeqa failing when meta-gplv2 isn't around.

Qi, if no one comes up with a better idea,
please test that as well as:

  $ oe-selftest -r bbtests.BitbakeTests.test_non_gplv3
so that Richard doesn't burn another build cycle on it.

This isn't urgent but it would be nice to resolve the issue properly for M3 without driving Richard around the bend!

Thanks,
../Randy

Quoted 5 lines

Cheers,

Richard

Signature · 2 lines
# Randy MacLeod
# Wind River Linux
author
#4 lore raw
Quoted 103 lines · Randy MacLeod
Quoted 42 lines · Richard Purdie
Quoted 32 lines · Chen Qi

From: Jackie Huang <[email protected]>

On other distros like ubuntu/centos, libnss-db usually provides:
- The libraries
- The Makefile to create database
(in /var/db for centos, /var/lib/misc/ for ubuntu)
- The makedb command (it's in glibc-common for centos7)

What we had is:
- The libraries are in glibc-extra-nss
- The Makefile is removed
- The makedb command is in glibc-utils (lack of dependency)

So when glibc-extra-nss is installed but glibc-utils is not, we see error like:
nscd[165]: 165 checking for monitored file `/var/db/group.db': No such file or directory
nscd[165]: 165 checking for monitored file `/var/db/passwd.db': No such file or directory

And there is not an easy way to create these databases.

To fix the issue:
- Re-package the libraries into libnss-db
- Don't remove the Makefile and add it in libnss-db
- Add RDEPENDS for libnss-db on glibc-utils and make

Notes:
For external toolchain, an extra package 'libnss-db' need to be provided
If replacing glibc from core.

Quoted 5 lines
Signed-off-by Jackie Huang <[email protected]>
Signed-off-by Chen Qi <[email protected]>
---
  meta/recipes-core/glibc/glibc-package.inc | 5 +++--
  1 file changed, 3 insertions(+), 2 deletions(-)

This breaks on the autobuilder in oe-selftest:

2018-08-15 13:09:15,551 - oe-selftest - INFO - ======================================================================

...

Quoted 8 lines

NOTE: Resolving any missing task queue dependencies
ERROR: Nothing RPROVIDES 'make' (but /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/recipes-core/glibc/glibc_2.28.bb RDEPENDS on or otherwise requires it) make was skipped: it has an incompatible license: GPLv3 & LGPLv2
NOTE: Runtime target 'make' is unbuildable, removing...
Missing or unbuildable dependency chain was: ['make']
ERROR: Required build target 'selftest-ed' has no buildable providers.
Missing or unbuildable dependency chain was: ['selftest-ed', 'virtual/libc', 'make']

Quoted 6 lines

Summary: There were 2 ERROR messages shown, returning a non-zero exit code.

https://autobuilder.yocto.io/builders/nightly-oe-selftest/builds/1281/steps/Running%20oe-selftest/logs/stdio

We really don't want a make dependency in glibc :(

Switching from RDEPENDS to RSUGGESTS seems like a reasonable solution. The GPLv2 build would work but libnss-db would fail unless meta-gplv2 were used to get an old version of make.

One way to avoid needing make is to demand that the bdb files are generated during the build. That would be nice but it shouldn't be a requirement since it's not the typical workflow and since the db will usually be dynamic.

History and more details below.

../Randy

To support on target generation of the bdb files that libnss-db uses then we need to:
- separate the libnss-db files into a libnss-db pkg
- *suggest* a 'make' as described below:

Back in January, Jackie had replied to:
[OE-core] [PATCH 0/2 v2] glibc: fixes for nscd and libnss-db

with:

Quoted 6 lines

Sorry, I didn't noticed that make is gplv3. I change the dependency to RSUGGESTS and the test passed:

$ oe-selftest -r bbtests.BitbakeTests.test_non_gplv3
...
2018-01-16 21:46:22,540 - oe-selftest - INFO - oe-selftest - OK - All required tests passed

but of course that papers over the failure by allowing the build to pass even though runtime needs to include make to be able to rebuild the db(s). And of course libnss-db isn't even being tested yet so once we add that to oeqa, the test will fail for GPLv2-only images where meta-gplv2 isn't added to the list of layers. I'm fine with such oeqa failing when meta-gplv2 isn't around.

Qi, if no one comes up with a better idea,
please test that as well as:

Quoted 17 lines
  $ oe-selftest -r bbtests.BitbakeTests.test_non_gplv3
so that Richard doesn't burn another build cycle on it.

This isn't urgent but it would be nice to resolve the issue properly for M3 without driving Richard around the bend!

Thanks,
../Randy

Quoted 5 lines

Cheers,

Richard

Sorry for breaking the autobuilder.

I checked the Makefile and found that it's a bunch of `awd [expression] /etc/xxx | makedb -o /var/db/xxx.db -'.
I also checked the glibc repo and this Makefile is quite stable. So I think we can use a shell script to get rid of the make dependency. And it should not create maintenance problem.

I'll send out V2.
I've tested it locally and it should pass the oe-selftest.

Best Regards,
Chen Qi