glibc: fixes for nscd and libnss-db

From: Jackie Huang <[email protected]>

v2 comments:
Add a note for external toolchain in the commit log.

--
The following changes since commit 514a808f21c37b6ad704ce397bb2740ecc9a93bc:

ref-manual: Updates to "Image Generation" section. (2017-10-30 15:55:08 +0000)

are available in the git repository at:

git://git.pokylinux.org/poky-contrib.git jhuang0/d_nscd_171102_0
http://git.pokylinux.org/cgit.cgi//log/?h=jhuang0/d_nscd_171102_0

Jackie Huang (2):
glibc: re-package for libnss-db glibc/nscd: do not cache for netgroup by default

 meta/recipes-core/glibc/glibc-package.inc | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

2.11.0

From: Jackie Huang <[email protected]>

On other distros like ubuntu/centos, libnss-db usually provides:
- The libraries
- The Makefile to create database
(in /var/db for centos, /var/lib/misc/ for ubuntu)
- The makedb command (it's in glibc-common for centos7)

What we had is:
- The libraries are in glibc-extra-nss
- The Makefile is removed
- The makedb command is in glibc-utils (lack of dependency)

So when glibc-extra-nss is installed but glibc-utils is not,
we see error like:
nscd[165]: 165 checking for monitored file /var/db/group.db': No such file or directory nscd[165]: 165 checking for monitored file /var/db/passwd.db': No such file or directory

And there is not an easy way to create these databases.

To fix the issue:
- Re-package the libraries into libnss-db
- Don't remove the Makefile and add it in libnss-db
- Add RDEPENDS for libnss-db on glibc-utils and make

Notes:
For external toolchain, an extra package 'libnss-db' need to be provided If replacing glibc from core.

---
 meta/recipes-core/glibc/glibc-package.inc | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

index df3db2cc45..1c3782dbb0 100644
@@ -1,6 +1,6 @@
 INHIBIT_SYSROOT_STRIP = "1"
 
-PACKAGES = "${PN}-dbg catchsegv sln nscd ldd tzcode glibc-thread-db ${PN}-pic libcidn libmemusage libsegfault ${PN}-pcprofile libsotruss ${PN} ${PN}-utils glibc-extra-nss ${PN}-dev ${PN}-staticdev ${PN}-doc"
+PACKAGES = "${PN}-dbg catchsegv sln nscd ldd tzcode glibc-thread-db ${PN}-pic libcidn libmemusage libnss-db libsegfault ${PN}-pcprofile libsotruss ${PN} ${PN}-utils glibc-extra-nss ${PN}-dev ${PN}-staticdev ${PN}-doc"
 
 # The ld.so in this glibc supports the GNU_HASH
 RPROVIDES_${PN} = "eglibc rtld(GNU_HASH)"

@@ -23,6 +23,8 @@ FILES_ldd = "${bindir}/ldd"
 FILES_libsegfault = "${base_libdir}/libSegFault*"
 FILES_libcidn = "${base_libdir}/libcidn-*.so ${base_libdir}/libcidn.so.*"
 FILES_libmemusage = "${base_libdir}/libmemusage.so"
+FILES_libnss-db = "${base_libdir}/libnss_db.so.* ${base_libdir}/libnss_db-*.so ${localstatedir}/db/Makefile"
+RDEPENDS_libnss-db = "${PN}-utils make"
 FILES_glibc-extra-nss = "${base_libdir}/libnss_*-*.so ${base_libdir}/libnss_*.so.*"
 FILES_sln = "${base_sbindir}/sln"
 FILES_${PN}-pic = "${libdir}/*_pic.a ${libdir}/*_pic.map ${libdir}/libc_pic/*.o"

@@ -59,7 +61,6 @@ inherit libc-common multilib_header
 
 do_install_append () {
 	rm -f ${D}${sysconfdir}/localtime
-	rm -rf ${D}${localstatedir}
 
 	# remove empty glibc dir
 	if [ -d ${D}${libexecdir} ]; then

2.11.0

From: Jackie Huang <[email protected]>

We don't have /etc/netgroup by default, so do not
cache for netgroup by default to avoid:
nscd[529]: 529 disabled inotify-based monitoring for file /etc/netgroup': No such file or directory nscd[529]: 529 stat failed for file /etc/netgroup'; will try again later: No such file or directory

---
 meta/recipes-core/glibc/glibc-package.inc | 1 +
 1 file changed, 1 insertion(+)

index 1c3782dbb0..c4ca94c981 100644
@@ -98,6 +98,7 @@ do_install_append () {
 	install -m 0755 ${S}/nscd/nscd.init ${D}${sysconfdir}/init.d/nscd
 	install -m 0755 ${S}/nscd/nscd.conf ${D}${sysconfdir}/nscd.conf
 	sed -i "s%daemon%start-stop-daemon --start --exec%g" ${D}${sysconfdir}/init.d/nscd
+	sed -i "s|\(enable-cache\t\+netgroup\t\+\)yes|\1no|" ${D}${sysconfdir}/nscd.conf
 
 	install -d ${D}${systemd_unitdir}/system
 	install -m 0644 ${S}/nscd/nscd.service ${D}${systemd_unitdir}/system/

2.11.0

Ping, I didn't see any objection on this, but it's not merged yet, do I miss anything?

Thanks,
Jackie

When we test it we see:

WARNING: glibc-2.26-r0 do_package: glibc-extra-nss-2.26 was registered as shlib provider for libnss_db.so.2, changing it to libnss-db-2.26 because it was built later

so it looks like you swapped one race for another?

Cheers,

Richard

Sorry, but I haven't seen this warning in our builds after this patch merged in our local branch for two weeks. Maybe we missed some cases in our test builds, so could you show me how to reproduce it? Thanks!

This patch is simply re-package libnss_db.so* from glibc-extra-nss into libnss-db, I don't understand why libnss_db.so* is still provided by both of them, did I miss anything when I do a re-packaging for a recipe?

Thanks,
Jackie

I was able to reproduce this by starting with an existing build without the patch, then applying the patch. The error shows once during the rebuild and then not again. Its therefore probably less of an issue (but does raise a question about why that is happening).

The new make dependency does cause an additional failure though:

$ oe-selftest -r bbtests.BitbakeTests.test_non_gplv3

2018-01-12 08:31:25,196 - oe-selftest - INFO - test_non_gplv3 (bbtests.BitbakeTests)
2018-01-12 08:34:02,940 - oe-selftest - INFO -  ... FAIL
2018-01-12 08:34:02,941 - oe-selftest - INFO - ======================================================================
2018-01-12 08:34:02,941 - oe-selftest - INFO - FAIL: test_non_gplv3 (bbtests.BitbakeTests)
2018-01-12 08:34:02,941 - oe-selftest - INFO - ----------------------------------------------------------------------
2018-01-12 08:34:02,941 - oe-selftest - INFO - Traceback (most recent call last):

  File "/media/build1/poky/meta/lib/oeqa/core/decorator/__init__.py", line 32, in wrapped_f
    return func(*args, **kwargs)
  File "/media/build1/poky/meta/lib/oeqa/selftest/cases/bbtests.py", line 237, in test_non_gplv3 self.assertEqual(result.status, 0, "Bitbake failed, exit code %s, output %s" % (result.status, result.output))
AssertionError: 1 != 0 : Bitbake failed, exit code 1, output Parsing recipes...done.
Parsing of 829 .bb files complete (0 cached, 829 parsed). 1287 targets, 99 skipped, 0 masked, 0 errors. Removing 35 recipes from the core2-64 sysroot...done.
Removing 35 recipes from the qemux86-64 sysroot...done.
NOTE: Resolving any missing task queue dependencies
ERROR: Nothing RPROVIDES 'make' (but /media/build1/poky/meta/recipes-core/glibc/glibc_2.26.bb RDEPENDS on or otherwise requires it) make was skipped: it has an incompatible license: GPLv3 & LGPLv2
NOTE: Runtime target 'make' is unbuildable, removing...
Missing or unbuildable dependency chain was: ['make']
ERROR: Required build target 'selftest-ed' has no buildable providers.
Missing or unbuildable dependency chain was: ['selftest-ed', 'virtual/libc', 'make']

Cheers,

Richard

Sorry, I didn't noticed that make is gplv3. I change the dependency to RSUGGESTS and the test passed:

$ oe-selftest -r bbtests.BitbakeTests.test_non_gplv3
2018-01-16 21:24:54,920 - oe-selftest - WARNING - meta-selftest layer not found in BBLAYERS, adding it
2018-01-16 21:25:05,010 - oe-selftest - INFO - Adding layer libraries:
2018-01-16 21:25:05,011 - oe-selftest - INFO -  /buildarea2/jhuang0/git/yp/poky/meta/lib
2018-01-16 21:25:05,011 - oe-selftest - INFO -  /buildarea2/jhuang0/git/yp/poky/meta-yocto-bsp/lib
2018-01-16 21:25:05,011 - oe-selftest - INFO -  /buildarea2/jhuang0/git/yp/poky/meta-selftest/lib
2018-01-16 21:25:05,012 - oe-selftest - INFO - Running bitbake -p
2018-01-16 21:25:10,565 - oe-selftest - INFO - Adding: "include selftest.inc" in /buildarea2/jhuang0/yp/y_x64_180112/conf/local.conf
2018-01-16 21:25:10,566 - oe-selftest - INFO - Adding: "include bblayers.inc" in bblayers.conf
2018-01-16 21:25:10,566 - oe-selftest - INFO - test_non_gplv3 (bbtests.BitbakeTests)
2018-01-16 21:46:22,538 - oe-selftest - INFO -  ... ok
2018-01-16 21:46:22,539 - oe-selftest - INFO - ----------------------------------------------------------------------
2018-01-16 21:46:22,539 - oe-selftest - INFO - Ran 1 test in 1271.974s
2018-01-16 21:46:22,539 - oe-selftest - INFO - OK
2018-01-16 21:46:22,539 - oe-selftest - INFO - RESULTS:
2018-01-16 21:46:22,539 - oe-selftest - INFO - RESULTS - bbtests.BitbakeTests.test_non_gplv3 - Testcase 1119: PASSED
2018-01-16 21:46:22,540 - oe-selftest - INFO - SUMMARY:
2018-01-16 21:46:22,540 - oe-selftest - INFO - oe-selftest () - Ran 1 test in 1271.975s
2018-01-16 21:46:22,540 - oe-selftest - INFO - oe-selftest - OK - All required tests passed

Thanks,
Jackie