Skip to contentSkip to conversation

Change the way of CONFFILES handling

PATCH New Chen Qi 1 participant 5 messages Feb 17, 2015

lore
Share & tools
lore

Chen Qi

author
#0 lore raw

v3 cover Change the way of CONFFILES handling

The following changes since commit d5af8539c0a1718a7254bcdcfa973e3c887dfbd6:

syslinux: support ext2/3/4 device (2015-02-15 08:11:19 +0000)

are available in the git repository at:

git://git.openembedded.org/openembedded-core-contrib ChenQi/CONFFILES
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=ChenQi/CONFFILES

Chen Qi (4):
packaging: allow globs in CONFFILES
package_manager.py: use 'purge' instead of 'remove' in case of deb update-rc.d: use '-f' option in updatercd_postrm useradd.bbclass: avoid do_rootfs error for debian package backend

 meta/classes/package.bbclass     | 79 +++++++++++++++++++++++++++++++---------
 meta/classes/package_deb.bbclass |  2 +-
 meta/classes/package_ipk.bbclass |  2 +-
 meta/classes/package_rpm.bbclass |  2 +-
 meta/classes/update-rc.d.bbclass |  4 +-
 meta/classes/useradd.bbclass     | 10 +++++
 meta/lib/oe/package_manager.py   |  4 +-
 7 files changed, 78 insertions(+), 25 deletions(-)
Signature · 1 line
1.9.1
author
#1 lore raw

PATCH v3 1/4 packaging: allow globs in CONFFILES

Allow globs in CONFFILES.

This patch changes the way of CONFFILES handling. After this change, the CONFFILES can take the same form as FILES. That means, we don't have to list a bunch of files for CONFFILES. It will just be expanded like the FILES variable.

We don't assume default value for CONFFILES in OE. But distro vendors could provide a default value for CONFFILES in their distro configuration file like below.

    CONFFILES = "${sysconfdir}"

In this way, files under /etc are treated as configuration files by default. Of course, setting CONFFILES in recipes take precedence over the CONFFILES. For example, if the recipe author decides that package A should only treat files under ${sysconfdir}/default/ as config files, he/she can write like this.

    CONFFILES_A = "${sysconfdir}/default"

[YOCTO #5200]

---
 meta/classes/package.bbclass     | 79 +++++++++++++++++++++++++++++++---------
 meta/classes/package_deb.bbclass |  2 +-
 meta/classes/package_ipk.bbclass |  2 +-
 meta/classes/package_rpm.bbclass |  2 +-
 4 files changed, 64 insertions(+), 21 deletions(-)
meta/classes/package.bbclass +61 -18
index f6c92cb..9f64ed7 100644
@@ -239,6 +239,66 @@ python () {
         d.appendVarFlag('do_package', 'deptask', " do_packagedata")
 }
 
+# Get a list of files from file vars by searching files under current working directory
+# The list contains symlinks, directories and normal files.
+def files_from_filevars(filevars):
+    import os,glob
+    cpath = oe.cachedpath.CachedPath()
+    files = []
+    for f in filevars:
+        if os.path.isabs(f):
+            f = '.' + f
+        if not f.startswith("./"):
+            f = './' + f
+        globbed = glob.glob(f)
+        if globbed:
+            if [ f ] != globbed:
+                files += globbed
+                continue
+        files.append(f)
+
+    for f in files:
+        if not cpath.islink(f):
+            if cpath.isdir(f):
+                newfiles = [ os.path.join(f,x) for x in os.listdir(f) ]
+                if newfiles:
+                    files += newfiles
+
+    return files
+
+# Called in package_<rpm,ipk,deb>.bbclass to get the correct list of configuration files
+def get_conffiles(pkg, d):
+    pkgdest = d.getVar('PKGDEST', True)
+    root = os.path.join(pkgdest, pkg)
+    cwd = os.getcwd()
+    os.chdir(root)
+
+    conffiles = d.getVar('CONFFILES_%s' % pkg, True);
+    if conffiles == None:
+        conffiles = d.getVar('CONFFILES', True)
+    if conffiles == None:
+        conffiles = ""
+    conffiles = conffiles.split()
+    conf_orig_list = files_from_filevars(conffiles)
+
+    # Remove links and directories from conf_orig_list to get conf_list which only contains normal files
+    conf_list = []
+    for f in conf_orig_list:
+        if os.path.isdir(f):
+            continue
+        if os.path.islink(f):
+            continue
+        if not os.path.exists(f):
+            continue
+        conf_list.append(f)
+
+    # Remove the leading './'
+    for i in range(0, len(conf_list)):
+        conf_list[i] = conf_list[i][1:]
+
+    os.chdir(cwd)
+    return conf_list
+
 def splitdebuginfo(file, debugfile, debugsrcdir, sourcefile, d):
     # Function to split a single file into two components, one is the stripped
     # target system binary, the other contains any debugging information. The
@@ -1009,26 +1069,9 @@ python populate_packages () {
             filesvar.replace("//", "/")
 
         origfiles = filesvar.split()
-        files = []
-        for file in origfiles:
-            if os.path.isabs(file):
-                file = '.' + file
-            if not file.startswith("./"):
-                file = './' + file
-            globbed = glob.glob(file)
-            if globbed:
-                if [ file ] != globbed:
-                    files += globbed
-                    continue
-            files.append(file)
+        files = files_from_filevars(origfiles)
 
         for file in files:
-            if not cpath.islink(file):
-                if cpath.isdir(file):
-                    newfiles =  [ os.path.join(file,x) for x in os.listdir(file) ]
-                    if newfiles:
-                        files += newfiles
-                        continue
             if (not cpath.islink(file)) and (not cpath.exists(file)):
                 continue
             if file in seen:
meta/classes/package_deb.bbclass +1 -1
index 5b5f7e2..9d7c59b 100644
@@ -262,7 +262,7 @@ python do_package_deb () {
             scriptfile.close()
             os.chmod(os.path.join(controldir, script), 0755)
 
-        conffiles_str = localdata.getVar("CONFFILES", True)
+        conffiles_str = ' '.join(get_conffiles(pkg, d))
         if conffiles_str:
             try:
                 conffiles = open(os.path.join(controldir, 'conffiles'), 'w')
meta/classes/package_ipk.bbclass +1 -1
index 44fd3eb..dba6804 100644
@@ -226,7 +226,7 @@ python do_package_ipk () {
             scriptfile.close()
             os.chmod(os.path.join(controldir, script), 0755)
 
-        conffiles_str = localdata.getVar("CONFFILES", True)
+        conffiles_str = ' '.join(get_conffiles(pkg, d))
         if conffiles_str:
             try:
                 conffiles = open(os.path.join(controldir, 'conffiles'), 'w')
meta/classes/package_rpm.bbclass +1 -1
index 92ddf7a..b87e634 100644
@@ -324,7 +324,7 @@ python write_specfile () {
 
         bb.data.update_data(localdata)
 
-        conffiles = (localdata.getVar('CONFFILES', True) or "").split()
+        conffiles = get_conffiles(pkg, d)
         dirfiles = localdata.getVar('DIRFILES', True)
         if dirfiles is not None:
             dirfiles = dirfiles.split()
Signature · 1 line
1.9.1
author
#2 lore raw

PATCH v3 2/4 package_manager.py: use 'purge' instead of 'remove' in case of deb

In reply to Chen Qi

We need to use 'purge' instead of 'remove' for debian package backend when removing packages at rootfs time. This is because that 'remove' command for 'dpkg' and 'apt-get' does not remove configuration files.

---
 meta/lib/oe/package_manager.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
meta/lib/oe/package_manager.py +2 -2
index fcf05dc..070b79d 100644
@@ -1628,10 +1628,10 @@ class DpkgPM(PackageManager):
     def remove(self, pkgs, with_dependencies=True):
         if with_dependencies:
             os.environ['APT_CONFIG'] = self.apt_conf_file
-            cmd = "%s remove %s" % (self.apt_get_cmd, ' '.join(pkgs))
+            cmd = "%s purge %s" % (self.apt_get_cmd, ' '.join(pkgs))
         else:
             cmd = "%s --admindir=%s/var/lib/dpkg --instdir=%s" \
-                  " -r --force-depends %s" % \
+                  " -P --force-depends %s" % \
                   (bb.utils.which(os.getenv('PATH'), "dpkg"),
                    self.target_rootfs, self.target_rootfs, ' '.join(pkgs))
 
Signature · 1 line
1.9.1
author
#3 lore raw

PATCH v3 3/4 update-rc.d: use '-f' option in updatercd_postrm

In reply to Chen Qi

Use '-f' ('--force') option so that while removing packages using deb, we don't fail because of the following error:

update-rc.d: $initd/$bn exists during rc.d purge (use -f to force)

Using '-f' option would make this a warning but continue to execute.

update-rc.d: $initd/$bn exists during rc.d purge (continuing)

We need this option because dpkg package backend have special handling for configuration files. And if files under /etc/init.d are treated as configuration files, we will have errors.

---
 meta/classes/update-rc.d.bbclass | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
meta/classes/update-rc.d.bbclass +2 -2
index bc1aa7d..a9c0323 100644
@@ -48,9 +48,9 @@ fi
 updatercd_postrm() {
 if type update-rc.d >/dev/null 2>/dev/null; then
 	if [ -n "$D" ]; then
-		OPT="-r $D"
+		OPT="-f -r $D"
 	else
-		OPT=""
+		OPT="-f"
 	fi
 	update-rc.d $OPT ${INITSCRIPT_NAME} remove
 fi
Signature · 1 line
1.9.1
author
#4 lore raw

PATCH v3 4/4 useradd.bbclass: avoid do_rootfs error for debian package backend

In reply to Chen Qi

If /etc/login.defs is treated as a configuration file, then we would meet errors at do_rootfs time telling us that useradd/groupadd cannot execute correctly.

This is because the dpkg handles config file specially, the login.defs is temporarily renamed as login.defs.dpkg-new.

How ubuntu deals the user/group adding problem? They do it at postinst of the package. And, the postinst script of a package would possibly do `chown' of its files or directories.

The above strategy is not suitable for OE. Because we do chown in do_install and add user/group in preinst scripts of the packages.

That's why we need this patch so that do_rootfs don't fail.

---
 meta/classes/useradd.bbclass | 10 ++++++++++
 1 file changed, 10 insertions(+)
meta/classes/useradd.bbclass +10 -0
index 0b9a843..e443f84 100644
@@ -24,6 +24,16 @@ if test "x$D" != "x"; then
 	# Installing into a sysroot
 	SYSROOT="$D"
 	OPT="--root $D"
+
+	# Make sure login.defs is there, this is to make debian package backend work
+	# correctly while doing rootfs.
+	# The problem here is that if /etc/login.defs is treated as a config file for
+	# shadow package, then while performing preinsts for packages that depend on
+	# shadow, there might only be /etc/login.def.dpkg-new there in root filesystem.
+	if [ ! -e $D${sysconfdir}/login.defs -a -e $D${sysconfdir}/login.defs.dpkg-new ]; then
+	    cp $D${sysconfdir}/login.defs.dpkg-new $D${sysconfdir}/login.defs
+	fi
+
 	# user/group lookups should match useradd/groupadd --root
 	export PSEUDO_PASSWD="$SYSROOT:${STAGING_DIR_NATIVE}"
 fi
Signature · 1 line
1.9.1