Persistent /var/log support

The following changes since commit a675b2c89e477af088faee9b3be96eae19a85f0b:

sanity.bbclass: fix logging of an error (2016-11-15 15:18:50 +0000)

are available in the git repository at:

git://git.openembedded.org/openembedded-core-contrib ChenQi/persistent-var-log
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=ChenQi/persistent-var-log

Chen Qi (4):
bitbake.conf: add VOLATILE_LOG_DIR variable
base-files: respect VOLATILE_LOG_DIR
initscripts: support persistent /var/log
package.bbclass: support persistent /var/log

 meta/classes/package.bbclass                       |  2 +-
 meta/conf/bitbake.conf                             |  3 +
 meta/files/fs-perms-persistent-log.txt             | 69 ++++++++++++++++++++++
 meta/recipes-core/base-files/base-files_3.0.14.bb  |  4 +-
 .../initscripts/initscripts-1.0/volatiles          |  1 -
 meta/recipes-core/initscripts/initscripts_1.0.bb   |  3 +
 6 files changed, 78 insertions(+), 4 deletions(-)
 create mode 100644 meta/files/fs-perms-persistent-log.txt

1.9.1

The default value is "yes" which results in the /var/log being a link pointing to /var/volatile/log which is on tmpfs.

The user could override this value to "no" which causes /var/log to be a directory on persistent storage.

[YOCTO #6132]

---
 meta/conf/bitbake.conf | 3 +++
 1 file changed, 3 insertions(+)

index 1472e8f..794f422 100644
@@ -83,6 +83,9 @@ USRBINPATH_class-nativesdk = "/usr/bin"
 # Root home directory
 ROOT_HOME ??= "/home/root"
 
+# If set to "yes", /var/log links to /var/volatile/log; otherwise, /var/log is on persistent storage.
+VOLATILE_LOG_DIR ?= "yes"
+
 ##################################################################
 # Architecture-dependent build variables.
 ##################################################################

1.9.1

Respect VOLATILE_LOG_DIR variable. In this way, if the user overrides this variable to be "no", /var/log on the final image would reside on persistent storage.

[YOCTO #6132]

---
 meta/recipes-core/base-files/base-files_3.0.14.bb | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

index 5333110..7cc8e8c 100644
@@ -41,7 +41,7 @@ dirs755 = "/bin /boot /dev ${sysconfdir} ${sysconfdir}/default \
            ${localstatedir}/backups ${localstatedir}/lib \
            /sys ${localstatedir}/lib/misc ${localstatedir}/spool \
            ${localstatedir}/volatile \
-           ${localstatedir}/volatile/log \
+           ${localstatedir}/${@['', 'volatile/'][d.getVar('VOLATILE_LOG_DIR', True) == 'yes']}log \
            /home ${prefix}/src ${localstatedir}/local \
            /media"
 

@@ -52,7 +52,7 @@ dirs755-lsb = "/srv  \
                ${prefix}/lib/locale"
 dirs2775-lsb = "/var/mail"
 
-volatiles = "log tmp"
+volatiles = "${@['', 'log'][d.getVar('VOLATILE_LOG_DIR', True) == 'yes']} tmp"
 conffiles = "${sysconfdir}/debian_version ${sysconfdir}/host.conf \
              ${sysconfdir}/issue /${sysconfdir}/issue.net \
              ${sysconfdir}/nsswitch.conf ${sysconfdir}/profile \

1.9.1

I find the "if/else" expression easier to read, there's no need for an explicit d.getVar(), and there's an utility method for variables which are meant to be booleans:
${@ 'volatile/' if oe.types.boolean('${VOLATILE_LOG_DIR'}) else '' }

Using the utility function ensures that typos are caught and allows several other values besides yes/no.

The change itself of course works either way. Just my 2 cents ;-}

Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.

Respect VOLATILE_VAR_LOG variable so that if it's set to "no", we could have persistent /var/log on the final image.

[YOCTO #6132]

---
 meta/recipes-core/initscripts/initscripts-1.0/volatiles | 1 -
 meta/recipes-core/initscripts/initscripts_1.0.bb        | 3 +++
 2 files changed, 3 insertions(+), 1 deletion(-)

index 297245d..6cccab7 100644
@@ -25,7 +25,6 @@ d root root 1777 /run/lock none
 d root root 0755 /var/volatile/log none
 d root root 1777 /var/volatile/tmp none
 l root root 1777 /var/lock /run/lock
-l root root 0755 /var/log /var/volatile/log
 l root root 0755 /var/run /run
 l root root 1777 /var/tmp /var/volatile/tmp
 l root root 1777 /tmp /var/tmp

index 8f110b0..14cdf6a 100644
@@ -102,6 +102,9 @@ do_install () {
 	install -m 0755    ${WORKDIR}/read-only-rootfs-hook.sh ${D}${sysconfdir}/init.d
 	install -m 0755    ${WORKDIR}/save-rtc.sh	${D}${sysconfdir}/init.d
 	install -m 0644    ${WORKDIR}/volatiles		${D}${sysconfdir}/default/volatiles/00_core
+	if [ "${VOLATILE_LOG_DIR}" = "log" ]; then
+		echo "l root root 0755 /var/log /var/volatile/log" >> ${D}${sysconfdir}/default/volatiles/00_core
+	fi
 	install -m 0755    ${WORKDIR}/dmesg.sh		${D}${sysconfdir}/init.d
 	install -m 0644    ${WORKDIR}/logrotate-dmesg.conf ${D}${sysconfdir}/
 

1.9.1

Why are you checking for "log" here? I thought VOLATILE_LOG_DIR was supposed to be a boolean?

Using oe.types.boolean in shell code is a bit awkward, but would be necessary for the sake of consistency when allowing VOLATILE_LOG_DIR to have more than just yes/no. Untested (relies on bool->str conversion):

if [ ${@ oe.types.boolean('${VOLATILE_LOG_DIR}') } = True ]; then

Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.

Thanks for your suggestion.

I'll send out V2.

Best Regards,

Chen Qi

Add a new file, fs-perms-persistent-log.txt, which treats /var/log as a directory instead of a link.

Modify package.bbclass to use this file if VOLATILE_LOG_DIR is not set to "yes".

[YOCTO #6132]

---
 meta/classes/package.bbclass           |  2 +-
 meta/files/fs-perms-persistent-log.txt | 69 ++++++++++++++++++++++++++++++++++
 2 files changed, 70 insertions(+), 1 deletion(-)
 create mode 100644 meta/files/fs-perms-persistent-log.txt

index a6f0a7a..9a423f8 100644
@@ -733,7 +733,7 @@ python fixup_perms () {
         bbpath = d.getVar('BBPATH', True)
         fs_perms_tables = d.getVar('FILESYSTEM_PERMS_TABLES', True)
         if not fs_perms_tables:
-            fs_perms_tables = 'files/fs-perms.txt'
+            fs_perms_tables = ['files/fs-perms-persistent-log.txt', 'files/fs-perms.txt'][d.getVar('VOLATILE_LOG_DIR', True) == 'yes']
         for conf_file in fs_perms_tables.split():
             str += " %s" % bb.utils.which(bbpath, conf_file)
         return str

1.9.1

Ack from me, I don't know if this is the best way to do it, but it is way way way too hard to get persistent logging in OE. I've had to implement it in numerous ways, multiple times over the past few years and it shouldn't be that hard.