V2 Incremental image generation(rpm based rootfs)

Incremental image generation, the rootfs would be totally removed and re-created in the second generation by default, but with INC_IMAGE_GEN = "1", the rootfs would be kept, and will do update(remove/add some pkgs) on it.
NOTE: This is not suggested when you want to create a productive rootfs

For example:
1) Add the follow config option to a conf file:

    INC_IMAGE_GEN = "1"

2) bitbake core-image-sato
modify a package
bitbake core-image-sato

The rootfs would not be totally removed and re-created in the second generation, it would be simply updated based on the "package".

Implatation:
1) Figure out the pkg which need to be removed or re-installed, then use 'rpm -e to remove the old one. Use the rpm's BUILDTIME to determine which pkg has been rebuilt.

2) Figure out the pkg which is newly added, and use 'rpm -U' to install it.

This only for the rpm based rootfs, I don't know whether we also need this for ipk or deb based rootfs.

Incremental image generation, the rootfs would be totally removed and re-created in the second generation by default, but with INC_IMAGE_GEN = "1", the rootfs would be kept, and will do update(remove/add some pkgs) on it.
NOTE: This is not suggested when you want to create a productive rootfs

For example:
1) Add the follow config option to a conf file:

     INC_IMAGE_GEN = "1"

2) bitbake core-image-sato
modify a package
bitbake core-image-sato

The rootfs would not be totally removed and re-created in the second generation, it would be simply updated based on the "package".

Implatation:
1) Figure out the pkg which need to be removed or re-installed, then use 'rpm -e to remove the old one. Use the rpm's BUILDTIME to determine which pkg has been rebuilt.

2) Figure out the pkg which is newly added, and use 'rpm -U' to install it.

This only for the rpm based rootfs, I don't know whether we also need this for ipk or deb based rootfs.

[YOCTO #1651]

Signed-off-by Robert Yang <[email protected]>

---
 meta/classes/image.bbclass       |   11 +++++-
 meta/classes/package_rpm.bbclass |   72 ++++++++++++++++++++++++++++++++-----
 2 files changed, 71 insertions(+), 12 deletions(-)

meta/classes/image.bbclass +9 -2

index 865d430..71b60f3 100644
@@ -134,15 +134,22 @@ do_rootfs[umask] = 022
 
 fakeroot do_rootfs () {
 	#set -x
-	rm -rf ${IMAGE_ROOTFS}
+    # When use the incremental image generation, don't remove the rootfs
+    if [ "${INC_IMAGE_GEN}" != "1" ]; then
+        rm -rf ${IMAGE_ROOTFS}
+    fi
 	rm -rf ${MULTILIB_TEMP_ROOTFS}
 	mkdir -p ${IMAGE_ROOTFS}
 	mkdir -p ${DEPLOY_DIR_IMAGE}
 
 	cp ${COREBASE}/meta/files/deploydir_readme.txt ${DEPLOY_DIR_IMAGE}/README_-_DO_NOT_DELETE_FILES_IN_THIS_DIRECTORY.txt
 
-	if [ "${USE_DEVFS}" != "1" ]; then
+    # If "${IMAGE_ROOTFS}/dev" exists, then the device had been made by
+    # the previous build
+	if [ "${USE_DEVFS}" != "1" -a ! -r "${IMAGE_ROOTFS}/dev" ]; then
 		for devtable in ${@get_devtable_list(d)}; do
+            # Always return ture since there maybe already one when use the
+            # incremental image generation
 			makedevs -r ${IMAGE_ROOTFS} -D $devtable
 		done
 	fi

meta/classes/package_rpm.bbclass +35 -0

index d03dc3f..541d351 100644
@@ -147,6 +147,66 @@ resolve_package_rpm () {
 	echo $pkg_name
 }
 
+# rpm common command and options
+rpm_common_comand () {
+
+    local target_rootfs="${INSTALL_ROOTFS_RPM}"
+    local extra_args="$@"
+
+    ${RPM} --root ${target_rootfs} \
+        --predefine "_rpmds_sysinfo_path ${target_rootfs}/etc/rpm/sysinfo" \
+        --predefine "_rpmrc_platform_path ${target_rootfs}/etc/rpm/platform" \
+        -D "_var ${localstatedir}" \
+        -D "_dbpath ${rpmlibdir}" \
+        --noparentdirs --nolinktos \
+        -D "__dbi_txn create nofsync private" \
+        -D "_cross_scriptlet_wrapper ${WORKDIR}/scriptlet_wrapper" $extra_args
+}
+
+# install or remove the pkg
+rpm_update_pkg () {
+
+    local target_rootfs="${INSTALL_ROOTFS_RPM}"
+
+    # Save the rpm's build time for incremental image generation, and the file
+    # would be moved to ${T}
+    rm -f ${target_rootfs}/install/total_solution_bt.manifest
+    for i in `cat ${target_rootfs}/install/total_solution.manifest`; do
+        # Use "rpm" rather than "${RPM}" here, since we don't need the
+        # '--dbpath' option
+        echo "$i `rpm -qp --qf '%{BUILDTIME}\n' $i`" >> \
+            ${target_rootfs}/install/total_solution_bt.manifest
+    done
+
+    # Only install the different pkgs if incremental image generation is set
+    if [ "${INC_IMAGE_GEN}" = "1" -a -f ${T}/total_solution_bt.manifest ]; then
+        cur_list="${target_rootfs}/install/total_solution_bt.manifest"
+        pre_list="${T}/total_solution_bt.manifest"

Quoted 6 lines · Robert Yang

Incremental image generation, the rootfs would be totally removed and re-created in the second generation by default, but with INC_IMAGE_GEN = "1", the rootfs would be kept, and will do update(remove/add some pkgs) on it.
it.

Quoted 2 lines

This only for the rpm based rootfs, I don't know whether we also need this for ipk or deb based rootfs.

As Koen points out, this should be called INC_RPM_IMAGE_GEN since it only works for rpm at present. It would be good to make this work on the other package backends. If we keep the generic variable name, the other package backends could error out saying they don't support it.

Quoted 4 lines

[YOCTO #1651]

Signed-off-by Robert Yang<[email protected]>

---

meta/classes/package_rpm.bbclass resolve_package_rpm () { · line 194

Quoted 4 lines

meta/classes/package_rpm.bbclass +4 -0

+
+        # Attempt to remove unwanted pkgs, the scripts(pre, post, etc.) has not
+        # been run by now, so don't have to run them(preun, postun, etc.) when
+        # erase the pkg

Really? I'd have thought it possible the scripts have run during the previous do_rootfs?

Otherwise the patch looks reasonable to me.

Cheers,

Richard