Skip to contentSkip to conversation

add two zram IMAGE_FEATUREs

RFC PATCH In review Trevor Woerner 2 participants 4 messages Aug 18, 2021

lore
Share & tools
lore
#0 lore raw

RFC PATCH v2 1/1 add two zram IMAGE_FEATUREs

The zram kernel module creates RAM-based block devices named /dev/zram<id> (<id> = 0, 1, ...). Pages written to these disks are compressed and stored in memory itself. These disks allow very fast I/O and compression provides good amounts of memory savings. Some of the use cases include /tmp storage, use as swap disks, various caches under /var and maybe many more.

If the user wants to switch from using uncompressed tmpfs to compressed zram-based in-RAM temporary filesystems then add "zram-tmpfs" to IMAGE_FEATURES. Currently the base install uses tmpfs filesystems for /run and /var/volatile. By default the /run filesystem will be 10MiB in size and the /var/volatile (which holds /tmp, among others) will be 30MiB in size. The following variables can be tweaked to adjust these sizes:
ZRAM_RUN_SIZE
ZRAM_TMP_SIZE

If the user wants to enable an in-RAM, compressed, zram-backed SWAP for their system, simply add "zram-swap" to IMAGE_FEATURES. The size of the swap will be based on a percentage of the total system RAM which the user can adjust by modifying the following variable:
ZRAM_SWAP_PERCENT

By default ZRAM_SWAP_PERCENT is set to "100" which means the initial SWAP size will be the same size as the total system RAM.

The kernel provides several compression algorithms from which the user can choose. By default the "lzo-rle" compression is used, but the user can modify the following variable to choose from any one of the other available compression algorithms:
ZRAM_ALGORITHM

Currently the set of available compression algorithms include:

	lzo
	lzo-rle
	lz4
	lz4hc
	842
	zstd

!!!NOTE!!!
this is a WIP patch submitted for RFC
this patch only works, currently, with sysvinit
but I wanted to get feedback on the general approach before tackling systemd

---
NOTE:
- this patch relies on there being a zram.scc available in the yocto-kernel-cache repository, a patch for which was submitted moments ago
https://lists.yoctoproject.org/g/linux-yocto/message/10293

changes in v2
- move the check of the algorithm outside the check for zram-tmpfs since the algorithm applies to both zram-tmpfs and zram-swap
- add checks on the /var/volatile and /run sizes to make sure they're greater than 0

---
 meta/classes/core-image.bbclass               |  4 ++
 meta/classes/image.bbclass                    |  2 +-
 meta/classes/rootfs-postcommands.bbclass      | 35 +++++++++++++++
 .../initscripts/initscripts-1.0/mountall.sh   | 27 +++++++++++-
 .../packagegroup-core-zram-swap.bb            | 10 +++++
 .../packagegroup-core-zram-tmpfs.bb           |  9 ++++
 .../sysvinit/sysvinit/rcS-default             |  8 ++++
 meta/recipes-core/sysvinit/sysvinit_2.99.bb   | 44 ++++++++++++++++++-
 meta/recipes-kernel/linux/linux-yocto.inc     |  2 +
 9 files changed, 137 insertions(+), 4 deletions(-)
 create mode 100644 meta/recipes-core/packagegroups/packagegroup-core-zram-swap.bb
 create mode 100644 meta/recipes-core/packagegroups/packagegroup-core-zram-tmpfs.bb
meta/classes/core-image.bbclass +4 -0
index 84fd3eeb38..ed6184d11c 100644
@@ -39,6 +39,8 @@
 # - read-only-rootfs    - tweaks an image to support read-only rootfs
 # - stateless-rootfs    - systemctl-native not run, image populated by systemd at runtime
 # - splash              - bootup splash screen
+# - zram-tmpfs          - use a zram disk for temporary filesystems instead of tmpfs
+# - zram-swap           - use a zram-backed swap
 #
 FEATURE_PACKAGES_weston = "packagegroup-core-weston"
 FEATURE_PACKAGES_x11 = "packagegroup-core-x11"
@@ -53,6 +55,8 @@ FEATURE_PACKAGES_nfs-server = "packagegroup-core-nfs-server"
 FEATURE_PACKAGES_nfs-client = "packagegroup-core-nfs-client"
 FEATURE_PACKAGES_ssh-server-dropbear = "packagegroup-core-ssh-dropbear"
 FEATURE_PACKAGES_ssh-server-openssh = "packagegroup-core-ssh-openssh"
+FEATURE_PACKAGES_zram-tmpfs = "packagegroup-core-zram-tmpfs"
+FEATURE_PACKAGES_zram-swap = "packagegroup-core-zram-swap"
 FEATURE_PACKAGES_hwcodecs = "${MACHINE_HWCODECS}"
 
 
meta/classes/image.bbclass +1 -1
index d76895178f..57178baace 100644
@@ -33,7 +33,7 @@ INHIBIT_DEFAULT_DEPS = "1"
 # IMAGE_FEATURES may contain any available package group
 IMAGE_FEATURES ?= ""
 IMAGE_FEATURES[type] = "list"
-IMAGE_FEATURES[validitems] += "debug-tweaks read-only-rootfs read-only-rootfs-delayed-postinsts stateless-rootfs empty-root-password allow-empty-password allow-root-login post-install-logging"
+IMAGE_FEATURES[validitems] += "debug-tweaks read-only-rootfs read-only-rootfs-delayed-postinsts stateless-rootfs empty-root-password allow-empty-password allow-root-login post-install-logging zram-tmpfs zram-swap"
 
 # Generate companion debugfs?
 IMAGE_GEN_DEBUGFS ?= "0"
meta/classes/rootfs-postcommands.bbclass +35 -0
index c5746eba13..a1e00aeebc 100644
@@ -17,6 +17,12 @@ ROOTFS_POSTPROCESS_COMMAND += "rootfs_update_timestamp; "
 # Tweak the mount options for rootfs in /etc/fstab if read-only-rootfs is enabled
 ROOTFS_POSTPROCESS_COMMAND += '${@bb.utils.contains("IMAGE_FEATURES", "read-only-rootfs", "read_only_rootfs_hook; ", "",d)}'
 
+# Use zram for temporary filesystems instead of tmpfs
+ROOTFS_POSTPROCESS_COMMAND += '${@bb.utils.contains("IMAGE_FEATURES", "zram-tmpfs", "zram_tmpfs_hook; ", "",d)}'
+
+# Use a zram-backed swap
+ROOTFS_POSTPROCESS_COMMAND += '${@bb.utils.contains("IMAGE_FEATURES", "zram-swap", "zram_swap_hook; ", "",d)}'
+
 # We also need to do the same for the kernel boot parameters,
 # otherwise kernel or initramfs end up mounting the rootfs read/write
 # (the default) if supported by the underlying storage.
@@ -143,6 +149,35 @@ read_only_rootfs_hook () {
 	fi
 }
 
+#
+# A hook function to support zram-tmpfs IMAGE_FEATURES
+#
+zram_tmpfs_hook () {
+	# Remove tmpfs entries from fstab
+	if [ -f ${IMAGE_ROOTFS}/etc/fstab ]; then
+		sed -n -i -e '/tmpfs/!p' ${IMAGE_ROOTFS}/etc/fstab
+	fi
+
+	# Enable zram when sysvinit is used
+	if ${@bb.utils.contains("DISTRO_FEATURES", "sysvinit", "true", "false", d)}; then
+		if [ -e ${IMAGE_ROOTFS}/etc/default/rcS ]; then
+			sed -i 's/USE_ZRAM_TMPFS=no/USE_ZRAM_TMPFS=yes/' ${IMAGE_ROOTFS}/etc/default/rcS
+		fi
+	fi
+}
+
+#
+# A hook function to support using a zram-backed swap
+#
+zram_swap_hook () {
+	# Enable zram when sysvinit is used
+	if ${@bb.utils.contains("DISTRO_FEATURES", "sysvinit", "true", "false", d)}; then
+		if [ -e ${IMAGE_ROOTFS}/etc/default/rcS ]; then
+			sed -i 's/USE_ZRAM_SWAP=no/USE_ZRAM_SWAP=yes/' ${IMAGE_ROOTFS}/etc/default/rcS
+		fi
+	fi
+}
+
 #
 # This function is intended to disallow empty root password if 'debug-tweaks' is not in IMAGE_FEATURES.
 #
meta/recipes-core/initscripts/initscripts-1.0/mountall.sh +26 -1
index 2839d57cbe..996709bce6 100755
@@ -35,11 +35,36 @@ if [ ! -p "$INITCTL" ] && [ "${INIT_SYSTEM}" = "sysvinit" ]; then
 		[ -n "$PID" ] && kill -s USR1 "$PID"
 fi
 
+# If the user wants to use zram tmpfs
+# create and mount those now
+if [ "$USE_ZRAM_TMPFS"z = "yes"z ]; then
+	ZDEVICE=$(zramctl -f -s ${ZRAM_TMP_SIZE}MiB -a ${ZRAM_ALGORITHM})
+	if [ $? -eq 0 ]; then
+		mkfs.ext2 $ZDEVICE
+		mount $ZDEVICE /var/volatile
+	fi
+	ZDEVICE=$(zramctl -f -s ${ZRAM_RUN_SIZE}MiB -a ${ZRAM_ALGORITHM})
+	if [ $? -eq 0 ]; then
+		mkfs.ext2 $ZDEVICE
+		mount $ZDEVICE /run
+	fi
+fi
+
 #
 # Execute swapon command again, in case we want to swap to
 # a file on a now mounted filesystem.
 #
-[ -x /sbin/swapon ] && swapon -a
+if [ "$USE_ZRAM_SWAP"z = "yes"z ]; then
+	MEMTOTAL=$(grep MemTotal /proc/meminfo | awk ' { print $2 } ')
+	MEMZRAM=$((${MEMTOTAL}*${ZRAM_SWAP_PERCENT}/100))
+	ZDEVICE=$(zramctl -f -s ${MEMZRAM}KiB -a ${ZRAM_ALGORITHM})
+	if [ $? -eq 0 ]; then
+		mkswap -L "zram-swap" $ZDEVICE
+		swapon -p 100 $ZDEVICE
+	fi
+else
+	[ -x /sbin/swapon ] && swapon -a
+fi
 
 : exit 0
 
meta/recipes-core/packagegroups/packagegroup-core-zram-swap.bb new file +10 -0
new file mode 100644
index 0000000000..fa6da61b47
@@ -0,0 +1,10 @@
+SUMMARY = "Use zram-backed swap"
+PR = "r1"
+
+inherit packagegroup
+
+RDEPENDS:${PN} = " \
+	util-linux-zramctl \
+	util-linux-swaponoff \
+	util-linux-mkswap \
+	"
meta/recipes-core/packagegroups/packagegroup-core-zram-tmpfs.bb new file +9 -0
new file mode 100644
index 0000000000..84b6642340
@@ -0,0 +1,9 @@
+SUMMARY = "Use zram for temporary directories instead of tmpfs"
+PR = "r1"
+
+inherit packagegroup
+
+RDEPENDS:${PN} = " \
+	util-linux-zramctl \
+	e2fsprogs-mke2fs \
+	"
meta/recipes-core/sysvinit/sysvinit/rcS-default +8 -0
index f7c4a2f841..91739183d9 100644
@@ -34,3 +34,11 @@ INIT_SYSTEM=sysvinit
 PSPLASH_FIFO_DIR=/mnt
 # psplash textual updates knob
 PSPLASH_TEXT_UPDATES=#PSPLASH_TEXT#
+# zram
+# zram can be used for swap or tmp filesystems (or both)
+ZRAM_ALGORITHM=#ZRAM_ALGORITHM#
+USE_ZRAM_SWAP=#USE_ZRAM_SWAP#
+ZRAM_SWAP_PERCENT=#ZRAM_SWAP_PERCENT#
+USE_ZRAM_TMPFS=#USE_ZRAM_TMPFS#
+ZRAM_TMP_SIZE=#ZRAM_TMP_SIZE#
+ZRAM_RUN_SIZE=#ZRAM_RUN_SIZE#
meta/recipes-core/sysvinit/sysvinit_2.99.bb +42 -2

56 lines not on this page · raw message

meta/recipes-kernel/linux/linux-yocto.inc +2 -0
index 331727d62c..66aa8df4a3 100644
@@ -35,6 +35,8 @@ KERNEL_FEATURES:append:qemuall=" features/debug/printk.scc"
 
 KERNEL_FEATURES:append = " ${@bb.utils.contains('MACHINE_FEATURES', 'numa', 'features/numa/numa.scc', '', d)}"
 KERNEL_FEATURES:append = " ${@bb.utils.contains('MACHINE_FEATURES', 'vfat', 'cfg/fs/vfat.scc', '', d)}"
+KERNEL_FEATURES:append = " ${@bb.utils.contains('IMAGE_FEATURES', 'zram-tmpfs', 'features/zram/zram.scc', '', d)}"
+KERNEL_FEATURES:append = " ${@bb.utils.contains('IMAGE_FEATURES', 'zram-swap', 'features/zram/zram.scc', '', d)}"
 
 # A KMACHINE is the mapping of a yocto $MACHINE to what is built
 # by the kernel. This is typically the branch that should be built,
Signature · 1 line
2.30.0.rc0
#1 lore raw
Quoted 3 lines · Trevor Woerner

this is a WIP patch submitted for RFC
this patch only works, currently, with sysvinit
but I wanted to get feedback on the general approach before tackling systemd

How about using/extending the zram package in meta-openembedded? Would that be an option?

#2 lore raw
Quoted 7 lines · Sinan Kaya
Quoted 3 lines · Trevor Woerner

this is a WIP patch submitted for RFC
this patch only works, currently, with sysvinit
but I wanted to get feedback on the general approach before tackling systemd

How about using/extending the zram package in meta-openembedded? Would that be an option?

When I found the zram package in meta-openembedded I had thought that my work was done. I figured I'd add it to my image and move on. Unfortunately the zram package in meta-openembedded is woefully inadequate :-(

In fact, if my patch gets accepted into oe-core, my next patch would be to remove this package from meta-openembedded ;-)

So far I have only considered the case where the user wants to use zram with sysvinit. I haven't looked at the systemd case yet, but if it has as many holes as the sysvinit case…

If you look at the package in meta-openembedded you'll see that it:
- tries to do its work using start/stop initscripts
- doesn't even try to handle the case where the user wants to use zram instead of tmpfs, and that's not a coincidence

A running system will happily incorporate added SWAP anytime that SWAP is added. Simply prepare the partition for SWAP as appropriate then "swapon" and everything is happy. Substituting zram for tmpfs, however is a lot more finicky. You can't simply wait until after tmpfs has been used for your various tmp partitions then mount new partitions on top of the existing /var/volatile and /run. This will cause your logger to get upset, and it will cause pretty much every currently-running program to not be very happy. Programs that are started and put their pids in /run and various things in /tmp don't usually react favourably when those filesystems have something else mounted on top of them during their execution.

Therefore in order to properly switch from tmpfs to zram for tmp filesystems:
- /etc/fstab has to be modified to not try to mount tmpfs things to /var/volatile and /run
- the place where "mount -a" is called has to be modified to setup, format, and mount zram-based disks. specifying zram disks is not something you can simply tweak /etc/fstab to do. according to the kernel documentation there is no
zram /var/volatile zram,ext2 defaults 0 2
line that can be added to /etc/fstab to get zram-based filesystems to mount automatically via "mount -a"

Because this change is much more invasive than simply running a script at a given runlevel I came to the conclusion that an IMAGE_FEATURE would be a better fit for this sort of feature.

I also took the opportunity to parameterize various parts of the configuration to make it easy for the user to either accept the sane defaults, or easily tweak them to their liking. I added checks to these parameters so that they're valid/sane. I added the feature in a way that any required packages are added to the image automatically without the user having to do anything. I also tied into the kernel's "kmeta config" system so that simply enabling this IMAGE_FEATURE will also tweak the kernel config to add the necessary options (again, without user intervention).

Also, if you look at the existing sysvinit script in meta-openembedded you'll notice that it jumps through some hoops trying to figure out how many cores/cpus are on the system, then it uses that number as a basis for how many SWAPs to create. You can tell this is a very old script because, according to the kernel documentation, this is something that was required a long time ago, but has long since is no longer required. Simply create one SWAP or one zram-backed /tmp mount and the number of "streams" is automatically handled internally in the kernel:

	root@rock-pi-e:~# zramctl
	NAME       ALGORITHM DISKSIZE  DATA COMPR TOTAL STREAMS MOUNTPOINT
	/dev/zram2 lz4         975.6M    4K   73B   12K       4 [SWAP]
	/dev/zram1 lz4            10M  400K 16.1K  240K       4 /run
	/dev/zram0 lz4            30M   68K  1.6K   68K       4 /var/volatile

So, in conclusion, the stuff on meta-openembmedded's zram package is:
- old
- has very few knobs for the user to tweak
- doesn't handle the zram-tmpfs case
- doesn't update your kernel config automatically

(at least for the sysvinit portion)

#3 lore raw
Quoted 11 lines · Trevor Woerner
root@rock-pi-e:~# zramctl
	NAME       ALGORITHM DISKSIZE  DATA COMPR TOTAL STREAMS MOUNTPOINT
	/dev/zram2 lz4         975.6M    4K   73B   12K       4 [SWAP]
	/dev/zram1 lz4            10M  400K 16.1K  240K       4 /run
	/dev/zram0 lz4            30M   68K  1.6K   68K       4 /var/volatile

So, in conclusion, the stuff on meta-openembmedded's zram package is:
- old
- has very few knobs for the user to tweak
- doesn't handle the zram-tmpfs case
- doesn't update your kernel config automatically

I see your point. zram package is focused on swap partitions only. You seem to go beyond swap partitions and tap into tmpfs next.