initramfs: kunit tests and cleanups
Subsystems (4)
v4 cover initramfs: kunit tests and cleanups
This patchset adds basic kunit test coverage for initramfs unpacking and cleans up some minor buffer handling issues / inefficiencies.
Changes since v3:
- Drop shared unpack buffer changes
+ rework into initramfs: allocate heap buffers together (patch 5/8)
+ extra review complexity wasn't worth the tiny boot-time heap saving
- move hardlink hash leak repro into first initramfs_test patch
- add note regarding kunit section=.data -> section=.init.text warning
Changes since v2 (patch 2 only):
- fix !CONFIG_INITRAMFS_PRESERVE_MTIME kunit test checks
- add test MODULE_DESCRIPTION(), as suggested by Jeff Johnson
- add some missing headers, reported by kernel test robot
Changes since v1 (RFC):
- rebase atop v6.12-rc6 and filename field overrun fix from
https://lore.kernel.org/r/[email protected]
- add unit test coverage (new patches 1 and 2)
- add patch: fix hardlink hash leak without TRAILER
- rework patch: avoid static buffer for error message
+ drop unnecessary message propagation
- drop patch: cpio_buf reuse for built-in and bootloader initramfs
+ no good justification for the change
Feedback appreciated.
David Disseldorp (8):
init: add initramfs_internal.h
initramfs_test: kunit tests for initramfs unpacking
vsprintf: add simple_strntoul
initramfs: avoid memcpy for hex header fields
initramfs: allocate heap buffers together
initramfs: reuse name_len for dir mtime tracking
initramfs: fix hardlink hash leak without TRAILER
initramfs: avoid static buffer for error message
include/linux/kstrtox.h | 1 + init/.kunitconfig | 3 + init/Kconfig | 7 + init/Makefile | 1 + init/initramfs.c | 66 ++++---- init/initramfs_internal.h | 8 + init/initramfs_test.c | 407 ++++++++++++++++++++++++++++++++++++++++++++++ lib/vsprintf.c | 7 + 8 files changed, 472 insertions(+), 28 deletions(-) create mode 100644 init/.kunitconfig create mode 100644 init/initramfs_internal.h create mode 100644 init/initramfs_test.c
PATCH v4 1/8 init: add initramfs_internal.h
The new header only exports a single unpack function and a CPIO_HDRLEN constant for future test use.
--- init/initramfs.c | 16 +++++++++++++--- init/initramfs_internal.h | 8 ++++++++ 2 files changed, 21 insertions(+), 3 deletions(-) create mode 100644 init/initramfs_internal.h
init/initramfs.c +13 -3
@@ -20,6 +20,7 @@#include <linux/security.h> #include "do_mounts.h" +#include "initramfs_internal.h" static __initdata bool csum_present;static __initdata u32 io_csum;
@@ -256,7 +257,7 @@ static __initdata char *header_buf, *symlink_buf, *name_buf;static int __init do_start (void ) { -read_into (header_buf,110 , GotHeader); +read_into (header_buf,CPIO_HDRLEN , GotHeader);return 0 ; }
@@ -497,14 +498,23 @@ static unsigned long my_inptr __initdata; /* index of next byte to be processed#include <linux/decompress/generic.h> -static char *__init unpack_to_rootfs (char *buf,unsigned long len) +/** +* unpack_to_rootfs - decompress and extract an initramfs archive +* @buf: input initramfs archive to extract +* @len: length of initramfs data to process +* +* Returns: NULL for success or an error message string +* +* This symbol shouldn't be used externally. It's available for unit tests. +*/ +char *__init unpack_to_rootfs (char *buf,unsigned long len) {long written;decompress_fn decompress;const char *compress_name;static __initdata char msg_buf[64 ]; - header_buf =kmalloc (110 ,GFP_KERNEL ); + header_buf =kmalloc (CPIO_HDRLEN ,GFP_KERNEL ); symlink_buf =kmalloc (PATH_MAX +N_ALIGN (PATH_MAX ) +1 ,GFP_KERNEL ); name_buf =kmalloc (N_ALIGN (PATH_MAX ),GFP_KERNEL );
init/initramfs_internal.h new file +8 -0
@@ -0,0 +1,8 @@ +// SPDX-License-Identifier: GPL-2.0 +#ifndef __INITRAMFS_INTERNAL_H__ +#define __INITRAMFS_INTERNAL_H__ + +char *unpack_to_rootfs (char *buf,unsigned long len); +#define CPIO_HDRLEN 110 + +#endif
Signature · 1 line
2.43.0
PATCH v4 2/8 initramfs_test: kunit tests for initramfs unpacking
In reply to David DisseldorpProvide some basic initramfs unpack sanity tests covering:
- simple file / dir extraction
- filename field overrun, as reported and fixed separately via
https://lore.kernel.org/r/[email protected]
- "070702" cpio data checksums
- hardlinks
These tests introduce new modpost warnings for initramfs_test_cases section=.data -> section=.init.text run_case() hooks. The kunit_case/_suite struct cannot be marked as __initdata as this will be used in debugfs to retrieve results after a test has run.
--- init/.kunitconfig | 3 + init/Kconfig | 7 + init/Makefile | 1 + init/initramfs_test.c | 407 ++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 418 insertions(+) create mode 100644 init/.kunitconfig create mode 100644 init/initramfs_test.c
init/.kunitconfig new file +3 -0
@@ -0,0 +1,3 @@ +CONFIG_KUNIT=y +CONFIG_BLK_DEV_INITRD=y +CONFIG_INITRAMFS_TEST=y
init/Kconfig +7 -0
@@ -1454,6 +1454,13 @@ config INITRAMFS_PRESERVE_MTIME If unsure, say Y. +config INITRAMFS_TEST + bool "Test initramfs cpio archive extraction" if !KUNIT_ALL_TESTS + depends on BLK_DEV_INITRD && KUNIT=y + default KUNIT_ALL_TESTS + help + Build KUnit tests for initramfs. See Documentation/dev-tools/kunit + choice prompt "Compiler optimization level" default CC_OPTIMIZE_FOR_PERFORMANCE
init/Makefile +1 -0
@@ -12,6 +12,7 @@ else obj-$(CONFIG_BLK_DEV_INITRD) += initramfs.o endif obj-$(CONFIG_GENERIC_CALIBRATE_DELAY) += calibrate.o +obj-$(CONFIG_INITRAMFS_TEST) += initramfs_test.o obj-y += init_task.o
init/initramfs_test.c new file +407 -0
407 lines not on this page · raw message
Signature · 1 line
2.43.0
PATCH v4 3/8 vsprintf: add simple_strntoul
In reply to David Disseldorpcpio extraction currently does a memcpy to ensure that the archive hex fields are null terminated for simple_strtoul(). simple_strntoul() will allow us to avoid the memcpy.
--- include/linux/kstrtox.h | 1 + lib/vsprintf.c | 7 +++++++ 2 files changed, 8 insertions(+)
include/linux/kstrtox.h +1 -0
@@ -143,6 +143,7 @@ static inline int __must_check kstrtos32_from_user(const char __user *s, size_t*/ extern unsigned long simple_strtoul (const char *,char **,unsigned int ); +extern unsigned long simple_strntoul (const char *,char **,unsigned int ,size_t );extern long simple_strtol (const char *,char **,unsigned int );extern unsigned long long simple_strtoull (const char *,char **,unsigned int );extern long long simple_strtoll (const char *,char **,unsigned int );
lib/vsprintf.c +7 -0
@@ -114,6 +114,13 @@ unsigned long simple_strtoul(const char *cp, char **endp, unsigned int base) }EXPORT_SYMBOL (simple_strtoul); +unsigned long simple_strntoul (const char *cp,char **endp,unsigned int base, +size_t max_chars) +{ +return simple_strntoull (cp, endp, base, max_chars); +} +EXPORT_SYMBOL (simple_strntoul); +/** * simple_strtol - convert a string to a signed long * @cp: The start of the string
Signature · 1 line
2.43.0
PATCH v4 4/8 initramfs: avoid memcpy for hex header fields
In reply to David Disseldorpnewc/crc cpio headers contain a bunch of 8-character hexadecimal fields which we convert via simple_strtoul(), following memcpy() into a zero-terminated stack buffer. The new simple_strntoul() helper allows us to pass in max_chars=8 to avoid zero-termination and memcpy().
--- init/initramfs.c | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-)
init/initramfs.c +3 -6
@@ -189,14 +189,11 @@ static __initdata u32 hdr_csum;static void __init parse_header (char *s) {unsigned long parsed[13 ]; -char buf[9 ];int i; - buf[8 ] ='\0' ; -for (i =0 , s +=6 ; i <13 ; i++, s +=8 ) { -memcpy (buf, s,8 ); - parsed[i] =simple_strtoul (buf,NULL ,16 ); - } +for (i =0 , s +=6 ; i <13 ; i++, s +=8 ) + parsed[i] =simple_strntoul (s,NULL ,16 ,8 ); + ino = parsed[0 ]; mode = parsed[1 ]; uid = parsed[2 ];
Signature · 1 line
2.43.0
PATCH v4 5/8 initramfs: allocate heap buffers together
In reply to David Disseldorpheader_buf, symlink_buf and name_buf all share the same lifecycle so needn't be allocated / freed separately. This change leads to a minor reduction in .text size:
before:
text data bss dec hex filename
7914 1110 8 9032 2348 init/initramfs.o
after:
text data bss dec hex filename
7854 1110 8 8972 230c init/initramfs.o
A previous iteration of this patch reused a single buffer instead of three, given that buffer use is state-sequential (GotHeader, GotName, GotSymlink). However, the slight decrease in heap use during early boot isn't really worth the extra review complexity.
--- init/initramfs.c | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-)
init/initramfs.c +11 -8
@@ -510,14 +510,19 @@ char * __init unpack_to_rootfs(char *buf, unsigned long len)decompress_fn decompress;const char *compress_name;static __initdata char msg_buf[64 ]; +struct { +char header[CPIO_HDRLEN ]; +char symlink[PATH_MAX +N_ALIGN (PATH_MAX ) +1 ]; +char name[N_ALIGN (PATH_MAX )]; + } *bufs =kmalloc (sizeof (*bufs),GFP_KERNEL ); - header_buf =kmalloc (CPIO_HDRLEN ,GFP_KERNEL ); - symlink_buf =kmalloc (PATH_MAX +N_ALIGN (PATH_MAX ) +1 ,GFP_KERNEL ); - name_buf =kmalloc (N_ALIGN (PATH_MAX ),GFP_KERNEL ); - -if (!header_buf || !symlink_buf || !name_buf) +if (!bufs)panic_show_mem ("can't allocate buffers" ); + header_buf = bufs->header; + symlink_buf = bufs->symlink; + name_buf = bufs->name; + state = Start; this_header =0 ; message =NULL ;
@@ -560,9 +565,7 @@ char * __init unpack_to_rootfs(char *buf, unsigned long len) len -= my_inptr; }dir_utime (); -kfree (name_buf); -kfree (symlink_buf); -kfree (header_buf); +kfree (bufs);return message; }
Signature · 1 line
2.43.0
PATCH v4 6/8 initramfs: reuse name_len for dir mtime tracking
In reply to David DisseldorpWe already have a nulterm-inclusive, checked name_len for the directory name, so use that instead of calling strlen().
--- init/initramfs.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-)
init/initramfs.c +3 -4
@@ -144,9 +144,8 @@ struct dir_entry {char name[]; }; -static void __init dir_add (const char *name,time64_t mtime) +static void __init dir_add (const char *name,size_t nlen,time64_t mtime) { -size_t nlen =strlen (name) +1 ;struct dir_entry *de; de =kmalloc (sizeof (struct dir_entry ) + nlen,GFP_KERNEL );
@@ -170,7 +169,7 @@ static void __init dir_utime(void)#else static void __init do_utime (char *filename,time64_t mtime) {}static void __init do_utime_path (const struct path *path,time64_t mtime) {} -static void __init dir_add (const char *name,time64_t mtime) {} +static void __init dir_add (const char *name,size_t nlen,time64_t mtime) {}static void __init dir_utime (void ) {}#endif
@@ -394,7 +393,7 @@ static int __init do_name(void)init_mkdir (collected, mode);init_chown (collected, uid, gid,0 );init_chmod (collected, mode); -dir_add (collected, mtime); +dir_add (collected, name_len, mtime); }else if (S_ISBLK (mode) ||S_ISCHR (mode) ||S_ISFIFO (mode) ||S_ISSOCK (mode)) {if (maybe_link () ==0 ) {
Signature · 1 line
2.43.0
PATCH v4 7/8 initramfs: fix hardlink hash leak without TRAILER
In reply to David DisseldorpCovered in Documentation/driver-api/early-userspace/buffer-format.rst , initramfs archives can carry an optional "TRAILER!!!" entry which serves as a boundary for collecting and associating hardlinks with matching inode and major / minor device numbers.
Although optional, if hardlinks are found in an archive without a subsequent "TRAILER!!!" entry then the hardlink state hash table is leaked, e.g. unfixed kernel, with initramfs_test.c hunk applied only:
unreferenced object 0xffff9405408cc000 (size 8192):
comm "kunit_try_catch", pid 53, jiffies 4294892519
hex dump (first 32 bytes):
01 00 00 00 01 00 00 00 00 00 00 00 ff 81 00 00 ................
00 00 00 00 00 00 00 00 69 6e 69 74 72 61 6d 66 ........initramf
backtrace (crc a9fb0ee0):
[<0000000066739faa>] __kmalloc_cache_noprof+0x11d/0x250
[<00000000fc755219>] maybe_link.part.5+0xbc/0x120
[<000000000526a128>] do_name+0xce/0x2f0
[<00000000145c1048>] write_buffer+0x22/0x40
[<000000003f0b4f32>] unpack_to_rootfs+0xf9/0x2a0
[<00000000d6f7e5af>] initramfs_test_hardlink+0xe3/0x3f0
[<0000000014fde8d6>] kunit_try_run_case+0x5f/0x130
[<00000000dc9dafc5>] kunit_generic_run_threadfn_adapter+0x18/0x30
[<000000001076c239>] kthread+0xc8/0x100
[<00000000d939f1c1>] ret_from_fork+0x2b/0x40
[<00000000f848ad1a>] ret_from_fork_asm+0x1a/0x30Fix this by calling free_hash() after initramfs buffer processing in unpack_to_rootfs(). An extra hardlink_seen global is added as an optimization to avoid walking the 32 entry hash array unnecessarily. The expectation is that a "TRAILER!!!" entry will normally be present, and initramfs hardlinks are uncommon.
There is one user facing side-effect of this fix: hardlinks can currently be associated across built-in and external initramfs archives, *if* the built-in initramfs archive lacks a "TRAILER!!!" terminator. I'd consider this cross-archive association broken, but perhaps it's used.
--- init/initramfs.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
init/initramfs.c +6 -1
@@ -76,6 +76,7 @@ static __initdata struct hash {struct hash *next;char name[N_ALIGN (PATH_MAX )]; } *head[32 ]; +static __initdata bool hardlink_seen;static inline int hash (int major,int minor,int ino) {
@@ -109,19 +110,21 @@ static char __init *find_link(int major, int minor, int ino,strcpy (q->name, name); q->next =NULL ; *p = q; + hardlink_seen = true;return NULL ; }static void __init free_hash (void ) {struct hash **p, *q; -for (p = head; p < head +32 ; p++) { +for (p = head; hardlink_seen && p < head +32 ; p++) {while (*p) { q = *p; *p = q->next;kfree (q); } } + hardlink_seen = false; }#ifdef CONFIG_INITRAMFS_PRESERVE_MTIME
@@ -564,6 +567,8 @@ char * __init unpack_to_rootfs(char *buf, unsigned long len) len -= my_inptr; }dir_utime (); +/* free any hardlink state collected without optional TRAILER!!! */ +free_hash ();kfree (bufs);return message; }
Signature · 1 line
2.43.0
PATCH v4 8/8 initramfs: avoid static buffer for error message
In reply to David DisseldorpThe dynamic error message printed if CONFIG_RD_$ALG compression support is missing needn't be propagated up to the caller via a static buffer. Print it immediately via pr_err() and set @message to a const string to flag error.
Before:
text data bss dec hex filename
8006 1118 8 9132 23ac init/initramfs.o
After:
text data bss dec hex filename
7938 1022 8 8968 2308 init/initramfs.o
--- init/initramfs.c | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-)
init/initramfs.c +3 -7
@@ -511,7 +511,6 @@ char * __init unpack_to_rootfs(char *buf, unsigned long len)long written;decompress_fn decompress;const char *compress_name; -static __initdata char msg_buf[64 ];struct {char header[CPIO_HDRLEN ];char symlink[PATH_MAX +N_ALIGN (PATH_MAX ) +1 ];
@@ -552,12 +551,9 @@ char * __init unpack_to_rootfs(char *buf, unsigned long len)if (res)error ("decompressor failed" ); }else if (compress_name) { -if (!message) { -snprintf (msg_buf,sizeof msg_buf, -"compression method %s not configured" , - compress_name); - message = msg_buf; - } +pr_err ("compression method %s not configured\n" , + compress_name); +error ("decompressor failed" ); }else error ("invalid magic at start of compressed archive" );if (state != Reset)
Signature · 1 line
2.43.0
Quoted 11 lines · David Disseldorp
This patchset adds basic kunit test coverage for initramfs unpacking and cleans up some minor buffer handling issues / inefficiencies.
Changes since v3:
- Drop shared unpack buffer changes
+ rework into initramfs: allocate heap buffers together (patch 5/8)
+ extra review complexity wasn't worth the tiny boot-time heap saving
- move hardlink hash leak repro into firstinitramfs_testpatch
- add note regarding kunit section=.data -> section=.init.text warning[...]
Ok, let's see if this breaks any testsuites.
---
Applied to the vfs-6.15.initramfs branch of the vfs/vfs.git tree. Patches in the vfs-6.15.initramfs branch should appear in linux-next soon.
Please report any outstanding bugs that were missed during review in a new review to the original patch series allowing us to drop it.
It's encouraged to provide Acked-bys and Reviewed-bys even though the patch has now been applied. If possible patch trailers will be updated.
Note that commit hashes shown below are subject to change due to rebase, trailer updates or similar. If in doubt, please check the listed branch.
tree: https://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs.git branch: vfs-6.15.initramfs
[1/8] init: add initramfs_internal.h
https://git.kernel.org/vfs/vfs/c/5f469c4f7167 [2/8] initramfs_test: kunit tests for initramfs unpacking https://git.kernel.org/vfs/vfs/c/b6736cfccb58 [3/8] vsprintf: add simple_strntoul https://git.kernel.org/vfs/vfs/c/dfa63602367a [4/8] initramfs: avoid memcpy for hex header fields https://git.kernel.org/vfs/vfs/c/2c4babf8d182 [5/8] initramfs: allocate heap buffers together https://git.kernel.org/vfs/vfs/c/b40c5e61f940 [6/8] initramfs: reuse name_len for dir mtime tracking https://git.kernel.org/vfs/vfs/c/aeff5090a5ea [7/8] initramfs: fix hardlink hash leak without TRAILER https://git.kernel.org/vfs/vfs/c/15e2de59de08 [8/8] initramfs: avoid static buffer for error message https://git.kernel.org/vfs/vfs/c/154c1e422ffe
PATCH 1/1 initramfs_test: flag kunit_case __refdata to suppress warning
We already have a comment regarding why .data -> .init references are present for initramfs_test_cases[]. This allows us to suppress the modpost warning.
---
@Christian: feel free to squash this in with commit b6736cfccb582 ("initramfs_test: kunit tests for initramfs unpacking") in your vfs-6.15.initramfs branch if you like (and remove "These tests introduce new modpost warnings..." from the commit msg).
init/initramfs_test.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
init/initramfs_test.c +1 -1
@@ -387,7 +387,7 @@ static void __init initramfs_test_many(struct kunit *test)* The kunit_case/_suite struct cannot be marked as __initdata as this will be * used in debugfs to retrieve results after test has run. */ -static struct kunit_case initramfs_test_cases[] = { +static struct kunit_case __refdata initramfs_test_cases[] = {KUNIT_CASE (initramfs_test_extract),KUNIT_CASE (initramfs_test_fname_overrun),KUNIT_CASE (initramfs_test_data),
Signature · 1 line
2.43.0
Re: [PATCH] initramfs_test: flag kunit_case __refdata to suppress warning
Quoted 5 lines · David Disseldorp
We already have a comment regarding why .data -> .init references are present for initramfs_test_cases[]. This allows us to suppress the modpost warning.
Folded, thanks!
---
Applied to the vfs-6.15.initramfs branch of the vfs/vfs.git tree. Patches in the vfs-6.15.initramfs branch should appear in linux-next soon.
Please report any outstanding bugs that were missed during review in a new review to the original patch series allowing us to drop it.
It's encouraged to provide Acked-bys and Reviewed-bys even though the patch has now been applied. If possible patch trailers will be updated.
Note that commit hashes shown below are subject to change due to rebase, trailer updates or similar. If in doubt, please check the listed branch.
tree: https://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs.git branch: vfs-6.15.initramfs
[1/1] initramfs_test: flag kunit_case __refdata to suppress warning
(no commit info)