Skip to contentSkip to conversation

initramfs: move unnecessary memcmp from hot path

PATCH RESEND In review David Disseldorp 2 participants 6 messages Jul 21, 2021 – Aug 4, 2021

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH RESEND 1/3 initramfs: move unnecessary memcmp from hot path

do_header() is called for each cpio entry and first checks for "newc" magic before parsing further. The magic check includes a special case error message if POSIX.1 ASCII (cpio -H odc) magic is detected. This special case POSIX.1 check needn't be done in the hot path, so move it under the non-newc-magic error path.

---
 init/initramfs.c | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)
init/initramfs.c +4 -5
index af27abc59643..f01590cefa2d 100644
@@ -256,12 +256,11 @@ static int __init do_collect(void)
 
 static int __init do_header(void)
 {
-	if (memcmp(collected, "070707", 6)==0) {
-		error("incorrect cpio method used: use -H newc option");
-		return 1;
-	}
 	if (memcmp(collected, "070701", 6)) {
-		error("no cpio magic");
+		if (memcmp(collected, "070707", 6) == 0)
+			error("incorrect cpio method used: use -H newc option");
+		else
+			error("no cpio magic");
 		return 1;
 	}
 	parse_header(collected);
Signature · 1 line
2.26.2
#1 lore raw

PATCH v2 3/3 docs: remove mention of "crc" cpio format support

init/initramfs.c only supports extraction of cpio archives carrying the "newc" header magic ("070701"). Remove statements indicating support for the "crc" cpio format.

---
 .../early-userspace/buffer-format.rst         | 24 +++++++------------
 1 file changed, 8 insertions(+), 16 deletions(-)

Version 2:
- reword initramfs padding description, as suggested by Matthew Wilcox

Documentation/driver-api/early-userspace/buffer-format.rst +8 -16
index 7f74e301fdf3..0df76bca444c 100644
@@ -14,10 +14,10 @@ is different.  The initramfs buffer contains an archive which is
 expanded into a ramfs filesystem; this document details the format of
 the initramfs buffer format.
 
-The initramfs buffer format is based around the "newc" or "crc" CPIO
-formats, and can be created with the cpio(1) utility.  The cpio
-archive can be compressed using gzip(1).  One valid version of an
-initramfs buffer is thus a single .cpio.gz file.
+The initramfs buffer format is based around the "newc" CPIO format, and
+can be created with the cpio(1) utility.  The cpio archive can be
+compressed using gzip(1).  One valid version of an initramfs buffer is
+thus a single .cpio.gz file.
 
 The full format of the initramfs buffer is defined by the following
 grammar, where::
@@ -40,9 +40,8 @@ grammar, where::
 
 
 In human terms, the initramfs buffer contains a collection of
-compressed and/or uncompressed cpio archives (in the "newc" or "crc"
-formats); arbitrary amounts zero bytes (for padding) can be added
-between members.
+compressed and/or uncompressed cpio archives (in the "newc" format),
+with arbitrary amount of zero-byte padding between members.
 
 The cpio "TRAILER!!!" entry (cpio end-of-archive) is optional, but is
 not ignored; see "handling of hard links" below.
@@ -55,7 +54,7 @@ by the ASCII string "000012ac"):
 ============= ================== ==============================================
 Field name    Field size	 Meaning
 ============= ================== ==============================================
-c_magic	      6 bytes		 The string "070701" or "070702"
+c_magic	      6 bytes		 The string "070701"
 c_ino	      8 bytes		 File inode number
 c_mode	      8 bytes		 File mode and permissions
 c_uid	      8 bytes		 File uid
@@ -68,8 +67,7 @@ c_min	      8 bytes		 Minor part of file device number
 c_rmaj	      8 bytes		 Major part of device node reference
 c_rmin	      8 bytes		 Minor part of device node reference
 c_namesize    8 bytes		 Length of filename, including final \0
-c_chksum      8 bytes		 Checksum of data field if c_magic is 070702;
-				 otherwise zero
+c_chksum      8 bytes		 Ignored; reserved for unsupported "crc" format
 ============= ================== ==============================================
 
 The c_mode field matches the contents of st_mode returned by stat(2)
@@ -78,12 +76,6 @@ on Linux, and encodes the file type and file permissions.
 The c_filesize should be zero for any file which is not a regular file
 or symlink.
 
-The c_chksum field contains a simple 32-bit unsigned sum of all the
-bytes in the data field.  cpio(1) refers to this as "crc", which is
-clearly incorrect (a cyclic redundancy check is a different and
-significantly stronger integrity check), however, this is the
-algorithm used.
-
 If the filename is "TRAILER!!!" this is actually an end-of-archive
 marker; the c_filesize for an end-of-archive marker must be zero.
 
Signature · 1 line
2.26.2
#2 lore raw

PATCH RESEND 2/3 initramfs: print helpful cpio error on "crc" magic

In reply to David Disseldorp

Contrary to the buffer-format.rst documentation, initramfs cpio extraction does not support "crc" archives, which carry "070702" header magic. Make it a little clearer that "newc" (magic="070701") is the only supported cpio format, by extending the POSIX.1 ASCII (magic="070707") specific error message to also cover "crc" magic.

---
 init/initramfs.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
init/initramfs.c +1 -1
index f01590cefa2d..19b1c70446fc 100644
@@ -257,7 +257,7 @@ static int __init do_collect(void)
 static int __init do_header(void)
 {
 	if (memcmp(collected, "070701", 6)) {
-		if (memcmp(collected, "070707", 6) == 0)
+		if (memcmp(collected, "0707", 4) == 0)
 			error("incorrect cpio method used: use -H newc option");
 		else
 			error("no cpio magic");
Signature · 1 line
2.26.2
#3 lore raw
In reply to David Disseldorp

Ping, any feedback on this change?

I think it's a no brainer, but for kicks I ran a few unrealistic micro benchmarks on my laptop. Extraction time for a cpio image with 1M+ directories improved by 5ms (pre: 14.614s, post: 14.609s), when averaged across 20 runs of:

  qemu-system-x86_64 -machine accel=kvm -smp cpus=1 -m 10240 \
        -kernel ~/linux/arch/x86/boot/bzImage \
        -initrd ./initrds/gen_cpio.out \
        -append "initramfs_async=0 console=ttyS0 panic=0" -nographic \
        | awk '/Trying to unpack rootfs/ {start_ts = $2};
               /Freeing initrd memory/ {end_ts = $2}
               END {printf "%f\n", end_ts - start_ts}'

Cheers, David

Quoted 32 lines · David Disseldorp

do_header() is called for each cpio entry and first checks for "newc" magic before parsing further. The magic check includes a special case error message if POSIX.1 ASCII (cpio -H odc) magic is detected. This special case POSIX.1 check needn't be done in the hot path, so move it under the non-newc-magic error path.

---
 init/initramfs.c | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)
init/initramfs.c +4 -5
index af27abc59643..f01590cefa2d 100644
@@ -256,12 +256,11 @@ static int __init do_collect(void)
 
 static int __init do_header(void)
 {
-	if (memcmp(collected, "070707", 6)==0) {
-		error("incorrect cpio method used: use -H newc option");
-		return 1;
-	}
 	if (memcmp(collected, "070701", 6)) {
-		error("no cpio magic");
+		if (memcmp(collected, "070707", 6) == 0)
+			error("incorrect cpio method used: use -H newc option");
+		else
+			error("no cpio magic");
 		return 1;
 	}
 	parse_header(collected);
maintainer
#4 lore raw
Quoted 6 lines · David Disseldorp

Ping, any feedback on this change?

I think it's a no brainer, but for kicks I ran a few unrealistic micro benchmarks on my laptop. Extraction time for a cpio image with 1M+ directories improved by 5ms (pre: 14.614s, post: 14.609s), when averaged across 20 runs of:

Quoted 7 lines
  qemu-system-x86_64 -machine accel=kvm -smp cpus=1 -m 10240 \
        -kernel ~/linux/arch/x86/boot/bzImage \
        -initrd ./initrds/gen_cpio.out \
        -append "initramfs_async=0 console=ttyS0 panic=0" -nographic \
        | awk '/Trying to unpack rootfs/ {start_ts = $2};
               /Freeing initrd memory/ {end_ts = $2}
               END {printf "%f\n", end_ts - start_ts}'

What was the dispersion for those runs?

#5 lore raw
Quoted 15 lines · Al Viro
Quoted 6 lines · David Disseldorp

Ping, any feedback on this change?

I think it's a no brainer, but for kicks I ran a few unrealistic micro benchmarks on my laptop. Extraction time for a cpio image with 1M+ directories improved by 5ms (pre: 14.614s, post: 14.609s), when averaged across 20 runs of:

Quoted 7 lines
  qemu-system-x86_64 -machine accel=kvm -smp cpus=1 -m 10240 \
        -kernel ~/linux/arch/x86/boot/bzImage \
        -initrd ./initrds/gen_cpio.out \
        -append "initramfs_async=0 console=ttyS0 panic=0" -nographic \
        | awk '/Trying to unpack rootfs/ {start_ts = $2};
               /Freeing initrd memory/ {end_ts = $2}
               END {printf "%f\n", end_ts - start_ts}'  
Quoted 1 line

What was the dispersion for those runs?

Too high for the 5ms to be considered statistically significant. Std deviations were pre: 171ms, post: 214ms... <sigh> I'll redo this on a proper test rig.

Cheers, David