initramfs: move unnecessary memcmp from hot path
Subsystems (2)
PATCH RESEND 1/3 initramfs: move unnecessary memcmp from hot path
do_header() is called for each cpio entry and first checks for "newc" magic before parsing further. The magic check includes a special case error message if POSIX.1 ASCII (cpio -H odc) magic is detected. This special case POSIX.1 check needn't be done in the hot path, so move it under the non-newc-magic error path.
--- init/initramfs.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-)
init/initramfs.c +4 -5
@@ -256,12 +256,11 @@ static int __init do_collect(void)static int __init do_header (void ) { -if (memcmp (collected,"070707" ,6 )==0 ) { -error ("incorrect cpio method used: use -H newc option" ); -return 1 ; - }if (memcmp (collected,"070701" ,6 )) { -error ("no cpio magic" ); +if (memcmp (collected,"070707" ,6 ) ==0 ) +error ("incorrect cpio method used: use -H newc option" ); +else +error ("no cpio magic" );return 1 ; }parse_header (collected);
Signature · 1 line
2.26.2
PATCH v2 3/3 docs: remove mention of "crc" cpio format support
init/initramfs.c only supports extraction of cpio archives carrying the "newc" header magic ("070701"). Remove statements indicating support for the "crc" cpio format.
--- .../early-userspace/buffer-format.rst | 24 +++++++------------ 1 file changed, 8 insertions(+), 16 deletions(-)
Version 2:
- reword initramfs padding description, as suggested by Matthew Wilcox
Documentation/driver-api/early-userspace/buffer-format.rst +8 -16
@@ -14,10 +14,10 @@ is different. The initramfs buffer contains an archive which is expanded into a ramfs filesystem; this document details the format of the initramfs buffer format. -The initramfs buffer format is based around the "newc" or "crc" CPIO -formats, and can be created with the cpio(1) utility. The cpio -archive can be compressed using gzip(1). One valid version of an -initramfs buffer is thus a single .cpio.gz file. +The initramfs buffer format is based around the "newc" CPIO format, and +can be created with the cpio(1) utility. The cpio archive can be +compressed using gzip(1). One valid version of an initramfs buffer is +thus a single .cpio.gz file. The full format of the initramfs buffer is defined by the following grammar, where::
@@ -40,9 +40,8 @@ grammar, where:: In human terms, the initramfs buffer contains a collection of -compressed and/or uncompressed cpio archives (in the "newc" or "crc" -formats); arbitrary amounts zero bytes (for padding) can be added -between members. +compressed and/or uncompressed cpio archives (in the "newc" format), +with arbitrary amount of zero-byte padding between members. The cpio "TRAILER!!!" entry (cpio end-of-archive) is optional, but is not ignored; see "handling of hard links" below.
@@ -55,7 +54,7 @@ by the ASCII string "000012ac"): ============= ================== ============================================== Field name Field size Meaning ============= ================== ============================================== -c_magic 6 bytes The string "070701" or "070702" +c_magic 6 bytes The string "070701" c_ino 8 bytes File inode number c_mode 8 bytes File mode and permissions c_uid 8 bytes File uid
@@ -68,8 +67,7 @@ c_min 8 bytes Minor part of file device number c_rmaj 8 bytes Major part of device node reference c_rmin 8 bytes Minor part of device node reference c_namesize 8 bytes Length of filename, including final \0 -c_chksum 8 bytes Checksum of data field if c_magic is 070702; - otherwise zero +c_chksum 8 bytes Ignored; reserved for unsupported "crc" format ============= ================== ============================================== The c_mode field matches the contents of st_mode returned by stat(2)
@@ -78,12 +76,6 @@ on Linux, and encodes the file type and file permissions. The c_filesize should be zero for any file which is not a regular file or symlink. -The c_chksum field contains a simple 32-bit unsigned sum of all the -bytes in the data field. cpio(1) refers to this as "crc", which is -clearly incorrect (a cyclic redundancy check is a different and -significantly stronger integrity check), however, this is the -algorithm used. - If the filename is "TRAILER!!!" this is actually an end-of-archive marker; the c_filesize for an end-of-archive marker must be zero.
Signature · 1 line
2.26.2
PATCH RESEND 2/3 initramfs: print helpful cpio error on "crc" magic
In reply to David DisseldorpContrary to the buffer-format.rst documentation, initramfs cpio extraction does not support "crc" archives, which carry "070702" header magic. Make it a little clearer that "newc" (magic="070701") is the only supported cpio format, by extending the POSIX.1 ASCII (magic="070707") specific error message to also cover "crc" magic.
--- init/initramfs.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
init/initramfs.c +1 -1
@@ -257,7 +257,7 @@ static int __init do_collect(void)static int __init do_header (void ) {if (memcmp (collected,"070701" ,6 )) { -if (memcmp (collected,"070707" ,6 ) ==0 ) +if (memcmp (collected,"0707" ,4 ) ==0 )error ("incorrect cpio method used: use -H newc option" );else error ("no cpio magic" );
Signature · 1 line
2.26.2
Ping, any feedback on this change?
I think it's a no brainer, but for kicks I ran a few unrealistic micro benchmarks on my laptop. Extraction time for a cpio image with 1M+ directories improved by 5ms (pre: 14.614s, post: 14.609s), when averaged across 20 runs of:
qemu-system-x86_64 -machine accel=kvm -smp cpus=1 -m 10240 \
-kernel ~/linux/arch/x86/boot/bzImage \
-initrd ./initrds/gen_cpio.out \
-append "initramfs_async=0 console=ttyS0 panic=0" -nographic \
| awk '/Trying to unpack rootfs/ {start_ts = $2};
/Freeing initrd memory/ {end_ts = $2}
END {printf "%f\n", end_ts - start_ts}'Cheers, David
Quoted 32 lines · David Disseldorp
do_header()is called for each cpio entry and first checks for "newc" magic before parsing further. The magic check includes a special case error message if POSIX.1 ASCII (cpio -H odc) magic is detected. This special case POSIX.1 check needn't be done in the hot path, so move it under the non-newc-magic error path.Signed-off-by David Disseldorp <[email protected]>--- init/initramfs.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-)init/initramfs.c +4
-5@@ -256,12 +256,11 @@ static int __init do_collect(void)static int __init do_header (void ) { -if (memcmp (collected,"070707" ,6 )==0 ) { -error ("incorrect cpio method used: use -H newc option" ); -return 1 ; - }if (memcmp (collected,"070701" ,6 )) { -error ("no cpio magic" ); +if (memcmp (collected,"070707" ,6 ) ==0 ) +error ("incorrect cpio method used: use -H newc option" ); +else +error ("no cpio magic" );return 1 ; }parse_header (collected);
Al Viro
maintainerQuoted 6 lines · David Disseldorp
Ping, any feedback on this change?
I think it's a no brainer, but for kicks I ran a few unrealistic micro benchmarks on my laptop. Extraction time for a cpio image with 1M+ directories improved by 5ms (pre: 14.614s, post: 14.609s), when averaged across 20 runs of:
Quoted 7 lines
qemu-system-x86_64 -machine accel=kvm -smp cpus=1 -m 10240 \
-kernel ~/linux/arch/x86/boot/bzImage \
-initrd ./initrds/gen_cpio.out \
-append "initramfs_async=0 console=ttyS0 panic=0" -nographic \
| awk '/Trying to unpack rootfs/ {start_ts = $2};
/Freeing initrd memory/ {end_ts = $2}
END {printf "%f\n", end_ts - start_ts}'What was the dispersion for those runs?
Quoted 15 lines · Al Viro
Quoted 6 lines · David Disseldorp
Ping, any feedback on this change?
I think it's a no brainer, but for kicks I ran a few unrealistic micro benchmarks on my laptop. Extraction time for a cpio image with 1M+ directories improved by 5ms (pre: 14.614s, post: 14.609s), when averaged across 20 runs of:
Quoted 7 lines
qemu-system-x86_64 -machine accel=kvm -smp cpus=1 -m 10240 \ -kernel ~/linux/arch/x86/boot/bzImage \ -initrd ./initrds/gen_cpio.out \ -append "initramfs_async=0 console=ttyS0 panic=0" -nographic \ | awk '/Trying to unpack rootfs/ {start_ts = $2}; /Freeing initrd memory/ {end_ts = $2} END {printf "%f\n", end_ts - start_ts}'
Quoted 1 line
What was the dispersion for those runs?
Too high for the 5ms to be considered statistically significant. Std deviations were pre: 171ms, post: 214ms... <sigh> I'll redo this on a proper test rig.
Cheers, David