Upgrade the zlib_inflate library code to a recent version
PATCH 1/1 Upgrade the zlib_inflate library code to a recent version
Upgrade the zlib_inflate implementation in the kernel from a patched version 1.1.3 to a patched 1.2.3.
The code in the kernel is about seven years old and I noticed that the external zlib library's inflate performance was significantly faster (~50%) than the code in the kernel on ARM (and faster again on x86_32).
For comparison the newer deflate code is 20% slower on ARM and 50% slower on x86_32 but gives an approx 1% compression ratio improvement. I don't consider this to be an improvement for kernel use so have no plans to change the zlib_deflate code.
Various changes have been made to the zlib code in the kernel, the most significant being the extra functions/flush option used by ppp_deflate. This update reimplements the features PPP needs to ensure it continues to work.
This code has been tested on ARM under both JFFS2 (with zlib compression enabled) and ppp_deflate and on x86_32. JFFS2 sees an approx. 10% real world file read speed improvement.
This patch also removes ZLIB_VERSION as it no longer has a correct value. We don't need version checks anyway as the kernel's module handling will take care of that for us. This removal is also more in keeping with the zlib author's wishes
(http://www.zlib.net/zlib_faq.html#faq24) and I've added something to the zlib.h header to note its a modified version.
---
include/linux/zconf.h | 12 include/linux/zlib.h | 209 ++++--- include/linux/zutil.h | 12 lib/zlib_deflate/deflate.c | 25 lib/zlib_deflate/deflate_syms.c | 3 lib/zlib_inflate/Makefile | 4 lib/zlib_inflate/infblock.c | 365 ------------- lib/zlib_inflate/infblock.h | 48 - lib/zlib_inflate/infcodes.c | 202 ------- lib/zlib_inflate/infcodes.h | 33 - lib/zlib_inflate/inffast.c | 462 ++++++++++------ lib/zlib_inflate/inffast.h | 12 lib/zlib_inflate/inffixed.h | 94 +++ lib/zlib_inflate/inflate.c | 1114 +++++++++++++++++++++++++++++++--------- lib/zlib_inflate/inflate.h | 107 +++ lib/zlib_inflate/inflate_syms.c | 3 lib/zlib_inflate/inflate_sync.c | 152 ----- lib/zlib_inflate/inftrees.c | 677 ++++++++++-------------- lib/zlib_inflate/inftrees.h | 99 +-- lib/zlib_inflate/infutil.c | 88 --- lib/zlib_inflate/infutil.h | 178 ------ 21 files changed, 1886 insertions(+), 2013 deletions(-)
The patch is ~160kb so I'll link to it:
Quoted 3 lines · Richard Purdie
Upgrade the
zlib_inflateimplementation in the kernel from a patched version 1.1.3 to a patched 1.2.3.
s/1.1.3/1.1.4/
I once pulled all the bugfixes between the versions into the kernel.
Quoted 18 lines
The code in the kernel is about seven years old and I noticed that the external zlib library's inflate performance was significantly faster (~50%) than the code in the kernel on ARM (and faster again on
x86_32).For comparison the newer deflate code is 20% slower on ARM and 50% slower on
x86_32but gives an approx 1% compression ratio improvement. I don't consider this to be an improvement for kernel use so have no plans to change thezlib_deflatecode.Various changes have been made to the zlib code in the kernel, the most significant being the extra functions/flush option used by
ppp_deflate. This update reimplements the features PPP needs to ensure it continues to work.This code has been tested on ARM under both JFFS2 (with zlib compression enabled) and
ppp_deflateand onx86_32. JFFS2 sees an approx. 10% real world file read speed improvement.
Quoted 6 lines
This patch also removes
ZLIB_VERSIONas it no longer has a correct value. We don't need version checks anyway as the kernel's module handling will take care of that for us. This removal is also more in keeping with the zlib author's wishes
(http://www.zlib.net/zlib_faq.html#faq24) and I've added something to thezlib.hheader to note its a modified version.
Sounds good.
Review
Quoted 2 lines
(quoted diff) +2
-0+#if 0 +int zlib_inflatePrime(z_streamp strm, int bits, int value)
Was this code dead in 1.2.3 as well?
Quoted 1 line
+#ifndef
PKZIP_BUG_WORKAROUND
For the kernel, we can remove compat code against DOS compilers, etc. In this particular case, I believe we can just consider data compressed with PKZIP to be illegal and throw an error.
Overall the patch doesn't contain any obvious problems and seems to do what you described. If testing agrees as well, I'm all for merging it.
Jörn
Signature · 3 lines
It is better to die of hunger having lived without grief and fear, than to live with a troubled spirit amid abundance. -- Epictetus
Quoted 7 lines · Jörn Engel
Quoted 2 lines · Richard Purdie
Upgrade the
zlib_inflateimplementation in the kernel from a patched version 1.1.3 to a patched 1.2.3.s/1.1.3/1.1.4/
I once pulled all the bugfixes between the versions into the kernel.
That's not what the header says :)
Quoted 4 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+#if 0 +int zlib_inflatePrime(z_streamp strm, int bits, int value)Was this code dead in 1.2.3 as well?
This code was commented out in the kernel by Adrian Bunk as it has no users and I just followed that example when updating I'm sure if someone needed it, it could be enabled.
Quoted 5 lines
Quoted 1 line
+#ifndef
PKZIP_BUG_WORKAROUNDFor the kernel, we can remove compat code against DOS compilers, etc. In this particular case, I believe we can just consider data compressed with PKZIP to be illegal and throw an error.
Agreed, there are places it can be tidied up a bit further. I was trying not to make too many changes so any future updates against zlib are easier.
Richard
Quoted 10 lines · Richard Purdie
Quoted 7 lines · Jörn Engel
Quoted 2 lines · Richard Purdie
Upgrade the
zlib_inflateimplementation in the kernel from a patched version 1.1.3 to a patched 1.2.3.s/1.1.3/1.1.4/
I once pulled all the bugfixes between the versions into the kernel.
That's not what the header says :)
Well, the complete patch from 1.1.3 to 1.1.4 was rather big. Every single copyright line got updated to a current date, without any other changes in those files. I found it rather silly and took the bugfixes only. ;)
Quoted 8 lines
Quoted 4 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+#if 0 +int zlib_inflatePrime(z_streamp strm, int bits, int value)Was this code dead in 1.2.3 as well?
This code was commented out in the kernel by Adrian Bunk as it has no users and I just followed that example when updating I'm sure if someone needed it, it could be enabled.
My thinking was rather that we could remove it. #if 0 has an advantage in case you made a mistake - easy to revert. But after a while...
Quoted 9 lines
Quoted 5 lines
Quoted 1 line
+#ifndef
PKZIP_BUG_WORKAROUNDFor the kernel, we can remove compat code against DOS compilers, etc. In this particular case, I believe we can just consider data compressed with PKZIP to be illegal and throw an error.
Agreed, there are places it can be tidied up a bit further. I was trying not to make too many changes so any future updates against zlib are easier.
Fair enough.
Jörn
Signature · 2 lines
Data expands to fill the space available for storage. -- Parkinson's Law
Quoted 8 lines
Upgrade the
zlib_inflateimplementation in the kernel from a patched version 1.1.3 to a patched 1.2.3.The code in the kernel is about seven years old and I noticed that the external zlib library's inflate performance was significantly faster (~50%) than the code in the kernel on ARM (and faster again on
x86_32).
Any plans to move to something newer, e.g. bzip2 or LZMA?
Jan Engelhardt
Quoted 9 lines · Jan Engelhardt
Quoted 8 lines
Upgrade the
zlib_inflateimplementation in the kernel from a patched version 1.1.3 to a patched 1.2.3.The code in the kernel is about seven years old and I noticed that the external zlib library's inflate performance was significantly faster (~50%) than the code in the kernel on ARM (and faster again on
x86_32).Any plans to move to something newer, e.g. bzip2 or LZMA?
For PPP? Write a new RFC suggesting it and explaining how to deal with the "chunkiness" and additional memory overhead. For JFFS2? Bzlib compresses worse than zlib when dealing with small blocks. For...
There may be a couple of uses where zlib could be replaced by something else. But even ignoring the compatibility issue it isn't always that simple.
Jörn
Signature · 3 lines
With a PC, I always felt limited by the software available. On Unix, I am limited only by my knowledge. -- Peter J. Schoenster
Quoted 2 lines · Richard Purdie
Signed-off-by Richard Purdie <[email protected]>
Jörn
Signature · 3 lines
There is no worse hell than that provided by the regrets for wasted opportunities. -- Andre-Louis Moreau in Scarabouche