Skip to contentSkip to conversation

Upgrade the zlib_inflate library code to a recent version

PATCH In review Richard Purdie 3 participants 7 messages Apr 4, 2006 – Apr 5, 2006 Memory technology devices (MTD)

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Upgrade the zlib_inflate library code to a recent version

Upgrade the zlib_inflate implementation in the kernel from a patched version 1.1.3 to a patched 1.2.3.

The code in the kernel is about seven years old and I noticed that the external zlib library's inflate performance was significantly faster (~50%) than the code in the kernel on ARM (and faster again on x86_32).

For comparison the newer deflate code is 20% slower on ARM and 50% slower on x86_32 but gives an approx 1% compression ratio improvement. I don't consider this to be an improvement for kernel use so have no plans to change the zlib_deflate code.

Various changes have been made to the zlib code in the kernel, the most significant being the extra functions/flush option used by ppp_deflate. This update reimplements the features PPP needs to ensure it continues to work.

This code has been tested on ARM under both JFFS2 (with zlib compression enabled) and ppp_deflate and on x86_32. JFFS2 sees an approx. 10% real world file read speed improvement.

This patch also removes ZLIB_VERSION as it no longer has a correct value. We don't need version checks anyway as the kernel's module handling will take care of that for us. This removal is also more in keeping with the zlib author's wishes
(http://www.zlib.net/zlib_faq.html#faq24) and I've added something to the zlib.h header to note its a modified version.

---

 include/linux/zconf.h           |   12
 include/linux/zlib.h            |  209 ++++---
 include/linux/zutil.h           |   12
 lib/zlib_deflate/deflate.c      |   25
 lib/zlib_deflate/deflate_syms.c |    3
 lib/zlib_inflate/Makefile       |    4
 lib/zlib_inflate/infblock.c     |  365 -------------
 lib/zlib_inflate/infblock.h     |   48 -
 lib/zlib_inflate/infcodes.c     |  202 -------
 lib/zlib_inflate/infcodes.h     |   33 -
 lib/zlib_inflate/inffast.c      |  462 ++++++++++------
 lib/zlib_inflate/inffast.h      |   12
 lib/zlib_inflate/inffixed.h     |   94 +++
 lib/zlib_inflate/inflate.c      | 1114 +++++++++++++++++++++++++++++++---------
 lib/zlib_inflate/inflate.h      |  107 +++
 lib/zlib_inflate/inflate_syms.c |    3
 lib/zlib_inflate/inflate_sync.c |  152 -----
 lib/zlib_inflate/inftrees.c     |  677 ++++++++++--------------
 lib/zlib_inflate/inftrees.h     |   99 +--
 lib/zlib_inflate/infutil.c      |   88 ---
 lib/zlib_inflate/infutil.h      |  178 ------
 21 files changed, 1886 insertions(+), 2013 deletions(-)

The patch is ~160kb so I'll link to it:

http://www.o-hand.com/~richard/zlib_inflate-r3.patch

#1 lore raw
Quoted 3 lines · Richard Purdie

Upgrade the zlib_inflate implementation in the kernel from a patched version 1.1.3 to a patched 1.2.3.

s/1.1.3/1.1.4/

I once pulled all the bugfixes between the versions into the kernel.

Quoted 18 lines

The code in the kernel is about seven years old and I noticed that the external zlib library's inflate performance was significantly faster (~50%) than the code in the kernel on ARM (and faster again on x86_32).

For comparison the newer deflate code is 20% slower on ARM and 50% slower on x86_32 but gives an approx 1% compression ratio improvement. I don't consider this to be an improvement for kernel use so have no plans to change the zlib_deflate code.

Various changes have been made to the zlib code in the kernel, the most significant being the extra functions/flush option used by ppp_deflate. This update reimplements the features PPP needs to ensure it continues to work.

This code has been tested on ARM under both JFFS2 (with zlib compression enabled) and ppp_deflate and on x86_32. JFFS2 sees an approx. 10% real world file read speed improvement.

Quoted 6 lines

This patch also removes ZLIB_VERSION as it no longer has a correct value. We don't need version checks anyway as the kernel's module handling will take care of that for us. This removal is also more in keeping with the zlib author's wishes
(http://www.zlib.net/zlib_faq.html#faq24) and I've added something to the zlib.h header to note its a modified version.

Sounds good.

Review

Quoted 2 lines
(quoted diff) +2 -0
+#if 0
+int zlib_inflatePrime(z_streamp strm, int bits, int value)

Was this code dead in 1.2.3 as well?

Quoted 1 line

+#ifndef PKZIP_BUG_WORKAROUND

For the kernel, we can remove compat code against DOS compilers, etc. In this particular case, I believe we can just consider data compressed with PKZIP to be illegal and throw an error.

Overall the patch doesn't contain any obvious problems and seems to do what you described. If testing agrees as well, I'm all for merging it.

Jörn

Signature · 3 lines
It is better to die of hunger having lived without grief and fear,
than to live with a troubled spirit amid abundance.
-- Epictetus
#2 lore raw
Quoted 7 lines · Jörn Engel
Quoted 2 lines · Richard Purdie

Upgrade the zlib_inflate implementation in the kernel from a patched version 1.1.3 to a patched 1.2.3.

s/1.1.3/1.1.4/

I once pulled all the bugfixes between the versions into the kernel.

That's not what the header says :)

Quoted 4 lines

Review

Quoted 2 lines
(quoted diff) +2 -0
+#if 0
+int zlib_inflatePrime(z_streamp strm, int bits, int value)

Was this code dead in 1.2.3 as well?

This code was commented out in the kernel by Adrian Bunk as it has no users and I just followed that example when updating I'm sure if someone needed it, it could be enabled.

Quoted 5 lines
Quoted 1 line

+#ifndef PKZIP_BUG_WORKAROUND

For the kernel, we can remove compat code against DOS compilers, etc. In this particular case, I believe we can just consider data compressed with PKZIP to be illegal and throw an error.

Agreed, there are places it can be tidied up a bit further. I was trying not to make too many changes so any future updates against zlib are easier.

Richard

#3 lore raw
Quoted 10 lines · Richard Purdie
Quoted 7 lines · Jörn Engel
Quoted 2 lines · Richard Purdie

Upgrade the zlib_inflate implementation in the kernel from a patched version 1.1.3 to a patched 1.2.3.

s/1.1.3/1.1.4/

I once pulled all the bugfixes between the versions into the kernel.

That's not what the header says :)

Well, the complete patch from 1.1.3 to 1.1.4 was rather big. Every single copyright line got updated to a current date, without any other changes in those files. I found it rather silly and took the bugfixes only. ;)

Quoted 8 lines
Quoted 4 lines

Review

Quoted 2 lines
(quoted diff) +2 -0
+#if 0
+int zlib_inflatePrime(z_streamp strm, int bits, int value)

Was this code dead in 1.2.3 as well?

This code was commented out in the kernel by Adrian Bunk as it has no users and I just followed that example when updating I'm sure if someone needed it, it could be enabled.

My thinking was rather that we could remove it. #if 0 has an advantage in case you made a mistake - easy to revert. But after a while...

Quoted 9 lines
Quoted 5 lines
Quoted 1 line

+#ifndef PKZIP_BUG_WORKAROUND

For the kernel, we can remove compat code against DOS compilers, etc. In this particular case, I believe we can just consider data compressed with PKZIP to be illegal and throw an error.

Agreed, there are places it can be tidied up a bit further. I was trying not to make too many changes so any future updates against zlib are easier.

Fair enough.

Jörn

Signature · 2 lines
Data expands to fill the space available for storage.
-- Parkinson's Law
#4 lore raw
In reply to Richard Purdie
Quoted 8 lines

Upgrade the zlib_inflate implementation in the kernel from a patched version 1.1.3 to a patched 1.2.3.

The code in the kernel is about seven years old and I noticed that the external zlib library's inflate performance was significantly faster (~50%) than the code in the kernel on ARM (and faster again on x86_32).

Any plans to move to something newer, e.g. bzip2 or LZMA?

Jan Engelhardt

#5 lore raw
Quoted 9 lines · Jan Engelhardt
Quoted 8 lines

Upgrade the zlib_inflate implementation in the kernel from a patched version 1.1.3 to a patched 1.2.3.

The code in the kernel is about seven years old and I noticed that the external zlib library's inflate performance was significantly faster (~50%) than the code in the kernel on ARM (and faster again on x86_32).

Any plans to move to something newer, e.g. bzip2 or LZMA?

For PPP? Write a new RFC suggesting it and explaining how to deal with the "chunkiness" and additional memory overhead. For JFFS2? Bzlib compresses worse than zlib when dealing with small blocks. For...

There may be a couple of uses where zlib could be replaced by something else. But even ignoring the compatibility issue it isn't always that simple.

Jörn

Signature · 3 lines
With a PC, I always felt limited by the software available. On Unix, 
I am limited only by my knowledge.
-- Peter J. Schoenster