Skip to contentSkip to conversation

Update LZO compression

GIT PULL Markus F.X.J. Oberhumer 5 participants 8 messages Aug 21, 2012 – Oct 4, 2012 Btrfs file system

lore
Share & tools
lore
#0 lore raw

Hi all,

as suggested on the mailing list I have converted the updated LZO code into git, so please pull my "lzo-update" branch from

git://github.com/markus-oberhumer/linux.git lzo-update

You can browse the branch at

https://github.com/markus-oberhumer/linux/compare/lzo-update

I'd ask some official kernel maintainer for review and to push this into linux-next so that it will hopefully land in the 3.7 release.

Share and enjoy,
Markus

[ Changes in v2: Optimize code for CPUs with inefficient unaligned
access => significant speed increase on ARM ]

Quoted 20 lines · Markus F.X.J. Oberhumer

Hi all,

I finally have prepared a small package that updates the LZO version in the Linux kernel. Please get it from:

http://www.oberhumer.com/opensource/lzo/download/Testing/linux-kernel-lzo-20120716.tar.gz

As stated in the README this version is significantly faster (typically more than 2 times faster!) than the current version, has been thoroughly tested on x86_64/i386/powerpc platforms and is intended to get included into the official Linux 3.6 or 3.7 release.

I encourage all compression users to test and benchmark this new version, and I also would ask some official LZO maintainer to convert the updated source files into a GIT commit and possibly push it to Linus or linux-next.

Share and enjoy,
Markus

Signature · 1 line
Markus Oberhumer, <[email protected]>, http://www.oberhumer.com/
#1 lore raw
Quoted 2 lines · Markus F.X.J. Oberhumer

as suggested on the mailing list I have converted the updated LZO code into git, so please pull my "lzo-update" branch from

...

Quoted 2 lines

[ Changes in v2: Optimize code for CPUs with inefficient unaligned
access => significant speed increase on ARM ]

I can confirm that this new code runs at the same speed as the current lzo code in the Linux kernel on
my ARM926EJ-S based platform. I only tested decompression, using the attached hacky userspace code.

   # time ./lzo-bench/old/unlzop <lzoimage >/dev/null
   real    0m 0.29s
   # time ./lzo-bench/new/unlzop <lzoimage >/dev/null
   real    0m 0.29s

(where lzoimage is a Linux Image compressed with lzop)

So, from my side there are no more objections.
Thanks for doing this work, Markus.

Johannes
-------------- next part --------------
A non-text attachment was scrubbed...
Name: lzo-bench.tar.gz
Type: application/octet-stream
Size: 35783 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20120822/ee82ee5e/attachment-0001.obj>

#2 lore raw
In reply to Markus F.X.J. Oberhumer

Hi all,

is there some official maintainer who feels responsible for pushing the LZO update to Linus?

Otherwise I also could issue a pull-request on github, but somehow I don't think that this is the preferred workflow.

Cheers,
Markus

Quoted 45 lines · Markus F.X.J. Oberhumer

Hi all,

as suggested on the mailing list I have converted the updated LZO code into git, so please pull my "lzo-update" branch from

git://github.com/markus-oberhumer/linux.git lzo-update

You can browse the branch at

https://github.com/markus-oberhumer/linux/compare/lzo-update

I'd ask some official kernel maintainer for review and to push this into linux-next so that it will hopefully land in the 3.7 release.

Share and enjoy,
Markus

[ Changes in v2: Optimize code for CPUs with inefficient unaligned
access => significant speed increase on ARM ]

Quoted 20 lines · Markus F.X.J. Oberhumer

Hi all,

I finally have prepared a small package that updates the LZO version in the Linux kernel. Please get it from:

http://www.oberhumer.com/opensource/lzo/download/Testing/linux-kernel-lzo-20120716.tar.gz

As stated in the README this version is significantly faster (typically more than 2 times faster!) than the current version, has been thoroughly tested on x86_64/i386/powerpc platforms and is intended to get included into the official Linux 3.6 or 3.7 release.

I encourage all compression users to test and benchmark this new version, and I also would ask some official LZO maintainer to convert the updated source files into a GIT commit and possibly push it to Linus or linux-next.

Share and enjoy,
Markus

Signature · 1 line
Markus Oberhumer, <[email protected]>, http://www.oberhumer.com/
#3 lore raw

CC'in akpm.

Quoted 60 lines · Markus F.X.J. Oberhumer

Hi all,

is there some official maintainer who feels responsible for pushing the LZO update to Linus?

Otherwise I also could issue a pull-request on github, but somehow I don't think that this is the preferred workflow.

Cheers,
Markus

Quoted 17 lines · Markus F.X.J. Oberhumer

Hi all,

as suggested on the mailing list I have converted the updated LZO code into git, so please pull my "lzo-update" branch from

git://github.com/markus-oberhumer/linux.git lzo-update

You can browse the branch at

https://github.com/markus-oberhumer/linux/compare/lzo-update

I'd ask some official kernel maintainer for review and to push this into linux-next so that it will hopefully land in the 3.7 release.

Share and enjoy,
Markus

Quoted 28 lines

[ Changes in v2: Optimize code for CPUs with inefficient unaligned
access => significant speed increase on ARM ]

Quoted 20 lines · Markus F.X.J. Oberhumer

Hi all,

I finally have prepared a small package that updates the LZO version in the Linux kernel. Please get it from:

http://www.oberhumer.com/opensource/lzo/download/Testing/linux-kernel-lzo-20120716.tar.gz

As stated in the README this version is significantly faster (typically more than 2 times faster!) than the current version, has been thoroughly tested on x86_64/i386/powerpc platforms and is intended to get included into the official Linux 3.6 or 3.7 release.

I encourage all compression users to test and benchmark this new version, and I also would ask some official LZO maintainer to convert the updated source files into a GIT commit and possibly push it to Linus or linux-next.

Share and enjoy,
Markus

--
Markus Oberhumer, <[email protected]>, http://www.oberhumer.com/

Signature · 2 lines
Thanks,
//richard
#4 lore raw
Quoted 1 line · On Wed, 3 Oct 2012 12:48:46 +0200 richard -rw- weinberger…

CC'in akpm.

Thanks.

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

Hi all,

I finally have prepared a small package that updates the LZO version in the Linux kernel. Please get it from:

http://www.oberhumer.com/opensource/lzo/download/Testing/linux-kernel-lzo-20120716.tar.gz

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

As stated in the README this version is significantly faster (typically more than 2 times faster!) than the current version, has been thoroughly tested on x86_64/i386/powerpc platforms and is intended to get included into the official Linux 3.6 or 3.7 release.

I encourage all compression users to test and benchmark this new version, and I also would ask some official LZO maintainer to convert the updated source files into a GIT commit and possibly push it to Linus or linux-next.

No, lib/lzo has no identifiable maintainer. I suggest you proceed as follows:

- Post the entire patch series to lkml for review (I'd like a cc please)

- After that process has played out, ask Stephen to add this git tree to linux-next.

- After that process has played out, ask Linus to pull the tree during a merge window.

I haven't actually looked at the patches yet, but if they are as extensive as they sound, it would be appropriate for you become the formal maintainer of lib/lzo.

#5 lore raw
Quoted 4 lines

No, lib/lzo has no identifiable maintainer. I suggest you proceed as follows:

- Post the entire patch series to lkml for review (I'd like a cc please)

Already happened, multiple people reviewed and tested.

This was just the patch ping.

-Andi

Signature · 1 line
[email protected] -- Speaking for myself only.
#6 lore raw
Quoted 6 lines · On Wed, 3 Oct 2012 23:19:04 +0200 Andi Kleen <[email protected]> wrote:
Quoted 4 lines

No, lib/lzo has no identifiable maintainer. I suggest you proceed as follows:

- Post the entire patch series to lkml for review (I'd like a cc please)

Already happened, multiple people reviewed and tested.

um, I would not consider "Looks ok to me from a quick look." and "I couldn't tell from the github view, but I assume you follow standard coding style." to indicate a rigorous code review!

That's the problem with the git presentation: hardly anyone reads the patches and there is no patch for a reviewer to reply to.

So please send the patches out for review. One at a time, via email.

#7 lore raw
Quoted 7 lines
Quoted 3 lines
Quoted 1 line

- Post the entire patch series to lkml for review (I'd like a cc please)

Already happened, multiple people reviewed and tested.

um, I would not consider "Looks ok to me from a quick look." and "I couldn't tell from the github view, but I assume you follow standard coding style." to indicate a rigorous code review!

Let's put it like this. LZO is mature widely used production software and Markus wrote and maintains it. Essentially this is a release from him.

The main problem before with using his version directly was that it did not follow Linux coding standards, but that seems to be fixed now. It's also relatively simple and at least I didn't spot anything bad when I looked.

It was runtime tested by several people on different architectures. If it was fuzzed so we can assume reasonably good security.

-Andi

Signature · 1 line
[email protected] -- Speaking for myself only.