Skip to contentSkip to conversation

Add LZO1X compression support to the kernel

PATCH [email protected] 4 participants 4 messages May 12, 2007 – May 16, 2007

lore
Share & tools
lore
#0 lore raw

oh - and think of linux software suspend.
take a notebook with 2 GB of ram - that takes a while to write that to disk and read that back again. using lzo compression for this may probably halve the time for suspend/resume

using a fast compression scheme

Quoted 67 lines

-----Ursprüngliche Nachricht-----
Von: [email protected]
Gesendet: 11.05.07 22:48:15
An: [email protected]
CC: [email protected],[email protected]
Betreff: Re: [PATCH] Add LZO1X compression support to the kernel

Quoted 1 line

Why is this needed? What code plans to use it?

its pretty useful because its and a damn fast and damn cpu friendly compression alorithm.

afaik, there is already a least one linux kernel-feature (under development) which is using lzo compression:
see compressed caching project at http://linux-mm.org/CompressedCaching & http://linuxcompressed.sourceforge.net/

seems, they have also done porting it to the kernel, so there is probably choice between two implemetations to merge.

Quoted 1 line

How many buffer overruns are there in it?

i don`t know :)
but, from a user-perspective, lzo is really portable and seems to be a rock solid compression scheme. i`m sucessfully using it for years (lzop utility) and i know projects which compress gigabytes of data every day with lzop. furthermore, i know of at least 40 software projects using lzo compression, so this should have some level of maturity.

maybe i can add another software integrating lzo compression to the enumeration at http://www.lzop.de ? ;)

regards
roland

List:       linux-kernel
Subject:    Re: [PATCH] Add LZO1X compression support to the kernel
From:       Andrew Morton <akpm () linux-foundation ! org>
Date:       2007-05-10 6:21:29
Message-ID: 20070509232129.371f49d5.akpm () linux-foundation ! org

[Download message RAW]

Quoted 20 lines · On Wed, 02 May 2007 09:56:23 +0100 Richard Purdie <[email protected]>…

Current thinking is that lzo should get merged directly followed by the subsystem parts through their specific trees. It appears this should make it onto LKML despite the size so here goes.

Please keep in mind I haven't reformatted the LZO code itself as if I do so, it will make maintenance of it against any changes in LZO itself near impossible. In its current form, it should be possible to diff against upstream. All the bad formatting is confined to a handful of files in lib/lzo/ and the kernel interface should be clean.

I realise a maze of ifdefs still remain. I've already spent a lot of time removing a ton of them and going much further might start to affect diffability of the code - I hoping whats there is a good compromise.

I've asked the LZO author about the comments on lzo_copyright function but the code is GPLv2 licensed so is suitable for inclusion in the kernel.

Quoted 6 lines

Add LZO1X compression/decompression support to the kernel.

This is based on the standard userspace lzo library, particularly minilzo with the headers much trimmed down and simplified for kernel use. Its structured so that it should still diff with the userspace version for ease of future updating.

Quoted 5 lines

Well that's attractive-looking code.

Why is this needed? What code plans to use it?

How many buffer overruns are there in it?

_____________________________________________________________________
Der WEB.DE SmartSurfer hilft bis zu 70% Ihrer Onlinekosten zu sparen!
http://smartsurfer.web.de/?mc=100071&distributionid=000000000066

#1 lore raw
Quoted 4 lines · On Sat, 12 May 2007 12:41:03 +0200 [email protected] wrote:

oh - and think of linux software suspend.
take a notebook with 2 GB of ram - that takes a while to write that to disk and read that back again. using lzo compression for this may probably halve the time for suspend/resume

There were already some attempts on merging of the lzf algorithm:

http://lkml.org/lkml/2006/6/26/215

Also, ffmpeg/libavcodec contains a much cleaner implementation of LZO decompressor, with comparable performance (but no compressor yet I am afraid).

Signature · 1 line
Jindrich Makovicka
#2 lore raw

Hi Jindrich,

Quoted 11 lines · Jindrich Makovicka
Quoted 4 lines · On Sat, 12 May 2007 12:41:03 +0200 [email protected] wrote:

oh - and think of linux software suspend.
take a notebook with 2 GB of ram - that takes a while to write that to disk and read that back again. using lzo compression for this may probably halve the time for suspend/resume

There were already some attempts on merging of the lzf algorithm:

http://lkml.org/lkml/2006/6/26/215

Hmmm ... that thread had some really neat and simple kernel-compliant code, but it wanted to bring in a different compression algo :-) LZF, LZO, LZW are all different.

Quoted 1 line

Also, ffmpeg/libavcodec contains a much cleaner implementation of LZO

Yeah, I hope the discussion here shifts (after the copyright issues are resolved,
of course) to the submitted code itself. It's a compression library also linked to
cryptoapi, so would definitely find users for itself -- this was later submitted as
a patchset containing the JFFS2 glue code too.

But what I'm not sure about is the suitability of _this_ implementation to be merged AS IS (~2500 lines of non-kernel-style, and even pointless, code is no small thing) without a proper conversion / port to the kernel.

The upsides of a proper port would be nice: I suspect ~500 LOC in those two headers would vanish, and perhaps an equal number in lzo.c (it even re-invents memset, memcpy, etc for itself!).

The downside, as Richard said, would be that the kernel's LZO is no longer diff-able with minilzo (the userspace library) which means someone who takes on maintainership of this will have more work to do. (but how much more would it be anyway?)

Quoted 2 lines

decompressor, with comparable performance (but no compressor yet I am afraid).

Satyam

#3 lore raw
In reply to [email protected]
Quoted 3 lines · On Sat 2007-05-12 12:41:03, [email protected] wrote:

oh - and think of linux software suspend.
take a notebook with 2 GB of ram - that takes a while to write that to disk and read that back again. using lzo compression for this may probably halve the time for suspend/resume

suspend.sf.net. We don't need lzo in kernel to do compressed hibernation.

Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html