Skip to contentSkip to conversation

Compression algorithm in cloop

DISCUSSION Phillip Lougher 5 participants 5 messages Aug 11, 2004

lore
Share & tools
lore
#0 lore raw
Quoted 5 lines

Hello,

I am trying to do some experiment on compression ratios with cloop. I know that currently cloop uses zlib. How can I change it to other algorithms?

Changing the algorithm from gzip is going to be probably unpopular. Cloop uses the gzip deflate library inside the kernel shared by a large number of other programs. To change the algorithm you'll have to add more (private) decompression code to the kernel. This is a retrograde step because the shared library was only introduced in 2.4.17 to avoid lots of private copies of gzip.

Quoted 2 lines

Where should I start from? Really a newbie to this, appreciate any comments and suggestions!!

There has been discussion on this list before about adding bzip2 support to the kernel. Do a search on the list for this.

Regards

Phillip

#1 lore raw
Quoted 5 lines · Phillip Lougher
Quoted 2 lines

Where should I start from? Really a newbie to this, appreciate any comments and suggestions!!

There has been discussion on this list before about adding bzip2 support to the kernel. Do a search on the list for this.

Long, long ago there was something called e2compr. They had bzip2 in the kernel. I remember 40 seconds mc start time on disk compressed with bzip2 on i486-DX 40 MHz... :-)

Grzegorz Kulewski

#2 lore raw

On Wed, 11 Aug 2004 19:21:50 +0200 (CEST) Grzegorz Kulewski wrote:

| On Wed, 11 Aug 2004, Phillip Lougher wrote:
| 
| >> Where should I start from? Really a newbie to this,
| >> appreciate any comments and suggestions!!
| >
| > There has been discussion on this list before about adding
| > bzip2 support to the kernel.  Do a search on the list for this.
| 
| Long, long ago there was something called e2compr. They had bzip2 in the 
| kernel. I remember 40 seconds mc start time on disk compressed with bzip2 
| on i486-DX 40 MHz... :-)

still at
http://e2compr.sourceforge.net/

--
~Randy

#3 lore raw
In reply to Phillip Lougher

If I don't want to build a new compression library and port it to kernel, all I want to do is to try out the idea -- whether it could work on cloop, whether the compression ratio is acceptable, can I just change the cloop code? In other words, I just need a loopback block device with other compression scheme.

I know that adding the new algorithm support to kernel would be standard way to do this and would be helpful to more people, it's just that I don't have that time :(

Quoted 23 lines · Phillip Lougher
Quoted 5 lines

Hello,

I am trying to do some experiment on compression ratios with cloop. I know that currently cloop uses zlib. How can I change it to other algorithms?

Changing the algorithm from gzip is going to be probably unpopular. Cloop uses the gzip deflate library inside the kernel shared by a large number of other programs. To change the algorithm you'll have to add more (private) decompression code to the kernel. This is a retrograde step because the shared library was only introduced in 2.4.17 to avoid lots of private copies of gzip.

Quoted 2 lines

Where should I start from? Really a newbie to this, appreciate any comments and suggestions!!

There has been discussion on this list before about adding bzip2 support to the kernel. Do a search on the list for this.

Regards

Phillip

#4 lore raw
Quoted 2 lines · Lei Yang

If I don't want to build a new compression library and port it to kernel, all I want to do is to try out the idea -- whether it could work on cloop, whether the compression ratio is acceptable, can I just change the cloop code? In other words, I just need a loopback block device with other compression scheme.

If you just want to try out some wacky ideas, I suggest you take a look at the network block device.

It is a block device that sends all read/write requests through a socket
so that this requests can be fulfilled by a user space deamon.

This is not very good in terms of performance, but it is much easier to develop this kind of thing in userspace, without all the stack / memory / atomicity / concurrency problems :)

Signature · 2 lines
Paulo Marques - www.grupopie.com
"In a world without walls and fences who needs windows and gates?"