Skip to contentSkip to conversation

random minor benchmark: Re: Copy 20 tarfiles: ext2 vs (reiser4, unixfile) vs (reiser4,cryptcompress)

DISCUSSION Hans Reiser 6 participants 18 messages Jan 25, 2006 – Feb 20, 2006 1 message not in the archive

lore
Share & tools
lore
#1 lore raw
In reply to a message not in the archive

Notice how CPU speed (and number of cpus) completely determines compression performance.

cryptcompress refers to the reiser4 compression plugin, (unix file) refers to the reiser4 non-compressing plugin.

Quoted 40 lines · Edward Shishkin

Here are the tests that vs asked for:
Creation (dd) of 20 tarfiles (the original 200M file is in ramfs)
Kernel: 2.6.15-mm4 + current git snapshot of reiser4

------------------------------------------

Laputa workstation
Uni Intel Pentium 4 (2.26 GHz) 512M RAM

ext2:
real 2m, 15s
sys 0m, 14s

reiser4(unix file)
real 2m, 7s
sys 0m, 23s

reiser4(cryptcompress, lzo1, 64K)
real 2m, 13s
sys 0m, 11s
------------------------------------------

Belka workstation
Dual Intel Xeon (2.4GHz) 1G RAM

ext2:
real 2m, 16s
sys 0m, 10s

reiser4(unix file)
real 2m, 14s
sys 0m, 17s

reiser4(cryptcompress, lzo1, 64K)
real 1m, 35s
sys 0m, 14s

#2 lore raw
Quoted 28 lines · Hans Reiser

Notice how CPU speed (and number of cpus) completely determines compression performance.

cryptcompress refers to the reiser4 compression plugin, (unix file) refers to the reiser4 non-compressing plugin.

Quoted 20 lines · Edward Shishkin

Here are the tests that vs asked for:
Creation (dd) of 20 tarfiles (the original 200M file is in ramfs)
Kernel: 2.6.15-mm4 + current git snapshot of reiser4

------------------------------------------

Laputa workstation
Uni Intel Pentium 4 (2.26 GHz) 512M RAM

ext2:
real 2m, 15s
sys 0m, 14s

reiser4(unix file)
real 2m, 7s
sys 0m, 23s

reiser4(cryptcompress, lzo1, 64K)
real 2m, 13s
sys 0m, 11s

Just curious - does your crypt plugin reside in user space?

Signature · 1 line
Jens Axboe
#3 lore raw
In reply to Jens Axboe
Quoted 30 lines · Jens Axboe
Quoted 27 lines · Hans Reiser

Notice how CPU speed (and number of cpus) completely determines compression performance.

cryptcompress refers to the reiser4 compression plugin, (unix file) refers to the reiser4 non-compressing plugin.

Quoted 17 lines · Edward Shishkin

Here are the tests that vs asked for:
Creation (dd) of 20 tarfiles (the original 200M file is in ramfs)
Kernel: 2.6.15-mm4 + current git snapshot of reiser4

------------------------------------------

Laputa workstation
Uni Intel Pentium 4 (2.26 GHz) 512M RAM

ext2:
real 2m, 15s
sys 0m, 14s

reiser4(unix file)
real 2m, 7s
sys 0m, 23s

Quoted 10 lines
Quoted 5 lines
Quoted 5 lines

reiser4(cryptcompress, lzo1, 64K)
real 2m, 13s
sys 0m, 11s

Just curious - does your crypt plugin reside in user space?

Nop.
This is just wrappers for linux crypto api, zlib, etc.. so user time is zero and not interesting.

Edward.

#15 lore raw
In reply to Danny Milosavljevic
Quoted 11 lines · Danny Milosavljevic

Hi,

Nice :)

Just curious, is there a description how to enable cryptcompress for files somewhere? (or is it still bleeding-edge ? :))

cheers,
Danny

Hello.
If you have a free partition and a time to report about possible bugs, we'll send you a setup against 2.6.15-mm4 with the description.

Thanks,
Edward.

#4 lore raw
In reply to Edward Shishkin
Quoted 38 lines · Edward Shishkin
Quoted 36 lines · Jens Axboe
Quoted 32 lines · Hans Reiser

Notice how CPU speed (and number of cpus) completely determines compression performance.

cryptcompress refers to the reiser4 compression plugin, (unix file) refers to the reiser4 non-compressing plugin.

Quoted 22 lines · Edward Shishkin

Here are the tests that vs asked for:
Creation (dd) of 20 tarfiles (the original 200M file is in ramfs)
Kernel: 2.6.15-mm4 + current git snapshot of reiser4

------------------------------------------

Laputa workstation
Uni Intel Pentium 4 (2.26 GHz) 512M RAM

ext2:
real 2m, 15s
sys 0m, 14s

reiser4(unix file)
real 2m, 7s
sys 0m, 23s

reiser4(cryptcompress, lzo1, 64K)
real 2m, 13s
sys 0m, 11s

Quoted 8 lines
Quoted 4 lines

Just curious - does your crypt plugin reside in user space?

Nop.
This is just wrappers for linux crypto api, zlib, etc.. so user time is zero and not interesting.

Then why is the sys time lower than the "plain" writes on ext2 and reiser4? Surely compressing isn't for free, yet the sys time is lower on the compression write than the others.

Signature · 1 line
Jens Axboe
#5 lore raw
Quoted 60 lines · Jens Axboe
Quoted 51 lines · Edward Shishkin
Quoted 34 lines · Jens Axboe
Quoted 29 lines · Hans Reiser

Notice how CPU speed (and number of cpus) completely determines compression performance.

cryptcompress refers to the reiser4 compression plugin, (unix file) refers to the reiser4 non-compressing plugin.

Quoted 17 lines · Edward Shishkin

Here are the tests that vs asked for:
Creation (dd) of 20 tarfiles (the original 200M file is in ramfs)
Kernel: 2.6.15-mm4 + current git snapshot of reiser4

------------------------------------------

Laputa workstation
Uni Intel Pentium 4 (2.26 GHz) 512M RAM

ext2:
real 2m, 15s
sys 0m, 14s

reiser4(unix file)
real 2m, 7s
sys 0m, 23s

Quoted 13 lines
Quoted 7 lines
Quoted 7 lines

reiser4(cryptcompress, lzo1, 64K)
real 2m, 13s
sys 0m, 11s

Just curious - does your crypt plugin reside in user space?

Quoted 5 lines

Nop.
This is just wrappers for linux crypto api, zlib, etc.. so user time is zero and not interesting.

Quoted 6 lines

Then why is the sys time lower than the "plain" writes on ext2 and reiser4? Surely compressing isn't for free, yet the sys time is lower on the compression write than the others.

I guess this is because real compression is going in background flush, not in sys_write->write_cryptcompress (which just copies user's data to page cache). So in this case we have something very similar to ext2. Reiser4 plain write (write_unix_file) is more complex, and currently we try to reduce its sys time.

Edward.

#13 lore raw
In reply to Jens Axboe
Quoted 30 lines · Jens Axboe
Quoted 27 lines · Hans Reiser

Notice how CPU speed (and number of cpus) completely determines compression performance.

cryptcompress refers to the reiser4 compression plugin, (unix file) refers to the reiser4 non-compressing plugin.

Quoted 17 lines · Edward Shishkin

Here are the tests that vs asked for:
Creation (dd) of 20 tarfiles (the original 200M file is in ramfs)
Kernel: 2.6.15-mm4 + current git snapshot of reiser4

------------------------------------------

Laputa workstation
Uni Intel Pentium 4 (2.26 GHz) 512M RAM

ext2:
real 2m, 15s
sys 0m, 14s

reiser4(unix file)
real 2m, 7s
sys 0m, 23s

Quoted 10 lines
Quoted 5 lines
Quoted 5 lines

reiser4(cryptcompress, lzo1, 64K)
real 2m, 13s
sys 0m, 11s

Just curious - does your crypt plugin reside in user space?

No, kernel. It would have to encrypt+compress with every write to be in user space, we encrypt+compress only at flush time, and that is a key optimization (encryption is disabled at the moment due to needing a little API work, but....) for file sets that are cachable.

#6 lore raw
In reply to Edward Shishkin
Quoted 12 lines · Edward Shishkin

I guess this is because real compression is going in background flush, not in sys_write->write_cryptcompress (which just copies user's data to page cache). So in this case we have something very similar to ext2. Reiser4 plain write (write_unix_file) is more complex, and currently we try to reduce its sys time.

Edward.

Which means that only real time is a meaningful measurement.....

#8 lore raw
In reply to Edward Shishkin
Quoted 69 lines · Edward Shishkin
Quoted 60 lines · Jens Axboe
Quoted 51 lines · Edward Shishkin
Quoted 34 lines · Jens Axboe
Quoted 29 lines · Hans Reiser

Notice how CPU speed (and number of cpus) completely determines compression performance.

cryptcompress refers to the reiser4 compression plugin, (unix file) refers to the reiser4 non-compressing plugin.

Quoted 17 lines · Edward Shishkin

Here are the tests that vs asked for:
Creation (dd) of 20 tarfiles (the original 200M file is in ramfs)
Kernel: 2.6.15-mm4 + current git snapshot of reiser4

------------------------------------------

Laputa workstation
Uni Intel Pentium 4 (2.26 GHz) 512M RAM

ext2:
real 2m, 15s
sys 0m, 14s

reiser4(unix file)
real 2m, 7s
sys 0m, 23s

Quoted 13 lines
Quoted 7 lines
Quoted 7 lines

reiser4(cryptcompress, lzo1, 64K)
real 2m, 13s
sys 0m, 11s

Just curious - does your crypt plugin reside in user space?

Quoted 5 lines

Nop.
This is just wrappers for linux crypto api, zlib, etc.. so user time is zero and not interesting.

Quoted 6 lines

Then why is the sys time lower than the "plain" writes on ext2 and reiser4? Surely compressing isn't for free, yet the sys time is lower on the compression write than the others.

Quoted 5 lines

I guess this is because real compression is going in background flush, not in sys_write->write_cryptcompress (which just copies user's data to page cache). So in this case we have something very similar to ext2. Reiser4 plain write (write_unix_file) is more complex, and currently we try to reduce its sys time.

So the systime quoted above is basically useless, it doesn't reflect the real time spent in the kernel by far. I think you should note that when you post these scores, otherwise you're really showing a skewed picture.

Signature · 1 line
Jens Axboe
#7 lore raw
In reply to Hans Reiser
Quoted 9 lines · Hans Reiser
Quoted 7 lines · Edward Shishkin

I guess this is because real compression is going in background flush, not in sys_write->write_cryptcompress (which just copies user's data to page cache). So in this case we have something very similar to ext2. Reiser4 plain write (write_unix_file) is more complex, and currently we try to reduce its sys time.

Quoted 6 lines
Quoted 5 lines

Edward.

Which means that only real time is a meaningful measurement.....

Indeed. I guess the compression stuff cost is hard to quantify, since it has cache effects on the rest of the system in addition to costing CPU cycles on its own.

A profile of, say, dbench with and without compression would be interesting to see. And the actual dbench reults, naturally :-)

Signature · 1 line
Jens Axboe
#9 lore raw
In reply to Jens Axboe
Quoted 7 lines · Jens Axboe

So the systime quoted above is basically useless, it doesn't reflect the

real time spent in the kernel by far. I think you should note that when you post these scores, otherwise you're really showing a skewed picture.

He wasn't expecting me to post the benchmark, and I frankly only looked at the real time and forgot there was a systime in there that needed cutting out when I posted it. My error, not his. I must say though, the real time makes me quite happy, especially considering how CPUs are just going to keep getting faster.

Best,

Hans

#10 lore raw
Quoted 9 lines · Hans Reiser
Quoted 7 lines · Jens Axboe

So the systime quoted above is basically useless, it doesn't reflect the

real time spent in the kernel by far. I think you should note that when you post these scores, otherwise you're really showing a skewed picture.

Quoted 5 lines

He wasn't expecting me to post the benchmark, and I frankly only looked at the real time and forgot there was a systime in there that needed cutting out when I posted it. My error, not his. I must say though, the real time makes me quite happy, especially considering how CPUs are just going to keep getting faster.

Yeah and that's ok, I was just interested in seeing some more interesting compression benchmarks so I wondered if you had done that.

Signature · 1 line
Jens Axboe
#11 lore raw
Quoted 6 lines · Jens Axboe

Yeah and that's ok, I was just interested in seeing some more interesting compression benchmarks so I wondered if you had done that.

I think "random minor benchmark" was an apt description, yes.;-)

First we will debug it fully. Then we will figure out how to change mongo so that the files do not consist entirely of the letter a as their contents, and run mongo on it. Probably we will find some way to slice up a linux kernel tar file into files of random sizes, and assume that is a fair thing to let it compress during mongo. Then, just so people won't think mongo is slanted in our favor we will do some cp -r's of large numbers of linux kernel source trees and time that also.

Hans

#12 lore raw
In reply to Jens Axboe

Hello,

Quoted 33 lines · Jens Axboe
Quoted 32 lines · Edward Shishkin
Quoted 30 lines · Jens Axboe
Quoted 27 lines · Hans Reiser

Notice how CPU speed (and number of cpus) completely determines compression performance.

cryptcompress refers to the reiser4 compression plugin, (unix file) refers to the reiser4 non-compressing plugin.

Quoted 20 lines · Edward Shishkin

Here are the tests that vs asked for:
Creation (dd) of 20 tarfiles (the original 200M file is in ramfs)
Kernel: 2.6.15-mm4 + current git snapshot of reiser4

------------------------------------------

Laputa workstation
Uni Intel Pentium 4 (2.26 GHz) 512M RAM

ext2:
real 2m, 15s
sys 0m, 14s

reiser4(unix file)
real 2m, 7s
sys 0m, 23s

reiser4(cryptcompress, lzo1, 64K)
real 2m, 13s
sys 0m, 11s

Just curious - does your crypt plugin reside in user space?

Quoted 7 lines
Quoted 3 lines

Nop.
This is just wrappers for linux crypto api, zlib, etc.. so user time is zero and not interesting.

Then why is the sys time lower than the "plain" writes on ext2 and reiser4? Surely compressing isn't for free, yet the sys time is lower on the compression write than the others.

I guess the compression was done by the background writeout daemon. CPU utilization numbers would say more than sys time.

Signature · 1 line
Alex.
#16 lore raw
In reply to Edward Shishkin

Hi,

Am Donnerstag, den 26.01.2006, 21:41 +0300 schrieb Edward Shishkin:

Quoted 14 lines
Quoted 11 lines · Danny Milosavljevic

Hi,

Nice :)

Just curious, is there a description how to enable cryptcompress for files somewhere? (or is it still bleeding-edge ? :))

cheers,
Danny

Quoted 4 lines

Hello.
If you have a free partition and a time to report about possible bugs, we'll send you a setup against 2.6.15-mm4 with the description.

yes, now I do :)

Quoted 3 lines

Thanks,
Edward.

cheers,
Danny

#17 lore raw

Hi again,

Wow does that mean cryptocompress is almost ready for primetime (next months or so)?!
I've been waiting for something like this for ages, it would be so great if that could turn into reality soon.

I am quite enthusiastic how it will perform on the Dual-Athlon64 + 4x Software-Raid-5 (PATA) I plan to buy.

Thank you many, many times!!

lg Clemens

#18 lore raw
Quoted 7 lines · Clemens Eisserer

Hi again,

Wow does that mean cryptocompress is almost ready for primetime (next months or so)?!
I've been waiting for something like this for ages, it would be so great if that could turn into reality soon.

Quoted 10 lines

I am quite enthusiastic how it will perform on the Dual-Athlon64 + 4x Software-Raid-5 (PATA) I plan to buy.

Thank you many, many times!!

lg Clemens

It should work well for that hardware configuration. See our website for new benchmarks using cryptcompress.