Skip to contentSkip to conversation

Ext4 benchmarks

DISCUSSION Jean-Pierre Dion 7 participants 13 messages Mar 20, 2007 – Apr 4, 2007

lore
Share & tools
lore

Jean-Pierre Dion

author
#0 lore raw

Hi all,

we already discussed during the conf calls what
benchmarks should be ran on ext4.

As we have OLS paper on the table we were thinking here at Bull what bench t run and on which kernel.

If we want trying to compare ext3 and ext4, I guess we should at least show that :
- ext4 has equivalent perfs than ext3,
- improvements done for ext3 are still in ext4 (mb alloc, del alloc...).

So we were wondering what's best to do :
- run on 2.6.19 (includes del alloc and mb alloc if I am not wrong),
- run on 2.6.20 (lacks mb alloc),

- select relevant benchs (iozone...).

What do you think ?

Thanks.

jean-pierre

#1 lore raw
Quoted 5 lines · Jean-Pierre Dion

Hi all,

we already discussed during the conf calls what
benchmarks should be ran on ext4.

Quoted 7 lines

As we have OLS paper on the table we were thinking here at Bull what bench t run and on which kernel.

If we want trying to compare ext3 and ext4, I guess we should at least show that :
- ext4 has equivalent perfs than ext3,

Define equivalent performance.

Are the workloads only going to be focused on single repetitive operations or simulation of actual desktop/server environments? How about performance on an aged filesystem?

Quoted 6 lines

- improvements done for ext3 are still in ext4 (mb alloc, del alloc...).

So we were wondering what's best to do :
- run on 2.6.19 (includes del alloc and mb alloc if I am not wrong),
- run on 2.6.20 (lacks mb alloc),

What about system configurations? While a desktop configuration would be easy to come by, a server configuration needs a bit more thought. Will ext4 perform better than ext3 in a wide range of storage configuration that scale from a couple thousands IOPS to several hundred thousand IOPS?

Having baseline data on other filesystems like XFS or JFS would be interesting as well to see how well ext4 stacks up to the competition. :)

Quoted 2 lines

- select relevant benchs (iozone...).

I haven checked IOzone in quite a bit but last time I checked FFSB had a couple of capabilities that are not available in IOzone like multi threading on a shared data set and a very customizable IO operations to attempt to simulate real IO patterns seen on workloads. Might be worth a look if your interested in compiling a very comprehensive set of results

Quoted 7 lines

What do you think ?

Thanks.

jean-pierre

-JRS

author
#2 lore raw

Hi Jose,

thank you for the feedback.

We took your remarks into account and we are doing some perfs with iozone (close to desktop activity, mono-thread) and ffsb (allows to run benchs in a multi-thread activity like a server does, different blocks sizes...).

We compare ext3 and ext4 (with extents, w/ and w/o del alloc...)...

We will publish the results on bullopensource.org

jean-pierre

Quoted 41 lines · Jose R. Santos
Quoted 5 lines · Jean-Pierre Dion

Hi all,

we already discussed during the conf calls what
benchmarks should be ran on ext4.

Quoted 7 lines

As we have OLS paper on the table we were thinking here at Bull what bench t run and on which kernel.

If we want trying to compare ext3 and ext4, I guess we should at least show that :
- ext4 has equivalent perfs than ext3,

Define equivalent performance.

Are the workloads only going to be focused on single repetitive operations or simulation of actual desktop/server environments? How about performance on an aged filesystem?

Quoted 6 lines

- improvements done for ext3 are still in ext4 (mb alloc, del alloc...).

So we were wondering what's best to do :
- run on 2.6.19 (includes del alloc and mb alloc if I am not wrong),
- run on 2.6.20 (lacks mb alloc),

What about system configurations? While a desktop configuration would be easy to come by, a server configuration needs a bit more thought. Will ext4 perform better than ext3 in a wide range of storage configuration that scale from a couple thousands IOPS to several hundred thousand IOPS?

Having baseline data on other filesystems like XFS or JFS would be interesting as well to see how well ext4 stacks up to the competition. :)

Quoted 2 lines

- select relevant benchs (iozone...).

I haven checked IOzone in quite a bit but last time I checked FFSB had a couple of capabilities that are not available in IOzone like multi threading on a shared data set and a very customizable IO operations to attempt to simulate real IO patterns seen on workloads. Might be worth a look if your interested in compiling a very comprehensive set of results

Quoted 7 lines

What do you think ?

Thanks.

jean-pierre

-JRS

#3 lore raw
Quoted 8 lines · Jean-Pierre Dion

Hi Jose,

thank you for the feedback.

We took your remarks into account and we are doing some perfs with iozone (close to desktop activity, mono-thread) and ffsb (allows to run benchs in a multi-thread activity like a server does, different blocks sizes...).

Quoted 7 lines

We compare ext3 and ext4 (with extents, w/ and w/o del alloc...)...

We will publish the results on bullopensource.org

jean-pierre

I also have a benchmark that stresses a heavy synchronous write workload that I can run.

ric

author
#4 lore raw

Hi Ric,

that may be useful. Checking with the team here and tell you.

Thanks.

jean-pierre

Quoted 22 lines · Ric Wheeler
Quoted 8 lines · Jean-Pierre Dion

Hi Jose,

thank you for the feedback.

We took your remarks into account and we are doing some perfs with iozone (close to desktop activity, mono-thread) and ffsb (allows to run benchs in a multi-thread activity like a server does, different blocks sizes...).

Quoted 7 lines

We compare ext3 and ext4 (with extents, w/ and w/o del alloc...)...

We will publish the results on bullopensource.org

jean-pierre

I also have a benchmark that stresses a heavy synchronous write workload that I can run.

ric

#5 lore raw
In reply to Jean-Pierre Dion
Quoted 4 lines · Jean-Pierre Dion

Hi Jose,

thank you for the feedback.

Quoted 8 lines

We took your remarks into account and we are doing some perfs with iozone (close to desktop activity, mono-thread) and ffsb (allows to run benchs in a multi-thread activity like a server does, different blocks sizes...).

We compare ext3 and ext4 (with extents, w/ and w/o del alloc...)...

We will publish the results on bullopensource.org

Hi Jean-Pierre,

While it may be to late for the purposes of your OLS paper, one thing that doesn't seem to be getting much attention is the performance of a file system while doing many meta-data operations or throughput testing during heavy journal log activity. I believe that IOzone is very limited in testing this and FFSB isn't much better at it either. Eventually, I plan to add support in FFSB to create workload profile were one can select a weight balance of these types of operations.

Is this something that you are already doing for this round of testing?

-JRS

#6 lore raw
Quoted 4 lines · Jose R. Santos

While it may be to late for the purposes of your OLS paper, one thing that doesn't seem to be getting much attention is the performance of a file system while doing many meta-data operations or throughput testing during heavy journal log activity.

Back in 2005, Alex sent a patch to monitor journal activity through procfs:
http://marc.info/?l=linux-ext4&m=113538565128617&w=2

We have been using this patch for more than 1 year and it is very useful for investigating performance issues.
Does anybody know why this patch hasn't found its way into the mainline? Maybe it could be merged in jbd2?

Johann

#7 lore raw
Quoted 6 lines · Johann Lombardi
Quoted 4 lines · Jose R. Santos

While it may be to late for the purposes of your OLS paper, one thing that doesn't seem to be getting much attention is the performance of a file system while doing many meta-data operations or throughput testing during heavy journal log activity.

Quoted 7 lines

Back in 2005, Alex sent a patch to monitor journal activity through procfs:
http://marc.info/?l=linux-ext4&m=113538565128617&w=2

We have been using this patch for more than 1 year and it is very useful for investigating performance issues.
Does anybody know why this patch hasn't found its way into the mainline? Maybe it could be merged in jbd2?

Yes, we use this functionality failry often, so having it in jbd2 would be quite useful.

Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.

author
#8 lore raw
In reply to Jose R. Santos

Hi Jose,

I have to check with the team here.
Will tell you.

Thanks.

jean-pierre

Quoted 22 lines · Jose R. Santos
Quoted 4 lines · Jean-Pierre Dion

Hi Jose,

thank you for the feedback.

Quoted 8 lines

We took your remarks into account and we are doing some perfs with iozone (close to desktop activity, mono-thread) and ffsb (allows to run benchs in a multi-thread activity like a server does, different blocks sizes...).

We compare ext3 and ext4 (with extents, w/ and w/o del alloc...)...

We will publish the results on bullopensource.org

Hi Jean-Pierre,

While it may be to late for the purposes of your OLS paper, one thing that doesn't seem to be getting much attention is the performance of a file system while doing many meta-data operations or throughput testing during heavy journal log activity. I believe that IOzone is very limited in testing this and FFSB isn't much better at it either. Eventually, I plan to add support in FFSB to create workload profile were one can select a weight balance of these types of operations.

Is this something that you are already doing for this round of testing?

-JRS

#10 lore raw
Quoted 2 lines · Cordenner jean noel

here is the first results of the round:
http://www.bullopensource.org/ext4/20070404/

Jean Noel,
thank you for the test results. It is always nice to see that ext4 is doing so well compared to ext3 and XFS.

Ming Ming,
it should be possible to just include the mballoc+delalloc patches that Jean Noel used into the upstream ext4 patch series. When Alex or Christoph get a chance to do the VFS delalloc rewrite we can move to that new patch, but until then it seems pointless to not include this functionality which improves the performance so much.

Also, if we include those patches the mballoc and delalloc features (along with extents) should be enabled by default if INCOMPAT_EXTENTS is in the superblock unless:
- "noextents", "nomballoc", or "nodelalloc" mount options are given
- delalloc needs to be disabled if blocksize != PAGE_SIZE

Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.

#11 lore raw
Quoted 8 lines · Andreas Dilger
Quoted 2 lines · Cordenner jean noel

here is the first results of the round:
http://www.bullopensource.org/ext4/20070404/

Jean Noel,
thank you for the test results. It is always nice to see that ext4 is doing so well compared to ext3 and XFS.

Quoted 8 lines

Ming Ming,
it should be possible to just include the mballoc+delalloc patches that Jean Noel used into the upstream ext4 patch series. When Alex or Christoph get a chance to do the VFS delalloc rewrite we can move to that new patch, but until then it seems pointless to not include this functionality which improves the performance so much.

From the bull website it said the test is based on 2.6.21-rc4 kernel +

delalloc patch. I don't think that includes the mballoc patch.

Quoted 4 lines

Also, if we include those patches the mballoc and delalloc features (along with extents) should be enabled by default if INCOMPAT_EXTENTS is in the superblock unless:
- "noextents", "nomballoc", or "nodelalloc" mount options are given

I just added noextents and nodelalloc mount options in the 2.6.21-rc5 version ext4 patch queue.

But we should keep delalloc with nomballoc. The current delalloc patch in ext4 tree plays well without mballoc. We still could do multiple block allocations with delayed allocation, though not as smart as Alex's mballoc.

Quoted 2 lines

- delalloc needs to be disabled if blocksize != PAGE_SIZE

I believe the current ext4 delalloc code turns off delalloc in this case already.

Quoted 10 lines

Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.

-
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#12 lore raw

ext4 patch queue update

Ted,

I Just rebased the ext4 patch queue to 2.6.21-rc5,

# Added Reserve high 32 bit for 64 bit inode version i_version_hi.patch

# Add mount option to turn off extents
ext4_noextent_mount_opt.patch

# Add mount option to turn off delayed allocation
ext4_nodelalloc_mount_opt.patch

And move nanosecond patch before the delayed allocation patches.

fsx test passed on ppc64,x86 and x86_64.
http://repo.or.cz/w/ext4-patch-queue.git

# Rebased the patches to 2.6.21-rc4

# New patch to fix whitespace before applying new patches whitespace.patch

# Replaced truncated beginning comments
extent-overlap-bugfix

persistent_allocation_1_ioctl_and_unitialized_extents

# Fixed an endian error persistent_allocation_2_support_for_writing_to_unitialized_extent

# updated to latest version
nanosecond_timestamps.patch
# i_verion
# Reserve high 32 bit for 64 bit inode version
# Missing the full inode version patch
i_version_hi.patch

# Add mount option to turn off extents
ext4_noextent_mount_opt.patch

##############################################################
# Unstable patches
# Note: still lots of outstanding comments from linux-ext4 list, 12/2006 # Missing signed-off-by:
booked-page-flag.patch

# Missing signed-off-by:
ext4-block-reservation.patch

# fixed a bunch of endianness errors reported by sparse
# Needs a signed-off-by from Alex, then can add shaggy's
ext4-delayed-allocation.patch

ext4-delalloc-extents-48bit.patch

# Add mount option to turn off delayed allocation
ext4_nodelalloc_mount_opt.patch