Skip to contentSkip to conversation

Bad performance results with the latest git patches

DISCUSSION Valerie Clement 3 participants 4 messages Sep 7, 2007 – Sep 10, 2007

lore
Share & tools
lore
#0 lore raw

Hi Alex,

running ffsb tests (large files creation) on my system with the latest ext4 git patches against a 2.6.23-rc4 kernel, I've got very bad performance results: the I/O throughput measured on an ext4 filesystem is ten times lower than those measured on an XFS filesystem on the same machine. I have mounted the ext4 filesytem with mballoc, delalloc and data=writeback options.

dmesg output shows plenty of error messages like this:

EXT4-fs error (device sdc): ext4_ext_search_right: bad header in inode #3745797: unexpected eh_depth - magic f30a, entries 81, max 84(0), depth 0(1) EXT4-fs error (device sdc): ext4_ext_search_right: bad header in inode #3745797: unexpected eh_depth - magic f30a, entries 81, max 84(0), depth 0(1) EXT4-fs error (device sdc): ext4_ext_search_right: bad header in inode #3745797: unexpected eh_depth - magic f30a, entries 81, max 84(0), depth 0(1) EXT4-fs error (device sdc): ext4_ext_search_right: bad header in inode #3745797: unexpected eh_depth - magic f30a, entries 81, max 84(0), depth 0(1)

If I removed the second call to ext4_ext_check_header() in the ext4_ext_search_right() function, the problem disappears, no more error messages and better throughput values close to those measured on the XFS filesystem. It seems that the depth value passed in argument is buggy.

In a previous line,
while (++depth < path->p_depth) {
the depth value is incremented even if we don't enter the loop. Is it the problem ?

Could you check this?
Thanks,
Valérie

#1 lore raw
Quoted 8 lines · Valerie Clement

running ffsb tests (large files creation) on my system with the latest ext4 git patches against a 2.6.23-rc4 kernel, I've got very bad performance results: the I/O throughput measured on an ext4 filesystem is ten times lower than those measured on an XFS filesystem on the same machine.
I have mounted the ext4 filesytem with mballoc, delalloc and data=writeback options.

Quoted 5 lines

dmesg output shows plenty of error messages like this:

EXT4-fs error (device sdc): ext4_ext_search_right: bad header in inode #3745797: unexpected eh_depth - magic f30a, entries 81, max 84(0), depth 0(1)

It is the message that likely slows down the filesystem.

Quoted 11 lines

If I removed the second call to ext4_ext_check_header() in the ext4_ext_search_right() function, the problem disappears, no more error messages and better throughput values close to those measured on the XFS filesystem.
It seems that the depth value passed in argument is buggy.

In a previous line,
while (++depth < path->p_depth) {

the depth value is incremented even if we don't enter the loop. Is it the problem ?

No, because the "depth" value is not used after the loop is done. The problem is really that the depth decreases down the tree instead of increasing. This is not immediately seen during testing because mballoc does a good job of merging extents and files have to be very fragmented (heavy multi-threaded IO) and/or very large (>512MB) before the index grows outside the inode (depth > 0).

The problem was in the recently-added "extent sanity checks", and has also been fixed:

fs/ext3/extents.c +1 -1
diff -u linux-2.6.18.8/fs/ext3/extents.c linux-2.6.18.8/fs/ext3/extents.c
@@ -1069,7 +1069,7 @@ ext3_ext_search_right(struct inode *inode,
 		if (bh == NULL)
 			return -EIO;
 		eh = ext_block_hdr(bh);
-		if (ext3_ext_check_header(inode, eh, depth)) {
+		if (ext3_ext_check_header(inode, eh, path->p_depth - depth)) {
 			brelse(bh);
 			return -EIO;
 		}

Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.

#2 lore raw
Quoted 37 lines · Andreas Dilger
Quoted 6 lines · Valerie Clement

running ffsb tests (large files creation) on my system with the latest ext4 git patches against a 2.6.23-rc4 kernel, I've got very bad performance results: the I/O throughput measured on an ext4 filesystem is ten times lower than those measured on an XFS filesystem on the same machine. I have mounted the ext4 filesytem with mballoc, delalloc and data=writeback options.

dmesg output shows plenty of error messages like this:

EXT4-fs error (device sdc): ext4_ext_search_right: bad header in inode #3745797: unexpected eh_depth - magic f30a, entries 81, max 84(0), depth 0(1)

It is the message that likely slows down the filesystem.

Quoted 7 lines

If I removed the second call to ext4_ext_check_header() in the ext4_ext_search_right() function, the problem disappears, no more error messages and better throughput values close to those measured on the XFS filesystem. It seems that the depth value passed in argument is buggy.

In a previous line,
while (++depth < path->p_depth) {

the depth value is incremented even if we don't enter the loop. Is it the problem ?

No, because the "depth" value is not used after the loop is done. The problem is really that the depth decreases down the tree instead of increasing. This is not immediately seen during testing because mballoc does a good job of merging extents and files have to be very fragmented (heavy multi-threaded IO) and/or very large (>512MB) before the index grows outside the inode (depth > 0).

The problem was in the recently-added "extent sanity checks", and has also been fixed:

diff -u linux-2.6.18.8/fs/ext3/extents.c linux-2.6.18.8/fs/ext3/extents.c

fs/ext3/extents.c ext3_ext_search_right(struct inode *inode, · line 1075

Quoted 8 lines
fs/ext3/extents.c +1 -1
@@ -1069,7 +1069,7 @@ ext3_ext_search_right(struct inode *inode,
 		if (bh == NULL)
 			return -EIO;
 		eh = ext_block_hdr(bh);
-		if (ext3_ext_check_header(inode, eh, depth)) {
+		if (ext3_ext_check_header(inode, eh, path->p_depth - depth)) {
 			brelse(bh);
 			return -EIO;
 		}

s/ext3/ext4 :)

I will merge it with new-extent-function.patch. Will also add the above signed-off-by:

-aneesh

#3 lore raw
In reply to Andreas Dilger
Quoted 12 lines · Andreas Dilger

The problem was in the recently-added "extent sanity checks", and has also been fixed:

diff -u linux-2.6.18.8/fs/ext3/extents.c linux-2.6.18.8/fs/ext3/extents.c

fs/ext3/extents.c ext3_ext_search_right(struct inode *inode, · line 1075

Quoted 8 lines
fs/ext3/extents.c +1 -1
@@ -1069,7 +1069,7 @@ ext3_ext_search_right(struct inode *inode,
 		if (bh == NULL)
 			return -EIO;
 		eh = ext_block_hdr(bh);
-		if (ext3_ext_check_header(inode, eh, depth)) {
+		if (ext3_ext_check_header(inode, eh, path->p_depth - depth)) {
 			brelse(bh);
 			return -EIO;
 		}

Thanks Andreas, I confirm this patch fixes the problem.
Valérie