Skip to contentSkip to conversation

updated data=ordered patch for 2.6.3

PATCH In review Chris Mason 6 participants 18 messages Feb 19, 2004 – Mar 3, 2004 1 message not in the archive

lore
Share & tools
lore

Chris Mason

author
#0 lore raw

PATCH 1/1 updated data=ordered patch for 2.6.3

Hello everyone,

I've updated the reiserfs logging speedups and data=ordered support to 2.6.3, and fixed a few bugs:

i_block count not properly updated on files with holes oops when the disk is full
small files were not being packed into tails

Also, the code has some extra performance tweaks to smooth out performance both with and without data=ordered. There are new mechanisms to trigger metadata/commit block writeback and to help throttle writers. The goal is to reduce the huge bursts of io during a commit and during data=ordered writeback.

I'm very interested in benchmarks here and info about how the patches feel for interactive performance.

Please note that data=ordered is not the default yet, you have to mount with -o data=ordered to get it.

You can get the patches from:

ftp.suse.com/pub/people/mason/patches/data-logging/experimental/2.6.3

(once the suse mirror copies it over there)

-chris

#1 lore raw

Am Donnerstag, 19. Februar 2004 05:36 schrieb Chris Mason:

Quoted 9 lines

Hello everyone,

I've updated the reiserfs logging speedups and data=ordered support to 2.6.3, and fixed a few bugs:

i_block count not properly updated on files with holes oops when the disk is full
small files were not being packed into tails

Quoted 5 lines

Also, the code has some extra performance tweaks to smooth out performance both with and without data=ordered. There are new mechanisms to trigger metadata/commit block writeback and to help throttle writers. The goal is to reduce the huge bursts of io during a commit and during data=ordered writeback.

On-top of SuSE 9.0, kernel 2.6.3-0:

01-reiserfs-journal-writer

Apply clean.

02-reiserfs-nesting
03-reiserfs-iosize

Are in.

04-reiserfs-logging
05-reiserfs-jh-3

Need some handwork due to ACLs.

You've forgotten in linux/fs/buffer.c
EXPORT_SYMBOL(try_to_release_page);
again ;-)

And the FIX in stree.c:

fs/reiserfs/stree.c 2004-02-16 09:27:24.166887040 -0500 renamed from fs/reiserfs/stree.c.1 2004-02-16 09:26:33.643567752 -0500 +1 -1
@@ -1754,7 +1754,7 @@
          reiserfs_update_sd(th, p_s_inode) ;
 
          journal_end(th, p_s_inode->i_sb, orig_len_alloc) ;
-         journal_begin(th, p_s_inode->i_sb, orig_len_alloc) ;
+         journal_begin(th, p_s_inode->i_sb, JOURNAL_PER_BALANCE_CNT * 5) ;
          reiserfs_update_inode_transaction(p_s_inode) ;
        }
     } while ( n_file_size > ROUND_UP (n_new_file_size) &&
Quoted 2 lines

I'm very interested in benchmarks here and info about how the patches feel for interactive performance.

Not, yet.
But same as with stree.c on 2.6.2-6.

Greetings,
Dieter

author
#2 lore raw
Quoted 4 lines · Dieter Nützel

You've forgotten in linux/fs/buffer.c
EXPORT_SYMBOL(try_to_release_page);
again ;-)

Fixed in a new upload.

Quoted 2 lines

And the FIX in stree.c:

Also fixed. Patches 00* will get submitted to Andrew as soon as namesys reviews them.

There are going into the suse kernel now too, it's time for some broader testing.

-chris

#3 lore raw
Quoted 9 lines · Chris Mason

Also fixed. Patches 00* will get submitted to Andrew as soon as namesys reviews them.

There are going into the suse kernel now too, it's time for some broader testing.

-chris

Those patches make use of the ->journal_info field ... which is on the
other hand removed in the reiser4 kernel patch, so both patches conflict :-/
Is there any way to get both reiser4 and reiser3+data-logging in the
same kernel ? Would adding back the journal_info field be enough ?

Compilation with a reiser4-patched kernel fails with:

/usr/src/linux$ make

  CHK     include/linux/version.h
  CC      scripts/empty.o
  MKELF   scripts/elfconfig.h
  HOSTCC  scripts/file2alias.o
  HOSTCC  scripts/modpost.o
  HOSTCC  scripts/sumversion.o
  HOSTLD  scripts/modpost
  CC      arch/i386/kernel/asm-offsets.s
  CHK     include/asm-i386/asm_offsets.h
  CHK     include/linux/compile.h
  CC      fs/buffer.o
  CC      fs/reiserfs/bitmap.o

In file included from fs/reiserfs/bitmap.c:8:
include/linux/reiserfs_fs.h: In function `reiserfs_transaction_running':
include/linux/reiserfs_fs.h:1753: error: structure has no member named `journal_info' make[2]: *** [fs/reiserfs/bitmap.o] Error 1
make[1]: *** [fs/reiserfs] Error 2
make: *** [fs] Error 2

/usr/src/linux$ zgrep journal_info ../2004.02.06.all.diff.gz
- current->journal_info = handle;
- current->journal_info = NULL;

-       current->journal_info = NULL;
-       .journal_info   = NULL,
-       return current->journal_info;
-       void *journal_info;
+       result = init_journal_info(s);

+ done_journal_info(s);
+ done_journal_info(s);

#4 lore raw
In reply to Chris Mason

Hi,

Quoted 5 lines

Also, the code has some extra performance tweaks to smooth out performance both with and without data=ordered. There are new mechanisms to trigger metadata/commit block writeback and to help throttle writers. The goal is to reduce the huge bursts of io during a commit and during data=ordered writeback.

It seems you introduced a bug here. I installed the patches yesterday and found a lockup on my notebook when running lilo (with /boot on the root reiserfs filesystem).

A SysRq-T showed that lilo is stuck in fsync:

lilo:
schedule
sleep_on
default_wake_function
sys_sched_yield
let_transaction_grow
__commit_trans_jl
reiserfs_sync_file
sys_fsync

reiserfs/0:
worker_thread
flush_async_commits
default_wake_function
...

The problem is 100% reproducable here. Perhaps it has something to do with tail unpacking? This is strange:

After copying the bzImage file to /boot and running lilo he can add the copied bzImage file, add the old file and then hangs on the third one. But after rebooting and running lilo he already hangs on the first one. After copying the file again he again passes the first two and hangs on the third kernel to add.

The whole filesystem locks up when this happens, not just the lilo process.

author
#5 lore raw
Quoted 8 lines · Christophe Saout

Hi,

Quoted 5 lines

Also, the code has some extra performance tweaks to smooth out performance both with and without data=ordered. There are new mechanisms to trigger metadata/commit block writeback and to help throttle writers. The goal is to reduce the huge bursts of io during a commit and during data=ordered writeback.

Quoted 6 lines

It seems you introduced a bug here. I installed the patches yesterday and found a lockup on my notebook when running lilo (with /boot on the root reiserfs filesystem).

A SysRq-T showed that lilo is stuck in fsync:

Ugh, I use grub so I haven't tried lilo. Could you please send me the full sysrq-t, this is probably something stupid.

-chris

A message that is not in the archive (sent off-list or not yet received): [email protected]

author
#7 lore raw
Quoted 8 lines · Christophe Saout

Am Mo, den 01.03.2004 schrieb Chris Mason um 15:01:

Quoted 6 lines
Quoted 5 lines

It seems you introduced a bug here. I installed the patches yesterday and found a lockup on my notebook when running lilo (with /boot on the root reiserfs filesystem).

A SysRq-T showed that lilo is stuck in fsync:

Quoted 8 lines
Quoted 2 lines

Ugh, I use grub so I haven't tried lilo. Could you please send me the full sysrq-t, this is probably something stupid.

Yes. I could reproduce it by simply creating a dummy /boot volume on reiserfs. I copied the content of /boot, ran lilo and it hung again. The other reiserfs filesystems were still usable (but a global sync hangs afterwards). I also attached a bzipped strace of the lilo process.

Ok, thanks. The problem is in reiserfs_unpack(), which needs updating for the patch. Fixing.

-chris

#8 lore raw

Am Montag, 1. März 2004 15:38 schrieb Chris Mason:

Quoted 11 lines · Christophe Saout
Quoted 10 lines

Am Mo, den 01.03.2004 schrieb Chris Mason um 15:01:

Quoted 8 lines
Quoted 5 lines

It seems you introduced a bug here. I installed the patches yesterday and found a lockup on my notebook when running lilo (with /boot on the root reiserfs filesystem).

A SysRq-T showed that lilo is stuck in fsync:

Ugh, I use grub so I haven't tried lilo. Could you please send me the full sysrq-t, this is probably something stupid.

Quoted 7 lines
Quoted 4 lines

Yes. I could reproduce it by simply creating a dummy /boot volume on reiserfs. I copied the content of /boot, ran lilo and it hung again. The other reiserfs filesystems were still usable (but a global sync hangs afterwards). I also attached a bzipped strace of the lilo process.

Ok, thanks. The problem is in reiserfs_unpack(), which needs updating for the patch. Fixing.

I'll test this under SuSE 2.6.3-7 (with lilo).
Or is it in?

Thanks,
Dieter

#9 lore raw
In reply to Chris Mason

Am Montag, 1. März 2004 15:38 schrieb Chris Mason:

Quoted 11 lines · Christophe Saout
Quoted 10 lines

Am Mo, den 01.03.2004 schrieb Chris Mason um 15:01:

Quoted 8 lines
Quoted 5 lines

It seems you introduced a bug here. I installed the patches yesterday and found a lockup on my notebook when running lilo (with /boot on the root reiserfs filesystem).

A SysRq-T showed that lilo is stuck in fsync:

Ugh, I use grub so I haven't tried lilo. Could you please send me the full sysrq-t, this is probably something stupid.

Quoted 7 lines
Quoted 4 lines

Yes. I could reproduce it by simply creating a dummy /boot volume on reiserfs. I copied the content of /boot, ran lilo and it hung again. The other reiserfs filesystems were still usable (but a global sync hangs afterwards). I also attached a bzipped strace of the lilo process.

Ok, thanks. The problem is in reiserfs_unpack(), which needs updating for the patch. Fixing.

Progress?

I'll try on SuSE 2.6.3-16.

Thanks,
Dieter

author
#10 lore raw
Quoted 17 lines · Dieter Nützel

Am Montag, 1. März 2004 15:38 schrieb Chris Mason:

Quoted 16 lines · Christophe Saout
Quoted 14 lines

Am Mo, den 01.03.2004 schrieb Chris Mason um 15:01:

Quoted 8 lines
Quoted 5 lines

It seems you introduced a bug here. I installed the patches yesterday and found a lockup on my notebook when running lilo (with /boot on the root reiserfs filesystem).

A SysRq-T showed that lilo is stuck in fsync:

Ugh, I use grub so I haven't tried lilo. Could you please send me the full sysrq-t, this is probably something stupid.

Yes. I could reproduce it by simply creating a dummy /boot volume on reiserfs. I copied the content of /boot, ran lilo and it hung again. The other reiserfs filesystems were still usable (but a global sync hangs afterwards). I also attached a bzipped strace of the lilo process.

Quoted 6 lines
Quoted 2 lines

Ok, thanks. The problem is in reiserfs_unpack(), which needs updating for the patch. Fixing.

Progress?

I'll try on SuSE 2.6.3-16.

Not fixed there yet. Slight delay yesterday since I'm trying to roll out the fix and some quota patches. Should be ready this afternoon.

-chris

#11 lore raw

Am Dienstag, 2. März 2004 19:16 schrieb Chris Mason:

Quoted 22 lines · Dieter Nützel
Quoted 21 lines

Am Montag, 1. März 2004 15:38 schrieb Chris Mason:

Quoted 19 lines · Christophe Saout
Quoted 7 lines

Am Mo, den 01.03.2004 schrieb Chris Mason um 15:01:

Quoted 6 lines
Quoted 5 lines

It seems you introduced a bug here. I installed the patches yesterday and found a lockup on my notebook when running lilo (with /boot on the root reiserfs filesystem).

A SysRq-T showed that lilo is stuck in fsync:

Quoted 8 lines
Quoted 2 lines

Ugh, I use grub so I haven't tried lilo. Could you please send me the full sysrq-t, this is probably something stupid.

Yes. I could reproduce it by simply creating a dummy /boot volume on reiserfs. I copied the content of /boot, ran lilo and it hung again. The other reiserfs filesystems were still usable (but a global sync hangs afterwards). I also attached a bzipped strace of the lilo process.

Ok, thanks. The problem is in reiserfs_unpack(), which needs updating for the patch. Fixing.

Quoted 6 lines
Quoted 3 lines

Progress?

I'll try on SuSE 2.6.3-16.

Not fixed there yet. Slight delay yesterday since I'm trying to roll out the fix and some quota patches. Should be ready this afternoon.

OK, I start with the older...

-Dieter

#12 lore raw

Am Dienstag, 2. März 2004 19:22 schrieb Dieter Nützel:

Quoted 23 lines

Am Dienstag, 2. März 2004 19:16 schrieb Chris Mason:

Quoted 22 lines · Dieter Nützel
Quoted 21 lines

Am Montag, 1. März 2004 15:38 schrieb Chris Mason:

Quoted 19 lines · Christophe Saout
Quoted 7 lines

Am Mo, den 01.03.2004 schrieb Chris Mason um 15:01:

Quoted 6 lines
Quoted 5 lines

It seems you introduced a bug here. I installed the patches yesterday and found a lockup on my notebook when running lilo (with /boot on the root reiserfs filesystem).

A SysRq-T showed that lilo is stuck in fsync:

Quoted 8 lines
Quoted 2 lines

Ugh, I use grub so I haven't tried lilo. Could you please send me the full sysrq-t, this is probably something stupid.

Yes. I could reproduce it by simply creating a dummy /boot volume on reiserfs. I copied the content of /boot, ran lilo and it hung again. The other reiserfs filesystems were still usable (but a global sync hangs afterwards). I also attached a bzipped strace of the lilo process.

Ok, thanks. The problem is in reiserfs_unpack(), which needs updating for the patch. Fixing.

Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

Progress?

I'll try on SuSE 2.6.3-16.

Not fixed there yet. Slight delay yesterday since I'm trying to roll out the fix and some quota patches. Should be ready this afternoon.

OK, I start with the older...

Ups, data-logging IS already in SuSE's 2.6.3-16 kernel ;-)

What's needed? --- The reiserfs_unpack() fix and your speed up patch?

[-]

Quoted 5 lines

Also, the code has some extra performance tweaks to smooth out performance both with and without data=ordered. There are new mechanisms to trigger metadata/commit block writeback and to help throttle writers. The goal is to reduce the huge bursts of io during a commit and during data=ordered writeback.

[-]

00x-xxx?

Thanks,
Dieter

author
#13 lore raw
Quoted 2 lines · Dieter Nützel

Ups, data-logging IS already in SuSE's 2.6.3-16 kernel ;-)

Yes ;-)

Quoted 2 lines

What's needed? --- The reiserfs_unpack() fix and your speed up patch?

suse only needs the reiserfs_unpack() fix.

-chris

#14 lore raw

Am Dienstag, 2. März 2004 21:06 schrieb Chris Mason:

Quoted 4 lines · Dieter Nützel
Quoted 1 line

Ups, data-logging IS already in SuSE's 2.6.3-16 kernel ;-)

Yes ;-)

Brave men.

Some numbers:
time dd if=/dev/zero of=sparse bs=1 seek=200G count=1 1+0 Records ein
1+0 Records aus
0.001u 2.592s 0:04.39 58.9% 0+0k 0+0io 0pf+0w
time rm sparse
0.000u 4.516s 0:04.52 99.7% 0+0k 0+0io 0pf+0w

Mouse movement and X server (screen update) works but is very slow (choppy) for the "know" four seconds.
But much better than before.

Quoted 3 lines
Quoted 1 line

What's needed? --- The reiserfs_unpack() fix and your speed up patch?

suse only needs the reiserfs_unpack() fix.

But it works like a charm without it. ;-)

Now, let's see to get it in...

Thank you!

-Dieter

#15 lore raw

Am Dienstag, 2. März 2004 22:06 schrieb Dieter Nützel:

Quoted 20 lines

Am Dienstag, 2. März 2004 21:06 schrieb Chris Mason:

Quoted 4 lines · Dieter Nützel
Quoted 1 line

Ups, data-logging IS already in SuSE's 2.6.3-16 kernel ;-)

Yes ;-)

Brave men.

Some numbers:
time dd if=/dev/zero of=sparse bs=1 seek=200G count=1 1+0 Records ein
1+0 Records aus
0.001u 2.592s 0:04.39 58.9% 0+0k 0+0io 0pf+0w
time rm sparse
0.000u 4.516s 0:04.52 99.7% 0+0k 0+0io 0pf+0w

Mouse movement and X server (screen update) works but is very slow (choppy) for the "know" four seconds.
But much better than before.

Quoted 5 lines
Quoted 3 lines
Quoted 1 line

What's needed? --- The reiserfs_unpack() fix and your speed up patch?

suse only needs the reiserfs_unpack() fix.

But it works like a charm without it. ;-)

Argh, ... partially... 8-(

lilo... do NOT work with the "new" (2.6.3-16) kernel as we know... But I could boot with my old one and could create the new initrd. Let me know if you have something to test.

-Dieter

#16 lore raw
In reply to Dieter Nützel
Quoted 16 lines · Dieter Nützel

Am Dienstag, 2. März 2004 21:06 schrieb Chris Mason:

Quoted 4 lines · Dieter Nützel
Quoted 1 line

Ups, data-logging IS already in SuSE's 2.6.3-16 kernel ;-)

Yes ;-)

Brave men.

Some numbers:
time dd if=/dev/zero of=sparse bs=1 seek=200G count=1 1+0 Records ein
1+0 Records aus
0.001u 2.592s 0:04.39 58.9% 0+0k 0+0io 0pf+0w
time rm sparse
0.000u 4.516s 0:04.52 99.7% 0+0k 0+0io 0pf+0w

Quoted 4 lines

Mouse movement and X server (screen update) works but is very slow (choppy) for the "know" four seconds.
But much better than before.

Oh, part of the delete code was missed in the latency fixes. I'll add a few more schedules.

Sorry for the delay, I'm chasing a strange bug, not sure if it is mine yet.

-chris

#17 lore raw
In reply to Dieter Nützel

On Tuesday 02 March 2004 20:53, Dieter Nützel wrote:

Hi Dieter,

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

I'll try on SuSE 2.6.3-16.

sorry for my ignorance, but where do you find 2.6.3-_16_? I only find -0

ciao, Marc

#18 lore raw

Am Mittwoch, 3. März 2004 10:13 schrieb Marc-Christian Petersen:

Quoted 7 lines · Dieter Nützel

Hi Dieter,

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

I'll try on SuSE 2.6.3-16.

sorry for my ignorance, but where do you find 2.6.3-_16_? I only find -0

ftp://sunsite.informatik.rwth-aachen.de/pub/Linux/suse/people/kraxel

Or the "original site" and its mirrors.

Regards,
Dieter