Allow ext4 to run without a journal.
PATCH 1/1 Allow ext4 to run without a journal.
A few weeks ago I posted a patch for discussion that allowed ext4 to run without a journal. Since that time I've integrated the excellent comments from Andreas and fixed several serious bugs. We're currently running with this patch and generating some performance numbers against both ext2 (with backported reservations code) and ext4 with and without a journal. It just so happens that running without a journal is slightly faster for most everything.
We did
iozone -T -t 4 s 2g -r 256k -T -I -i0 -i1 -i2
which creates 4 threads, each of which create and do reads and writes on a 2G file, with a buffer size of 256K, using O_DIRECT for all file opens to bypass the page cache. Results:
ext2 ext4, default ext4, no journal
initial writes 13.0 MB/s 15.4 MB/s 15.7 MB/s rewrites 13.1 MB/s 15.6 MB/s 15.9 MB/s reads 15.2 MB/s 16.9 MB/s 17.2 MB/s re-reads 15.3 MB/s 16.9 MB/s 17.2 MB/s random readers 5.6 MB/s 5.6 MB/s 5.7 MB/s random writers 5.1 MB/s 5.3 MB/s 5.4 MB/s
So it seems that, so far, this was a useful exercise.
The patch follows. It was
---
fs/ext4/balloc.c | 4 +- fs/ext4/ext4_jbd2.c | 60 +++++++++++++------ fs/ext4/ext4_jbd2.h | 95 ++++++++++++++++++++++++++---- fs/ext4/ext4_sb.h | 1 + fs/ext4/extents.c | 12 ++-- fs/ext4/ialloc.c | 25 ++++---- fs/ext4/inode.c | 130 +++++++++++++++++++++++++++------------- fs/ext4/ioctl.c | 2 +- fs/ext4/mballoc.c | 17 +++--- fs/ext4/migrate.c | 5 +- fs/ext4/namei.c | 50 ++++++++-------- fs/ext4/resize.c | 31 +++++----- fs/ext4/super.c | 163 +++++++++++++++++++++++++++++++++++++-------------- fs/ext4/xattr.c | 21 ++++--- 14 files changed, 419 insertions(+), 197 deletions(-)
fs/ext4/balloc.c +2 -2
@@ -531,11 +531,11 @@ do_more:/* We dirtied the bitmap block */ BUFFER_TRACE (bitmap_bh,"dirtied bitmap block" ); - err =ext4_journal_dirty_metadata (handle, bitmap_bh); + err =ext4_handle_dirty_metadata (handle,NULL , bitmap_bh);/* And the group descriptor block */ BUFFER_TRACE (gd_bh,"dirtied group descriptor block" ); - ret =ext4_journal_dirty_metadata (handle, gd_bh); + ret =ext4_handle_dirty_metadata (handle,NULL , gd_bh);if (!err) err = ret; *pdquot_freed_blocks += group_freed;
fs/ext4/ext4_jbd2.c +42 -18
95 lines not on this page · raw message
fs/ext4/ext4_jbd2.h +82 -13
165 lines not on this page · raw message
fs/ext4/ext4_sb.h +1 -0
@@ -28,6 +28,7 @@* fourth extended-fs super-block data in memory */ struct ext4_sb_info { +int *s_nojournal_flag;/* Null to indicate "not a handle" */ unsigned long s_desc_size;/* Size of a group descriptor in bytes */ unsigned long s_inodes_per_block;/* Number of inodes per block */ unsigned long s_blocks_per_group;/* Number of blocks in a group */
fs/ext4/extents.c +7 -5
@@ -97,6 +97,8 @@ static int ext4_ext_journal_restart(handle_t *handle, int needed) {int err; +if (!ext4_handle_valid (handle)) +return 0 ;if (handle->h_buffer_credits > needed)return 0 ; err =ext4_journal_extend (handle, needed);
@@ -134,7 +136,7 @@ static int ext4_ext_dirty(handle_t *handle, struct inode *inode,int err;if (path->p_bh) {/* path points to block */ - err =ext4_journal_dirty_metadata (handle, path->p_bh); + err =ext4_handle_dirty_metadata (handle, inode, path->p_bh); }else {/* path points to leaf/index in inode body */ err =ext4_mark_inode_dirty (handle, inode);
@@ -780,7 +782,7 @@ static int ext4_ext_split(handle_t *handle, struct inode *inode,set_buffer_uptodate (bh);unlock_buffer (bh); - err =ext4_journal_dirty_metadata (handle, bh); + err =ext4_handle_dirty_metadata (handle, inode, bh);if (err) goto cleanup;brelse (bh);
@@ -859,7 +861,7 @@ static int ext4_ext_split(handle_t *handle, struct inode *inode,set_buffer_uptodate (bh);unlock_buffer (bh); - err =ext4_journal_dirty_metadata (handle, bh); + err =ext4_handle_dirty_metadata (handle, inode, bh);if (err) goto cleanup;brelse (bh);
@@ -955,7 +957,7 @@ static int ext4_ext_grow_indepth(handle_t *handle, struct inode *inode,set_buffer_uptodate (bh);unlock_buffer (bh); - err =ext4_journal_dirty_metadata (handle, bh); + err =ext4_handle_dirty_metadata (handle, inode, bh);if (err) goto out;
@@ -2950,7 +2952,7 @@ void ext4_ext_truncate(struct inode *inode)* transaction synchronous. */ if (IS_SYNC (inode)) - handle->h_sync =1 ; +ext4_handle_sync (handle); out_stop:up_write (&EXT4_I (inode)->i_data_sem);
fs/ext4/ialloc.c +13 -12
@@ -253,12 +253,12 @@ void ext4_free_inode(handle_t *handle, struct inode *inode)spin_unlock (sb_bgl_lock (sbi, flex_group)); } } -BUFFER_TRACE (bh2,"call ext4_journal_dirty_metadata" ); - err =ext4_journal_dirty_metadata (handle, bh2); +BUFFER_TRACE (bh2,"call ext4_handle_dirty_metadata" ); + err =ext4_handle_dirty_metadata (handle,NULL , bh2);if (!fatal) fatal = err; } -BUFFER_TRACE (bitmap_bh,"call ext4_journal_dirty_metadata" ); - err =ext4_journal_dirty_metadata (handle, bitmap_bh); +BUFFER_TRACE (bitmap_bh,"call ext4_handle_dirty_metadata" ); + err =ext4_handle_dirty_metadata (handle,NULL , bitmap_bh);if (!fatal) fatal = err; sb->s_dirt =1 ;
@@ -656,15 +656,16 @@ repeat_in_this_group: ino, bitmap_bh->b_data)) {/* we won it */ BUFFER_TRACE (bitmap_bh, -"call ext4_journal_dirty_metadata" ); - err =ext4_journal_dirty_metadata (handle, +"call ext4_handle_dirty_metadata" ); + err =ext4_handle_dirty_metadata (handle, + inode, bitmap_bh);if (err) goto fail; goto got; }/* we lost it */ -jbd2_journal_release_buffer (handle, bitmap_bh); +ext4_handle_release_buffer (handle, bitmap_bh);if (++ino <EXT4_INODES_PER_GROUP (sb)) goto repeat_in_this_group;
@@ -726,7 +727,8 @@ got:/* Don't need to dirty bitmap block if we didn't change it */ if (free) {BUFFER_TRACE (block_bh,"dirty block bitmap" ); - err =ext4_journal_dirty_metadata (handle, block_bh); + err =ext4_handle_dirty_metadata (handle, +NULL , block_bh); }brelse (block_bh);
@@ -771,8 +773,8 @@ got: } gdp->bg_checksum =ext4_group_desc_csum (sbi, group, gdp);spin_unlock (sb_bgl_lock (sbi, group)); -BUFFER_TRACE (bh2,"call ext4_journal_dirty_metadata" ); - err =ext4_journal_dirty_metadata (handle, bh2); +BUFFER_TRACE (bh2,"call ext4_handle_dirty_metadata" ); + err =ext4_handle_dirty_metadata (handle,NULL , bh2);if (err) goto fail;percpu_counter_dec (&sbi->s_freeinodes_counter);
@@ -825,7 +827,7 @@ got:ext4_set_inode_flags (inode);if (IS_DIRSYNC (inode)) - handle->h_sync =1 ; +ext4_handle_sync (handle);insert_inode_hash (inode);spin_lock (&sbi->s_next_gen_lock); inode->i_generation = sbi->s_next_generation++;
@@ -1024,4 +1026,3 @@ unsigned long ext4_count_dirs(struct super_block * sb) }return count; } -
fs/ext4/inode.c +87 -43
310 lines not on this page · raw message
fs/ext4/ioctl.c +1 -1
@@ -99,7 +99,7 @@ long ext4_ioctl(struct file *filp, unsigned int cmd, unsigned long arg) goto flags_out; }if (IS_SYNC (inode)) - handle->h_sync =1 ; +ext4_handle_sync (handle); err =ext4_reserve_inode_write (handle, inode, &iloc);if (err) goto flags_err;
fs/ext4/mballoc.c +9 -8
@@ -2551,7 +2551,8 @@ int ext4_mb_init(struct super_block *sb, int needs_recovery)ext4_mb_init_per_dev_proc (sb);ext4_mb_history_init (sb); - sbi->s_journal->j_commit_callback = release_blocks_on_commit; +if (sbi->s_journal) + sbi->s_journal->j_commit_callback = release_blocks_on_commit;printk (KERN_INFO "EXT4-fs: mballoc enabled\n" );return 0 ;
@@ -2852,7 +2853,7 @@ ext4_mb_mark_diskspace_used(struct ext4_allocation_context *ac,mb_set_bits (sb_bgl_lock (sbi, ac->ac_b_ex.fe_group), bitmap_bh->b_data, ac->ac_b_ex.fe_start, ac->ac_b_ex.fe_len); - err =ext4_journal_dirty_metadata (handle, bitmap_bh); + err =ext4_handle_dirty_metadata (handle,NULL , bitmap_bh);if (!err) err = -EAGAIN ; goto out_err;
@@ -2899,10 +2900,10 @@ ext4_mb_mark_diskspace_used(struct ext4_allocation_context *ac,spin_unlock (sb_bgl_lock (sbi, flex_group)); } - err =ext4_journal_dirty_metadata (handle, bitmap_bh); + err =ext4_handle_dirty_metadata (handle,NULL , bitmap_bh);if (err) goto out_err; - err =ext4_journal_dirty_metadata (handle, gdp_bh); + err =ext4_handle_dirty_metadata (handle,NULL , gdp_bh); out_err: sb->s_dirt =1 ;
@@ -4412,7 +4413,7 @@ ext4_mb_free_metadata(handle_t *handle, struct ext4_buddy *e4b,struct rb_node **n = &db->bb_free_root.rb_node, *node;struct rb_node *parent =NULL , *new_node; - +BUG_ON (!ext4_handle_valid (handle));BUG_ON (e4b->bd_bitmap_page ==NULL );BUG_ON (e4b->bd_buddy_page ==NULL );
@@ -4598,7 +4599,7 @@ do_more:/* We dirtied the bitmap block */ BUFFER_TRACE (bitmap_bh,"dirtied bitmap block" ); - err =ext4_journal_dirty_metadata (handle, bitmap_bh); + err =ext4_handle_dirty_metadata (handle,NULL , bitmap_bh);if (ac) { ac->ac_b_ex.fe_group = block_group;
@@ -4607,7 +4608,7 @@ do_more:ext4_mb_store_history (ac); } -if (metadata) { +if (metadata &&ext4_handle_valid (handle)) {/* blocks being freed are metadata. these blocks shouldn't * be used until this transaction is committed */ ext4_mb_free_metadata (handle, &e4b, block_group, bit, count);
@@ -4637,7 +4638,7 @@ do_more:/* And the group descriptor block */ BUFFER_TRACE (gd_bh,"dirtied group descriptor block" ); - ret =ext4_journal_dirty_metadata (handle, gd_bh); + ret =ext4_handle_dirty_metadata (handle,NULL , gd_bh);if (!err) err = ret;
fs/ext4/migrate.c +3 -2
@@ -59,7 +59,8 @@ static int finish_range(handle_t *handle, struct inode *inode,/* * Make sure the credit we accumalated is not really high */ -if (needed && handle->h_buffer_credits >=EXT4_RESERVE_TRANS_BLOCKS ) { +if (needed &&ext4_handle_has_enough_credits (handle, +EXT4_RESERVE_TRANS_BLOCKS )) { retval =ext4_journal_restart (handle, needed);if (retval) goto err_out;
@@ -229,7 +230,7 @@ static int extend_credit_for_blkdel(handle_t *handle, struct inode *inode) {int retval =0 , needed; -if (handle->h_buffer_credits >EXT4_RESERVE_TRANS_BLOCKS ) +if (ext4_handle_has_enough_credits (handle,EXT4_RESERVE_TRANS_BLOCKS +1 ))return 0 ;/* * We are freeing a blocks. During this we touch
fs/ext4/namei.c +25 -25
166 lines not on this page · raw message
fs/ext4/resize.c +16 -15
@@ -149,7 +149,7 @@ static int extend_or_restart_transaction(handle_t *handle, int thresh, {int err; -if (handle->h_buffer_credits >= thresh) +if (ext4_handle_has_enough_credits (handle, thresh))return 0 ; err =ext4_journal_extend (handle,EXT4_MAX_TRANS_DATA );
@@ -232,7 +232,7 @@ static int setup_new_group_blocks(struct super_block *sb,memcpy (gdb->b_data, sbi->s_group_desc[i]->b_data, gdb->b_size);set_buffer_uptodate (gdb);unlock_buffer (gdb); -ext4_journal_dirty_metadata (handle, gdb); +ext4_handle_dirty_metadata (handle,NULL , gdb);ext4_set_bit (bit, bh->b_data);brelse (gdb); }
@@ -251,7 +251,7 @@ static int setup_new_group_blocks(struct super_block *sb, err =PTR_ERR (bh); goto exit_bh; } -ext4_journal_dirty_metadata (handle, gdb); +ext4_handle_dirty_metadata (handle,NULL , gdb);ext4_set_bit (bit, bh->b_data);brelse (gdb); }
@@ -276,7 +276,7 @@ static int setup_new_group_blocks(struct super_block *sb, err =PTR_ERR (it); goto exit_bh; } -ext4_journal_dirty_metadata (handle, it); +ext4_handle_dirty_metadata (handle,NULL , it);brelse (it);ext4_set_bit (bit, bh->b_data); }
@@ -286,7 +286,7 @@ static int setup_new_group_blocks(struct super_block *sb,mark_bitmap_end (input->blocks_count,EXT4_BLOCKS_PER_GROUP (sb), bh->b_data); -ext4_journal_dirty_metadata (handle, bh); +ext4_handle_dirty_metadata (handle,NULL , bh);brelse (bh);/* Mark unused entries in inode bitmap used */
@@ -299,7 +299,7 @@ static int setup_new_group_blocks(struct super_block *sb,mark_bitmap_end (EXT4_INODES_PER_GROUP (sb),EXT4_BLOCKS_PER_GROUP (sb), bh->b_data); -ext4_journal_dirty_metadata (handle, bh); +ext4_handle_dirty_metadata (handle,NULL , bh); exit_bh:brelse (bh);
@@ -486,12 +486,12 @@ static int add_new_gdb(handle_t *handle, struct inode *inode,* reserved inode, and will become GDT blocks (primary and backup). */ data[gdb_num %EXT4_ADDR_PER_BLOCK (sb)] =0 ; -ext4_journal_dirty_metadata (handle, dind); +ext4_handle_dirty_metadata (handle,NULL , dind);brelse (dind); inode->i_blocks -= (gdbackups +1 ) * sb->s_blocksize >>9 ;ext4_mark_iloc_dirty (handle, inode, &iloc);memset ((*primary)->b_data,0 , sb->s_blocksize); -ext4_journal_dirty_metadata (handle, *primary); +ext4_handle_dirty_metadata (handle,NULL , *primary); o_group_desc =EXT4_SB (sb)->s_group_desc; memcpy(n_group_desc, o_group_desc,
@@ -502,7 +502,7 @@ static int add_new_gdb(handle_t *handle, struct inode *inode,kfree (o_group_desc);le16_add_cpu (&es->s_reserved_gdt_blocks,-1 ); -ext4_journal_dirty_metadata (handle,EXT4_SB (sb)->s_sbh); +ext4_handle_dirty_metadata (handle,NULL ,EXT4_SB (sb)->s_sbh);return 0 ;
@@ -618,7 +618,7 @@ static int reserve_backup_gdb(handle_t *handle, struct inode *inode,primary[i]->b_blocknr, gdbackups, blk + primary[i]->b_blocknr); */ data[gdbackups] =cpu_to_le32 (blk + primary[i]->b_blocknr); - err2 =ext4_journal_dirty_metadata (handle, primary[i]); + err2 =ext4_handle_dirty_metadata (handle,NULL , primary[i]);if (!err) err = err2; }
@@ -676,7 +676,8 @@ static void update_backups(struct super_block *sb,struct buffer_head *bh;/* Out of journal space, and can't get more - abort - so sad */ -if (handle->h_buffer_credits ==0 && +if (ext4_handle_valid (handle) && + handle->h_buffer_credits ==0 &&ext4_journal_extend (handle,EXT4_MAX_TRANS_DATA ) && (err =ext4_journal_restart (handle,EXT4_MAX_TRANS_DATA )))break ;
@@ -696,7 +697,7 @@ static void update_backups(struct super_block *sb,memset (bh->b_data + size,0 , rest);set_buffer_uptodate (bh);unlock_buffer (bh); -ext4_journal_dirty_metadata (handle, bh); +ext4_handle_dirty_metadata (handle,NULL , bh);brelse (bh); }if ((err2 =ext4_journal_stop (handle)) && !err)
@@ -915,7 +916,7 @@ int ext4_group_add(struct super_block *sb, struct ext4_new_group_data *input)/* Update the global fs size fields */ sbi->s_groups_count++; -ext4_journal_dirty_metadata (handle, primary); +ext4_handle_dirty_metadata (handle,NULL , primary);/* Update the reserved block counts only once the new group is * active. */
@@ -937,7 +938,7 @@ int ext4_group_add(struct super_block *sb, struct ext4_new_group_data *input)EXT4_INODES_PER_GROUP (sb); } -ext4_journal_dirty_metadata (handle, sbi->s_sbh); +ext4_handle_dirty_metadata (handle,NULL , sbi->s_sbh); sb->s_dirt =1 ; exit_journal:
@@ -1071,7 +1072,7 @@ int ext4_group_extend(struct super_block *sb, struct ext4_super_block *es, goto exit_put; }ext4_blocks_count_set (es, o_blocks_count + add); -ext4_journal_dirty_metadata (handle,EXT4_SB (sb)->s_sbh); +ext4_handle_dirty_metadata (handle,NULL ,EXT4_SB (sb)->s_sbh); sb->s_dirt =1 ;unlock_super (sb); ext4_debug("freeing blocks %llu through %llu\n", o_blocks_count,
fs/ext4/super.c +119 -44
350 lines not on this page · raw message
fs/ext4/xattr.c +12 -9
@@ -457,7 +457,7 @@ static void ext4_xattr_update_super_block(handle_t *handle,if (ext4_journal_get_write_access (handle,EXT4_SB (sb)->s_sbh) ==0 ) {EXT4_SET_COMPAT_FEATURE (sb,EXT4_FEATURE_COMPAT_EXT_ATTR ); sb->s_dirt =1 ; -ext4_journal_dirty_metadata (handle,EXT4_SB (sb)->s_sbh); +ext4_handle_dirty_metadata (handle,NULL ,EXT4_SB (sb)->s_sbh); } }
@@ -487,9 +487,9 @@ ext4_xattr_release_block(handle_t *handle, struct inode *inode,ext4_forget (handle,1 , inode, bh, bh->b_blocknr); }else {le32_add_cpu (&BHDR (bh)->h_refcount,-1 ); - error =ext4_journal_dirty_metadata (handle, bh); + error =ext4_handle_dirty_metadata (handle, inode, bh);if (IS_SYNC (inode)) - handle->h_sync =1 ; +ext4_handle_sync (handle);DQUOT_FREE_BLOCK (inode,1 );ea_bdebug (bh,"refcount now=%d; releasing" ,le32_to_cpu (BHDR (bh)->h_refcount));
@@ -724,8 +724,9 @@ ext4_xattr_block_set(handle_t *handle, struct inode *inode,if (error == -EIO ) goto bad_block;if (!error) - error =ext4_journal_dirty_metadata (handle, - bs->bh); + error =ext4_handle_dirty_metadata (handle, + inode, + bs->bh);if (error) goto cleanup; goto inserted;
@@ -794,8 +795,9 @@ inserted:ea_bdebug (new_bh,"reusing; refcount now=%d" ,le32_to_cpu (BHDR (new_bh)->h_refcount));unlock_buffer (new_bh); - error =ext4_journal_dirty_metadata (handle, - new_bh); + error =ext4_handle_dirty_metadata (handle, + inode, + new_bh);if (error) goto cleanup_dquot; }
@@ -833,7 +835,8 @@ getblk_failed:set_buffer_uptodate (new_bh);unlock_buffer (new_bh);ext4_xattr_cache_insert (new_bh); - error =ext4_journal_dirty_metadata (handle, new_bh); + error =ext4_handle_dirty_metadata (handle, + inode, new_bh);if (error) goto cleanup; }
@@ -1040,7 +1043,7 @@ ext4_xattr_set_handle(handle_t *handle, struct inode *inode, int name_index,*/ is.iloc.bh =NULL ;if (IS_SYNC (inode)) - handle->h_sync =1 ; +ext4_handle_sync (handle); } cleanup:
Quoted 7 lines · Frank Mayhar
A few weeks ago I posted a patch for discussion that allowed ext4 to run without a journal. Since that time I've integrated the excellent comments from Andreas and fixed several serious bugs. We're currently running with this patch and generating some performance numbers against both ext2 (with backported reservations code) and ext4 with and without a journal. It just so happens that running without a journal is slightly faster for most everything.
Has anyone had a chance to take a look at this? Andreas? Ted? Anyone?
Signature · 2 lines
Frank Mayhar <[email protected]> Google, Inc.
Theodore Tso
maintainerQuoted 10 lines · Frank Mayhar
Quoted 7 lines · Frank Mayhar
A few weeks ago I posted a patch for discussion that allowed ext4 to run without a journal. Since that time I've integrated the excellent comments from Andreas and fixed several serious bugs. We're currently running with this patch and generating some performance numbers against both ext2 (with backported reservations code) and ext4 with and without a journal. It just so happens that running without a journal is slightly faster for most everything.
Has anyone had a chance to take a look at this? Andreas? Ted? Anyone?
For some reason it didn't show up in my mailbox; maybe an errant SPAM checker grabbed it? (For some reason Herbert Xu's mail server recently blacklisted my mail server, so he can't get any e-mails from me, whether it's a reply to LKML or an invite to the kernel summit. More recently Baracuda Networks, which handles mit.edu's spam filtering, blacklisted the mail server for the vendor-sec mailing list. Go figure.)
Fortunately your patch did show up in patchwork:
http://patchwork.ozlabs.org/project/linux-ext4/
I've been travelling, so I haven't had a chance to look at it, although I did note that it had showed up over the weekend when I was going over the patch queue and reconciling it with Patchwork, and wondered why I hadn't seen earlier...
I will try to get it shortly.
Regards,
- Ted
Theodore Tso
maintainerNote, the patch allow-ext4-to-run-without-a-journal conflicted with a large number of other patches in the patch queue. I fixed it up as best I could, and reviewed all of the changed patches, and I think it's OK. Please be cautious when first testing this, though.
- Ted
Quoted 4 lines · Theodore Tso
Note, the patch allow-ext4-to-run-without-a-journal conflicted with a large number of other patches in the patch queue. I fixed it up as best I could, and reviewed all of the changed patches, and I think it's OK. Please be cautious when first testing this, though.
Yeah, sorry, for reasons that are probably obvious this patch was relative to the ext4 stable tree, not to the latest bits.
Signature · 2 lines
Frank Mayhar <[email protected]> Google, Inc.
Theodore Tso
maintainerfs/ext4/ext4_jbd2.h line 23
Quoted 10 lines · Frank Mayhar
fs/ext4/ext4_jbd2.h +2
-0@@ -19,6 +19,8 @@#include <linux/jbd2.h> #include "ext4.h" +#define EXT4_NOJOURNAL_MAGIC (void *)0x457874344e6a6e6c/* "Ext4Njnl" */ +
This will cause warnings on systems where pointers are 32-bits. I'd suggest using a 32-bit magic number. Also, the number you have picked is divisble by 4, which means at least in theory it could be a valid pointer. If we make EXT4_NOJOURNAL_MAGIC a 32-bit number, then a number which is divisible by 4 definitely could be a valid pointer on a 64-bit system. I'd suggest keeping it simple, and simply using 0x1. This is known not be a valid pointer, since it's not 32-bit word aligned, and there is precedent for using such a number --- for example, SIG_IGN is ((void *) 1) on many architectures, and page 0 is generally unmapped to catch null pointer dereferences. So I'd suggest keeping things simple instead of using a fancy char string.
Also, why not use EXT4_NOJOURNAL_MAGIC constant directly, instead of assigning it as a constant value to s_nojournal_flag in struct ext4_sb_info? That bloats the data structure for no good reason that I can tell.
fs/ext4/ext4_jbd2.h
Quoted 12 lines
fs/ext4/ext4_jbd2.h +12
-0+static inline int ext4_handle_dirty_metadata (handle_t *handle, +struct inode *inode,struct buffer_head *bh) +{ +int err; + +if (ext4_handle_valid (handle)) { + err =__ext4_journal_dirty_metadata (__func__, handle, bh); + }else { + err =__ext4_write_dirty_metadata (inode, bh); + } +return err; +}
I don't see the point in doing this as an inline function; either way, each time the inline function is executed, it will result a function call, either to __ext4_journal_dirty_metadata() or __ext4_write_dirty_metadata(), both of which are called exactly once by this inline function. So why not move the conditional and the contents of _ext4_write_dirty_metadata() and __ext4_journal_dirty_metadata() into __ext4_handle_dirty_metadata(), and then do this:
#define ext4_handle_dirty_metadata(handle, inode, bh) \ __ext4_handle_dirty_metadata(__func__, handle, inode, bh)
Ext4_handle_dirty_metadata() gets called a *lot*, so this will shrink the icache footprint of ext4 filesystem, and for CPU's that have decent branch prediction, centralizing the condition into a single location instead of an inline will also be a win, once again demonstrating that with modern CPU's, optimizations that minimize code size often also faster.
fs/ext4/ext4_jbd2.h int ext4_force_commit(struct super_block *sb); · line 278
Quoted 11 lines
/*
super.c*/fs/ext4/ext4_jbd2.h +3
-0@@ -208,6 +270,9 @@ int ext4_force_commit(struct super_block *sb);static inline int ext4_should_journal_data (struct inode *inode) { +BUG_ON (EXT4_JOURNAL (inode) ==NULL && + (test_opt (inode->i_sb,DATA_FLAGS ) ==EXT4_MOUNT_JOURNAL_DATA || +EXT4_I (inode)->i_flags &EXT4_JOURNAL_DATA_FL ));if (!S_ISREG (inode->i_mode))return 1 ;if (test_opt (inode->i_sb,DATA_FLAGS ) ==EXT4_MOUNT_JOURNAL_DATA )
This shouldn't be a BUG_ON; this gets triggered if there is an inode which is marked as needing data journalling. So it is not a *bug*, but at worst either a filesystem corruption, or more charitably a conflict between a mount option and a setting in an inode flag. In any case, these should not cause a BUG; at worst an ext4_error(), or ext4_warning(). However, today, if a filesystem with inodes requesting data journalling are mounted on ext2, the request is simply ignored. So I'd argue that a similar treatmeant should be used for ext4 filesystem running w/o a journal. At worst perhaps there should be a one-time ext4_warning(), but I'm not convinced even this is warranted. If the system administrator mounts a filesystem without a journal, it's pretty clear what was meant.
fs/ext4/ext4_jbd2.h static inline int ext4_should_journal_data(struct inode *inode) · line 288
Quoted 6 lines
fs/ext4/ext4_jbd2.h +2
-0@@ -219,6 +284,8 @@ static inline int ext4_should_journal_data(struct inode *inode)static inline int ext4_should_order_data (struct inode *inode) { +BUG_ON (EXT4_JOURNAL (inode) ==NULL && +EXT4_I (inode)->i_flags &EXT4_JOURNAL_DATA_FL );
See above discussion.
fs/ext4/ext4_jbd2.h static inline int ext4_should_journal_data(struct inode *inode)
Quoted 4 lines
fs/ext4/ext4_jbd2.h +2
-0static inline int ext4_should_writeback_data (struct inode *inode) { +BUG_ON (EXT4_JOURNAL (inode) ==NULL && +EXT4_I (inode)->i_flags &EXT4_JOURNAL_DATA_FL );
Again, see above.
fs/ext4/ext4_sb.h line 32
Quoted 8 lines
fs/ext4/ext4_sb.h +1
-0@@ -28,6 +28,7 @@* fourth extended-fs super-block data in memory */ struct ext4_sb_info { +int *s_nojournal_flag;/* Null to indicate "not a handle" */ unsigned long s_desc_size;/* Size of a group descriptor in bytes */
See above discussion; I don't understand why this is necessary, and why it is an int * here, especially given that it is assigned a constant value which is a (void *), and it is eventually cast into a (handle_t *).
fs/ext4/super.c +6 -1
@@ -2284,7 +2314,12 @@ static int ext4_fill_super(struct super_block *sb, void *data, int silent)printk (KERN_ERR "ext4: No journal on filesystem on %s\n" , sb->s_id); - goto failed_mount3; +clear_opt (sbi->s_mount_opt,DATA_FLAGS ); +set_opt (sbi->s_mount_opt,WRITEBACK_DATA ); + sbi->s_journal =NULL ; + es->s_state &=cpu_to_le16 (~EXT4_VALID_FS ); + needs_recovery =0 ; + goto no_journal; }
In the case where there is no journal, please remove the KERN_ERR printk; that's just excess noise, and some day, Google System Administrators will thank you for getting rid of it. :-)
@@ -2748,6 +2798,9 @@ static int ext4_create_journal(struct super_block *sb,
EXT4_SB(sb)->s_journal = journal;
+ if (journal_inum == 0)
+ return 0;
+
ext4_update_dynamic_rev(sb); EXT4_SET_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_RECOVER); EXT4_SET_COMPAT_FEATURE(sb, EXT4_FEATURE_COMPAT_HAS_JOURNAL);
If journal_inum == 0, ext4_create_journal will never be called (see line 2429 in ext4fill_super), so this check is not really necessary.
Actually, what we should probably do (as a separate patch before this one), is to remove ext4_create_journal() and the journal_inum mount option altogether. It's pointless code that was originally added to ext3 by Stephen Tweedie back when he wanted to convert ext2 filesystems to ext3 before e2fsprogs supported tune2fs -j. It's been obsolete for over eight years, in both ext3 and ext4.
In any case, these are all pretty tiny nits; on the whole I think this patch is quite clean, so I'll add it to the ext4 patch queue for testing. I would appreciate getting an updated patch which addresses these suggested cleanups, though. Also, although I haven't tested it, I suspect we need to add a check so that if there is no journal, and the EXT4_FEATURE_INCOMPAT_RECOVERY feature is set, the mount should be aborted with an error, instead of just clearning the recovery flag and moving on. In actual practice, such a combination should never happen, but if it does, failing the mount is probably a safer thing to do.
Regards,
- Ted
This is just a quick response to one issue; I need to look at the rest in more detail but they seem valid from a quick skim.
Quoted 4 lines · Theodore Tso
Also, why not use
EXT4_NOJOURNAL_MAGICconstant directly, instead of assigning it as a constant value tos_nojournal_flagin structext4_sb_info? That bloats the data structure for no good reason that I can tell.
The way this works (which I did at Andreas' suggestion) is that when there is no journal the "current handle" is really a pointer to ext4_sb_info. I need to distinguish that structure from a valid handle, so I added the magic field to do so; the first field of a handle_t is a pointer (to a transaction) so I need something I can distinguish from that. (And your comments regarding the magic value are very helpful, thanks.)
If you look at ext4_handle_valid you'll see where it's actually used. I admit this is very non-obvious (so far it has confused at least two different reviewers); if you have a more clear way to handle it I'm all ears.
Quoted 11 lines
Review
Quoted 8 lines
static inline int
ext4_should_journal_data(struct inode *inode){(quoted diff) +3
-0+ BUG_ON(EXT4_JOURNAL(inode) == NULL && + (test_opt(inode->i_sb, DATA_FLAGS) == EXT4_MOUNT_JOURNAL_DATA || + EXT4_I(inode)->i_flags & EXT4_JOURNAL_DATA_FL)); if (!S_ISREG(inode->i_mode)) return 1; if (test_opt(inode->i_sb, DATA_FLAGS) == EXT4_MOUNT_JOURNAL_DATA)This shouldn't be a
BUG_ON; this gets triggered if there is an inode which is marked as needing data journalling.
Yeah, I have an open issue to remove all of these BUG_ONs after we've had a chance to test this fairly thoroughly. In the meantime I was leaving them in as a sanity check. I'll remove them from the next version of the patch I send to you.
Quoted 3 lines
In the case where there is no journal, please remove the
KERN_ERRprintk; that's just excess noise, and some day, Google System Administrators will thank you for getting rid of it. :-)
Heh. You're probably right.
Quoted 6 lines
Actually, what we should probably do (as a separate patch before this one), is to remove
ext4_create_journal()and thejournal_inummount option altogether. It's pointless code that was originally added to ext3 by Stephen Tweedie back when he wanted to convert ext2 filesystems to ext3 before e2fsprogs supported tune2fs -j. It's been obsolete for over eight years, in both ext3 and ext4.
If by "we" you mean you, I'll just disregard those bits entirely. If you mean me, I'll chat with my manager but I think it's doable since it's pretty much just ripping out a bit of code.
Quoted 3 lines
In any case, these are all pretty tiny nits; on the whole I think this patch is quite clean, so I'll add it to the ext4 patch queue for testing.
Thanks! And thanks for the comments, I will address them and generate a new patch. Please think about the magic-number/ext4_sb_info thing, though, and get back to me.
Quoted 8 lines
I would appreciate getting an updated patch which addresses
these suggested cleanups, though. Also, although I haven't tested it, I suspect we need to add a check so that if there is no journal, and theEXT4_FEATURE_INCOMPAT_RECOVERYfeature is set, the mount should be aborted with an error, instead of just clearning the recovery flag and moving on. In actual practice, such a combination should never happen, but if it does, failing the mount is probably a safer thing to do.
Ah, good point. If we want to recover the journal and there isn't one that's probably bad, yes.
Signature · 2 lines
Frank Mayhar <[email protected]> Google, Inc.
Theodore Tso
maintainerPATCH 1/1 Allow ext4 to run without a journal.
In reply to Theodore TsoI've modified your patch to reflect the comments I made two weeks ago. Were there any other changes you have planned? I'd like to get this pushed to Linus soon....
- Ted
ext4: Allow ext4 to run without a journal
From: Frank Mayhar <[email protected]>
A few weeks ago I posted a patch for discussion that allowed ext4 to run without a journal. Since that time I've integrated the excellent comments from Andreas and fixed several serious bugs. We're currently running with this patch and generating some performance numbers against both ext2 (with backported reservations code) and ext4 with and without a journal. It just so happens that running without a journal is slightly faster for most everything.
We did
iozone -T -t 4 s 2g -r 256k -T -I -i0 -i1 -i2
which creates 4 threads, each of which create and do reads and writes on a 2G file, with a buffer size of 256K, using O_DIRECT for all file opens to bypass the page cache. Results:
ext2 ext4, default ext4, no journal
initial writes 13.0 MB/s 15.4 MB/s 15.7 MB/s rewrites 13.1 MB/s 15.6 MB/s 15.9 MB/s reads 15.2 MB/s 16.9 MB/s 17.2 MB/s re-reads 15.3 MB/s 16.9 MB/s 17.2 MB/s random readers 5.6 MB/s 5.6 MB/s 5.7 MB/s random writers 5.1 MB/s 5.3 MB/s 5.4 MB/s
So it seems that, so far, this was a useful exercise.
---
fs/ext4/balloc.c +2 -2
@@ -531,11 +531,11 @@ do_more:/* We dirtied the bitmap block */ BUFFER_TRACE (bitmap_bh,"dirtied bitmap block" ); - err =ext4_journal_dirty_metadata (handle, bitmap_bh); + err =ext4_handle_dirty_metadata (handle,NULL , bitmap_bh);/* And the group descriptor block */ BUFFER_TRACE (gd_bh,"dirtied group descriptor block" ); - ret =ext4_journal_dirty_metadata (handle, gd_bh); + ret =ext4_handle_dirty_metadata (handle,NULL , gd_bh);if (!err) err = ret; *pdquot_freed_blocks += group_freed;
fs/ext4/ext4_jbd2.c +58 -20
111 lines not on this page · raw message
fs/ext4/ext4_jbd2.h +68 -15
147 lines not on this page · raw message
fs/ext4/extents.c +7 -5
@@ -97,6 +97,8 @@ static int ext4_ext_journal_restart(handle_t *handle, int needed) {int err; +if (!ext4_handle_valid (handle)) +return 0 ;if (handle->h_buffer_credits > needed)return 0 ; err =ext4_journal_extend (handle, needed);
@@ -134,7 +136,7 @@ static int ext4_ext_dirty(handle_t *handle, struct inode *inode,int err;if (path->p_bh) {/* path points to block */ - err =ext4_journal_dirty_metadata (handle, path->p_bh); + err =ext4_handle_dirty_metadata (handle, inode, path->p_bh); }else {/* path points to leaf/index in inode body */ err =ext4_mark_inode_dirty (handle, inode);
@@ -780,7 +782,7 @@ static int ext4_ext_split(handle_t *handle, struct inode *inode,set_buffer_uptodate (bh);unlock_buffer (bh); - err =ext4_journal_dirty_metadata (handle, bh); + err =ext4_handle_dirty_metadata (handle, inode, bh);if (err) goto cleanup;brelse (bh);
@@ -859,7 +861,7 @@ static int ext4_ext_split(handle_t *handle, struct inode *inode,set_buffer_uptodate (bh);unlock_buffer (bh); - err =ext4_journal_dirty_metadata (handle, bh); + err =ext4_handle_dirty_metadata (handle, inode, bh);if (err) goto cleanup;brelse (bh);
@@ -955,7 +957,7 @@ static int ext4_ext_grow_indepth(handle_t *handle, struct inode *inode,set_buffer_uptodate (bh);unlock_buffer (bh); - err =ext4_journal_dirty_metadata (handle, bh); + err =ext4_handle_dirty_metadata (handle, inode, bh);if (err) goto out;
@@ -2947,7 +2949,7 @@ void ext4_ext_truncate(struct inode *inode)* transaction synchronous. */ if (IS_SYNC (inode)) - handle->h_sync =1 ; +ext4_handle_sync (handle); out_stop:up_write (&EXT4_I (inode)->i_data_sem);
fs/ext4/ialloc.c +13 -12
@@ -253,12 +253,12 @@ void ext4_free_inode(handle_t *handle, struct inode *inode)spin_unlock (sb_bgl_lock (sbi, flex_group)); } } -BUFFER_TRACE (bh2,"call ext4_journal_dirty_metadata" ); - err =ext4_journal_dirty_metadata (handle, bh2); +BUFFER_TRACE (bh2,"call ext4_handle_dirty_metadata" ); + err =ext4_handle_dirty_metadata (handle,NULL , bh2);if (!fatal) fatal = err; } -BUFFER_TRACE (bitmap_bh,"call ext4_journal_dirty_metadata" ); - err =ext4_journal_dirty_metadata (handle, bitmap_bh); +BUFFER_TRACE (bitmap_bh,"call ext4_handle_dirty_metadata" ); + err =ext4_handle_dirty_metadata (handle,NULL , bitmap_bh);if (!fatal) fatal = err; sb->s_dirt =1 ;
@@ -656,15 +656,16 @@ repeat_in_this_group: ino, bitmap_bh->b_data)) {/* we won it */ BUFFER_TRACE (bitmap_bh, -"call ext4_journal_dirty_metadata" ); - err =ext4_journal_dirty_metadata (handle, +"call ext4_handle_dirty_metadata" ); + err =ext4_handle_dirty_metadata (handle, + inode, bitmap_bh);if (err) goto fail; goto got; }/* we lost it */ -jbd2_journal_release_buffer (handle, bitmap_bh); +ext4_handle_release_buffer (handle, bitmap_bh);if (++ino <EXT4_INODES_PER_GROUP (sb)) goto repeat_in_this_group;
@@ -726,7 +727,8 @@ got:/* Don't need to dirty bitmap block if we didn't change it */ if (free) {BUFFER_TRACE (block_bh,"dirty block bitmap" ); - err =ext4_journal_dirty_metadata (handle, block_bh); + err =ext4_handle_dirty_metadata (handle, +NULL , block_bh); }brelse (block_bh);
@@ -771,8 +773,8 @@ got: } gdp->bg_checksum =ext4_group_desc_csum (sbi, group, gdp);spin_unlock (sb_bgl_lock (sbi, group)); -BUFFER_TRACE (bh2,"call ext4_journal_dirty_metadata" ); - err =ext4_journal_dirty_metadata (handle, bh2); +BUFFER_TRACE (bh2,"call ext4_handle_dirty_metadata" ); + err =ext4_handle_dirty_metadata (handle,NULL , bh2);if (err) goto fail;percpu_counter_dec (&sbi->s_freeinodes_counter);
@@ -825,7 +827,7 @@ got:ext4_set_inode_flags (inode);if (IS_DIRSYNC (inode)) - handle->h_sync =1 ; +ext4_handle_sync (handle);insert_inode_hash (inode);spin_lock (&sbi->s_next_gen_lock); inode->i_generation = sbi->s_next_generation++;
@@ -1024,4 +1026,3 @@ unsigned long ext4_count_dirs(struct super_block * sb) }return count; } -
fs/ext4/inode.c +87 -43
310 lines not on this page · raw message
fs/ext4/ioctl.c +1 -1
@@ -99,7 +99,7 @@ long ext4_ioctl(struct file *filp, unsigned int cmd, unsigned long arg) goto flags_out; }if (IS_SYNC (inode)) - handle->h_sync =1 ; +ext4_handle_sync (handle); err =ext4_reserve_inode_write (handle, inode, &iloc);if (err) goto flags_err;
fs/ext4/mballoc.c +9 -8
@@ -2553,7 +2553,8 @@ int ext4_mb_init(struct super_block *sb, int needs_recovery)ext4_mb_init_per_dev_proc (sb);ext4_mb_history_init (sb); - sbi->s_journal->j_commit_callback = release_blocks_on_commit; +if (sbi->s_journal) + sbi->s_journal->j_commit_callback = release_blocks_on_commit;printk (KERN_INFO "EXT4-fs: mballoc enabled\n" );return 0 ;
@@ -2854,7 +2855,7 @@ ext4_mb_mark_diskspace_used(struct ext4_allocation_context *ac,mb_set_bits (sb_bgl_lock (sbi, ac->ac_b_ex.fe_group), bitmap_bh->b_data, ac->ac_b_ex.fe_start, ac->ac_b_ex.fe_len); - err =ext4_journal_dirty_metadata (handle, bitmap_bh); + err =ext4_handle_dirty_metadata (handle,NULL , bitmap_bh);if (!err) err = -EAGAIN ; goto out_err;
@@ -2901,10 +2902,10 @@ ext4_mb_mark_diskspace_used(struct ext4_allocation_context *ac,spin_unlock (sb_bgl_lock (sbi, flex_group)); } - err =ext4_journal_dirty_metadata (handle, bitmap_bh); + err =ext4_handle_dirty_metadata (handle,NULL , bitmap_bh);if (err) goto out_err; - err =ext4_journal_dirty_metadata (handle, gdp_bh); + err =ext4_handle_dirty_metadata (handle,NULL , gdp_bh); out_err: sb->s_dirt =1 ;
@@ -4414,7 +4415,7 @@ ext4_mb_free_metadata(handle_t *handle, struct ext4_buddy *e4b,struct rb_node **n = &db->bb_free_root.rb_node, *node;struct rb_node *parent =NULL , *new_node; - +BUG_ON (!ext4_handle_valid (handle));BUG_ON (e4b->bd_bitmap_page ==NULL );BUG_ON (e4b->bd_buddy_page ==NULL );
@@ -4600,7 +4601,7 @@ do_more:/* We dirtied the bitmap block */ BUFFER_TRACE (bitmap_bh,"dirtied bitmap block" ); - err =ext4_journal_dirty_metadata (handle, bitmap_bh); + err =ext4_handle_dirty_metadata (handle,NULL , bitmap_bh);if (ac) { ac->ac_b_ex.fe_group = block_group;
@@ -4609,7 +4610,7 @@ do_more:ext4_mb_store_history (ac); } -if (metadata) { +if (metadata &&ext4_handle_valid (handle)) {/* blocks being freed are metadata. these blocks shouldn't * be used until this transaction is committed */ ext4_mb_free_metadata (handle, &e4b, block_group, bit, count);
@@ -4639,7 +4640,7 @@ do_more:/* And the group descriptor block */ BUFFER_TRACE (gd_bh,"dirtied group descriptor block" ); - ret =ext4_journal_dirty_metadata (handle, gd_bh); + ret =ext4_handle_dirty_metadata (handle,NULL , gd_bh);if (!err) err = ret;
fs/ext4/migrate.c +3 -2
@@ -59,7 +59,8 @@ static int finish_range(handle_t *handle, struct inode *inode,/* * Make sure the credit we accumalated is not really high */ -if (needed && handle->h_buffer_credits >=EXT4_RESERVE_TRANS_BLOCKS ) { +if (needed &&ext4_handle_has_enough_credits (handle, +EXT4_RESERVE_TRANS_BLOCKS )) { retval =ext4_journal_restart (handle, needed);if (retval) goto err_out;
@@ -229,7 +230,7 @@ static int extend_credit_for_blkdel(handle_t *handle, struct inode *inode) {int retval =0 , needed; -if (handle->h_buffer_credits >EXT4_RESERVE_TRANS_BLOCKS ) +if (ext4_handle_has_enough_credits (handle,EXT4_RESERVE_TRANS_BLOCKS +1 ))return 0 ;/* * We are freeing a blocks. During this we touch
fs/ext4/namei.c +25 -25
166 lines not on this page · raw message
fs/ext4/resize.c +16 -15
@@ -149,7 +149,7 @@ static int extend_or_restart_transaction(handle_t *handle, int thresh, {int err; -if (handle->h_buffer_credits >= thresh) +if (ext4_handle_has_enough_credits (handle, thresh))return 0 ; err =ext4_journal_extend (handle,EXT4_MAX_TRANS_DATA );
@@ -232,7 +232,7 @@ static int setup_new_group_blocks(struct super_block *sb,memcpy (gdb->b_data, sbi->s_group_desc[i]->b_data, gdb->b_size);set_buffer_uptodate (gdb);unlock_buffer (gdb); -ext4_journal_dirty_metadata (handle, gdb); +ext4_handle_dirty_metadata (handle,NULL , gdb);ext4_set_bit (bit, bh->b_data);brelse (gdb); }
@@ -251,7 +251,7 @@ static int setup_new_group_blocks(struct super_block *sb, err =PTR_ERR (bh); goto exit_bh; } -ext4_journal_dirty_metadata (handle, gdb); +ext4_handle_dirty_metadata (handle,NULL , gdb);ext4_set_bit (bit, bh->b_data);brelse (gdb); }
@@ -276,7 +276,7 @@ static int setup_new_group_blocks(struct super_block *sb, err =PTR_ERR (it); goto exit_bh; } -ext4_journal_dirty_metadata (handle, it); +ext4_handle_dirty_metadata (handle,NULL , it);brelse (it);ext4_set_bit (bit, bh->b_data); }
@@ -286,7 +286,7 @@ static int setup_new_group_blocks(struct super_block *sb,mark_bitmap_end (input->blocks_count,EXT4_BLOCKS_PER_GROUP (sb), bh->b_data); -ext4_journal_dirty_metadata (handle, bh); +ext4_handle_dirty_metadata (handle,NULL , bh);brelse (bh);/* Mark unused entries in inode bitmap used */
@@ -299,7 +299,7 @@ static int setup_new_group_blocks(struct super_block *sb,mark_bitmap_end (EXT4_INODES_PER_GROUP (sb),EXT4_BLOCKS_PER_GROUP (sb), bh->b_data); -ext4_journal_dirty_metadata (handle, bh); +ext4_handle_dirty_metadata (handle,NULL , bh); exit_bh:brelse (bh);
@@ -486,12 +486,12 @@ static int add_new_gdb(handle_t *handle, struct inode *inode,* reserved inode, and will become GDT blocks (primary and backup). */ data[gdb_num %EXT4_ADDR_PER_BLOCK (sb)] =0 ; -ext4_journal_dirty_metadata (handle, dind); +ext4_handle_dirty_metadata (handle,NULL , dind);brelse (dind); inode->i_blocks -= (gdbackups +1 ) * sb->s_blocksize >>9 ;ext4_mark_iloc_dirty (handle, inode, &iloc);memset ((*primary)->b_data,0 , sb->s_blocksize); -ext4_journal_dirty_metadata (handle, *primary); +ext4_handle_dirty_metadata (handle,NULL , *primary); o_group_desc =EXT4_SB (sb)->s_group_desc; memcpy(n_group_desc, o_group_desc,
@@ -502,7 +502,7 @@ static int add_new_gdb(handle_t *handle, struct inode *inode,kfree (o_group_desc);le16_add_cpu (&es->s_reserved_gdt_blocks,-1 ); -ext4_journal_dirty_metadata (handle,EXT4_SB (sb)->s_sbh); +ext4_handle_dirty_metadata (handle,NULL ,EXT4_SB (sb)->s_sbh);return 0 ;
@@ -618,7 +618,7 @@ static int reserve_backup_gdb(handle_t *handle, struct inode *inode,primary[i]->b_blocknr, gdbackups, blk + primary[i]->b_blocknr); */ data[gdbackups] =cpu_to_le32 (blk + primary[i]->b_blocknr); - err2 =ext4_journal_dirty_metadata (handle, primary[i]); + err2 =ext4_handle_dirty_metadata (handle,NULL , primary[i]);if (!err) err = err2; }
@@ -676,7 +676,8 @@ static void update_backups(struct super_block *sb,struct buffer_head *bh;/* Out of journal space, and can't get more - abort - so sad */ -if (handle->h_buffer_credits ==0 && +if (ext4_handle_valid (handle) && + handle->h_buffer_credits ==0 &&ext4_journal_extend (handle,EXT4_MAX_TRANS_DATA ) && (err =ext4_journal_restart (handle,EXT4_MAX_TRANS_DATA )))break ;
@@ -696,7 +697,7 @@ static void update_backups(struct super_block *sb,memset (bh->b_data + size,0 , rest);set_buffer_uptodate (bh);unlock_buffer (bh); -ext4_journal_dirty_metadata (handle, bh); +ext4_handle_dirty_metadata (handle,NULL , bh);brelse (bh); }if ((err2 =ext4_journal_stop (handle)) && !err)
@@ -916,7 +917,7 @@ int ext4_group_add(struct super_block *sb, struct ext4_new_group_data *input)/* Update the global fs size fields */ sbi->s_groups_count++; -ext4_journal_dirty_metadata (handle, primary); +ext4_handle_dirty_metadata (handle,NULL , primary);/* Update the reserved block counts only once the new group is * active. */
@@ -938,7 +939,7 @@ int ext4_group_add(struct super_block *sb, struct ext4_new_group_data *input)EXT4_INODES_PER_GROUP (sb); } -ext4_journal_dirty_metadata (handle, sbi->s_sbh); +ext4_handle_dirty_metadata (handle,NULL , sbi->s_sbh); sb->s_dirt =1 ; exit_journal:
@@ -1072,7 +1073,7 @@ int ext4_group_extend(struct super_block *sb, struct ext4_super_block *es, goto exit_put; }ext4_blocks_count_set (es, o_blocks_count + add); -ext4_journal_dirty_metadata (handle,EXT4_SB (sb)->s_sbh); +ext4_handle_dirty_metadata (handle,NULL ,EXT4_SB (sb)->s_sbh); sb->s_dirt =1 ;unlock_super (sb); ext4_debug("freeing blocks %llu through %llu\n", o_blocks_count,
fs/ext4/super.c +117 -48
340 lines not on this page · raw message
fs/ext4/xattr.c +12 -9
@@ -457,7 +457,7 @@ static void ext4_xattr_update_super_block(handle_t *handle,if (ext4_journal_get_write_access (handle,EXT4_SB (sb)->s_sbh) ==0 ) {EXT4_SET_COMPAT_FEATURE (sb,EXT4_FEATURE_COMPAT_EXT_ATTR ); sb->s_dirt =1 ; -ext4_journal_dirty_metadata (handle,EXT4_SB (sb)->s_sbh); +ext4_handle_dirty_metadata (handle,NULL ,EXT4_SB (sb)->s_sbh); } }
@@ -487,9 +487,9 @@ ext4_xattr_release_block(handle_t *handle, struct inode *inode,ext4_forget (handle,1 , inode, bh, bh->b_blocknr); }else {le32_add_cpu (&BHDR (bh)->h_refcount,-1 ); - error =ext4_journal_dirty_metadata (handle, bh); + error =ext4_handle_dirty_metadata (handle, inode, bh);if (IS_SYNC (inode)) - handle->h_sync =1 ; +ext4_handle_sync (handle);DQUOT_FREE_BLOCK (inode,1 );ea_bdebug (bh,"refcount now=%d; releasing" ,le32_to_cpu (BHDR (bh)->h_refcount));
@@ -724,8 +724,9 @@ ext4_xattr_block_set(handle_t *handle, struct inode *inode,if (error == -EIO ) goto bad_block;if (!error) - error =ext4_journal_dirty_metadata (handle, - bs->bh); + error =ext4_handle_dirty_metadata (handle, + inode, + bs->bh);if (error) goto cleanup; goto inserted;
@@ -794,8 +795,9 @@ inserted:ea_bdebug (new_bh,"reusing; refcount now=%d" ,le32_to_cpu (BHDR (new_bh)->h_refcount));unlock_buffer (new_bh); - error =ext4_journal_dirty_metadata (handle, - new_bh); + error =ext4_handle_dirty_metadata (handle, + inode, + new_bh);if (error) goto cleanup_dquot; }
@@ -833,7 +835,8 @@ getblk_failed:set_buffer_uptodate (new_bh);unlock_buffer (new_bh);ext4_xattr_cache_insert (new_bh); - error =ext4_journal_dirty_metadata (handle, new_bh); + error =ext4_handle_dirty_metadata (handle, + inode, new_bh);if (error) goto cleanup; }
@@ -1040,7 +1043,7 @@ ext4_xattr_set_handle(handle_t *handle, struct inode *inode, int name_index,*/ is.iloc.bh =NULL ;if (IS_SYNC (inode)) - handle->h_sync =1 ; +ext4_handle_sync (handle); } cleanup:
Quoted 3 lines · Theodore Tso
I've modified your patch to reflect the comments I made two weeks ago. Were there any other changes you have planned? I'd like to get this pushed to Linus soon....
My apologies, I have a new patch that has been tested quite a bit more (although I still wouldn't say quite thoroughly yet, but close), all I need to do is set it up correctly for you guys and send it to the list. Unfortunately other stuff (including the holidays) has gotten in the way. Expect it on Monday or Tuesday. Note that this version will have a handful of small but important bugs fixed, as well.
One question: Shold I make the patch relative to the stable tree or to the latest bits?
Signature · 2 lines
Frank Mayhar <[email protected]> Google, Inc.
Theodore Tso
maintainerQuoted 12 lines · Frank Mayhar
Quoted 3 lines · Theodore Tso
I've modified your patch to reflect the comments I made two weeks ago. Were there any other changes you have planned? I'd like to get this pushed to Linus soon....
My apologies, I have a new patch that has been tested quite a bit more (although I still wouldn't say quite thoroughly yet, but close), all I need to do is set it up correctly for you guys and send it to the list. Unfortunately other stuff (including the holidays) has gotten in the way. Expect it on Monday or Tuesday. Note that this version will have a handful of small but important bugs fixed, as well.
Quoted 2 lines
One question: Shold I make the patch relative to the stable tree or to the latest bits?
At this point can you make your patch relative to whatever you used last time (and remind me what it is)? Or, can you give me a delta between what you did last time and what you have now that has the fixes? The problem is that the patch touches so much of ext4 that it's quite painful to maintain it in the ext4 patch queue --- it either invalidates a large number of patches after the fact, or the patch needs a huge amount of work to fix up patch conflicts if it is applied afterwards --- and if it is placed in the middle of the patch queue, both conditions apply.
So I will have to do a lot of fix ups either way. So unless you're willing to pull down the ext4 patch queue, at:
git://repo.or.cz/ext4-patch-queue.git
... and worry about making it apply in place, it's probably easier for you to give me delta's versus the last patch, and I'll integrate in the changes by hand.
- Ted
Theodore Tso
maintainerHere is an updated patch that adds a check to make sure the right thing happens if an ext3 filesystem that needs recovery is mounted -o noload. This also fixes a potential oops in ext4_mark_recovery_complete() journal doesn't exist, and we avoid adding and remove orphans from the orphan list when the journal doesn't exist.
- Ted
ext4: Allow ext4 to run without a journal
From: Frank Mayhar <[email protected]>
A few weeks ago I posted a patch for discussion that allowed ext4 to run without a journal. Since that time I've integrated the excellent comments from Andreas and fixed several serious bugs. We're currently running with this patch and generating some performance numbers against both ext2 (with backported reservations code) and ext4 with and without a journal. It just so happens that running without a journal is slightly faster for most everything.
We did
iozone -T -t 4 s 2g -r 256k -T -I -i0 -i1 -i2
which creates 4 threads, each of which create and do reads and writes on a 2G file, with a buffer size of 256K, using O_DIRECT for all file opens to bypass the page cache. Results:
ext2 ext4, default ext4, no journal
initial writes 13.0 MB/s 15.4 MB/s 15.7 MB/s rewrites 13.1 MB/s 15.6 MB/s 15.9 MB/s reads 15.2 MB/s 16.9 MB/s 17.2 MB/s re-reads 15.3 MB/s 16.9 MB/s 17.2 MB/s random readers 5.6 MB/s 5.6 MB/s 5.7 MB/s random writers 5.1 MB/s 5.3 MB/s 5.4 MB/s
So it seems that, so far, this was a useful exercise.
fs/ext4/balloc.c +2 -2
@@ -531,11 +531,11 @@ do_more:/* We dirtied the bitmap block */ BUFFER_TRACE (bitmap_bh,"dirtied bitmap block" ); - err =ext4_journal_dirty_metadata (handle, bitmap_bh); + err =ext4_handle_dirty_metadata (handle,NULL , bitmap_bh);/* And the group descriptor block */ BUFFER_TRACE (gd_bh,"dirtied group descriptor block" ); - ret =ext4_journal_dirty_metadata (handle, gd_bh); + ret =ext4_handle_dirty_metadata (handle,NULL , gd_bh);if (!err) err = ret; *pdquot_freed_blocks += group_freed;
fs/ext4/ext4_jbd2.c +63 -20
116 lines not on this page · raw message
fs/ext4/ext4_jbd2.h +68 -15
147 lines not on this page · raw message
fs/ext4/extents.c +7 -5
@@ -97,6 +97,8 @@ static int ext4_ext_journal_restart(handle_t *handle, int needed) {int err; +if (!ext4_handle_valid (handle)) +return 0 ;if (handle->h_buffer_credits > needed)return 0 ; err =ext4_journal_extend (handle, needed);
@@ -134,7 +136,7 @@ static int ext4_ext_dirty(handle_t *handle, struct inode *inode,int err;if (path->p_bh) {/* path points to block */ - err =ext4_journal_dirty_metadata (handle, path->p_bh); + err =ext4_handle_dirty_metadata (handle, inode, path->p_bh); }else {/* path points to leaf/index in inode body */ err =ext4_mark_inode_dirty (handle, inode);
@@ -780,7 +782,7 @@ static int ext4_ext_split(handle_t *handle, struct inode *inode,set_buffer_uptodate (bh);unlock_buffer (bh); - err =ext4_journal_dirty_metadata (handle, bh); + err =ext4_handle_dirty_metadata (handle, inode, bh);if (err) goto cleanup;brelse (bh);
@@ -859,7 +861,7 @@ static int ext4_ext_split(handle_t *handle, struct inode *inode,set_buffer_uptodate (bh);unlock_buffer (bh); - err =ext4_journal_dirty_metadata (handle, bh); + err =ext4_handle_dirty_metadata (handle, inode, bh);if (err) goto cleanup;brelse (bh);
@@ -955,7 +957,7 @@ static int ext4_ext_grow_indepth(handle_t *handle, struct inode *inode,set_buffer_uptodate (bh);unlock_buffer (bh); - err =ext4_journal_dirty_metadata (handle, bh); + err =ext4_handle_dirty_metadata (handle, inode, bh);if (err) goto out;
@@ -2947,7 +2949,7 @@ void ext4_ext_truncate(struct inode *inode)* transaction synchronous. */ if (IS_SYNC (inode)) - handle->h_sync =1 ; +ext4_handle_sync (handle); out_stop:up_write (&EXT4_I (inode)->i_data_sem);
fs/ext4/ialloc.c +13 -12
@@ -253,12 +253,12 @@ void ext4_free_inode(handle_t *handle, struct inode *inode)spin_unlock (sb_bgl_lock (sbi, flex_group)); } } -BUFFER_TRACE (bh2,"call ext4_journal_dirty_metadata" ); - err =ext4_journal_dirty_metadata (handle, bh2); +BUFFER_TRACE (bh2,"call ext4_handle_dirty_metadata" ); + err =ext4_handle_dirty_metadata (handle,NULL , bh2);if (!fatal) fatal = err; } -BUFFER_TRACE (bitmap_bh,"call ext4_journal_dirty_metadata" ); - err =ext4_journal_dirty_metadata (handle, bitmap_bh); +BUFFER_TRACE (bitmap_bh,"call ext4_handle_dirty_metadata" ); + err =ext4_handle_dirty_metadata (handle,NULL , bitmap_bh);if (!fatal) fatal = err; sb->s_dirt =1 ;
@@ -656,15 +656,16 @@ repeat_in_this_group: ino, bitmap_bh->b_data)) {/* we won it */ BUFFER_TRACE (bitmap_bh, -"call ext4_journal_dirty_metadata" ); - err =ext4_journal_dirty_metadata (handle, +"call ext4_handle_dirty_metadata" ); + err =ext4_handle_dirty_metadata (handle, + inode, bitmap_bh);if (err) goto fail; goto got; }/* we lost it */ -jbd2_journal_release_buffer (handle, bitmap_bh); +ext4_handle_release_buffer (handle, bitmap_bh);if (++ino <EXT4_INODES_PER_GROUP (sb)) goto repeat_in_this_group;
@@ -726,7 +727,8 @@ got:/* Don't need to dirty bitmap block if we didn't change it */ if (free) {BUFFER_TRACE (block_bh,"dirty block bitmap" ); - err =ext4_journal_dirty_metadata (handle, block_bh); + err =ext4_handle_dirty_metadata (handle, +NULL , block_bh); }brelse (block_bh);
@@ -771,8 +773,8 @@ got: } gdp->bg_checksum =ext4_group_desc_csum (sbi, group, gdp);spin_unlock (sb_bgl_lock (sbi, group)); -BUFFER_TRACE (bh2,"call ext4_journal_dirty_metadata" ); - err =ext4_journal_dirty_metadata (handle, bh2); +BUFFER_TRACE (bh2,"call ext4_handle_dirty_metadata" ); + err =ext4_handle_dirty_metadata (handle,NULL , bh2);if (err) goto fail;percpu_counter_dec (&sbi->s_freeinodes_counter);
@@ -825,7 +827,7 @@ got:ext4_set_inode_flags (inode);if (IS_DIRSYNC (inode)) - handle->h_sync =1 ; +ext4_handle_sync (handle);insert_inode_hash (inode);spin_lock (&sbi->s_next_gen_lock); inode->i_generation = sbi->s_next_generation++;
@@ -1024,4 +1026,3 @@ unsigned long ext4_count_dirs(struct super_block * sb) }return count; } -
fs/ext4/inode.c +87 -43
310 lines not on this page · raw message
fs/ext4/ioctl.c +1 -1
@@ -99,7 +99,7 @@ long ext4_ioctl(struct file *filp, unsigned int cmd, unsigned long arg) goto flags_out; }if (IS_SYNC (inode)) - handle->h_sync =1 ; +ext4_handle_sync (handle); err =ext4_reserve_inode_write (handle, inode, &iloc);if (err) goto flags_err;
fs/ext4/mballoc.c +9 -8
@@ -2553,7 +2553,8 @@ int ext4_mb_init(struct super_block *sb, int needs_recovery)ext4_mb_init_per_dev_proc (sb);ext4_mb_history_init (sb); - sbi->s_journal->j_commit_callback = release_blocks_on_commit; +if (sbi->s_journal) + sbi->s_journal->j_commit_callback = release_blocks_on_commit;printk (KERN_INFO "EXT4-fs: mballoc enabled\n" );return 0 ;
@@ -2854,7 +2855,7 @@ ext4_mb_mark_diskspace_used(struct ext4_allocation_context *ac,mb_set_bits (sb_bgl_lock (sbi, ac->ac_b_ex.fe_group), bitmap_bh->b_data, ac->ac_b_ex.fe_start, ac->ac_b_ex.fe_len); - err =ext4_journal_dirty_metadata (handle, bitmap_bh); + err =ext4_handle_dirty_metadata (handle,NULL , bitmap_bh);if (!err) err = -EAGAIN ; goto out_err;
@@ -2901,10 +2902,10 @@ ext4_mb_mark_diskspace_used(struct ext4_allocation_context *ac,spin_unlock (sb_bgl_lock (sbi, flex_group)); } - err =ext4_journal_dirty_metadata (handle, bitmap_bh); + err =ext4_handle_dirty_metadata (handle,NULL , bitmap_bh);if (err) goto out_err; - err =ext4_journal_dirty_metadata (handle, gdp_bh); + err =ext4_handle_dirty_metadata (handle,NULL , gdp_bh); out_err: sb->s_dirt =1 ;
@@ -4414,7 +4415,7 @@ ext4_mb_free_metadata(handle_t *handle, struct ext4_buddy *e4b,struct rb_node **n = &db->bb_free_root.rb_node, *node;struct rb_node *parent =NULL , *new_node; - +BUG_ON (!ext4_handle_valid (handle));BUG_ON (e4b->bd_bitmap_page ==NULL );BUG_ON (e4b->bd_buddy_page ==NULL );
@@ -4600,7 +4601,7 @@ do_more:/* We dirtied the bitmap block */ BUFFER_TRACE (bitmap_bh,"dirtied bitmap block" ); - err =ext4_journal_dirty_metadata (handle, bitmap_bh); + err =ext4_handle_dirty_metadata (handle,NULL , bitmap_bh);if (ac) { ac->ac_b_ex.fe_group = block_group;
@@ -4609,7 +4610,7 @@ do_more:ext4_mb_store_history (ac); } -if (metadata) { +if (metadata &&ext4_handle_valid (handle)) {/* blocks being freed are metadata. these blocks shouldn't * be used until this transaction is committed */ ext4_mb_free_metadata (handle, &e4b, block_group, bit, count);
@@ -4639,7 +4640,7 @@ do_more:/* And the group descriptor block */ BUFFER_TRACE (gd_bh,"dirtied group descriptor block" ); - ret =ext4_journal_dirty_metadata (handle, gd_bh); + ret =ext4_handle_dirty_metadata (handle,NULL , gd_bh);if (!err) err = ret;
fs/ext4/migrate.c +3 -2
@@ -59,7 +59,8 @@ static int finish_range(handle_t *handle, struct inode *inode,/* * Make sure the credit we accumalated is not really high */ -if (needed && handle->h_buffer_credits >=EXT4_RESERVE_TRANS_BLOCKS ) { +if (needed &&ext4_handle_has_enough_credits (handle, +EXT4_RESERVE_TRANS_BLOCKS )) { retval =ext4_journal_restart (handle, needed);if (retval) goto err_out;
@@ -229,7 +230,7 @@ static int extend_credit_for_blkdel(handle_t *handle, struct inode *inode) {int retval =0 , needed; -if (handle->h_buffer_credits >EXT4_RESERVE_TRANS_BLOCKS ) +if (ext4_handle_has_enough_credits (handle,EXT4_RESERVE_TRANS_BLOCKS +1 ))return 0 ;/* * We are freeing a blocks. During this we touch
fs/ext4/namei.c +31 -25
184 lines not on this page · raw message
fs/ext4/resize.c +16 -15
@@ -149,7 +149,7 @@ static int extend_or_restart_transaction(handle_t *handle, int thresh, {int err; -if (handle->h_buffer_credits >= thresh) +if (ext4_handle_has_enough_credits (handle, thresh))return 0 ; err =ext4_journal_extend (handle,EXT4_MAX_TRANS_DATA );
@@ -232,7 +232,7 @@ static int setup_new_group_blocks(struct super_block *sb,memcpy (gdb->b_data, sbi->s_group_desc[i]->b_data, gdb->b_size);set_buffer_uptodate (gdb);unlock_buffer (gdb); -ext4_journal_dirty_metadata (handle, gdb); +ext4_handle_dirty_metadata (handle,NULL , gdb);ext4_set_bit (bit, bh->b_data);brelse (gdb); }
@@ -251,7 +251,7 @@ static int setup_new_group_blocks(struct super_block *sb, err =PTR_ERR (bh); goto exit_bh; } -ext4_journal_dirty_metadata (handle, gdb); +ext4_handle_dirty_metadata (handle,NULL , gdb);ext4_set_bit (bit, bh->b_data);brelse (gdb); }
@@ -276,7 +276,7 @@ static int setup_new_group_blocks(struct super_block *sb, err =PTR_ERR (it); goto exit_bh; } -ext4_journal_dirty_metadata (handle, it); +ext4_handle_dirty_metadata (handle,NULL , it);brelse (it);ext4_set_bit (bit, bh->b_data); }
@@ -286,7 +286,7 @@ static int setup_new_group_blocks(struct super_block *sb,mark_bitmap_end (input->blocks_count,EXT4_BLOCKS_PER_GROUP (sb), bh->b_data); -ext4_journal_dirty_metadata (handle, bh); +ext4_handle_dirty_metadata (handle,NULL , bh);brelse (bh);/* Mark unused entries in inode bitmap used */
@@ -299,7 +299,7 @@ static int setup_new_group_blocks(struct super_block *sb,mark_bitmap_end (EXT4_INODES_PER_GROUP (sb),EXT4_BLOCKS_PER_GROUP (sb), bh->b_data); -ext4_journal_dirty_metadata (handle, bh); +ext4_handle_dirty_metadata (handle,NULL , bh); exit_bh:brelse (bh);
@@ -486,12 +486,12 @@ static int add_new_gdb(handle_t *handle, struct inode *inode,* reserved inode, and will become GDT blocks (primary and backup). */ data[gdb_num %EXT4_ADDR_PER_BLOCK (sb)] =0 ; -ext4_journal_dirty_metadata (handle, dind); +ext4_handle_dirty_metadata (handle,NULL , dind);brelse (dind); inode->i_blocks -= (gdbackups +1 ) * sb->s_blocksize >>9 ;ext4_mark_iloc_dirty (handle, inode, &iloc);memset ((*primary)->b_data,0 , sb->s_blocksize); -ext4_journal_dirty_metadata (handle, *primary); +ext4_handle_dirty_metadata (handle,NULL , *primary); o_group_desc =EXT4_SB (sb)->s_group_desc; memcpy(n_group_desc, o_group_desc,
@@ -502,7 +502,7 @@ static int add_new_gdb(handle_t *handle, struct inode *inode,kfree (o_group_desc);le16_add_cpu (&es->s_reserved_gdt_blocks,-1 ); -ext4_journal_dirty_metadata (handle,EXT4_SB (sb)->s_sbh); +ext4_handle_dirty_metadata (handle,NULL ,EXT4_SB (sb)->s_sbh);return 0 ;
@@ -618,7 +618,7 @@ static int reserve_backup_gdb(handle_t *handle, struct inode *inode,primary[i]->b_blocknr, gdbackups, blk + primary[i]->b_blocknr); */ data[gdbackups] =cpu_to_le32 (blk + primary[i]->b_blocknr); - err2 =ext4_journal_dirty_metadata (handle, primary[i]); + err2 =ext4_handle_dirty_metadata (handle,NULL , primary[i]);if (!err) err = err2; }
@@ -676,7 +676,8 @@ static void update_backups(struct super_block *sb,struct buffer_head *bh;/* Out of journal space, and can't get more - abort - so sad */ -if (handle->h_buffer_credits ==0 && +if (ext4_handle_valid (handle) && + handle->h_buffer_credits ==0 &&ext4_journal_extend (handle,EXT4_MAX_TRANS_DATA ) && (err =ext4_journal_restart (handle,EXT4_MAX_TRANS_DATA )))break ;
@@ -696,7 +697,7 @@ static void update_backups(struct super_block *sb,memset (bh->b_data + size,0 , rest);set_buffer_uptodate (bh);unlock_buffer (bh); -ext4_journal_dirty_metadata (handle, bh); +ext4_handle_dirty_metadata (handle,NULL , bh);brelse (bh); }if ((err2 =ext4_journal_stop (handle)) && !err)
@@ -916,7 +917,7 @@ int ext4_group_add(struct super_block *sb, struct ext4_new_group_data *input)/* Update the global fs size fields */ sbi->s_groups_count++; -ext4_journal_dirty_metadata (handle, primary); +ext4_handle_dirty_metadata (handle,NULL , primary);/* Update the reserved block counts only once the new group is * active. */
@@ -938,7 +939,7 @@ int ext4_group_add(struct super_block *sb, struct ext4_new_group_data *input)EXT4_INODES_PER_GROUP (sb); } -ext4_journal_dirty_metadata (handle, sbi->s_sbh); +ext4_handle_dirty_metadata (handle,NULL , sbi->s_sbh); sb->s_dirt =1 ; exit_journal:
@@ -1072,7 +1073,7 @@ int ext4_group_extend(struct super_block *sb, struct ext4_super_block *es, goto exit_put; }ext4_blocks_count_set (es, o_blocks_count + add); -ext4_journal_dirty_metadata (handle,EXT4_SB (sb)->s_sbh); +ext4_handle_dirty_metadata (handle,NULL ,EXT4_SB (sb)->s_sbh); sb->s_dirt =1 ;unlock_super (sb); ext4_debug("freeing blocks %llu through %llu\n", o_blocks_count,
fs/ext4/super.c +135 -65
399 lines not on this page · raw message
fs/ext4/xattr.c +12 -9
@@ -457,7 +457,7 @@ static void ext4_xattr_update_super_block(handle_t *handle,if (ext4_journal_get_write_access (handle,EXT4_SB (sb)->s_sbh) ==0 ) {EXT4_SET_COMPAT_FEATURE (sb,EXT4_FEATURE_COMPAT_EXT_ATTR ); sb->s_dirt =1 ; -ext4_journal_dirty_metadata (handle,EXT4_SB (sb)->s_sbh); +ext4_handle_dirty_metadata (handle,NULL ,EXT4_SB (sb)->s_sbh); } }
@@ -487,9 +487,9 @@ ext4_xattr_release_block(handle_t *handle, struct inode *inode,ext4_forget (handle,1 , inode, bh, bh->b_blocknr); }else {le32_add_cpu (&BHDR (bh)->h_refcount,-1 ); - error =ext4_journal_dirty_metadata (handle, bh); + error =ext4_handle_dirty_metadata (handle, inode, bh);if (IS_SYNC (inode)) - handle->h_sync =1 ; +ext4_handle_sync (handle);DQUOT_FREE_BLOCK (inode,1 );ea_bdebug (bh,"refcount now=%d; releasing" ,le32_to_cpu (BHDR (bh)->h_refcount));
@@ -724,8 +724,9 @@ ext4_xattr_block_set(handle_t *handle, struct inode *inode,if (error == -EIO ) goto bad_block;if (!error) - error =ext4_journal_dirty_metadata (handle, - bs->bh); + error =ext4_handle_dirty_metadata (handle, + inode, + bs->bh);if (error) goto cleanup; goto inserted;
@@ -794,8 +795,9 @@ inserted:ea_bdebug (new_bh,"reusing; refcount now=%d" ,le32_to_cpu (BHDR (new_bh)->h_refcount));unlock_buffer (new_bh); - error =ext4_journal_dirty_metadata (handle, - new_bh); + error =ext4_handle_dirty_metadata (handle, + inode, + new_bh);if (error) goto cleanup_dquot; }
@@ -833,7 +835,8 @@ getblk_failed:set_buffer_uptodate (new_bh);unlock_buffer (new_bh);ext4_xattr_cache_insert (new_bh); - error =ext4_journal_dirty_metadata (handle, new_bh); + error =ext4_handle_dirty_metadata (handle, + inode, new_bh);if (error) goto cleanup; }
@@ -1040,7 +1043,7 @@ ext4_xattr_set_handle(handle_t *handle, struct inode *inode, int name_index,*/ is.iloc.bh =NULL ;if (IS_SYNC (inode)) - handle->h_sync =1 ; +ext4_handle_sync (handle); } cleanup:
Quoted 16 lines · Frank Mayhar
A few weeks ago I posted a patch for discussion that allowed ext4 to run without a journal. Since that time I've integrated the excellent comments from Andreas and fixed several serious bugs. We're currently running with this patch and generating some performance numbers against both ext2 (with backported reservations code) and ext4 with and without a journal. It just so happens that running without a journal is slightly faster for most everything.
We did
iozone -T -t 4 s 2g -r 256k -T -I -i0 -i1 -i2which creates 4 threads, each of which create and do reads and writes on a 2G file, with a buffer size of 256K, using
O_DIRECTfor all file opens to bypass the page cache. Results:ext2 ext4, default ext4, no journal
Quoted 8 lines
initial writes 13.0 MB/s 15.4 MB/s 15.7 MB/s rewrites 13.1 MB/s 15.6 MB/s 15.9 MB/s reads 15.2 MB/s 16.9 MB/s 17.2 MB/s re-reads 15.3 MB/s 16.9 MB/s 17.2 MB/s random readers 5.6 MB/s 5.6 MB/s 5.7 MB/s random writers 5.1 MB/s 5.3 MB/s 5.4 MB/sSo it seems that, so far, this was a useful exercise.
This would imply that there is very little value in keeping ext2 or ext3 around for the long term anymore (though of course I'm not going to suggest that we get rid of them today). It is now possible to mount ext2 filesystems directly with ext4, and it seems clear that ext4 is noticably faster, even with a journal.
Quoted 1 line
+#define
EXT4_NOJOURNAL_MAGIC(void *)0x457874344e6a6e6c /* "Ext4Njnl" */
While I wouldn't go so far as Ted suggests to use "1" as the magic value, it is definitely reasonable to not use a valid pointer value (so it should be an odd number) and keeping it out of the kernel address space is also useful. Ensuring that it is not a valid pointer is sufficient, because the first element of struct handle_s is a pointer and it can never be an odd number due to alignment requirements.
Quoted 4 lines
Quoted 4 lines
If you look at
ext4_handle_validyou'll see where it's actually used. I admit this is very non-obvious (so far it has confused at least two different reviewers); if you have a more clear way to handle it I'm all ears.
Adding a clear comments explaining this (not just a one-liner) at ext4_sb_info so that no others are so confused about this.
fs/ext4/super.c handle_t *ext4_journal_start_sb(struct super_block *sb, int nblocks) · line 152
Quoted 6 lines
fs/ext4/super.c +6
-0+/* +* We're not journaling. Return a pointer to our flag that +* indicates that fact. +*/ + current->journal_info = (handle_t *)&EXT4_SB (sb)->s_nojournal_flag; +return current->journal_info;
Since current->journal_info is a void * you shouldn't need to cast here.
Also, I thought there was somewhere that the "handle" was turned back into ext4_sb_info again? I can't see this after looking through the code. The EXT4_NOJOURNAL_MAGIC and s_nojournal_flag is only used to determine if a handle is not valid, and I can't see anywhere that a handle is turned into ext4_sb_info.
If there is nowhere that you need to translate a handle into a superblock anymore then having a pointer to a static variable would be sufficient. There is no need for a magic value, and dereferencing it will give an oops:
static int dummy_handle = 0x0bad;
void *ext4_no_journal_in_use = &dummy_handle;
static inline int ext4_handle_valid(handle_t *handle)
{
if (handle == ext4_no_journal_in_use)
return 0;
return 1;
}handle_t *ext4_journal_start_sb(struct super_block *sb, int nblocks) {
/* * We're not journaling. Return a pointer to our flag that * indicates that fact. */ current->journal_info = ext4_no_journal_in_use; return current->journal_info; }
Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.
Jan Kara
reviewerQuoted 16 lines
A few weeks ago I posted a patch for discussion that allowed ext4 to run without a journal. Since that time I've integrated the excellent comments from Andreas and fixed several serious bugs. We're currently running with this patch and generating some performance numbers against both ext2 (with backported reservations code) and ext4 with and without a journal. It just so happens that running without a journal is slightly faster for most everything.
We did
iozone -T -t 4 s 2g -r 256k -T -I -i0 -i1 -i2which creates 4 threads, each of which create and do reads and writes on a 2G file, with a buffer size of 256K, using
O_DIRECTfor all file opens to bypass the page cache. Results:ext2 ext4, default ext4, no journal
Quoted 8 lines
initial writes 13.0 MB/s 15.4 MB/s 15.7 MB/s rewrites 13.1 MB/s 15.6 MB/s 15.9 MB/s reads 15.2 MB/s 16.9 MB/s 17.2 MB/s re-reads 15.3 MB/s 16.9 MB/s 17.2 MB/s random readers 5.6 MB/s 5.6 MB/s 5.7 MB/s random writers 5.1 MB/s 5.3 MB/s 5.4 MB/sSo it seems that, so far, this was a useful exercise.
Interesting although I'm not that surprised because those tests seem
to do a lot of data changes (which are never journaled in fact) and tiny amount of metadata changes. If you run some benchmark doing lots of directory operations, I guess the numbers would be considerably different. Maybe trying dbench (I know it's kind of stupid ;) or postmark will show the differences better.
Honza
Signature · 2 lines
Jan Kara <[email protected]> SuSE CR Labs
Quoted 5 lines · Jan Kara
Interesting although I'm not that surprised because those tests seem
to do a lot of data changes (which are never journaled in fact) and tiny amount of metadata changes. If you run some benchmark doing lots of directory operations, I guess the numbers would be considerably different.
Actually we have some compile bench numbers (coming to this list soon) that also surprised us.
The stages of compile bench that I believe are dominated by directory operations are also
showing improvements without the journal.
Quoted 2 lines
Maybe trying dbench (I know it's kind of stupid ;) or postmark will show the differences better.
I admit also we see huge variance using dbench on subsequent runs. To the point where I don't know how much I trust it's numbers. Is this a tool that people on this list have a lot of faith in?
mrubin
Jan Kara
reviewerQuoted 11 lines · Jan Kara
Quoted 5 lines
Interesting although I'm not that surprised because those tests seem
to do a lot of data changes (which are never journaled in fact) and tiny amount of metadata changes. If you run some benchmark doing lots of directory operations, I guess the numbers would be considerably different.Actually we have some compile bench numbers (coming to this list soon) that also surprised us.
The stages of compile bench that I believe are dominated by directory operations are also showing improvements without the journal.
Yes, compilebench excercises directory operations quite heavily so
that is the kind of benchmark I'd be interested in :).
Quoted 6 lines
Quoted 2 lines
Maybe trying dbench (I know it's kind of stupid ;) or postmark will show the differences better.
I admit also we see huge variance using dbench on subsequent runs. To the point where I don't know how much I trust it's numbers. Is this a tool that people on this list have a lot of faith in?
Not really (at least as far as I'm concerned :). I had to deal with
dbench recently tracking some reported performance regression and lost a lot of my faith in it ;). It measures something but the numbers vary greatly and also the load it puts on the filesystem is not much realistic (lots of rewrites of the same pages and file deletes just after creation). I think compilebench is a better choice, I just didn't remember its name when I was writing the email.
Honza
Signature · 2 lines
Jan Kara <[email protected]> SuSE CR Labs
I should be able to post some numbers we're seeing for compilebench on various filesystems next week, including ext4 without a journal (Frank's patch).
Curt
Quoted 26 lines · Michael Rubin
Quoted 5 lines · Jan Kara
Interesting although I'm not that surprised because those tests seem
to do a lot of data changes (which are never journaled in fact) and tiny amount of metadata changes. If you run some benchmark doing lots of directory operations, I guess the numbers would be considerably different.Actually we have some compile bench numbers (coming to this list soon) that also surprised us.
The stages of compile bench that I believe are dominated by directory operations are also
showing improvements without the journal.Quoted 2 lines
Maybe trying dbench (I know it's kind of stupid ;) or postmark will show the differences better.
I admit also we see huge variance using dbench on subsequent runs. To the point where I don't know how much I trust it's numbers. Is this a tool that people on this list have a lot of faith in?
mrubin
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html