libext2fs: fix parents when modifying extents
PATCH 1/4 libext2fs: fix parents when modifying extents
From: Darrick J. Wong <[email protected]>
In ext2fs_extent_set_bmap() and ext2fs_punch_extent(), fix the parents when altering either end of an extent so that the parent nodes reflect the added mapping.
There's a slight complication to using fix_parents: if there are two mappings to an lblk in the tree, the value of handle->path->curr can point to either extent afterwards), which is documented in a comment.
Some additional color commentary from Darrick:
In the _set_bmap() case, I noticed that the "remapping last block in extent" case would produce symptoms if we are trying to remap a block from "extent" to "next_extent", and the two extents are pointed to by different index nodes. _extent_replace(..., next_extent) updates e_lblk in the leaf extent, but because there's no _extent_fix_parents() call, the index nodes never get updated.
In the _punch_extent() case, we conclude that we need to split an extent into two pieces since we're punching out the middle. If the extent is the last extent in the block, the second extent will be inserted into a new leaf node block. Without _fix_parents(), the index node doesn't seem to get updated.
--- lib/ext2fs/extent.c | 30 ++++++++++++++++++++++++------ lib/ext2fs/punch.c | 14 ++++++++++---- 2 files changed, 34 insertions(+), 10 deletions(-)
lib/ext2fs/extent.c +24 -6
@@ -734,7 +734,14 @@ errcode_t ext2fs_extent_goto(ext2_extent_handle_t handle,* and so on. * * Safe to call for any position in node; if not at the first entry, -* will simply return. +* it will simply return. +* +* Note a subtlety of this function -- if there happen to be two extents +* mapping the same lblk and someone calls fix_parents on the second of the two +* extents, the position of the extent handle after the call will be the second +* extent if nothing happened, or the first extent if something did. A caller +* in this situation must use ext2fs_extent_goto() after calling this function. +* Or simply don't map the same lblk with two extents, ever. */ errcode_t ext2fs_extent_fix_parents (ext2_extent_handle_t handle) {
@@ -1424,17 +1431,25 @@ errcode_t ext2fs_extent_set_bmap(ext2_extent_handle_t handle, &next_extent);if (retval) goto done; - retval =ext2fs_extent_fix_parents (handle); -if (retval) - goto done; }else retval =ext2fs_extent_insert (handle,EXT2_EXTENT_INSERT_AFTER , &newextent);if (retval) goto done; -/* Now pointing at inserted extent; move back to prev */ + retval =ext2fs_extent_fix_parents (handle); +if (retval) + goto done; +/* +* Now pointing at inserted extent; move back to prev. +* +* We cannot use EXT2_EXTENT_PREV to go back; note the +* subtlety in the comment for fix_parents(). +*/ + retval =ext2fs_extent_goto (handle, logical); +if (retval) + goto done; retval =ext2fs_extent_get (handle, -EXT2_EXTENT_PREV_LEAF , +EXT2_EXTENT_CURRENT , &extent);if (retval) goto done;
@@ -1467,6 +1482,9 @@ errcode_t ext2fs_extent_set_bmap(ext2_extent_handle_t handle, 0, &newextent);if (retval) goto done; + retval =ext2fs_extent_fix_parents (handle); +if (retval) + goto done; retval =ext2fs_extent_get (handle,EXT2_EXTENT_NEXT_LEAF , &extent);
lib/ext2fs/punch.c +10 -4
@@ -343,10 +343,16 @@ static errcode_t ext2fs_punch_extent(ext2_filsys fs, ext2_ino_t ino, EXT2_EXTENT_INSERT_AFTER, &newex);if (retval) goto errout; -/* Now pointing at inserted extent; so go back */ - retval =ext2fs_extent_get (handle, -EXT2_EXTENT_PREV_LEAF , - &newex); + retval =ext2fs_extent_fix_parents (handle); +if (retval) + goto errout; +/* +* Now pointing at inserted extent; so go back. +* +* We cannot use EXT2_EXTENT_PREV to go back; note the +* subtlety in the comment for fix_parents(). +*/ + retval =ext2fs_extent_goto (handle, extent.e_lblk);if (retval) goto errout; }
Signature · 1 line
1.7.3.4
PATCH 2/4 LU-7368 e2fsck: skip quota update when interrupted
There is a bug in how e2fsck handles being interrupted by CTRL-C. If CTRL-C is pressed to kill e2fsck rather than e.g. kill -9, then the interrupt handler sets E2F_FLAG_CANCEL in the context but doesn't actually kill the process. Instead, e2fsck_pass1() checks this flag before processing the next inode.
If a filesystem is running in fix mode (e2fsck -fy) is interrupted, and the quota feature is enabled, then the quota file will still be written to disk even though the inode scan was not complete and the quota information is totally inaccurate. Even worse, if the Pass 1 inode and block scan was not finished, then the in-memory block bitmaps (which are used for block allocation during e2fsck) are also invalid, so any blocks allocated to the quota files may corrupt other files if those blocks were actually used.
e2fsck 1.42.13.wc3 (28-Aug-2015)
Pass 1: Checking inodes, blocks, and sizes
^C[QUOTA WARNING] Usage inconsistent for ID 0:
actual (6455296, 168) != expected (8568832, 231)
[QUOTA WARNING] Usage inconsistent for ID 695:
actual (614932320256, 63981) != expected (2102405386240, 176432)Update quota info for quota type 0? yes
[QUOTA WARNING] Usage inconsistent for ID 0:
actual (6455296, 168) != expected (8568832, 231)
[QUOTA WARNING] Usage inconsistent for ID 538:
actual (614932320256, 63981) != expected (2102405386240, 176432)Update quota info for quota type 1? yes
myth-OST0001: e2fsck canceled.
myth-OST0001: ***** FILE SYSTEM WAS MODIFIED *****
There may be a desire to flush out modified inodes and such that have been repaired, so that restarting an interrupted e2fsck will make progress, but the quota file update is plain wrong unless at least pass1 has finished, and the journal recreation is also dangerous if the block bitmaps have not been fully updated.
--- e2fsck/e2fsck.c | 2 -- e2fsck/e2fsck.h | 4 ++-- e2fsck/pass1.c | 4 +++- e2fsck/pass2.c | 10 +++++----- e2fsck/unix.c | 18 ++++++++++-------- 5 files changed, 20 insertions(+), 18 deletions(-)
e2fsck/e2fsck.c +0 -2
@@ -203,8 +203,6 @@ static pass_t e2fsck_passes[] = { e2fsck_pass1, e2fsck_pass2, e2fsck_pass3, e2fsck_pass4, e2fsck_pass5, 0 }; -#define E2F_FLAG_RUN_RETURN (E2F_FLAG_SIGNAL_MASK|E2F_FLAG_RESTART) -int e2fsck_run (e2fsck_t ctx) {int i;
e2fsck/e2fsck.h +2 -2
@@ -173,10 +173,10 @@ struct resource_track {*/ #define E2F_FLAG_ABORT 0x0001/* Abort signaled */ #define E2F_FLAG_CANCEL 0x0002/* Cancel signaled */ -#define E2F_FLAG_SIGNAL_MASK 0x0003 +#define E2F_FLAG_SIGNAL_MASK (E2F_FLAG_ABORT | E2F_FLAG_CANCEL)#define E2F_FLAG_RESTART 0x0004/* Restart signaled */ +#define E2F_FLAG_RUN_RETURN (E2F_FLAG_SIGNAL_MASK | E2F_FLAG_RESTART)#define E2F_FLAG_RESTART_LATER 0x0008/* Restart after all iterations done */ -#define E2F_FLAG_SETJMP_OK 0x0010/* Setjmp valid for abort */ #define E2F_FLAG_PROG_BAR 0x0020/* Progress bar on screen */
e2fsck/pass1.c +3 -1
@@ -766,7 +766,7 @@ void e2fsck_pass1(e2fsck_t ctx) inode, inode_size);ehandler_operation (old_op);if (ctx->flags &E2F_FLAG_SIGNAL_MASK ) -return ; + goto endit;if (pctx.errcode ==EXT2_ET_BAD_BLOCK_IN_INODE_TABLE ) {if (!ctx->inode_bb_map)alloc_bb_map (ctx);
@@ -1276,6 +1276,8 @@ endit:if ((ctx->flags &E2F_FLAG_SIGNAL_MASK ) ==0 )print_resource_track (ctx,_ ("Pass 1" ), &rtrack, ctx->fs->io); +else + ctx->invalid_bitmaps++; }/*
e2fsck/pass2.c +5 -5
@@ -148,14 +148,14 @@ void e2fsck_pass2(e2fsck_t ctx) cd.pctx.errcode =ext2fs_dblist_iterate2 (fs->dblist, check_dir_block, &cd); -if (ctx->flags &E2F_FLAG_SIGNAL_MASK || ctx->flags &E2F_FLAG_RESTART ) -return ; -if (ctx->flags &E2F_FLAG_RESTART_LATER ) { ctx->flags |=E2F_FLAG_RESTART ; -return ; + ctx->flags &= ~E2F_FLAG_RESTART_LATER ; } +if (ctx->flags &E2F_FLAG_RUN_RETURN ) +return ; +if (cd.pctx.errcode) {fix_problem (ctx,PR_2_DBLIST_ITERATE , &cd.pctx); ctx->flags |=E2F_FLAG_ABORT ;
@@ -739,7 +739,7 @@ static int check_dir_block(ext2_filsys fs, buf = cd->buf; ctx = cd->ctx; -if (ctx->flags &E2F_FLAG_SIGNAL_MASK || ctx->flags &E2F_FLAG_RESTART ) +if (ctx->flags &E2F_FLAG_RUN_RETURN )return DIRENT_ABORT ;if (ctx->progress && (ctx->progress)(ctx,2 , cd->count++, cd->max))
e2fsck/unix.c +10 -8
@@ -1667,8 +1667,15 @@ print_unsupp_features: } no_journal: -if (ctx->qctx) { +if (run_result &E2F_FLAG_ABORT ) { +fatal_error (ctx,_ ("aborted" )); + }else if (run_result &E2F_FLAG_CANCEL ) { +log_out (ctx,_ ("%s: e2fsck canceled.\n" ), ctx->device_name ? + ctx->device_name : ctx->filesystem_name); + exit_value |=FSCK_CANCELED ; + }else if (ctx->qctx && !ctx->invalid_bitmaps) {int i, needs_writeout; +for (i =0 ; i <MAXQUOTAS ; i++) {if (qtype !=-1 && qtype != i)continue ;
@@ -1695,18 +1702,13 @@ no_journal:ext2fs_close_free (&fs); goto restart; } -if (run_result &E2F_FLAG_ABORT ) -fatal_error (ctx,_ ("aborted" ));#ifdef MTRACE mtrace_print ("Cleanup" );#endif was_changed =ext2fs_test_changed (fs); -if (run_result &E2F_FLAG_CANCEL ) { -log_out (ctx,_ ("%s: e2fsck canceled.\n" ), ctx->device_name ? - ctx->device_name : ctx->filesystem_name); - exit_value |=FSCK_CANCELED ; - }else if (!(ctx->options &E2F_OPT_READONLY )) { +if (!(ctx->flags &E2F_FLAG_RUN_RETURN ) && + !(ctx->options &E2F_OPT_READONLY )) {if (ext2fs_test_valid (fs)) {if (!(sb->s_state &EXT2_VALID_FS )) exit_value |=FSCK_NONDESTRUCT ;
Signature · 1 line
1.7.3.4
Re: [PATCH 2/4] LU-7368 e2fsck: skip quota update when interrupted
Quoted 5 lines · Andreas Dilger
There is a bug in how e2fsck handles being interrupted by CTRL-C. If CTRL-C is pressed to kill e2fsck rather than e.g. kill -9, then the interrupt handler sets
E2F_FLAG_CANCELin the context but doesn't actually kill the process. Instead,e2fsck_pass1()checks this flag before processing the next inode.
Sigh, please remove the "LU-7368" label from the summary line, as that is for internal tracking and I thought I'd done that for the patches before sending them out.
If desired, this patch can be linked back to our bug tracker to get a full discussion of the bug as below:
Lustre-change: http://review.whamcloud.com/17150
Lustre-bug-id: https://jira.hpdd.intel.com/browse/LU-7368
For "libext2fs: fix block-mapped file punch" it would be:
Lustre-change: http://review.whamcloud.com/17152
Lustre-bug-id: https://jira.hpdd.intel.com/browse/LU-7381
and for "e2fsck: fix e2fsck -fD directory truncation" it is:
Lustre-change: http://review.whamcloud.com/17153
Lustre-bug-id: https://jira.hpdd.intel.com/browse/LU-7381
Cheers, Andreas
Quoted 170 lines
If a filesystem is running in fix mode (e2fsck -fy) is interrupted, and the quota feature is enabled, then the quota file will still be written to disk even though the inode scan was not complete and the quota information is totally inaccurate. Even worse, if the Pass 1 inode and block scan was not finished, then the in-memory block bitmaps (which are used for block allocation during e2fsck) are also invalid, so any blocks allocated to the quota files may corrupt other files if those blocks were actually used.
e2fsck 1.42.13.wc3 (28-Aug-2015)
Pass 1: Checking inodes, blocks, and sizes
^C[QUOTA WARNING] Usage inconsistent for ID 0:actual (6455296, 168) != expected (8568832, 231) [QUOTA WARNING] Usage inconsistent for ID 695: actual (614932320256, 63981) != expected (2102405386240, 176432)Update quota info for quota type 0? yes
[QUOTA WARNING] Usage inconsistent for ID 0:
actual (6455296, 168) != expected (8568832, 231) [QUOTA WARNING] Usage inconsistent for ID 538: actual (614932320256, 63981) != expected (2102405386240, 176432)Update quota info for quota type 1? yes
myth-OST0001: e2fsck canceled.
myth-OST0001: ***** FILE SYSTEM WAS MODIFIED *****There may be a desire to flush out modified inodes and such that have been repaired, so that restarting an interrupted e2fsck will make progress, but the quota file update is plain wrong unless at least pass1 has finished, and the journal recreation is also dangerous if the block bitmaps have not been fully updated.
Signed-off-by Andreas Dilger <[email protected]>---
e2fsck/e2fsck.c | 2 -- e2fsck/e2fsck.h | 4 ++-- e2fsck/pass1.c | 4 +++- e2fsck/pass2.c | 10 +++++----- e2fsck/unix.c | 18 ++++++++++-------- 5 files changed, 20 insertions(+), 18 deletions(-)e2fsck/e2fsck.c +0
-2@@ -203,8 +203,6 @@ static pass_t e2fsck_passes[] = { e2fsck_pass1, e2fsck_pass2, e2fsck_pass3, e2fsck_pass4, e2fsck_pass5, 0 }; -#define E2F_FLAG_RUN_RETURN (E2F_FLAG_SIGNAL_MASK|E2F_FLAG_RESTART) -int e2fsck_run (e2fsck_t ctx) {int i;e2fsck/e2fsck.h +2
-2@@ -173,10 +173,10 @@ struct resource_track {*/ #define E2F_FLAG_ABORT 0x0001/* Abort signaled */ #define E2F_FLAG_CANCEL 0x0002/* Cancel signaled */ -#define E2F_FLAG_SIGNAL_MASK 0x0003 +#define E2F_FLAG_SIGNAL_MASK (E2F_FLAG_ABORT | E2F_FLAG_CANCEL)#define E2F_FLAG_RESTART 0x0004/* Restart signaled */ +#define E2F_FLAG_RUN_RETURN (E2F_FLAG_SIGNAL_MASK | E2F_FLAG_RESTART)#define E2F_FLAG_RESTART_LATER 0x0008/* Restart after all iterations done */ -#define E2F_FLAG_SETJMP_OK 0x0010/* Setjmp valid for abort */ #define E2F_FLAG_PROG_BAR 0x0020/* Progress bar on screen */ e2fsck/pass1.c +3
-1@@ -766,7 +766,7 @@ void e2fsck_pass1(e2fsck_t ctx) inode, inode_size);ehandler_operation (old_op);if (ctx->flags &E2F_FLAG_SIGNAL_MASK ) -return ; + goto endit;if (pctx.errcode ==EXT2_ET_BAD_BLOCK_IN_INODE_TABLE ) {if (!ctx->inode_bb_map)alloc_bb_map (ctx);@@ -1276,6 +1276,8 @@ endit:if ((ctx->flags &E2F_FLAG_SIGNAL_MASK ) ==0 )print_resource_track (ctx,_ ("Pass 1" ), &rtrack, ctx->fs->io); +else + ctx->invalid_bitmaps++; }/* e2fsck/pass2.c +5
-5@@ -148,14 +148,14 @@ void e2fsck_pass2(e2fsck_t ctx) cd.pctx.errcode =ext2fs_dblist_iterate2 (fs->dblist, check_dir_block, &cd); -if (ctx->flags &E2F_FLAG_SIGNAL_MASK || ctx->flags &E2F_FLAG_RESTART ) -return ; -if (ctx->flags &E2F_FLAG_RESTART_LATER ) { ctx->flags |=E2F_FLAG_RESTART ; -return ; + ctx->flags &= ~E2F_FLAG_RESTART_LATER ; } +if (ctx->flags &E2F_FLAG_RUN_RETURN ) +return ; +if (cd.pctx.errcode) {fix_problem (ctx,PR_2_DBLIST_ITERATE , &cd.pctx); ctx->flags |=E2F_FLAG_ABORT ;@@ -739,7 +739,7 @@ static int check_dir_block(ext2_filsys fs, buf = cd->buf; ctx = cd->ctx; -if (ctx->flags &E2F_FLAG_SIGNAL_MASK || ctx->flags &E2F_FLAG_RESTART ) +if (ctx->flags &E2F_FLAG_RUN_RETURN )return DIRENT_ABORT ;if (ctx->progress && (ctx->progress)(ctx,2 , cd->count++, cd->max))e2fsck/unix.c +0
-0@@ -1667,8 +1667,15 @@ print_unsupp_features: }
no_journal:e2fsck/unix.c +7
-1-if (ctx->qctx) { +if (run_result &E2F_FLAG_ABORT ) { +fatal_error (ctx,_ ("aborted" )); + }else if (run_result &E2F_FLAG_CANCEL ) { +log_out (ctx,_ ("%s: e2fsck canceled.\n" ), ctx->device_name ? + ctx->device_name : ctx->filesystem_name); + exit_value |=FSCK_CANCELED ; + }else if (ctx->qctx && !ctx->invalid_bitmaps) {int i, needs_writeout; + for (i = 0; i < MAXQUOTAS; i++) { if (qtype != -1 && qtype != i) continue;e2fsck/unix.c +2
-8@@ -1695,18 +1702,13 @@ no_journal:ext2fs_close_free (&fs); goto restart; } -if (run_result &E2F_FLAG_ABORT ) -fatal_error (ctx,_ ("aborted" ));#ifdef MTRACE mtrace_print ("Cleanup" );#endif was_changed =ext2fs_test_changed (fs); -if (run_result &E2F_FLAG_CANCEL ) { -log_out (ctx,_ ("%s: e2fsck canceled.\n" ), ctx->device_name ? - ctx->device_name : ctx->filesystem_name); - exit_value |=FSCK_CANCELED ; - }else if (!(ctx->options &E2F_OPT_READONLY )) { +if (!(ctx->flags &E2F_FLAG_RUN_RETURN ) && + !(ctx->options &E2F_OPT_READONLY )) {if (ext2fs_test_valid (fs)) {if (!(sb->s_state &EXT2_VALID_FS )) exit_value |=FSCK_NONDESTRUCT ; --1.7.3.4
Quoted 4 lines
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Cheers, Andreas
Theodore Ts'o
AppliedRe: [PATCH 2/4] LU-7368 e2fsck: skip quota update when interrupted
In reply to Andreas DilgerThanks, applied.
- Ted
PATCH 3/4 libext2fs: fix block-mapped file punch
In reply to Andreas DilgerIf ext2fs_punch() is called with "end = ~0U" (which is natural from a programming POV) it tries to compute "count" based on "start" and "end", but doesn't quite get it right.
Fix ext2fs_punch() if "end" is anywhere beyond the 2^32 block limit so that the 32-bit calculations don't overflow. Pass "count=~0" in this case, and also handle that explicitly in ext2_punch_ind(). Since ext2_punch_ind() is itself a public function, so it makes sense to fix this in both places.
--- lib/ext2fs/punch.c | 18 +++++++++++++----- 1 files changed, 13 insertions(+), 5 deletions(-)
lib/ext2fs/punch.c +13 -5
@@ -119,6 +119,9 @@ static errcode_t ext2fs_punch_ind(ext2_filsys fs, struct ext2_inode *inode, } addr_per_block = (blk_t ) fs->blocksize >>2 ; +/* make sure that count + start doesn't overflow a 32-bit value */ +if (count == ~0U ) + count = ~0U - start;for (level =0 ; level <4 ; level++, max *= (blk64_t )addr_per_block) {#ifdef PUNCH_DEBUG
@@ -458,14 +461,19 @@ errcode_t ext2fs_punch(ext2_filsys fs, ext2_ino_t ino,if (start > ~0U )return 0 ; -if (end > ~0U ) - end = ~0U ; - count = ((end - start +1 ) < ~0U ) ? (end - start +1 ) : ~0U ; - retval =ext2fs_punch_ind (fs, inode, block_buf, - (blk_t ) start, count); +if (end >= ~0U || end - start +1 >= ~0U ) + count = ~0U ; +else + count = end - start +1 ; + retval =ext2fs_punch_ind (fs, inode, block_buf, + (blk_t )start, count); }if (retval)return retval; +#ifdef PUNCH_DEBUG +printf ("%u: write inode size now %u blocks %u\n" , + ino, inode->i_size, inode->i_blocks); +#endif return ext2fs_write_inode (fs, ino, inode); }
Signature · 1 line
1.7.3.4
Re: [PATCH 3/4] libext2fs: fix block-mapped file punch
Quoted 3 lines · Andreas Dilger
If
ext2fs_punch()is called with "end = ~0U" (which is natural from a programming POV) it tries to compute "count" based on "start" and "end", but doesn't quite get it right.
You should make clear that this only applies in the case where we are punching on an indirect block mapped file. I read this and alarm bells started going off since end is a blk64_t, and we can't do this for extent-mapped files.
Quoted 4 lines
Pass "count=~0"
in this case, and also handle that explicitly inext2_punch_ind(). Sinceext2_punch_ind()is itself a public function, so it makes sense to fix this in both places.
Um, ext2fs_punch()_ind is *not* a public function.
- Ted
Re: [PATCH 3/4] libext2fs: fix block-mapped file punch
In reply to Andreas DilgerQuoted 3 lines · Andreas Dilger
If
ext2fs_punch()is called with "end = ~0U" (which is natural from a programming POV) it tries to compute "count" based on "start" and "end", but doesn't quite get it right.
I don't object to ext2fs_punch handling 32-bit overflows correctly if it is being called on an indirect mapped file (where the logical block numbers are limited to 2**32). But it would be good if the patch description actually said that. Also, arguably this is still a bug in the caller, which should use ~0ULL to mean the end of the file.
Quoted 5 lines
Fix
ext2fs_punch()if "end" is anywhere beyond the 2^32 block limit so that the 32-bit calculations don't overflow. Pass "count=~0" in this case, and also handle that explicitly inext2_punch_ind(). Sinceext2_punch_ind()is itself a public function, so it makes sense to fix this in both places.
Um, ext2_punch_ind() is a static function....
Review static errcode_t ext2fs_punch_ind(ext2_filsys fs, struct ext2_inode *inode,
Quoted 1 line
(quoted diff) +0
-0@@ -119,6 +119,9 @@ static errcode_t ext2fs_punch_ind(ext2_filsys fs, struct ext2_inode *inode,
Cheers,
- Ted
PATCH 4/4 e2fsck: fix e2fsck -fD directory truncation
In reply to Andreas DilgerWhen an extent-mapped directory is compacted by "e2fsck -fD" and frees enough leaf blocks that it loses an extent tree index block, the old e2fsck_rehash_dir->ext2fs_block_iterate3->write_dir_block() code would not free the extent block, which would result in the extent tree becoming corrupted when it is written out.
Pass 1: Checking inodes, blocks, and sizes
Inode 17825800, end of extent exceeds allowed value
(logical block 710, physical block 570459684, len 1019)
This results in loss of a whole index block of directory leaf blocks and thousands of files in lost+found.
Fix e2fsck_rehash_dir() to call ext2fs_punch() to free the blocks at the end of the directory instead of trying to handle this itself while writing out the directory. That properly handles all of the cases of updating the extent tree as well as accounting for blocks that are released (both leaf blocks and index blocks).
Add a test case for compacting the directory to be smaller than the index block that originally caused the corruption.
--- e2fsck/rehash.c | 73 ++++++++++++++++++++++------------------ tests/f_extent_htree/expect.1 | 29 ++++++++++++++++ tests/f_extent_htree/expect.2 | 7 ++++ tests/f_extent_htree/image.gz | Bin 0 -> 10101 bytes tests/f_extent_htree/name | 1 + tests/f_extent_htree/script | 69 ++++++++++++++++++++++++++++++++++++++ tests/f_h_badnode/expect.1 | 2 +- tests/f_h_badnode/expect.2 | 2 +- 8 files changed, 148 insertions(+), 35 deletions(-) create mode 100644 tests/f_extent_htree/expect.1 create mode 100644 tests/f_extent_htree/expect.2 create mode 100644 tests/f_extent_htree/image.gz create mode 100644 tests/f_extent_htree/name create mode 100644 tests/f_extent_htree/script
e2fsck/rehash.c +40 -33
@@ -52,10 +52,13 @@#include "e2fsck.h" #include "problem.h" +#undef REHASH_DEBUG +struct fill_dir_struct {char *buf;struct ext2_inode *inode;errcode_t err; +ext2_ino_t ino;e2fsck_t ctx;struct hash_entry *harray;int max_array, num_array;
@@ -625,8 +628,8 @@ static errcode_t calculate_tree(ext2_filsys fs,struct write_dir_struct {struct out_dir *outdir;errcode_t err; +ext2_ino_t ino;e2fsck_t ctx; -blk64_t cleared; };/*
@@ -643,28 +646,35 @@ static int write_dir_block(ext2_filsys fs,blk64_t blk;char *dir; -if (*block_nr ==0 ) +#ifdef REHASH_DEBUG +printf ("%u: write_dir_block %lld:%lld" , wd->ino, blockcnt, *block_nr); +#endif +if (*block_nr ==0 ) { +#ifdef REHASH_DEBUG +printf (" - skip\n" ); +#endif return 0 ; + } +/* Don't free blocks at the end of the directory, they will be +* truncated by the caller. */ if (blockcnt >= wd->outdir->num) { -e2fsck_read_bitmaps (wd->ctx); - blk = *block_nr; -/* -* In theory, we only release blocks from the end of the -* directory file, so it's fine to clobber a whole cluster at -* once. -*/ -if (blk %EXT2FS_CLUSTER_RATIO (fs) ==0 ) { -ext2fs_block_alloc_stats2 (fs, blk,-1 ); - wd->cleared++; - } - *block_nr =0 ; -return BLOCK_CHANGED ; +#ifdef REHASH_DEBUG +printf (" - not freed\n" ); +#endif +return 0 ; } -if (blockcnt <0 ) +if (blockcnt <0 ) { +#ifdef REHASH_DEBUG +printf (" - skip\n" ); +#endif return 0 ; + } dir = wd->outdir->buf + (blockcnt * fs->blocksize); wd->err =ext2fs_write_dir_block3 (fs, *block_nr, dir,0 ); +#ifdef REHASH_DEBUG +printf (" - write (%d)\n" , wd->err); +#endif if (wd->err)return BLOCK_ABORT ;return 0 ;
@@ -684,10 +694,10 @@ static errcode_t write_directory(e2fsck_t ctx, ext2_filsys fs, wd.outdir = outdir; wd.err =0 ; + wd.ino = ino; wd.ctx = ctx; - wd.cleared =0 ; - retval =ext2fs_block_iterate3 (fs, ino,0 ,0 , + retval =ext2fs_block_iterate3 (fs, ino,0 ,NULL , write_dir_block, &wd);if (retval)return retval;
@@ -699,14 +709,18 @@ static errcode_t write_directory(e2fsck_t ctx, ext2_filsys fs, inode.i_flags &= ~EXT2_INDEX_FL ; else inode.i_flags |= EXT2_INDEX_FL; - retval =ext2fs_inode_size_set (fs, &inode, - outdir->num * fs->blocksize); +#ifdef REHASH_DEBUG +printf ("%u: set inode size to %u blocks = %u bytes\n" , + ino, outdir->num, outdir->num * fs->blocksize); +#endif + retval =ext2fs_inode_size_set (fs, &inode, (ext2_off64_t )outdir->num * + fs->blocksize);if (retval)return retval; -ext2fs_iblk_sub_blocks (fs, &inode, wd.cleared); -e2fsck_write_inode (ctx, ino, &inode,"rehash_dir" ); -return 0 ; +/* ext2fs_punch() also calls ext2fs_write_inode() */ + retval =ext2fs_punch (fs, ino, &inode,NULL , outdir->num, ~0ULL ); +return retval; }errcode_t e2fsck_rehash_dir (e2fsck_t ctx,ext2_ino_t ino)
@@ -715,32 +729,25 @@ errcode_t e2fsck_rehash_dir(e2fsck_t ctx, ext2_ino_t ino)errcode_t retval;struct ext2_inode inode;char *dir_buf =0 ; -struct fill_dir_struct fd; -struct out_dir outdir; +struct fill_dir_struct fd = {NULL }; +struct out_dir outdir = {0 }; - outdir.max = outdir.num =0 ; - outdir.buf =0 ; - outdir.hashes =0 ;e2fsck_read_inode (ctx, ino, &inode,"rehash_dir" ); retval =ENOMEM ; - fd.harray =0 ; dir_buf =malloc (inode.i_size);if (!dir_buf) goto errout; fd.max_array = inode.i_size /32 ; - fd.num_array =0 ; fd.harray =malloc (fd.max_array *sizeof (struct hash_entry ));if (!fd.harray) goto errout; + fd.ino = ino; fd.ctx = ctx; fd.buf = dir_buf; fd.inode = &inode; - fd.err =0 ; - fd.dir_size =0 ; - fd.compress =0 ;if (!(fs->super->s_feature_compat &EXT2_FEATURE_COMPAT_DIR_INDEX ) || (inode.i_size / fs->blocksize) <2 ) fd.compress =1 ;
tests/f_extent_htree/expect.1 new file +29 -0
@@ -0,0 +1,29 @@ +Pass 1: Checking inodes, blocks, and sizes +Pass 2: Checking directory structure +Pass 3: Checking directory connectivity +Pass 3A: Optimizing directories +Pass 4: Checking reference counts +Pass 5: Checking group summary information + +test_filesys: ***** FILE SYSTEM WAS MODIFIED ***** + + 352 inodes used (41.12%, out of 856) + 0 non-contiguous files (0.0%) + 1 non-contiguous directory (0.3%) + # of inodes with ind/dind/tind blocks: 0/0/0 + Extent depth histogram: 342/1 + 586 blocks used (68.94%, out of 850) + 0 bad blocks + 0 large files + + 340 regular files + 3 directories + 0 character device files + 0 block device files + 0 fifos + 0 links + 0 symbolic links (0 fast symbolic links) + 0 sockets +------------ + 343 files +Exit status is 1
tests/f_extent_htree/expect.2 new file +7 -0
@@ -0,0 +1,7 @@ +Pass 1: Checking inodes, blocks, and sizes +Pass 2: Checking directory structure +Pass 3: Checking directory connectivity +Pass 4: Checking reference counts +Pass 5: Checking group summary information +test_filesys: 352/856 files (0.3% non-contiguous), 586/850 blocks +Exit status is 0
tests/f_extent_htree/image.gz new file binary +0 -0
tests/f_extent_htree/name new file +1 -0
@@ -0,0 +1 @@ +htree extent compression
tests/f_extent_htree/script new file +69 -0
69 lines not on this page · raw message
tests/f_h_badnode/expect.1 +1 -1
@@ -14,5 +14,5 @@ Pass 4: Checking reference counts Pass 5: Checking group summary information test_filesys: ***** FILE SYSTEM WAS MODIFIED ***** -test_filesys: 47730/100192 files (0.0% non-contiguous), 13551/31745 blocks +test_filesys: 47730/100192 files (0.0% non-contiguous), 13550/31745 blocks Exit status is 1
tests/f_h_badnode/expect.2 +1 -1
@@ -3,5 +3,5 @@ Pass 2: Checking directory structure Pass 3: Checking directory connectivity Pass 4: Checking reference counts Pass 5: Checking group summary information -test_filesys: 47730/100192 files (0.0% non-contiguous), 13551/31745 blocks +test_filesys: 47730/100192 files (0.0% non-contiguous), 13550/31745 blocks Exit status is 0
Signature · 1 line
1.7.3.4
Re: [PATCH 4/4] e2fsck: fix e2fsck -fD directory truncation
Quoted 10 lines · Andreas Dilger
When an extent-mapped directory is compacted by "e2fsck -fD" and frees enough leaf blocks that it loses an extent tree index block, the old
e2fsck_rehash_dir->ext2fs_block_iterate3->write_dir_block()code would not free the extent block, which would result in the extent tree becoming corrupted when it is written out.Pass 1: Checking inodes, blocks, and sizes Inode 17825800, end of extent exceeds allowed value
(logical block 710, physical block 570459684, len 1019)
Quoted 8 lines
This results in loss of a whole index block of directory leaf blocks and thousands of files in lost+found.
Fix
e2fsck_rehash_dir()to callext2fs_punch()to free the blocks at the end of the directory instead of trying to handle this itself while writing out the directory. That properly handles all of the cases of updating the extent tree as well as accounting for blocks that are released (both leaf blocks and index blocks).
There were a couple of bugs in the ext2fs_punch() code that also had to be fixed (see previous patches).
One patch is from Darrick on master, so you may prefer to just cherry-pick it directly. I didn't actually hit the problem it was fixing in my tests, but it seemed plausible and looked useful for the maint branch.
The second one fixes some existing bugs in the punch code for indirect block filesystems, which I hit in a few other tests (f_dir_optimize, f_dup_de, and others that are optimizing directories).
Quoted 2 lines
Add a test case for compacting the directory to be smaller than the index block that originally caused the corruption.
The test case for this patch was created using the supplied script, but depends on "mke2fs -d $root_dir" working, which doesn't exist on the maint branch. For maint I've included the resulting image.gz file (only about 10KB compressed) and a stub script file that just runs "e2fsck -fvyD" against it.
For merge into master, the test f_extent_htree script should be edited to remove the leading stub, and can generate the test filesystem itself (which also tests the "mke2fs -d" functionality and takes less space, but takes a couple more seconds to run).
Cheers, Andreas
Quoted 502 lines
Signed-off-by Andreas Dilger <[email protected]>---
e2fsck/rehash.c| 73
++++++++++++++++++++++------------------tests/f_extent_htree/expect.1 | 29 ++++++++++++++++ tests/f_extent_htree/expect.2 | 7 ++++ tests/f_extent_htree/image.gz | Bin 0 -> 10101 bytes tests/f_extent_htree/name | 1 + tests/f_extent_htree/script | 69 ++++++++++++++++++++++++++++++++++++++ tests/f_h_badnode/expect.1 | 2 +- tests/f_h_badnode/expect.2 | 2 +- 8 files changed, 148 insertions(+), 35 deletions(-)create mode 100644 tests/f_extent_htree/expect.1
create mode 100644 tests/f_extent_htree/expect.2
create mode 100644 tests/f_extent_htree/image.gz
create mode 100644 tests/f_extent_htree/name
create mode 100644 tests/f_extent_htree/scripte2fsck/rehash.c +28
-18@@ -52,10 +52,13 @@#include "e2fsck.h" #include "problem.h" +#undef REHASH_DEBUG +struct fill_dir_struct {char *buf;struct ext2_inode *inode;errcode_t err; +ext2_ino_t ino;e2fsck_t ctx;struct hash_entry *harray;int max_array, num_array;@@ -625,8 +628,8 @@ static errcode_t calculate_tree(ext2_filsys fs,struct write_dir_struct {struct out_dir *outdir;errcode_t err; +ext2_ino_t ino;e2fsck_t ctx; -blk64_t cleared; };/* @@ -643,28 +646,35 @@ static int write_dir_block(ext2_filsys fs,blk64_t blk;char *dir; -if (*block_nr ==0 ) +#ifdef REHASH_DEBUG +printf ("%u: write_dir_block %lld:%lld" , wd->ino, blockcnt, *block_nr); +#endif +if (*block_nr ==0 ) { +#ifdef REHASH_DEBUG +printf (" - skip\n" ); +#endif return 0 ; + } +/* Don't free blocks at the end of the directory, they will be +* truncated by the caller. */ if (blockcnt >= wd->outdir->num) { -e2fsck_read_bitmaps (wd->ctx); - blk = *block_nr; -/* -* In theory, we only release blocks from the end of the -* directory file, so it's fine to clobber a whole cluster at -* once. -*/ -if (blk %EXT2FS_CLUSTER_RATIO (fs) ==0 ) { -ext2fs_block_alloc_stats2 (fs, blk,-1 ); - wd->cleared++; - } - *block_nr =0 ; -return BLOCK_CHANGED ; +#ifdef REHASH_DEBUG +printf (" - not freed\n" ); +#endif +return 0 ; } -if (blockcnt <0 ) +if (blockcnt <0 ) { +#ifdef REHASH_DEBUG +printf (" - skip\n" ); +#endif return 0 ; + } dir = wd->outdir->buf + (blockcnt * fs->blocksize); wd->err =ext2fs_write_dir_block3 (fs, *block_nr, dir,0 ); +#ifdef REHASH_DEBUG +printf (" - write (%d)\n" , wd->err); +#endif if (wd->err)return BLOCK_ABORT ;return 0 ;@@ -684,10 +694,10 @@ static errcode_t write_directory(e2fsck_t ctx,ext2_filsys fs, wd.outdir = outdir; wd.err =0 ; + wd.ino = ino; wd.ctx = ctx; - wd.cleared =0 ; - retval =ext2fs_block_iterate3 (fs, ino,0 ,0 , + retval =ext2fs_block_iterate3 (fs, ino,0 ,NULL , write_dir_block, &wd);if (retval)return retval;e2fsck/rehash.c +9
-5@@ -699,14 +709,18 @@ static errcode_t write_directory(e2fsck_t ctx,ext2_filsys fs, inode.i_flags &= ~EXT2_INDEX_FL; else inode.i_flags |= EXT2_INDEX_FL; - retval =ext2fs_inode_size_set (fs, &inode, - outdir->num * fs->blocksize); +#ifdef REHASH_DEBUG +printf ("%u: set inode size to %u blocks = %u bytes\n" , + ino, outdir->num, outdir->num * fs->blocksize); +#endif + retval =ext2fs_inode_size_set (fs, &inode, (ext2_off64_t )outdir->num * + fs->blocksize);if (retval)return retval; -ext2fs_iblk_sub_blocks (fs, &inode, wd.cleared); -e2fsck_write_inode (ctx, ino, &inode,"rehash_dir" ); -return 0 ; +/* ext2fs_punch() also calls ext2fs_write_inode() */ + retval =ext2fs_punch (fs, ino, &inode,NULL , outdir->num, ~0ULL ); +return retval; }
errcode_te2fsck_rehash_dir(e2fsck_t ctx, ext2_ino_t ino)e2fsck/rehash.c +3
-10@@ -715,32 +729,25 @@ errcode_t e2fsck_rehash_dir(e2fsck_t ctx, ext2_ino_t ino) errcode_t retval;struct ext2_inode inode;char *dir_buf =0 ; -struct fill_dir_struct fd; -struct out_dir outdir; +struct fill_dir_struct fd = {NULL }; +struct out_dir outdir = {0 }; - outdir.max = outdir.num =0 ; - outdir.buf =0 ; - outdir.hashes =0 ;e2fsck_read_inode (ctx, ino, &inode,"rehash_dir" ); retval =ENOMEM ; - fd.harray =0 ; dir_buf =malloc (inode.i_size);if (!dir_buf) goto errout; fd.max_array = inode.i_size /32 ; - fd.num_array =0 ; fd.harray =malloc (fd.max_array *sizeof (struct hash_entry ));if (!fd.harray) goto errout; + fd.ino = ino; fd.ctx = ctx; fd.buf = dir_buf; fd.inode = &inode; - fd.err =0 ; - fd.dir_size =0 ; - fd.compress =0 ;if (!(fs->super->s_feature_compat &EXT2_FEATURE_COMPAT_DIR_INDEX ) || (inode.i_size / fs->blocksize) <2 )fd.compress = 1;tests/f_extent_htree/expect.1 new file +29
-0@@ -0,0 +1,29 @@ +Pass 1: Checking inodes, blocks, and sizes +Pass 2: Checking directory structure +Pass 3: Checking directory connectivity +Pass 3A: Optimizing directories +Pass 4: Checking reference counts +Pass 5: Checking group summary information + +test_filesys: ***** FILE SYSTEM WAS MODIFIED ***** + + 352 inodes used (41.12%, out of 856) + 0 non-contiguous files (0.0%) + 1 non-contiguous directory (0.3%) + # of inodes with ind/dind/tind blocks: 0/0/0 + Extent depth histogram: 342/1 + 586 blocks used (68.94%, out of 850) + 0 bad blocks + 0 large files + + 340 regular files + 3 directories + 0 character device files + 0 block device files + 0 fifos + 0 links + 0 symbolic links (0 fast symbolic links) + 0 sockets +------------ + 343 files +Exit status is 1tests/f_extent_htree/expect.2 new file +7
-0@@ -0,0 +1,7 @@ +Pass 1: Checking inodes, blocks, and sizes +Pass 2: Checking directory structure +Pass 3: Checking directory connectivity +Pass 4: Checking reference counts +Pass 5: Checking group summary information +test_filesys: 352/856 files (0.3% non-contiguous), 586/850 blocks +Exit status is 0tests/f_extent_htree/image.gz new file +0
-00000000000000000000000000000000000000000..284207efb12a04136c9f26c33f2197fe
d97cbd99
GIT binary patch
literal 10101 zcmbVRcR&-_*G2ugao1I>sGtd=t_{Tk0vd=2tZkLGA{HVFBBEd@5)3f9xS~Le*ib34 zV<jtyNC{&jumXvQ5CuY%paBCJLx4%0_dQra{P*(@Gs&BI^WMGZoO|wrk}zb*MN`+! z0|x~h@bUMXvv6-%h@aowP@h94Z4y%9#_G;yl^s6N%Cf8B>0egyp|$0gb{04Mb7ZIc z#y>Zn8*VmcK!NSp*|$%em@#AiF1Km%5AEmuRIn@f_K_p;$I7Au>B49mE!xDzBUci$ z9rbvHztHR9oUZh2WqP>Bgpnpii!--2jgM}-rV7+;T{mK;K}&nRtD?*pDx5&i!K~kzT0D2|TVrTAKwseD2#=5k-sP?CJpbihbeGuQrSVul{=g}FGy#Xt#i)#*nw2+y$kA4 zeZzy%)1x&1K3l^*tms@h<An}UM_3(w$DV0o=Y~9paDgV)?)8wyvhL6<-kdt6R<I z%{1x$*YrmlqIz9N!<-H<+P?T@(6jOnsXkoAor&r%(>Hf<4|Q!o%oj6N4H-83?ZX6( z%L7^1rX{?iex@(&UoGPc8VN%a=7&ARr-$h%3_+>+n~Ourr*<kkgNBtZP~Q&!-Lw| z*v;bBPYWJo4*nXvr@a1hO#<z5uCcJTSQ@NV7Vwcv&XLc);N#kC0}PGtgT-CjjH?T7 zlBivko9LZ}$DrU-!eQE0y-(aqr_Xb~Y-hbq->_L66Sb}r;2d)k!Iy-pG;kL&dc+W# z7nt(6R5RVu}M<IkUai3>p4C3(m{E(POTaSuB?u0x%KYKGZ1YUn>7WaXo45=LD%Oe zTv{>OrOsHjNFN+S*j9;;AT=WN(k8?XvBzD5_&S5KycOhK{-E10T}MaCiS{wkIi=2= zP5!hg%Z+J<vvJi%jWNOC?~5Eqadm-=3@XWh>-%VjeguGp6DyyPWPb_v)Lucc?J=f zsHO)R2#&3gR?8$x#>**zso*CWj-;wI!L~vqn?Y<#R7>}zB83d%XMr}@J*fqt37Pf zK|x{(lB?28x8MM>%z&GVz()8nLC)QiisY+Q_z{)Hf4m&zFo?KBHEn+?a*IJMhgWc} z3n~$}B(Ve>V-W8JTEssU%#;~I;n;!gh-kbVsbLb_g+lNbgBZtFqrOoF{;pK8NoL?p zK!F-Qm(L(*<3)hNR*UR~0M47s=AaRYYBZ1vFbg?&>Hy*dTG|33c*r0;64l6_R1hjN z(BX1kIRLRhi_RB<2MhvB6iYF73Hr_fjFK5><K^@{QHDA8WWa%|T9gjP3S^+z0dN!5 zyaht~?o_l!qX8+VxSnE;Jaa(8pB*AB#><2ErXq7>IQmef5ib%--!h4m337S~lMvgH z!C?Y%fHwsO|QgJ84e$Of4qWr7HN7RaP;9KdFQHg}N_g^Mv0Xt$Y*rCsp%XAa<$ zK#iVNX(ASpkwBT@K%yM+lo<*p$fXhoq|yP|8D&^5(C(IL;qH>za+)0(DP|hl5=+oH zl?JyJqPZ#qwwp>sv(=bC73g7-Ok^sGFOUei9f+0UL>k1L8)MfOopw11AeJsuS|>W zr4nleT8^Di3b)OGSN1~W8iP2SsOIiZ1!H7fL;|yZmqApq<!FIQ!x}HARWS*cIR{wF z4D#E%+Ek@1F1xVK&yw}0#i|xK@1RRNpNR7;T;wtFjyc%OC7)nnPHp-3S8lG?l1@+ zfp+<L6eLgJfDoCA9*_!V$qb%wTxxkZZL>-0yRzG0Jg}qsL%ln6KG2&$k9p%P$4J* zbKr#44uB~$j82q-I<{IGKn0`W2i#PU%y1TBg9TU3V-R-)T5O?^R-w_P1W^f6VhQa9 z(||4#g3}PB5KOj007J7JUP0(wWDwI6)uKg&R=-51qWMt?M}d|LQQ;4P3R&VP!@*#g z!3F|<wLnXUh=Oqu!zUnedjKlu1{>7!H8MjmO!pFjR%#~%=?uaY;t?X|D1+dz6=1YX zh3xx=P8871<)GF9*b6YQ3=TkIuw+gZz^(<K~NIaHg-a^9zw8^XoHuAb&FmI)tWe z4Dg)5K?O|1J$4D#IB1V{=%&YNx?d^^-*H@kAyZ^3dx){e4B|I|wg;OINT^J=O~f=P zEacLsOu~MF5QS7(1LIc#F_gz3<|L|vVJ?FYq#_nF1N@e2>r6aeC4l+u=b{+P&?At zQcxq%M$8wY_aMAeRGK<-4!Sp$HZL8#W*XppFPX$@fy{$lLZ43tyhJtTlZtrA3@ixa z5{*VGc0l0vq*4a}5n>9rVY-M6CJSV=w+T=K#f3o7!E(qsWhBLLLyVi_P~M4Nb^2 zlvv1V&zOYA0wI#6GDJYoR={tNa{&TvfrT8t4E!pS@iJ5zm()l-dms***O!!g=B;x zGekoOb5$A=)I6Au(<(huq|(^T6Czht22;4+VyKw&h2Ov#XCX(+9l$4nmIv>Eo4?0K z2yaNRBnaFIa>@ayq43S&aNIr}GDp@8&NWak=f{9xJ0X&yGI-1*gEk027@d=FyvzYi z?FOe5KycC^c{f1BLaO2x94Ma}$~+;sqB3-&!BwCJ7Bba0bV|+TT)!xT6O=TDOv^ox z3L@b64V4D64}oMa7KT22}orSawwjA4Fn2VpA0YeQ2I4CR3%MLy4n69X$@8vXCQH z4qzLECq!^2gTNs?ccubY8J9KUP9WBmKnI_6od;kWGDn>0F~guPnMXI7s^?GK17Y z-UCsnLPV*ra+qvc}s&x{+CI3Ldh+LACSf8RhmGEty-pG{ecv8XDa&20U0DS3}dSy zlL{TYauwHn!@riNoJ^Et9kyZNVLpgVMu5PG#Y>~5eT&Q5N($jL<O4zdLWwGEsu5Z zv4AK;us@Y(gM#-Bo{1c^28wC11CoL)i|KI}|KI<fBDZEm+nrsusSZr?+>}!dsy52l zH&$-qH5f^c9IxX~)%gYJ^aWCx=r9{MmX&rC6r{y2-MNPAd#Jyb)*jfBIv<Qsba2 z#DG76$!)~sEzRT$#iY~ph>rB%&xVZK!kGWrpz=p*7tCfyY&b%kgG<+AY<6Ryq>0 zxN30Is|kFTy^q;{4?d-^huZx;w6J>7oR*0lcT?i3okgw_r&>)3CtuJwnXWTNPK%+ zGYEZ$4bBAE#;--dbh~uh4Ztb;S<$-yf);1=p7z@D$ytIn<CYbg0XWZp=^eG?8Bu9; z@TmBSD*$EVE2|y9G~mExYez;inU@!8qo?Abv7@u_^cgs1Jj2#Hs15A$o^|PbyW>@ zVQI1W-^>1_Jk4*>cCQ-iI&wCfNk7I2eN|JVAEXs;sOL^jz>@8x%Uc^s@S&C44HnT4 z&K9H9o@(ou?R3Q+)mhzEn*<5=PmQQi-kIZKhTi85%VuUi!R+ay(W^Xmwq{QBfiuNe zOJJ3?_Z416mS#irf%Ibfm%s#T@7ugN+03M($FCZ1>SA!AaBHryb9OT>-MPjLATU6 zOY*3^@%Q}qa}!dU_vo#IGE*jHF$YOnFBYdX-Zprd^II4i)kTgO)ai#xfO1f=(;9 zS7Aa>tuJGnr1f&~)u2OVjc0;t_o#Sb*(@_)0WV-(+PEo-%5Akp?2ksuHmEo7$nI{ zuc*YJGuF!cDeKL+S5iXEBC6M1F3xDVb*9+ALm~2)Z?I9KUHM@zZ<fmOUbC@{JUJ zmS&lxHLW<gc@NioS71xt@%arNYoe@mQ6R~)Km~&QT(9cwoLp=uV@GmlFGhgGeTB zM+}P3r*$&AeOwI%<U8IcG~$Ynoh4|{sqR~=u6BP<>iK@R_ZuOM|XEW8g#X42>d4 z&EO6KvG^1N)Iz0uO1bCI20Cd|2w%HEMc~L)4;1V+k~D^HynfgQuyAS0lhAU!oppM~ z%#-E!3y2a-{_KJm_=fJhHegh;%G{IeFU8Gx9r@&_9{KCEWx_%)OPIF2<Jdn}*$U zGas#GTMk11Vt|rN$(;m}+MQ?Ww$nAyOS8;sP%(b)t(@1WHbpj*mj&58n~|0rOY zL8W17%2yG{Fq}D-y4d!z$Wf~?^_J}T3MpxOl^@1CIX%{QOi^1A~1Ve@3@H$L^S}q zS!7(~iF({JvrG-1zq}r3vIPuq2Hnhg4fJ6EnLjG;FP6z)E94)4bdC~lbyY!1>CYC z;qU@M4p&ki=Vm)IIpO{^c$h!Sd2})ze3&E_>6ugGOZhM%I}fXG(x;#%p_&y(j~ zaEtE%K2Np1a$rQ9F~D(m_9s7IWtBxTB3qga-)WjG2MJ@7q+%x^Qegbq%MV)@KF7iN zT{YhypUY#Jj!sIs7*}c>xO0PwFd*x#v%c$qn#qrFG>c@b!8AMvmqHW$3*7F1f&^2> zSK_0@Sep=Y*@g3Q-cTtRH?=ohM+&<@49;IUBV^A}xn;hE6f6al!DJ_sBb}5F>h47z z35HT32a-@MvkA9ccR4=wxf;ZVl<K}|TxlS{a3grw#hNS!FRoZ;QH)3_#KeEE*FwZN z1MZiJ6U#7{#ud{W52}g)N4DSxArkY)xhB0YcO5g$O8y3v)Pvrsg8|m51#g{{#+{kK zbk^dL^Ti;J3cls)w3Luy>V})~r6Y8}@|4?LvvfdT9ntVdUXS-3XmRl@g*%lHgA{l zn=HQua@Aq%#?&{rSKV;&IWHEu)Nkhe{V5$M&cEi>uB2TDZr{CgK+~o#Su_$+$sIz zqYhl1w4e>_H^}{*;Z+n;v8lrM8U|7ZRw4B2gsvhlYX))SYXP?vkEa20n=4v$)Kk? z)34f|^%K$zJ&=qu$$ZfXd2bg{|#30^*+7pYsvvUl|B+%Orngr7?&k{5f?58#`L&R zeSyn0q7eA^PLy8<He)-tY_Y5d^k4hZ-UG*~ElngHc%6pFq(exI?guEco^Sb<8(Z$$ znjXu9-hJr6^xmLf0U|#@pBZ1ii+t_2)q<4{fZnL<2SpJp<4)<W&y&M&m$0jVGlfGo zfmryysTkgtF<uE67%CZLB9g(x%*_6dR9La59qr-=dY-hMalbtuU|0AcM9LlP)wgk& zm>ghp+)4pa{Kvqgqp863j8*TZj}WwE5r4J&J}p7R?S{O8+p=zlRO+d^OM~<XJfPSj z-)Uw<z7%x@C4M!4bG^yxR;P5SdQ$xMica8iBRY7S5X3>mexpeT(6-{ck}5lZ%f%C^ zu7GmRp?^S7&y#NJA+yiBlgDNPPX8>ovSe<)18`6faQayXV9ejsn?ksg_Q+trBwh=c zA_mfBfO0;s7rINKl<2|r_|l*2;j7B5k}JXTvHhY9^+HSl2OYR_U;U505h?E@6kI4( zy6qSDMZ3snc1>+RAW;Kjk*Sq$zqhI*gHM*~VBpSPD;-h<Y)-7|WjjPHwTnNk4Zol^ zp!44B_W9L_jA%FX&n}pLvUJAh$*WwG7XGaVu1fHYFinpEt}BcvPE#5$z|3q&;xvQn zR>p5KW(m;<{ltA^LkL8M2n7H6Ne4DT=W?mPu*TTTyx^9(lYDKX9;|)d3D*7HJ9=77 zNENVY!k^%int|+*b^8>kl56|89N7k(Fnn2(BNU0jR<pmPLt(&T2XfW|2rgLWy1f>_ z?*f~BFT`0-scsKT-9BoOn<skbhw(0nvH8{g@gDLn+<i!J2thm9H+c_Lcg##Eq<<wG zhdeB3i=JylX721CQ3GPV{<99u8{)opTsoi}4jW+7SIu+?SBFFpAO=RwD(SaPnSlPO zQw{o2u^U=#fC49Yy90p5Vlcf=!{EY13jW!pO$&A4$?cNozkbw$aMNY|UykKPB$aMa z0@Gz_apNEx+ix5D^y$M6K$k)FT}(=bD!0zMoy|lD+Wj$odxL?4@_%NR`;9s35y@p% z*L#fKmOdSXI)m#B$m!;Z_iq5xvZT@)Sh-#rdy6e3O$^TR@oL%&_4m!0(t>)xWrk^K zeVfO02v5(?(m%(4f)CCd6?vRMC-#-5kUGGMk+3X7MnasGSWP*GfqTr3p(ZuR5mC=8 ziaJ3I4bbbnBRSGow_-QnZE&Ac(gEq4e+HsY#s(!kNvZ@TF0<bJMbHvQ)qI2NQg6t2 zSQvNNlj=!H8{O<fEVk_d&?4(vEAJ=7F9t#c7CaE)<J;kord%i5=0EfZAjWcTGL# zZVtu9dhib{VcqHCKgc6!zpl1YoWzZ=;P#iXNPu>+EUG}<!!Gau7M61u81$dt@8*eK z57wu}Mr)lq&B<l7eRe@>lKsI0|1MZZW}e>UoEh-PM}Vu0?iA$AsU?7O%>%R^GOm zGw9O2l(k-l<3Xn?8gs*|922DCHObb*3i}d2m4CjYa{~IN;%6_zgP^tLjm7znGZT_q zJ~vqtUiLe(H6Hnn(-H#1tL7x!yXUsHqA|zTc}_ygzNoyyn77Xy5Ub5*{kr5z~5JO z!JK=kxNcwl!h~GjRrf6a~oF5va_GC1#(}N(j0%)xIz}1{p7wX(yM;0tih=R}hW zq_OpTR4(RtS>p>^CAWC_KB{HHl6};92?cwpGZH-BG7p+@uc@?NQTc*#zN$;+^ozyv zy;1AT@EYa{GhD1v?kwIP4kLM61ZkFZwr$5sDlP{8Z=7O`#FCBPSK_wk@(w2!5?; zw=VLM&CX&L3Ox6y(#+}Siz5P~!p)@jRCZp}HL}2L=5fK4Y>f|lxu5E~IY-zy-Ijc| z*du_tD1lq4$+wPiktwn@W7)x_jgjV@vPRmzsNK*StJXlPEU*_-fVp%%VhD^1?EdjU z_m46K2VqZt+dbmlJ+i!eB&~a7w1NZnB^5wZ&7NM}J-!1mc=94o1xnx@PThBG_yX^M z=CT94r9lvUz!Abo*qm@9;f0WjM1)<bke-(rWX!(bW$M1xIHC@T6N{0SBjztdZzKzgjzkqQ~y)1W<87V#o&>NYCb;fQOgo&L}go=W~GCXZc$+RH-ik-RxFPyQ);=>Au%#B z2r3>PWc0VFl}-i1WF4=;wHx!UQN>>ubdVeMg1!EJ*a{1r@95vfd2hl@K|l<gD6 zG9S-|zGg-OwL}ZDJVYk2#QfJNKvPs5jn#gYl*3B;=coXo5gM3&ZK?VlcK(&shM^5K z(!<8NEX1o21J_T>d)zm4g|KgpB+!fTk9Lt8ZAUtuCuqZ0_inqpK&1mt4!~Z)xgAGE zp45ZY<Q&i!wRc4Z&)(it;>hpc*81W3JypvuljRdy6p{tPFIg0)dJ5-5O?!fzY<VZ zlzYQS69dxC6*qow2GP2KOZ$|l12}i1H61)KTn=1u=0TQcwL{x?bZ@h4!z+Z^@v-w zc!LX|FnTK=j663wc%2@Mz7+Q^4R=0H&<=@W<G)ZD%f#sKq(B;N<F}#8Hk|#ufGmO z2X7ha^@@5dz;eMUf;KC(-^|Idvk3s;SMX$@09+c1E-1aZAz7#Uu-2^XzH_dROF)% zI$+L~Pws4PNBZr^I}O;7thhCj?#6B#{;ohaw1Epu>(!S1ijyN1c1zkbkwa!Dl|sW| zdHbL5b557)A}SmJbb$4LT=I_c;JX$^))df!+ER-5!kj0Q1VL}Dz3u4sgS-z6)L(5 z&%lTV{o8?JmvaeL(ZzZNkX8uYERz~jVWK^qdrgmV@lGtCxA9YLwqxHaQeA=zMGz( z3pM!X&MgNyMQWn~x%0Art^xgR3^ou}rd?@qWaxLAzq4-R{UCsb^A~15rdOTs~V#L z#l5esfa&y?ZW|f!g8CAQJO!%w5zBPq_~od9~y5&CT3W1)J7x#y7s<;<Oh+!60;#{ z%dGSV0kS2rFPN~{8jHGFMTQ>d#*9%Ae*;9lg#p^ZENsnavHCDF>DUsZ%}}_p_U4Z7 zrL@~e@GXbC*W4Wy@aL9Kot;4GIHb%8C0*Ozajy$j^uGS}El6IU1P{SYcRQKgb7$o zex2tasD(sJXo$~oqp$p2MvYH-_0rdv7r+iz1PFo)x*?Gy;C~xqXC)L-?yp?BSyqq zL%;H89rFI98u&=uk^8;qG8F5_%L5;}*Hk?CySUcK_IZ(l%I*$~&e2gCeB*-?~@X z-H|-Z?j8fRJWtT7p~z@#&DO7QMzA#;9(l};Fy)n5m27UE%#eO#CpW1_S(>%D9#B$ z?T<vBNbpOV-U6=A=rc3jjyoSeQwon*N$X}UfT!@Yr}e|DGZ!~I+4_w7a#~v49xAZ; z<4Hd{iHhvUctNiKyU8%v<W24Gz#an5YOA_=ri^K$0#*WwzGJXSAI^IOsla*xJMzSt zKYd3)abf)EYUTVq9zS_>GuVyS>ocLw_L2B>9U)c6*o@Jhj2u$tmn3QiuH1fkGBKha zM&z#1+aG%P6@!wlcTW*LqMuY6B)C$@qCKO!ty}49YedfVxAM+Be64#-G4<t5H(v+X z;a&To=N3a{$vcEuLh{r{V{y<tjurL_bdPHTi1mW>$P=sXlwQjxPywcJ>4Q1+%dH^c zXzAc*X_T$Z-&p3od{0fc|n8VM|3Uv7^H|f_7@QRdx#~>&stLF0@Blq<Eum=;z6Y zL@wuM2>~li-=76M9k@MX0XOo*cA4TZ9Uc^oOWxfti*qr2xO?q<Q^rbt!&cbcI+2ok zCH}OiyNlq@R_ehanI1f!Hzp4*;cg#0dfFe>w6xf)gM<z#qn}*zOVFjH;6G@Y*>yK zT#6jBL&iD*$-Tb6khc>czO<m+b>mwCS^7**&{j}&1HXIO4&Wpy~kX7Y6M4F1US1L z(C7B|jPLP$MFYAMo&i~hEP&Mz9@_SuD^)2PHYCX|#yPBVyL)##Kwl}a*ggcpnDJBv zSheGLD7BkbSs5JwW$MV@64G;&YxI6NBV-XQhBucsEr*16?H@3GL;Rwsh=r!>?#x&a zOQ6eoy(rYd0BA)cuYE8^kGP|+)kDs{I09SZ|4y@EoqmtGCzVcx7>Pf*f(~oL)qXUD zScC1JQ@QK88&={sfaLQ-8Ad(Vx-EkZ^Kz@1@R;m>{Xk3Yu@)a;7%K_s4qeepTpIhj z7kbS=9f?M)^qm2>!dTZd2A;MkzKgmRo~Z0aBZk3dVzb-aCb*!r{XhxN-{64<<MZVB zS61RmxS+mL12-zY$%Q|t%j?;;8t{B0Rjut<N65EbF2VDr66~xdPK(2QeLrM4c4_* z+WdHG$d7Q>17&x^{s^11F%mXfGY)Umueo4%Zmk#9oxb&_EWeymzE|M!G>%miH(&t zrr8zI@%*zp?rA6d^6i8~NBW9oK9r)^%mS=%|K#mp!_RJ~K58j0uhg{fM|Q>{kTE_1 zo;Go>+%_Uh0p+JY^VhRj){rSlIk4Yx(>HJ<3I8n}ZKS1xlCJ7#5p+>xS?$J6Einac zYablS?Yv*ha>*eMv@E(C`Qms!cS-o4FMiLbzuCX@dQKQ<ef}HULP4lQ=|NeKmAvLK z)&~6OwN0FL3ft;vQSz<(b${Ta?SpcjEzYOp%FbFW5*z2n8M7+$^QFyiJ?y<b@a5qW zW>sf6XfHjUa^Q;o!6_C?JuF<>q8PsIzzzL?Xk9S>q4J*hj-1Zllv2vUw#O;?fGGTV z&Z1DHfzq%)FHgzQW|j@#)*KVJ=FmIFY6)Ioe4dv3#^}Lc8^q61=3Z3Q&#h~YS%3ZC z1COht?{ujpTc04jy7I?!!lmK$uNn{KgRsx-2U!x)gs^1+BwUVF=4M?_;gCEZ2j0}X zob7raI$Z(|g%^s2`SgE?yV2Veg!VidjHOR|%P%4Hxs}E@F>R*rs*Mq5=)n&``P>}! z5>igJ@$_a7N>+g~L&$8M7f~pAvq$yus8X88+)u(g7Y1+7%8DhnoEe)FI7yc2VgB{~ zAHJLnH8o*&F{A@%Js4h06JM9Q3P~Kd!jzXcMkj&rmbmm~VJtsHfDzBPc_EKB6W} z;)=IPnUCwz^U(t@l^NJgy?bS~Q&)}gi+dn9kE6sud;UVa+9*!=y<<O5U(rzT;l8on zA?~KU75hB%GDA2(X}o0DSU)Hn)EtrMCN>RE<L2W|Tkzcl<rs2884-Q+->0q;eW<V{ zc6gQXDEmSx@<cgZoLKaAh=hJ<XNCKudWVd9ho%(~_2-<)A?M0aSyJ-EHlo0^9BmpO z5tZi9^yXAP-67)L!3bn4x|YIr*o>!4jCtml7Ebvo#k-++k}0o^S<DCmjj^7Su+PSq zswp0?jS=>V<HN7%F{*7$$I2k*pC&Hq^2LerZ6P~6?U5aV?MVERbboK=ZhQNp1b=DR zT^7A-wj}DZlD8P7q<BP>R0dNDa|+7rKi#Z5{Dd*weBsAmarvc1AG#JwRHJeWuV*Y# zCI>sJtnHPaXue{<gyO01kjR?}*^{a!?|g1Z=Z>?Ya9RDqN+)?9P+<dwzZwga7s_F zU5>L7Cxjz(iL~<b#{gqJHR#MA7)J0tm8_jd%Ive=Cmx+EI=6vrenVfGYYQrN+BXEv z@lVGZlJkt@yrp=6pa^gFih6?V9$=GQ_1hwP%N(L$HjF-Htv#erLwa3$xk9L)wD#o zf@?%w=}q&sShcdnC!eN2pTV7=Jau+#Z2F{+r&4b1@P1NWq40E~=hsi-kiwYSUnDj( zpVHZ$Ut;)SErFhS204G8$Z0i;^U=OAhOLaFKj|_Ah67*j$>39Ojh*KQHIQ%Wou$9V zL_E?vRt6BP-SynyT)y*o#VOhEo=8-h{*MK_h110ff=^nE9FEn?*Iv@Ucd(~%CTfb= zO8N8nuVpD=NJZ8&x}=rRI$tl!E3RNAm<<gMFKJVG&MjPnTa-I#>QwoA&NNZ+JkCl z>1}1LOrUmT{0=3%lGZLwnW06PA>GlR)!1xS1Utg(IKsGRt6Ob*Cb0G=*@-=-{Sm! z5_h+UMn(3_UCq_~E3wG~FK&)idjDJPQS{EWV&Oa{3RqHFZgJ?cq*i)T96XKmLw; S1hD`^CoQ(LFj+Co<o^JlLF&8!literal 0
HcmV?d00001tests/f_extent_htree/name new file +1
-0@@ -0,0 +1 @@ +htree extent compressiontests/f_extent_htree/script new file +7
-0@@ -0,0 +1,69 @@ +#!/bin/bash + +FSCK_OPT="-fyvD" +. $cmd_dir/run_e2fsck + +exit $? +# This script depends on "mke2fs -d", which is only in master and notmaint,
tests/f_extent_htree/script new file +22
-0+# to populate the file directory tree poorly (namely that there are no +# contiguous blocks in the directory leaf and the extent tree is large). + +# Once the "mke2fs -d" option is available on the "maint" branch, the +# above few lines should be deleted, along with the "image.gz" file. + +TMPDIR=${TMPDIR:-"/tmp"} +OUT=$test_name.log + +FSCK_OPT="-fyvD" +SKIP_GUNZIP="true" + +NAMELEN=250 +SRC=$TMPDIR/$test_name.tmp +SUB=subdir +BASE=$SRC/$SUB/$(yes | tr -d '\n' | dd bs=$NAMELEN count=1 2> /dev/null) +TMPFILE=${TMPFILE:-"$TMPDIR/image"} +BSIZE=1024 + +> $OUT +mkdir -p $SRC/$SUB +# calculate the number of files needed to create the directory extenttree
tests/f_extent_htree/script new file +13
-0+# deep enough to exceed the in-inode index and spill into an index block. +# +# dirents per block * extents per block * (index blocks > i_blocks) +NUM=$(((BSIZE / (NAMELEN + 8)) * (BSIZE / 12) * 2)) +# Create source files. Unfortunately hard links will be copied as links, +# and blocks with only NULs will be turned into holes. +if [ ! -f $BASE.1 ]; then + for N in $(seq $NUM); do + echo "foo" > $BASE.$N + done >> $OUT +fi + +# make filesystem with enough inodes and blocks to hold all the testfiles
+> $TMPFILEtests/f_extent_htree/script new file +2
-0+NUM=$((NUM * 5 / 3)) +echo "mke2fs -b $BSIZE -O dir_index,extent -d$SRC -N$NUM $TMPFILE $NUM"Quoted 1 line
Quoted 1 line
$OUT
+$MKE2FS -b $BSIZE -O dir_index,extent -d$SRC -N$NUM $TMPFILE $NUM >> $OUT 2>&1
tests/f_extent_htree/script new file +4
-0+rm -r $SRC + +# Run e2fsck to convert dir to htree before deleting the files, as mke2fs +# doesn't do this. Run second e2fsck to verify there is no corruptionyet.
tests/f_extent_htree/script new file +10
-0+( + EXP1=$test_dir/expect.pre.1 + EXP2=$test_dir/expect.pre.2 + OUT1=$test_name.pre.1.log + OUT2=$test_name.pre.2.log + DESCRIPTION="$(cat $test_dir/name) setup" + . $cmd_dir/run_e2fsck +) + +# generate a list of filenames for debugfs to delete, one from each leafblock
tests/f_extent_htree/script new file +9
-0+DELETE_LIST=$TMPDIR/delete.$$ +$DEBUGFS -c -R "htree subdir" $TMPFILE 2>> $OUT | + grep -A2 "Reading directory block" | + awk '/yyyyy/ { print "rm '$SUB'/"$4 }' > $DELETE_LIST +$DEBUGFS -w -f $DELETE_LIST $TMPFILE >> $OUT 2>&1 +rm $DELETE_LIST +cp $TMPFILE $TMPFILE.sav + +. $cmd_dir/run_e2fscktests/f_h_badnode/expect.1 +0
-0@@ -14,5 +14,5 @@ Pass 4: Checking reference countsPass 5: Checking group summary information
test_filesys: ***** FILE SYSTEM WAS MODIFIED ***** -test_filesys: 47730/100192 files (0.0% non-contiguous), 13551/31745 blocks
+test_filesys: 47730/100192 files (0.0% non-contiguous), 13550/31745 blocks
Exit status is 1tests/f_h_badnode/expect.2 +0
-0@@ -3,5 +3,5 @@ Pass 2: Checking directory structure
Quoted 12 lines
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
-test_filesys: 47730/100192 files (0.0% non-contiguous), 13551/31745 blocks
+test_filesys: 47730/100192 files (0.0% non-contiguous), 13550/31745 blocks
Exit status is 0Signature · 1 line
1.7.3.4
Cheers, Andreas
Signature · 4 lines
Andreas Dilger Lustre Principal Architect Intel High Performance Data Division