Introduce parallel fsck to e2fsck pass1
v3 cover Introduce parallel fsck to e2fsck pass1
Currently it has been popular that single disk could be more than TiB, etc 16Tib with only one single disk, with this trend, one single filesystem could be larger and larger and easily reach PiB with LUN system.
The journal filesystem like ext4 need be offline to do regular check and repair from time to time, however the problem is e2fsck still do this using single thread, this could be challenging at scale for two reasons:
1) even with readahead, IO speed still limits several tens MiB per second.
2) could not utilize CPU cores.
It could be challenging to try multh-threads for all phase of e2fsck, but as first step, we might try this for most time-consuming pass1, according to our benchmarking it cost of 80% time for whole e2fck phase.
Pass1 is trying to scanning all valid inode of filesystem and check it one by one, and the patchset idea is trying to split these to different threads and trying to do this at the same time, we try to merge these inodes and corresponding inode's extent information after threads finish.
To simplify complexity and make it less error-prone, the fix is still serialized, since most of time there will be only minor errors for filesystem, what's important for us is parallel reading and checking.
Here is a benchmarking on our Lustre filesystem with 1.2 PiB OSD ext4 based filesystem:
DDN SFA18KE StorageServer
DCR(DeClustering RAID) with 162 x HGST 10TB NL-SAS Tested Server
A Virtual Machine running on SFA18KE
8 x CPU cores (Xeon(R) Gold 6140)
150GB memory
CentoOS7.7 (Lustre patched kernel)
Created 600 Million x 32K byte files.
Without Patch With Patch thr=64
pass1: 13079.66 488.57 seconds
Total: 15673.33 3188.42
We have 5x total time reduction of total time which is very inspiring.
I've tested the whole patch series using 'make test' of e2fsck itself, and i manually set default threads to 4 which still pass almost of test suite, failure cases are below:
f_h_badroot f_multithread f_multithread_logfile f_multithread_no f_multithread_ok
h_h_badroot failed because out of order checking output, and others are because of extra multiple threads log output.
The other test i've done to verify is:
1) filled a filesystem with different features enabled(encryption, shared xattr)
2) copied the above image out using e2image.
3) using e2fuzz to randomly corrupt above image, and then have another copy of corrupted image.
4) running e2fsck with single thread(without patches) and multiple threads with corrupted image respectively.
5) using valgrind to check any memory leak.
6) compare single thread and multiple threads fix results by mounting two images back and compare the directores and files.(size and name match)
Any review or comments are welcomed!
Thanks you very much!
changelog v2->v3:
1) Parallelize rw_bitmaps
2) Configuration to turn off parallel fsck.(default=on)
3) Add f_multithread_ok
4) Add annotations to e2fsck_struct instead of reshuffle
5) Fix memory leaks
6) Some more extra fixes
Andreas Dilger (2):
e2fsck: fix f_multithread_ok test
e2fsck: misc cleanups for pfsck
Li Xi (18):
e2fsck: add -m option for multithread
e2fsck: copy context when using multi-thread fsck
e2fsck: copy fs when using multi-thread fsck
e2fsck: add assert when copying context
e2fsck: copy bitmaps when copying context
e2fsck: open io-channel when copying fs
e2fsck: create logs for mult-threads
e2fsck: optionally configure one pfsck thread
e2fsck: add start/end group for thread
e2fsck: split groups to different threads
e2fsck: print thread log properly
e2fsck: do not change global variables
e2fsck: optimize the inserting of dir_info_db
e2fsck: merge dir_info after thread finishes
e2fsck: merge icounts after thread finishes
e2fsck: merge dblist after thread finishes
e2fsck: add debug codes for multiple threads
e2fsck: merge fs flags when threads finish
Saranya Muruganandam (2):
e2fsck: propagate number of threads
e2fsck: Annotating fields in e2fsck_struct
Wang Shilong (39):
e2fsck: clear icache when using multi-thread fsck
e2fsck: copy badblocks when copying fs
e2fsck: merge bitmaps after thread completes
e2fsck: rbtree bitmap for dir
e2fsck: merge badblocks after thread finishes
e2fsck: merge counts after threads finish
e2fsck: merge dx_dir_info after threads finish
e2fsck: merge dirs_to_hash when threads finish
e2fsck: merge context flags properly
e2fsck: merge quota context after threads finish
e2fsck: serialize fix operations
e2fsck: move some fixes out of parallel pthreads
e2fsck: split and merge invalid bitmaps
e2fsck: merge EA blocks properly
e2fsck: kickoff mutex lock for block found map
e2fsck: allow admin specify number of threads
e2fsck: adjust number of threads
e2fsck: fix readahead for pfsck of pass1
e2fsck: merge options after threads finish
e2fsck: reset lost_and_found after threads finish
e2fsck: merge extent depth count after threads finish
e2fsck: simplify e2fsck context merging codes
e2fsck: set E2F_FLAG_ALLOC_OK after threads
e2fsck: wait fix thread finish before checking
e2fsck: cleanup e2fsck_pass1_thread_join()
e2fsck: avoid too much memory allocation for pfsck
e2fsck: make default smallest RA size to 1M
ext2fs: parallel bitmap loading
e2fsck: update mmp block in one thread
e2fsck: reset @inodes_to_rebuild if restart
e2fsck: fix build for make rpm
e2fsck: move ext2fs_get_avg_group to rw_bitmaps.c
configure: enable pfsck by default
test: add pfsck test
e2fsck: fix race in ext2fs_read_bitmaps()
e2fsck: fix readahead for pass1 without pfsck
e2fsck: fix memory leaks with pfsck enabled
ext2fs: fix to set tail flags with pfsck enabled
e2fsck: update mmp block race
MCONFIG.in | 1 + configure | 90 +- configure.ac | 26 + e2fsck/Makefile.in | 9 +- e2fsck/dirinfo.c | 238 ++- e2fsck/dx_dirinfo.c | 64 + e2fsck/e2fsck.8.in | 8 +- e2fsck/e2fsck.c | 11 + e2fsck/e2fsck.h | 102 +- e2fsck/logfile.c | 13 +- e2fsck/pass1.c | 1766 ++++++++++++++++++++--- e2fsck/problem.c | 11 + e2fsck/problem.h | 3 + e2fsck/readahead.c | 4 + e2fsck/unix.c | 59 +- e2fsck/util.c | 193 ++- lib/config.h.in | 3 + lib/ext2fs/badblocks.c | 85 +- lib/ext2fs/bitmaps.c | 10 + lib/ext2fs/bitops.h | 2 + lib/ext2fs/blkmap64_rb.c | 65 + lib/ext2fs/bmap64.h | 4 + lib/ext2fs/dblist.c | 38 + lib/ext2fs/ext2_err.et.in | 3 + lib/ext2fs/ext2_io.h | 2 + lib/ext2fs/ext2fs.h | 19 +- lib/ext2fs/ext2fsP.h | 1 - lib/ext2fs/gen_bitmap64.c | 62 + lib/ext2fs/icount.c | 107 ++ lib/ext2fs/openfs.c | 48 +- lib/ext2fs/rw_bitmaps.c | 301 +++- lib/ext2fs/undo_io.c | 19 + lib/ext2fs/unix_io.c | 24 +- lib/support/mkquota.c | 39 + lib/support/quotaio.h | 3 + tests/f_itable_collision/expect.1 | 3 - tests/f_multithread/expect.1 | 25 + tests/f_multithread/expect.2 | 7 + tests/f_multithread/image.gz | 1 + tests/f_multithread/name | 1 + tests/f_multithread/script | 4 + tests/f_multithread_completion/expect.1 | 2 + tests/f_multithread_completion/expect.2 | 23 + tests/f_multithread_completion/image.gz | 1 + tests/f_multithread_completion/name | 1 + tests/f_multithread_completion/script | 4 + tests/f_multithread_logfile/expect.1 | 25 + tests/f_multithread_logfile/image.gz | 1 + tests/f_multithread_logfile/name | 1 + tests/f_multithread_logfile/script | 32 + tests/f_multithread_no/expect.1 | 26 + tests/f_multithread_no/expect.2 | 23 + tests/f_multithread_no/image.gz | 1 + tests/f_multithread_no/name | 1 + tests/f_multithread_no/script | 4 + tests/f_multithread_ok/expect.1 | 8 + tests/f_multithread_ok/image.gz | Bin 0 -> 796311 bytes tests/f_multithread_ok/name | 1 + tests/f_multithread_ok/script | 21 + tests/f_multithread_preen/expect.1 | 11 + tests/f_multithread_preen/expect.2 | 23 + tests/f_multithread_preen/image.gz | 1 + tests/f_multithread_preen/name | 1 + tests/f_multithread_preen/script | 4 + tests/f_multithread_yes/expect.1 | 2 + tests/f_multithread_yes/expect.2 | 23 + tests/f_multithread_yes/image.gz | 1 + tests/f_multithread_yes/name | 1 + tests/f_multithread_yes/script | 4 + tests/test_one.in | 8 + 70 files changed, 3335 insertions(+), 393 deletions(-) create mode 100644 tests/f_multithread/expect.1 create mode 100644 tests/f_multithread/expect.2 create mode 120000 tests/f_multithread/image.gz create mode 100644 tests/f_multithread/name create mode 100644 tests/f_multithread/script create mode 100644 tests/f_multithread_completion/expect.1 create mode 100644 tests/f_multithread_completion/expect.2 create mode 120000 tests/f_multithread_completion/image.gz create mode 100644 tests/f_multithread_completion/name create mode 100644 tests/f_multithread_completion/script create mode 100644 tests/f_multithread_logfile/expect.1 create mode 120000 tests/f_multithread_logfile/image.gz create mode 100644 tests/f_multithread_logfile/name create mode 100644 tests/f_multithread_logfile/script create mode 100644 tests/f_multithread_no/expect.1 create mode 100644 tests/f_multithread_no/expect.2 create mode 120000 tests/f_multithread_no/image.gz create mode 100644 tests/f_multithread_no/name create mode 100644 tests/f_multithread_no/script create mode 100644 tests/f_multithread_ok/expect.1 create mode 100644 tests/f_multithread_ok/image.gz create mode 100644 tests/f_multithread_ok/name create mode 100644 tests/f_multithread_ok/script create mode 100644 tests/f_multithread_preen/expect.1 create mode 100644 tests/f_multithread_preen/expect.2 create mode 120000 tests/f_multithread_preen/image.gz create mode 100644 tests/f_multithread_preen/name create mode 100644 tests/f_multithread_preen/script create mode 100644 tests/f_multithread_yes/expect.1 create mode 100644 tests/f_multithread_yes/expect.2 create mode 120000 tests/f_multithread_yes/image.gz create mode 100644 tests/f_multithread_yes/name create mode 100644 tests/f_multithread_yes/script
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 1/61 e2fsck: add -m option for multithread
From: Li Xi <[email protected]>
-m option is added but no actual functionality is added. This patch only adds the logic that when -m is specified, one of -p/-y/-n options should be specified. And when -m is specified, -C shouldn't be specified and the completion progress report won't be triggered by sending SIGUSR1/SIGUSR2 signals. This simplifies the implementation of multi-thread fsck in the future.
Completion progress support with multi-thread fsck will be added back after multi-thread fsck implementation is finished. Right now, disable it to simplify the implementation of multi-thread fsck.
--- e2fsck/e2fsck.h | 1 + e2fsck/unix.c | 31 ++++++++++++++++++++----- tests/f_multithread/expect.1 | 23 ++++++++++++++++++ tests/f_multithread/expect.2 | 7 ++++++ tests/f_multithread/image.gz | 1 + tests/f_multithread/name | 1 + tests/f_multithread/script | 4 ++++ tests/f_multithread_completion/expect.1 | 2 ++ tests/f_multithread_completion/expect.2 | 23 ++++++++++++++++++ tests/f_multithread_completion/image.gz | 1 + tests/f_multithread_completion/name | 1 + tests/f_multithread_completion/script | 4 ++++ tests/f_multithread_no/expect.1 | 24 +++++++++++++++++++ tests/f_multithread_no/expect.2 | 23 ++++++++++++++++++ tests/f_multithread_no/image.gz | 1 + tests/f_multithread_no/name | 1 + tests/f_multithread_no/script | 4 ++++ tests/f_multithread_preen/expect.1 | 11 +++++++++ tests/f_multithread_preen/expect.2 | 23 ++++++++++++++++++ tests/f_multithread_preen/image.gz | 1 + tests/f_multithread_preen/name | 1 + tests/f_multithread_preen/script | 4 ++++ tests/f_multithread_yes/expect.1 | 2 ++ tests/f_multithread_yes/expect.2 | 23 ++++++++++++++++++ tests/f_multithread_yes/image.gz | 1 + tests/f_multithread_yes/name | 1 + tests/f_multithread_yes/script | 4 ++++ 27 files changed, 217 insertions(+), 6 deletions(-) create mode 100644 tests/f_multithread/expect.1 create mode 100644 tests/f_multithread/expect.2 create mode 120000 tests/f_multithread/image.gz create mode 100644 tests/f_multithread/name create mode 100644 tests/f_multithread/script create mode 100644 tests/f_multithread_completion/expect.1 create mode 100644 tests/f_multithread_completion/expect.2 create mode 120000 tests/f_multithread_completion/image.gz create mode 100644 tests/f_multithread_completion/name create mode 100644 tests/f_multithread_completion/script create mode 100644 tests/f_multithread_no/expect.1 create mode 100644 tests/f_multithread_no/expect.2 create mode 120000 tests/f_multithread_no/image.gz create mode 100644 tests/f_multithread_no/name create mode 100644 tests/f_multithread_no/script create mode 100644 tests/f_multithread_preen/expect.1 create mode 100644 tests/f_multithread_preen/expect.2 create mode 120000 tests/f_multithread_preen/image.gz create mode 100644 tests/f_multithread_preen/name create mode 100644 tests/f_multithread_preen/script create mode 100644 tests/f_multithread_yes/expect.1 create mode 100644 tests/f_multithread_yes/expect.2 create mode 120000 tests/f_multithread_yes/image.gz create mode 100644 tests/f_multithread_yes/name create mode 100644 tests/f_multithread_yes/script
e2fsck/e2fsck.h +1 -0
@@ -177,6 +177,7 @@ struct resource_track {#define E2F_OPT_ICOUNT_FULLMAP 0x20000/* use an array for inode counts */ #define E2F_OPT_UNSHARE_BLOCKS 0x40000#define E2F_OPT_CLEAR_UNINIT 0x80000/* Hack to clear the uninit bit */ +#define E2F_OPT_MULTITHREAD 0x100000/* Use multiple threads to speedup */ /* * E2fsck flags
e2fsck/unix.c +25 -6
@@ -75,13 +75,14 @@ int journal_enable_debug = -1;static void usage (e2fsck_t ctx) {fprintf (stderr, -_ ("Usage: %s [-panyrcdfktvDFV] [-b superblock] [-B blocksize]\n" +_ ("Usage: %s [-pamnyrcdfktvDFV] [-b superblock] [-B blocksize]\n" "\t\t[-l|-L bad_blocks_file] [-C fd] [-j external_journal]\n" "\t\t[-E extended-options] [-z undo_file] device\n" ), ctx->program_name); fprintf(stderr, "%s", _("\nEmergency help:\n" " -p Automatic repair (no questions)\n" + " -m multiple threads to speedup fsck\n" " -n Make no changes to the filesystem\n" " -y Assume \"yes\" to all questions\n" " -c Check for bad blocks and add them to the badblock list\n"
@@ -847,7 +848,8 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx) phys_mem_kb =get_memory_size () /1024 ; ctx->readahead_kb = ~0ULL ; -while ((c =getopt (argc, argv,"panyrcC:B:dE:fvtFVM:b:I:j:P:l:L:N:SsDkz:" )) !=EOF ) + +while ((c =getopt (argc, argv,"pamnyrcC:B:dE:fvtFVM:b:I:j:P:l:L:N:SsDkz:" )) !=EOF )switch (c) {case 'C' : ctx->progress = e2fsck_update_progress;
@@ -888,6 +890,9 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx) } ctx->options |=E2F_OPT_PREEN ;break ; +case 'm' : + ctx->options |=E2F_OPT_MULTITHREAD ; +break ;case 'n' :if (ctx->options & (E2F_OPT_YES |E2F_OPT_PREEN )) goto conflict_opt;
@@ -1006,6 +1011,18 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx) _("The -n and -l/-L options are incompatible."));fatal_error (ctx,0 ); } +if (ctx->options &E2F_OPT_MULTITHREAD ) { +if ((ctx->options & (E2F_OPT_YES |E2F_OPT_NO |E2F_OPT_PREEN )) ==0 ) { +com_err (ctx->program_name,0 ,"%s" , +_ ("The -m option should be used together with one of -p/-y/-n options." )); +fatal_error (ctx,0 ); + } +if (ctx->progress) { +com_err (ctx->program_name,0 ,"%s" , +_ ("Only one of the options -C or -m may be specified." )); +fatal_error (ctx,0 ); + } + }if (ctx->options &E2F_OPT_NO ) ctx->options |=E2F_OPT_READONLY ;
@@ -1112,10 +1129,12 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx)#ifdef SA_RESTART sa.sa_flags =SA_RESTART ;#endif - sa.sa_handler = signal_progress_on; -sigaction (SIGUSR1 , &sa,0 ); - sa.sa_handler = signal_progress_off; -sigaction (SIGUSR2 , &sa,0 ); +if ((ctx->options &E2F_OPT_MULTITHREAD ) ==0 ) { + sa.sa_handler = signal_progress_on; +sigaction (SIGUSR1 , &sa,0 ); + sa.sa_handler = signal_progress_off; +sigaction (SIGUSR2 , &sa,0 ); + }#endif /* Update our PATH to include /sbin if we need to run badblocks */
tests/f_multithread/expect.1 new file +23 -0
@@ -0,0 +1,23 @@ +ext2fs_open2: Bad magic number in super-block +../e2fsck/e2fsck: Superblock invalid, trying backup blocks... +Pass 1: Checking inodes, blocks, and sizes +Pass 2: Checking directory structure +Pass 3: Checking directory connectivity +Pass 4: Checking reference counts +Pass 5: Checking group summary information +Free blocks count wrong for group #0 (7987, counted=7982). +Fix? yes + +Free blocks count wrong (11602, counted=11597). +Fix? yes + +Free inodes count wrong for group #0 (1493, counted=1488). +Fix? yes + +Free inodes count wrong (2997, counted=2992). +Fix? yes + + +test_filesys: ***** FILE SYSTEM WAS MODIFIED ***** +test_filesys: 16/3008 files (0.0% non-contiguous), 403/12000 blocks +Exit status is 1
tests/f_multithread/expect.2 new file +7 -0
@@ -0,0 +1,7 @@ +Pass 1: Checking inodes, blocks, and sizes +Pass 2: Checking directory structure +Pass 3: Checking directory connectivity +Pass 4: Checking reference counts +Pass 5: Checking group summary information +test_filesys: 16/3008 files (0.0% non-contiguous), 403/12000 blocks +Exit status is 0
tests/f_multithread/image.gz new file +1 -0
@@ -0,0 +1 @@ +../f_zero_super/image.gz
tests/f_multithread/name new file +1 -0
@@ -0,0 +1 @@ +test "e2fsck -m" option
tests/f_multithread/script new file +4 -0
@@ -0,0 +1,4 @@ +FSCK_OPT="-fy -m" +SECOND_FSCK_OPT=-yf + +. $cmd_dir/run_e2fsck
tests/f_multithread_completion/expect.1 new file +2 -0
@@ -0,0 +1,2 @@ +../e2fsck/e2fsck: Only one of the options -C or -m may be specified. +Exit status is 8
tests/f_multithread_completion/expect.2 new file +23 -0
@@ -0,0 +1,23 @@ +ext2fs_open2: Bad magic number in super-block +../e2fsck/e2fsck: Superblock invalid, trying backup blocks... +Pass 1: Checking inodes, blocks, and sizes +Pass 2: Checking directory structure +Pass 3: Checking directory connectivity +Pass 4: Checking reference counts +Pass 5: Checking group summary information +Free blocks count wrong for group #0 (7987, counted=7982). +Fix? yes + +Free blocks count wrong (11602, counted=11597). +Fix? yes + +Free inodes count wrong for group #0 (1493, counted=1488). +Fix? yes + +Free inodes count wrong (2997, counted=2992). +Fix? yes + + +test_filesys: ***** FILE SYSTEM WAS MODIFIED ***** +test_filesys: 16/3008 files (0.0% non-contiguous), 403/12000 blocks +Exit status is 1
tests/f_multithread_completion/image.gz new file +1 -0
@@ -0,0 +1 @@ +../f_zero_super/image.gz
tests/f_multithread_completion/name new file +1 -0
@@ -0,0 +1 @@ +test "e2fsck -m" option conflicts with "-C"
tests/f_multithread_completion/script new file +4 -0
@@ -0,0 +1,4 @@ +FSCK_OPT="-fy -m -C 1" +SECOND_FSCK_OPT=-yf + +. $cmd_dir/run_e2fsck
tests/f_multithread_no/expect.1 new file +24 -0
@@ -0,0 +1,24 @@ +ext2fs_open2: Bad magic number in super-block +../e2fsck/e2fsck: Superblock invalid, trying backup blocks... +Pass 1: Checking inodes, blocks, and sizes +Pass 2: Checking directory structure +Pass 3: Checking directory connectivity +Pass 4: Checking reference counts +Pass 5: Checking group summary information +Free blocks count wrong for group #0 (7987, counted=7982). +Fix? no + +Free blocks count wrong (11602, counted=11597). +Fix? no + +Free inodes count wrong for group #0 (1493, counted=1488). +Fix? no + +Free inodes count wrong (2997, counted=2992). +Fix? no + + +test_filesys: ********** WARNING: Filesystem still has errors ********** + +test_filesys: 11/3008 files (0.0% non-contiguous), 398/12000 blocks +Exit status is 4
tests/f_multithread_no/expect.2 new file +23 -0
@@ -0,0 +1,23 @@ +ext2fs_open2: Bad magic number in super-block +../e2fsck/e2fsck: Superblock invalid, trying backup blocks... +Pass 1: Checking inodes, blocks, and sizes +Pass 2: Checking directory structure +Pass 3: Checking directory connectivity +Pass 4: Checking reference counts +Pass 5: Checking group summary information +Free blocks count wrong for group #0 (7987, counted=7982). +Fix? yes + +Free blocks count wrong (11602, counted=11597). +Fix? yes + +Free inodes count wrong for group #0 (1493, counted=1488). +Fix? yes + +Free inodes count wrong (2997, counted=2992). +Fix? yes + + +test_filesys: ***** FILE SYSTEM WAS MODIFIED ***** +test_filesys: 16/3008 files (0.0% non-contiguous), 403/12000 blocks +Exit status is 1
tests/f_multithread_no/image.gz new file +1 -0
@@ -0,0 +1 @@ +../f_zero_super/image.gz
tests/f_multithread_no/name new file +1 -0
@@ -0,0 +1 @@ +test "e2fsck -m" option works with "-n"
tests/f_multithread_no/script new file +4 -0
@@ -0,0 +1,4 @@ +FSCK_OPT="-fn -m" +SECOND_FSCK_OPT=-yf + +. $cmd_dir/run_e2fsck
tests/f_multithread_preen/expect.1 new file +11 -0
@@ -0,0 +1,11 @@ +../e2fsck/e2fsck: Bad magic number in super-block while trying to open test.img +test_filesys: +The superblock could not be read or does not describe a valid ext2/ext3/ext4 +filesystem. If the device is valid and it really contains an ext2/ext3/ext4 +filesystem (and not swap or ufs or something else), then the superblock +is corrupt, and you might try running e2fsck with an alternate superblock: + e2fsck -b 8193 <device> + or + e2fsck -b 32768 <device> + +Exit status is 8
tests/f_multithread_preen/expect.2 new file +23 -0
@@ -0,0 +1,23 @@ +ext2fs_open2: Bad magic number in super-block +../e2fsck/e2fsck: Superblock invalid, trying backup blocks... +Pass 1: Checking inodes, blocks, and sizes +Pass 2: Checking directory structure +Pass 3: Checking directory connectivity +Pass 4: Checking reference counts +Pass 5: Checking group summary information +Free blocks count wrong for group #0 (7987, counted=7982). +Fix? yes + +Free blocks count wrong (11602, counted=11597). +Fix? yes + +Free inodes count wrong for group #0 (1493, counted=1488). +Fix? yes + +Free inodes count wrong (2997, counted=2992). +Fix? yes + + +test_filesys: ***** FILE SYSTEM WAS MODIFIED ***** +test_filesys: 16/3008 files (0.0% non-contiguous), 403/12000 blocks +Exit status is 1
tests/f_multithread_preen/image.gz new file +1 -0
@@ -0,0 +1 @@ +../f_zero_super/image.gz
tests/f_multithread_preen/name new file +1 -0
@@ -0,0 +1 @@ +test "e2fsck -m" option works with "-p"
tests/f_multithread_preen/script new file +4 -0
@@ -0,0 +1,4 @@ +FSCK_OPT="-fp -m" +SECOND_FSCK_OPT=-yf + +. $cmd_dir/run_e2fsck
tests/f_multithread_yes/expect.1 new file +2 -0
@@ -0,0 +1,2 @@ +../e2fsck/e2fsck: The -m option should be used together with one of -p/-y/-n options. +Exit status is 8
tests/f_multithread_yes/expect.2 new file +23 -0
@@ -0,0 +1,23 @@ +ext2fs_open2: Bad magic number in super-block +../e2fsck/e2fsck: Superblock invalid, trying backup blocks... +Pass 1: Checking inodes, blocks, and sizes +Pass 2: Checking directory structure +Pass 3: Checking directory connectivity +Pass 4: Checking reference counts +Pass 5: Checking group summary information +Free blocks count wrong for group #0 (7987, counted=7982). +Fix? yes + +Free blocks count wrong (11602, counted=11597). +Fix? yes + +Free inodes count wrong for group #0 (1493, counted=1488). +Fix? yes + +Free inodes count wrong (2997, counted=2992). +Fix? yes + + +test_filesys: ***** FILE SYSTEM WAS MODIFIED ***** +test_filesys: 16/3008 files (0.0% non-contiguous), 403/12000 blocks +Exit status is 1
tests/f_multithread_yes/image.gz new file +1 -0
@@ -0,0 +1 @@ +../f_zero_super/image.gz
tests/f_multithread_yes/name new file +1 -0
@@ -0,0 +1 @@ +test "e2fsck -m" option works with "-y"
tests/f_multithread_yes/script new file +4 -0
@@ -0,0 +1,4 @@ +FSCK_OPT="-f -m" +SECOND_FSCK_OPT=-yf + +. $cmd_dir/run_e2fsck
Signature · 1 line
2.29.2.299.gdc1121823c-goog
Re: [RFC PATCH v3 01/61] e2fsck: add -m option for multithread
This patch looks good to me. You can add -
Quoted 514 lines · Saranya Muruganandam
From: Li Xi <[email protected]>
-m option is added but no actual functionality is added. This patch only adds the logic that when -m is specified, one of
(quoted diff) +0
-2-p/-y/-n options should be specified. And when -m is specified, -C shouldn't be specified and the completion progress report won'tbe triggered by sending SIGUSR1/SIGUSR2 signals. This simplifies the implementation of multi-thread fsck in the future.
Completion progress support with multi-thread fsck will be added back after multi-thread fsck implementation is finished. Right now, disable it to simplify the implementation of multi-thread fsck.
Signed-off-by Li Xi <[email protected]>Signed-off-by Wang Shilong <[email protected]>Signed-off-by Saranya Muruganandam <[email protected]>--- e2fsck/e2fsck.h | 1 + e2fsck/unix.c | 31 ++++++++++++++++++++----- tests/f_multithread/expect.1 | 23 ++++++++++++++++++ tests/f_multithread/expect.2 | 7 ++++++ tests/f_multithread/image.gz | 1 + tests/f_multithread/name | 1 + tests/f_multithread/script | 4 ++++ tests/f_multithread_completion/expect.1 | 2 ++ tests/f_multithread_completion/expect.2 | 23 ++++++++++++++++++ tests/f_multithread_completion/image.gz | 1 + tests/f_multithread_completion/name | 1 + tests/f_multithread_completion/script | 4 ++++ tests/f_multithread_no/expect.1 | 24 +++++++++++++++++++ tests/f_multithread_no/expect.2 | 23 ++++++++++++++++++ tests/f_multithread_no/image.gz | 1 + tests/f_multithread_no/name | 1 + tests/f_multithread_no/script | 4 ++++ tests/f_multithread_preen/expect.1 | 11 +++++++++ tests/f_multithread_preen/expect.2 | 23 ++++++++++++++++++ tests/f_multithread_preen/image.gz | 1 + tests/f_multithread_preen/name | 1 + tests/f_multithread_preen/script | 4 ++++ tests/f_multithread_yes/expect.1 | 2 ++ tests/f_multithread_yes/expect.2 | 23 ++++++++++++++++++ tests/f_multithread_yes/image.gz | 1 + tests/f_multithread_yes/name | 1 + tests/f_multithread_yes/script | 4 ++++ 27 files changed, 217 insertions(+), 6 deletions(-) create mode 100644 tests/f_multithread/expect.1 create mode 100644 tests/f_multithread/expect.2 create mode 120000 tests/f_multithread/image.gz create mode 100644 tests/f_multithread/name create mode 100644 tests/f_multithread/script create mode 100644 tests/f_multithread_completion/expect.1 create mode 100644 tests/f_multithread_completion/expect.2 create mode 120000 tests/f_multithread_completion/image.gz create mode 100644 tests/f_multithread_completion/name create mode 100644 tests/f_multithread_completion/script create mode 100644 tests/f_multithread_no/expect.1 create mode 100644 tests/f_multithread_no/expect.2 create mode 120000 tests/f_multithread_no/image.gz create mode 100644 tests/f_multithread_no/name create mode 100644 tests/f_multithread_no/script create mode 100644 tests/f_multithread_preen/expect.1 create mode 100644 tests/f_multithread_preen/expect.2 create mode 120000 tests/f_multithread_preen/image.gz create mode 100644 tests/f_multithread_preen/name create mode 100644 tests/f_multithread_preen/script create mode 100644 tests/f_multithread_yes/expect.1 create mode 100644 tests/f_multithread_yes/expect.2 create mode 120000 tests/f_multithread_yes/image.gz create mode 100644 tests/f_multithread_yes/name create mode 100644 tests/f_multithread_yes/scripte2fsck/e2fsck.h +1
-0@@ -177,6 +177,7 @@ struct resource_track {#define E2F_OPT_ICOUNT_FULLMAP 0x20000/* use an array for inode counts */ #define E2F_OPT_UNSHARE_BLOCKS 0x40000#define E2F_OPT_CLEAR_UNINIT 0x80000/* Hack to clear the uninit bit */ +#define E2F_OPT_MULTITHREAD 0x100000/* Use multiple threads to speedup */ /* * E2fsck flags e2fsck/unix.c +25
-6@@ -75,13 +75,14 @@ int journal_enable_debug = -1;static void usage (e2fsck_t ctx) {fprintf (stderr, -_ ("Usage: %s [-panyrcdfktvDFV] [-b superblock] [-B blocksize]\n" +_ ("Usage: %s [-pamnyrcdfktvDFV] [-b superblock] [-B blocksize]\n" "\t\t[-l|-L bad_blocks_file] [-C fd] [-j external_journal]\n" "\t\t[-E extended-options] [-z undo_file] device\n" ), ctx->program_name); fprintf(stderr, "%s", _("\nEmergency help:\n" " -p Automatic repair (no questions)\n" + " -m multiple threads to speedup fsck\n" " -n Make no changes to the filesystem\n" " -y Assume \"yes\" to all questions\n" " -c Check for bad blocks and add them to the badblock list\n"@@ -847,7 +848,8 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx) phys_mem_kb =get_memory_size () /1024 ; ctx->readahead_kb = ~0ULL ; -while ((c =getopt (argc, argv,"panyrcC:B:dE:fvtFVM:b:I:j:P:l:L:N:SsDkz:" )) !=EOF ) + +while ((c =getopt (argc, argv,"pamnyrcC:B:dE:fvtFVM:b:I:j:P:l:L:N:SsDkz:" )) !=EOF )switch (c) {case 'C' : ctx->progress = e2fsck_update_progress;@@ -888,6 +890,9 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx) } ctx->options |=E2F_OPT_PREEN ;break ; +case 'm' : + ctx->options |=E2F_OPT_MULTITHREAD ; +break ;case 'n' :if (ctx->options & (E2F_OPT_YES |E2F_OPT_PREEN )) goto conflict_opt;@@ -1006,6 +1011,18 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx) _("The -n and -l/-L options are incompatible."));fatal_error (ctx,0 ); } +if (ctx->options &E2F_OPT_MULTITHREAD ) { +if ((ctx->options & (E2F_OPT_YES |E2F_OPT_NO |E2F_OPT_PREEN )) ==0 ) { +com_err (ctx->program_name,0 ,"%s" , +_ ("The -m option should be used together with one of -p/-y/-n options." )); +fatal_error (ctx,0 ); + } +if (ctx->progress) { +com_err (ctx->program_name,0 ,"%s" , +_ ("Only one of the options -C or -m may be specified." )); +fatal_error (ctx,0 ); + } + }if (ctx->options &E2F_OPT_NO ) ctx->options |=E2F_OPT_READONLY ;@@ -1112,10 +1129,12 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx)#ifdef SA_RESTART sa.sa_flags =SA_RESTART ;#endif - sa.sa_handler = signal_progress_on; -sigaction (SIGUSR1 , &sa,0 ); - sa.sa_handler = signal_progress_off; -sigaction (SIGUSR2 , &sa,0 ); +if ((ctx->options &E2F_OPT_MULTITHREAD ) ==0 ) { + sa.sa_handler = signal_progress_on; +sigaction (SIGUSR1 , &sa,0 ); + sa.sa_handler = signal_progress_off; +sigaction (SIGUSR2 , &sa,0 ); + }#endif /* Update our PATH to include /sbin if we need to run badblocks */ tests/f_multithread/expect.1 new file +23
-0@@ -0,0 +1,23 @@ +ext2fs_open2: Bad magic number in super-block +../e2fsck/e2fsck: Superblock invalid, trying backup blocks... +Pass 1: Checking inodes, blocks, and sizes +Pass 2: Checking directory structure +Pass 3: Checking directory connectivity +Pass 4: Checking reference counts +Pass 5: Checking group summary information +Free blocks count wrong for group #0 (7987, counted=7982). +Fix? yes + +Free blocks count wrong (11602, counted=11597). +Fix? yes + +Free inodes count wrong for group #0 (1493, counted=1488). +Fix? yes + +Free inodes count wrong (2997, counted=2992). +Fix? yes + + +test_filesys: ***** FILE SYSTEM WAS MODIFIED ***** +test_filesys: 16/3008 files (0.0% non-contiguous), 403/12000 blocks +Exit status is 1tests/f_multithread/expect.2 new file +7
-0@@ -0,0 +1,7 @@ +Pass 1: Checking inodes, blocks, and sizes +Pass 2: Checking directory structure +Pass 3: Checking directory connectivity +Pass 4: Checking reference counts +Pass 5: Checking group summary information +test_filesys: 16/3008 files (0.0% non-contiguous), 403/12000 blocks +Exit status is 0tests/f_multithread/image.gz new file +1
-0@@ -0,0 +1 @@ +../f_zero_super/image.gztests/f_multithread/name new file +1
-0@@ -0,0 +1 @@ +test "e2fsck -m" optiontests/f_multithread/script new file +4
-0@@ -0,0 +1,4 @@ +FSCK_OPT="-fy -m" +SECOND_FSCK_OPT=-yf + +. $cmd_dir/run_e2fscktests/f_multithread_completion/expect.1 new file +2
-0@@ -0,0 +1,2 @@ +../e2fsck/e2fsck: Only one of the options -C or -m may be specified. +Exit status is 8tests/f_multithread_completion/expect.2 new file +23
-0@@ -0,0 +1,23 @@ +ext2fs_open2: Bad magic number in super-block +../e2fsck/e2fsck: Superblock invalid, trying backup blocks... +Pass 1: Checking inodes, blocks, and sizes +Pass 2: Checking directory structure +Pass 3: Checking directory connectivity +Pass 4: Checking reference counts +Pass 5: Checking group summary information +Free blocks count wrong for group #0 (7987, counted=7982). +Fix? yes + +Free blocks count wrong (11602, counted=11597). +Fix? yes + +Free inodes count wrong for group #0 (1493, counted=1488). +Fix? yes + +Free inodes count wrong (2997, counted=2992). +Fix? yes + + +test_filesys: ***** FILE SYSTEM WAS MODIFIED ***** +test_filesys: 16/3008 files (0.0% non-contiguous), 403/12000 blocks +Exit status is 1tests/f_multithread_completion/image.gz new file +1
-0@@ -0,0 +1 @@ +../f_zero_super/image.gztests/f_multithread_completion/name new file +1
-0@@ -0,0 +1 @@ +test "e2fsck -m" option conflicts with "-C"tests/f_multithread_completion/script new file +4
-0@@ -0,0 +1,4 @@ +FSCK_OPT="-fy -m -C 1" +SECOND_FSCK_OPT=-yf + +. $cmd_dir/run_e2fscktests/f_multithread_no/expect.1 new file +24
-0@@ -0,0 +1,24 @@ +ext2fs_open2: Bad magic number in super-block +../e2fsck/e2fsck: Superblock invalid, trying backup blocks... +Pass 1: Checking inodes, blocks, and sizes +Pass 2: Checking directory structure +Pass 3: Checking directory connectivity +Pass 4: Checking reference counts +Pass 5: Checking group summary information +Free blocks count wrong for group #0 (7987, counted=7982). +Fix? no + +Free blocks count wrong (11602, counted=11597). +Fix? no + +Free inodes count wrong for group #0 (1493, counted=1488). +Fix? no + +Free inodes count wrong (2997, counted=2992). +Fix? no + + +test_filesys: ********** WARNING: Filesystem still has errors ********** + +test_filesys: 11/3008 files (0.0% non-contiguous), 398/12000 blocks +Exit status is 4tests/f_multithread_no/expect.2 new file +23
-0@@ -0,0 +1,23 @@ +ext2fs_open2: Bad magic number in super-block +../e2fsck/e2fsck: Superblock invalid, trying backup blocks... +Pass 1: Checking inodes, blocks, and sizes +Pass 2: Checking directory structure +Pass 3: Checking directory connectivity +Pass 4: Checking reference counts +Pass 5: Checking group summary information +Free blocks count wrong for group #0 (7987, counted=7982). +Fix? yes + +Free blocks count wrong (11602, counted=11597). +Fix? yes + +Free inodes count wrong for group #0 (1493, counted=1488). +Fix? yes + +Free inodes count wrong (2997, counted=2992). +Fix? yes + + +test_filesys: ***** FILE SYSTEM WAS MODIFIED ***** +test_filesys: 16/3008 files (0.0% non-contiguous), 403/12000 blocks +Exit status is 1tests/f_multithread_no/image.gz new file +1
-0@@ -0,0 +1 @@ +../f_zero_super/image.gztests/f_multithread_no/name new file +1
-0@@ -0,0 +1 @@ +test "e2fsck -m" option works with "-n"tests/f_multithread_no/script new file +4
-0@@ -0,0 +1,4 @@ +FSCK_OPT="-fn -m" +SECOND_FSCK_OPT=-yf + +. $cmd_dir/run_e2fscktests/f_multithread_preen/expect.1 new file +11
-0@@ -0,0 +1,11 @@ +../e2fsck/e2fsck: Bad magic number in super-block while trying to open test.img +test_filesys: +The superblock could not be read or does not describe a valid ext2/ext3/ext4 +filesystem. If the device is valid and it really contains an ext2/ext3/ext4 +filesystem (and not swap or ufs or something else), then the superblock +is corrupt, and you might try running e2fsck with an alternate superblock: + e2fsck -b 8193 <device> + or + e2fsck -b 32768 <device> + +Exit status is 8tests/f_multithread_preen/expect.2 new file +23
-0@@ -0,0 +1,23 @@ +ext2fs_open2: Bad magic number in super-block +../e2fsck/e2fsck: Superblock invalid, trying backup blocks... +Pass 1: Checking inodes, blocks, and sizes +Pass 2: Checking directory structure +Pass 3: Checking directory connectivity +Pass 4: Checking reference counts +Pass 5: Checking group summary information +Free blocks count wrong for group #0 (7987, counted=7982). +Fix? yes + +Free blocks count wrong (11602, counted=11597). +Fix? yes + +Free inodes count wrong for group #0 (1493, counted=1488). +Fix? yes + +Free inodes count wrong (2997, counted=2992). +Fix? yes + + +test_filesys: ***** FILE SYSTEM WAS MODIFIED ***** +test_filesys: 16/3008 files (0.0% non-contiguous), 403/12000 blocks +Exit status is 1tests/f_multithread_preen/image.gz new file +1
-0@@ -0,0 +1 @@ +../f_zero_super/image.gztests/f_multithread_preen/name new file +1
-0@@ -0,0 +1 @@ +test "e2fsck -m" option works with "-p"tests/f_multithread_preen/script new file +4
-0@@ -0,0 +1,4 @@ +FSCK_OPT="-fp -m" +SECOND_FSCK_OPT=-yf + +. $cmd_dir/run_e2fscktests/f_multithread_yes/expect.1 new file +2
-0@@ -0,0 +1,2 @@ +../e2fsck/e2fsck: The -m option should be used together with one of -p/-y/-n options. +Exit status is 8tests/f_multithread_yes/expect.2 new file +23
-0@@ -0,0 +1,23 @@ +ext2fs_open2: Bad magic number in super-block +../e2fsck/e2fsck: Superblock invalid, trying backup blocks... +Pass 1: Checking inodes, blocks, and sizes +Pass 2: Checking directory structure +Pass 3: Checking directory connectivity +Pass 4: Checking reference counts +Pass 5: Checking group summary information +Free blocks count wrong for group #0 (7987, counted=7982). +Fix? yes + +Free blocks count wrong (11602, counted=11597). +Fix? yes + +Free inodes count wrong for group #0 (1493, counted=1488). +Fix? yes + +Free inodes count wrong (2997, counted=2992). +Fix? yes + + +test_filesys: ***** FILE SYSTEM WAS MODIFIED ***** +test_filesys: 16/3008 files (0.0% non-contiguous), 403/12000 blocks +Exit status is 1tests/f_multithread_yes/image.gz new file +1
-0@@ -0,0 +1 @@ +../f_zero_super/image.gztests/f_multithread_yes/name new file +1
-0@@ -0,0 +1 @@ +test "e2fsck -m" option works with "-y"tests/f_multithread_yes/script new file +4
-1@@ -0,0 +1,4 @@ +FSCK_OPT="-f -m" +SECOND_FSCK_OPT=-yf + +. $cmd_dir/run_e2fsck --2.29.2.299.gdc1121823c-goog
Re: [RFC PATCH v3 01/61] e2fsck: add -m option for multithread
In reply to Saranya MuruganandamQuoted 9 lines · Saranya Muruganandam
From: Li Xi <[email protected]>
-m option is added but no actual functionality is added. This patch only adds the logic that when -m is specified, one of
(quoted diff) +0
-2-p/-y/-n options should be specified. And when -m is specified, -C shouldn't be specified and the completion progress report won'tbe triggered by sending SIGUSR1/SIGUSR2 signals. This simplifies the implementation of multi-thread fsck in the future.
Quoted 7 lines
Completion progress support with multi-thread fsck will be added back after multi-thread fsck implementation is finished. Right now, disable it to simplify the implementation of multi-thread fsck.
Signed-off-by Li Xi <[email protected]>Signed-off-by Wang Shilong <[email protected]>Signed-off-by Saranya Muruganandam <[email protected]>
I'm a bit surprised the changes to the e2fsck man page aren't inclulded in this patch. I see the man page changes were added in "e2fsck: misc cleanups for pfsck", which I would have merged into other patches, but that's a bit of a nit-pick. (The way I normally do these sorts of changes is to expert the patches using "git format-patch", and then I'll use an editor to move patch hunks around, and then apply them all using "git am". Or I'll using something like the "guilt" program, which is essentially quilt for git patch series.)
(Don't worry about making changes for this, unless you're going to be making lots of other changes to the patch series.)
- Ted
RFC PATCH v3 2/61 e2fsck: copy context when using multi-thread fsck
In reply to Saranya MuruganandamFrom: Li Xi <[email protected]>
This patch only copy the context to a new one when -m is enabled. It doesn't actually start any thread. When pass1 test finishes, the new context is copied back to the original context.
Since the signal handler only changes the original context, so add global_ctx in "struct e2fsck_struct" and use that to check whether there is any signal of canceling.
This patch handles the long jump properly so that all the existing tests can be passed even the context has been copied. Otherwise, test f_expisize_ea_del would fail when aborting.
--- e2fsck/pass1.c | 114 +++++++++++++++++++++++++++++++++++++++++++++---- e2fsck/unix.c | 1 + 2 files changed, 107 insertions(+), 8 deletions(-)
e2fsck/pass1.c +106 -8
@@ -1144,7 +1144,22 @@ static int quota_inum_is_reserved(ext2_filsys fs, ext2_ino_t ino)return 0 ; } -void e2fsck_pass1 (e2fsck_t ctx) +static int e2fsck_should_abort (e2fsck_t ctx) +{ +e2fsck_t global_ctx; + +if (ctx->flags &E2F_FLAG_SIGNAL_MASK ) +return 1 ; + +if (ctx->global_ctx) { + global_ctx = ctx->global_ctx; +if (global_ctx->flags &E2F_FLAG_SIGNAL_MASK ) +return 1 ; + } +return 0 ; +} + +void e2fsck_pass1_thread (e2fsck_t ctx) {int i;__u64 max_sizes;
@@ -1360,7 +1375,7 @@ void e2fsck_pass1(e2fsck_t ctx)if (ino > ino_threshold)pass1_readahead (ctx, &ra_group, &ino_threshold);ehandler_operation (old_op); -if (ctx->flags &E2F_FLAG_SIGNAL_MASK ) +if (e2fsck_should_abort (ctx)) goto endit;if (pctx.errcode ==EXT2_ET_BAD_BLOCK_IN_INODE_TABLE ) {/*
@@ -1955,7 +1970,7 @@ void e2fsck_pass1(e2fsck_t ctx)if (process_inode_count >= ctx->process_inode_size) {process_inodes (ctx, block_buf); -if (ctx->flags &E2F_FLAG_SIGNAL_MASK ) +if (e2fsck_should_abort (ctx)) goto endit; } }
@@ -2068,6 +2083,89 @@ endit:else ctx->invalid_bitmaps++; } + +static errcode_t e2fsck_pass1_thread_prepare (e2fsck_t global_ctx,e2fsck_t *thread_ctx) +{ +errcode_t retval; +e2fsck_t thread_context; + + retval =ext2fs_get_mem (sizeof (struct e2fsck_struct ), &thread_context); +if (retval) { +com_err (global_ctx->program_name, retval,"while allocating memory" ); +return retval; + } +memcpy (thread_context, global_ctx,sizeof (struct e2fsck_struct )); + thread_context->fs->priv_data = thread_context; + thread_context->global_ctx = global_ctx; + + *thread_ctx = thread_context; +return 0 ; +} + +static int e2fsck_pass1_thread_join (e2fsck_t global_ctx,e2fsck_t thread_ctx) +{ +int flags = global_ctx->flags; +#ifdef HAVE_SETJMP_H +jmp_buf old_jmp; + +memcpy (old_jmp, global_ctx->abort_loc,sizeof (jmp_buf)); +#endif +memcpy (global_ctx, thread_ctx,sizeof (struct e2fsck_struct )); +#ifdef HAVE_SETJMP_H +memcpy (global_ctx->abort_loc, old_jmp,sizeof (jmp_buf)); +#endif +/* Keep the global singal flags*/ + global_ctx->flags |= (flags &E2F_FLAG_SIGNAL_MASK ) | + (global_ctx->flags &E2F_FLAG_SIGNAL_MASK ); + + global_ctx->fs->priv_data = global_ctx; +ext2fs_free_mem (&thread_ctx); +return 0 ; +} + +void e2fsck_pass1_multithread (e2fsck_t ctx) +{ +errcode_t retval; +e2fsck_t thread_ctx; + + retval =e2fsck_pass1_thread_prepare (ctx, &thread_ctx); +if (retval) { +com_err (ctx->program_name,0 , +_ ("while preparing pass1 thread\n" )); + ctx->flags |=E2F_FLAG_ABORT ; +return ; + } + +#ifdef HAVE_SETJMP_H +/* +* When fatal_error() happens, jump to here. The thread +* context's flags will be saved, but its abort_loc will +* be overwritten by original jump buffer for the later +* tests. +*/ +if (setjmp (thread_ctx->abort_loc)) { + thread_ctx->flags &= ~E2F_FLAG_SETJMP_OK ; +e2fsck_pass1_thread_join (ctx, thread_ctx); +return ; + } + thread_ctx->flags |=E2F_FLAG_SETJMP_OK ; +#endif + +e2fsck_pass1_thread (thread_ctx); + retval =e2fsck_pass1_thread_join (ctx, thread_ctx); +if (retval) { +com_err (ctx->program_name,0 , +_ ("while joining pass1 thread\n" )); + ctx->flags |=E2F_FLAG_ABORT ; +return ; + } +} + +void e2fsck_pass1 (e2fsck_t ctx) +{ +e2fsck_pass1_multithread (ctx); +} +#undef FINISH_INODE_LOOP/*
@@ -2130,7 +2228,7 @@ static void process_inodes(e2fsck_t ctx, char *block_buf)ehandler_operation (buf);check_blocks (ctx, &pctx, block_buf, &inodes_to_process[i].ea_ibody_quota); -if (ctx->flags &E2F_FLAG_SIGNAL_MASK ) +if (e2fsck_should_abort (ctx))break ; } ctx->stashed_inode = old_stashed_inode;
@@ -3300,7 +3398,7 @@ static void check_blocks(e2fsck_t ctx, struct problem_context *pctx, inlinedata_fs =ext2fs_has_feature_inline_data (ctx->fs->super);if (check_ext_attr (ctx, pctx, block_buf, &ea_block_quota)) { -if (ctx->flags &E2F_FLAG_SIGNAL_MASK ) +if (e2fsck_should_abort (ctx)) goto out; pb.num_blocks +=EXT2FS_B2C (ctx->fs, ea_block_quota.blocks); }
@@ -3355,7 +3453,7 @@ static void check_blocks(e2fsck_t ctx, struct problem_context *pctx, }end_problem_latch (ctx,PR_LATCH_BLOCK );end_problem_latch (ctx,PR_LATCH_TOOBIG ); -if (ctx->flags &E2F_FLAG_SIGNAL_MASK ) +if (e2fsck_should_abort (ctx)) goto out;if (pctx->errcode)fix_problem (ctx,PR_1_BLOCK_ITERATE , pctx);
@@ -3836,7 +3934,7 @@ static int process_bad_block(ext2_filsys fs, *block_nr =0 ;return BLOCK_CHANGED ; } -if (ctx->flags &E2F_FLAG_SIGNAL_MASK ) +if (e2fsck_should_abort (ctx))return BLOCK_ABORT ; }else mark_block_used (ctx, blk);
@@ -3933,7 +4031,7 @@ static int process_bad_block(ext2_filsys fs, *block_nr =0 ;return BLOCK_CHANGED ; } -if (ctx->flags &E2F_FLAG_SIGNAL_MASK ) +if (e2fsck_should_abort (ctx))return BLOCK_ABORT ;return 0 ; }
e2fsck/unix.c +1 -0
@@ -1445,6 +1445,7 @@ int main (int argc, char *argv[]) }reserve_stdio_fds (); + ctx->global_ctx =NULL ;set_up_logging (ctx);if (ctx->logf) {int i;
Signature · 1 line
2.29.2.299.gdc1121823c-goog
Re: [RFC PATCH v3 02/61] e2fsck: copy context when using multi-thread fsck
Thanks for the patch. I noticed that this patch doesn't compile. It's because it needs "global_ctx" to be present in the e2fsck_struct but it gets added later in the patch series in the patch 1e1b0216 ("LU-8465 e2fsck: open io-channel when copying fs"). Given that this and also a couple of other patches need global_ctx, should we split 1e1b0216 ("LU-8465 e2fsck: open io-channel when copying fs") should we add global_ctx in this patch or as a separate patch before this?
Quoted 219 lines · Saranya Muruganandam
From: Li Xi <[email protected]>
This patch only copy the context to a new one when -m is enabled. It doesn't actually start any thread. When pass1 test finishes, the new context is copied back to the original context.
Since the signal handler only changes the original context, so add
global_ctxin "structe2fsck_struct" and use that to check whether there is any signal of canceling.This patch handles the long jump properly so that all the existing tests can be passed even the context has been copied. Otherwise, test
f_expisize_ea_delwould fail when aborting.Signed-off-by Li Xi <[email protected]>Signed-off-by Wang Shilong <[email protected]>Signed-off-by Saranya Muruganandam <[email protected]>--- e2fsck/pass1.c | 114 +++++++++++++++++++++++++++++++++++++++++++++---- e2fsck/unix.c | 1 + 2 files changed, 107 insertions(+), 8 deletions(-)e2fsck/pass1.c +106
-8@@ -1144,7 +1144,22 @@ static int quota_inum_is_reserved(ext2_filsys fs, ext2_ino_t ino)return 0 ; } -void e2fsck_pass1 (e2fsck_t ctx) +static int e2fsck_should_abort (e2fsck_t ctx) +{ +e2fsck_t global_ctx; + +if (ctx->flags &E2F_FLAG_SIGNAL_MASK ) +return 1 ; + +if (ctx->global_ctx) { + global_ctx = ctx->global_ctx; +if (global_ctx->flags &E2F_FLAG_SIGNAL_MASK ) +return 1 ; + } +return 0 ; +} + +void e2fsck_pass1_thread (e2fsck_t ctx) {int i;__u64 max_sizes;@@ -1360,7 +1375,7 @@ void e2fsck_pass1(e2fsck_t ctx)if (ino > ino_threshold)pass1_readahead (ctx, &ra_group, &ino_threshold);ehandler_operation (old_op); -if (ctx->flags &E2F_FLAG_SIGNAL_MASK ) +if (e2fsck_should_abort (ctx)) goto endit;if (pctx.errcode ==EXT2_ET_BAD_BLOCK_IN_INODE_TABLE ) {/* @@ -1955,7 +1970,7 @@ void e2fsck_pass1(e2fsck_t ctx)if (process_inode_count >= ctx->process_inode_size) {process_inodes (ctx, block_buf); -if (ctx->flags &E2F_FLAG_SIGNAL_MASK ) +if (e2fsck_should_abort (ctx)) goto endit; } }@@ -2068,6 +2083,89 @@ endit:else ctx->invalid_bitmaps++; } + +static errcode_t e2fsck_pass1_thread_prepare (e2fsck_t global_ctx,e2fsck_t *thread_ctx) +{ +errcode_t retval; +e2fsck_t thread_context; + + retval =ext2fs_get_mem (sizeof (struct e2fsck_struct ), &thread_context); +if (retval) { +com_err (global_ctx->program_name, retval,"while allocating memory" ); +return retval; + } +memcpy (thread_context, global_ctx,sizeof (struct e2fsck_struct )); + thread_context->fs->priv_data = thread_context; + thread_context->global_ctx = global_ctx; + + *thread_ctx = thread_context; +return 0 ; +} + +static int e2fsck_pass1_thread_join (e2fsck_t global_ctx,e2fsck_t thread_ctx) +{ +int flags = global_ctx->flags; +#ifdef HAVE_SETJMP_H +jmp_buf old_jmp; + +memcpy (old_jmp, global_ctx->abort_loc,sizeof (jmp_buf)); +#endif +memcpy (global_ctx, thread_ctx,sizeof (struct e2fsck_struct )); +#ifdef HAVE_SETJMP_H +memcpy (global_ctx->abort_loc, old_jmp,sizeof (jmp_buf)); +#endif +/* Keep the global singal flags*/ + global_ctx->flags |= (flags &E2F_FLAG_SIGNAL_MASK ) | + (global_ctx->flags &E2F_FLAG_SIGNAL_MASK ); + + global_ctx->fs->priv_data = global_ctx; +ext2fs_free_mem (&thread_ctx); +return 0 ; +} + +void e2fsck_pass1_multithread (e2fsck_t ctx) +{ +errcode_t retval; +e2fsck_t thread_ctx; + + retval =e2fsck_pass1_thread_prepare (ctx, &thread_ctx); +if (retval) { +com_err (ctx->program_name,0 , +_ ("while preparing pass1 thread\n" )); + ctx->flags |=E2F_FLAG_ABORT ; +return ; + } + +#ifdef HAVE_SETJMP_H +/* +* When fatal_error() happens, jump to here. The thread +* context's flags will be saved, but its abort_loc will +* be overwritten by original jump buffer for the later +* tests. +*/ +if (setjmp (thread_ctx->abort_loc)) { + thread_ctx->flags &= ~E2F_FLAG_SETJMP_OK ; +e2fsck_pass1_thread_join (ctx, thread_ctx); +return ; + } + thread_ctx->flags |=E2F_FLAG_SETJMP_OK ; +#endif + +e2fsck_pass1_thread (thread_ctx); + retval =e2fsck_pass1_thread_join (ctx, thread_ctx); +if (retval) { +com_err (ctx->program_name,0 , +_ ("while joining pass1 thread\n" )); + ctx->flags |=E2F_FLAG_ABORT ; +return ; + } +} + +void e2fsck_pass1 (e2fsck_t ctx) +{ +e2fsck_pass1_multithread (ctx); +} +#undef FINISH_INODE_LOOP/* @@ -2130,7 +2228,7 @@ static void process_inodes(e2fsck_t ctx, char *block_buf)ehandler_operation (buf);check_blocks (ctx, &pctx, block_buf, &inodes_to_process[i].ea_ibody_quota); -if (ctx->flags &E2F_FLAG_SIGNAL_MASK ) +if (e2fsck_should_abort (ctx))break ; } ctx->stashed_inode = old_stashed_inode;@@ -3300,7 +3398,7 @@ static void check_blocks(e2fsck_t ctx, struct problem_context *pctx, inlinedata_fs =ext2fs_has_feature_inline_data (ctx->fs->super);if (check_ext_attr (ctx, pctx, block_buf, &ea_block_quota)) { -if (ctx->flags &E2F_FLAG_SIGNAL_MASK ) +if (e2fsck_should_abort (ctx)) goto out; pb.num_blocks +=EXT2FS_B2C (ctx->fs, ea_block_quota.blocks); }@@ -3355,7 +3453,7 @@ static void check_blocks(e2fsck_t ctx, struct problem_context *pctx, }end_problem_latch (ctx,PR_LATCH_BLOCK );end_problem_latch (ctx,PR_LATCH_TOOBIG ); -if (ctx->flags &E2F_FLAG_SIGNAL_MASK ) +if (e2fsck_should_abort (ctx)) goto out;if (pctx->errcode)fix_problem (ctx,PR_1_BLOCK_ITERATE , pctx);@@ -3836,7 +3934,7 @@ static int process_bad_block(ext2_filsys fs, *block_nr =0 ;return BLOCK_CHANGED ; } -if (ctx->flags &E2F_FLAG_SIGNAL_MASK ) +if (e2fsck_should_abort (ctx))return BLOCK_ABORT ; }else mark_block_used (ctx, blk);@@ -3933,7 +4031,7 @@ static int process_bad_block(ext2_filsys fs, *block_nr =0 ;return BLOCK_CHANGED ; } -if (ctx->flags &E2F_FLAG_SIGNAL_MASK ) +if (e2fsck_should_abort (ctx))return BLOCK_ABORT ;return 0 ; }e2fsck/unix.c +1
-1@@ -1445,6 +1445,7 @@ int main (int argc, char *argv[]) }reserve_stdio_fds (); + ctx->global_ctx =NULL ;set_up_logging (ctx);if (ctx->logf) {int i; --2.29.2.299.gdc1121823c-goog
Re: [RFC PATCH v3 02/61] e2fsck: copy context when using multi-thread fsck
In reply to Saranya MuruganandamQuoted 6 lines · Saranya Muruganandam
From: Li Xi <[email protected]>
This patch only copy the context to a new one when -m is enabled. It doesn't actually start any thread. When pass1 test finishes, the new context is copied back to the original context.
Quoted 7 lines
Since the signal handler only changes the original context, so add
global_ctxin "structe2fsck_struct" and use that to check whether there is any signal of canceling.This patch handles the long jump properly so that all the existing tests can be passed even the context has been copied. Otherwise, test
f_expisize_ea_delwould fail when aborting.
The patch description is a bit misleading. What this is really about is adding the infrastructure to start and join threads in pass #1. Since we presumably will add multiple threading support for other passes, the one-line summary and commit description should make this clear, so that in the future, when a future developer is trying to examine the 60+ commits in this patch series, it will be a bit easier for them to understand what is happening.
It might also be good to explain the patch series that this only starts a single thread, since this patch series is really only about adding the multi-threading machinery.
Some questions which immediately come up is whether it makes sense to have this machinery in e2fsck/pass1.c, or whether some of this is more general and should be in e2fsck/util.c --- although obviously some portion of the code when we are merging the results from the pass will be pass specific. Of course, none of this is in this commit, so it's hard for me to see whether or not it will make sense in the long run.
We can refactor the code later, or in a future patch series, but the point is that it's hard for me to tell whether the existing function breakdown makes the most amount of sense in the first reading of the patches. If you have an opinion of what's the better way to do things, having looked at the whole patch series, feel free to move some of the refactoring into the earlier patches; the patch series doesn't have to reflect the developmental history of the changes, and for the purposes of patch review, it's often simpler when you change earlier patches to simplify things --- although that can make it harder since then you'll have to rework later patches. (If it's too hard, it's fine to leave things as they are.)
- Ted
Re: [RFC PATCH v3 02/61] e2fsck: copy context when using multi-thread fsck
In reply to Saranya MuruganandamQuoted 69 lines · Saranya Muruganandam
From: Li Xi <[email protected]>
This patch only copy the context to a new one when -m is enabled. It doesn't actually start any thread. When pass1 test finishes, the new context is copied back to the original context.
Since the signal handler only changes the original context, so add
global_ctxin "structe2fsck_struct" and use that to check whether there is any signal of canceling.This patch handles the long jump properly so that all the existing tests can be passed even the context has been copied. Otherwise, test
f_expisize_ea_delwould fail when aborting.Signed-off-by Li Xi <[email protected]>Signed-off-by Wang Shilong <[email protected]>Signed-off-by Saranya Muruganandam <[email protected]>--- e2fsck/pass1.c | 114 +++++++++++++++++++++++++++++++++++++++++++++---- e2fsck/unix.c | 1 + 2 files changed, 107 insertions(+), 8 deletions(-)e2fsck/pass1.c +18
-3@@ -1144,7 +1144,22 @@ static int quota_inum_is_reserved(ext2_filsys fs, ext2_ino_t ino)return 0 ; } -void e2fsck_pass1 (e2fsck_t ctx) +static int e2fsck_should_abort (e2fsck_t ctx) +{ +e2fsck_t global_ctx; + +if (ctx->flags &E2F_FLAG_SIGNAL_MASK ) +return 1 ; + +if (ctx->global_ctx) { + global_ctx = ctx->global_ctx; +if (global_ctx->flags &E2F_FLAG_SIGNAL_MASK ) +return 1 ; + } +return 0 ; +} + +void e2fsck_pass1_thread (e2fsck_t ctx) {int i;__u64 max_sizes;@@ -1360,7 +1375,7 @@ void e2fsck_pass1(e2fsck_t ctx)if (ino > ino_threshold)pass1_readahead (ctx, &ra_group, &ino_threshold);ehandler_operation (old_op); -if (ctx->flags &E2F_FLAG_SIGNAL_MASK ) +if (e2fsck_should_abort (ctx)) goto endit;if (pctx.errcode ==EXT2_ET_BAD_BLOCK_IN_INODE_TABLE ) {/* @@ -1955,7 +1970,7 @@ void e2fsck_pass1(e2fsck_t ctx)if (process_inode_count >= ctx->process_inode_size) {process_inodes (ctx, block_buf); -if (ctx->flags &E2F_FLAG_SIGNAL_MASK ) +if (e2fsck_should_abort (ctx)) goto endit; } }
e2fsck/pass1.c endit: · line 2092
Quoted 10 lines
e2fsck/pass1.c +7
-0@@ -2068,6 +2083,89 @@ endit:else ctx->invalid_bitmaps++; } + +static errcode_t e2fsck_pass1_thread_prepare (e2fsck_t global_ctx,e2fsck_t *thread_ctx) +{ +errcode_t retval; +e2fsck_t thread_context; + + retval =ext2fs_get_mem (sizeof (struct e2fsck_struct ), &thread_context);
Hm, so I guess the strategy here is that parallel e2fsck makes per-thread copies of the ext2_filsys and e2fsck_t global contexts? And then after the threaded parts complete, each thread merges its per-thread contexts back into the global one, right?
This means that we have to be careful to track which fields in those cloned contexts have been updated by the thread so that we can copy them back and not lose any data.
I'm wondering if for future maintainability it would be better to track the per-thread data in a separate structure to make it very explicit which data (sub)structures are effectively per-thread and hence don't require locking?
(I ask that mostly because I'm having a hard time figuring out which fields are supposed to be shared and which ones aren't...)
--D
Quoted 139 lines
e2fsck/pass1.c +81
-5+if (retval) { +com_err (global_ctx->program_name, retval,"while allocating memory" ); +return retval; + } +memcpy (thread_context, global_ctx,sizeof (struct e2fsck_struct )); + thread_context->fs->priv_data = thread_context; + thread_context->global_ctx = global_ctx; + + *thread_ctx = thread_context; +return 0 ; +} + +static int e2fsck_pass1_thread_join (e2fsck_t global_ctx,e2fsck_t thread_ctx) +{ +int flags = global_ctx->flags; +#ifdef HAVE_SETJMP_H +jmp_buf old_jmp; + +memcpy (old_jmp, global_ctx->abort_loc,sizeof (jmp_buf)); +#endif +memcpy (global_ctx, thread_ctx,sizeof (struct e2fsck_struct )); +#ifdef HAVE_SETJMP_H +memcpy (global_ctx->abort_loc, old_jmp,sizeof (jmp_buf)); +#endif +/* Keep the global singal flags*/ + global_ctx->flags |= (flags &E2F_FLAG_SIGNAL_MASK ) | + (global_ctx->flags &E2F_FLAG_SIGNAL_MASK ); + + global_ctx->fs->priv_data = global_ctx; +ext2fs_free_mem (&thread_ctx); +return 0 ; +} + +void e2fsck_pass1_multithread (e2fsck_t ctx) +{ +errcode_t retval; +e2fsck_t thread_ctx; + + retval =e2fsck_pass1_thread_prepare (ctx, &thread_ctx); +if (retval) { +com_err (ctx->program_name,0 , +_ ("while preparing pass1 thread\n" )); + ctx->flags |=E2F_FLAG_ABORT ; +return ; + } + +#ifdef HAVE_SETJMP_H +/* +* When fatal_error() happens, jump to here. The thread +* context's flags will be saved, but its abort_loc will +* be overwritten by original jump buffer for the later +* tests. +*/ +if (setjmp (thread_ctx->abort_loc)) { + thread_ctx->flags &= ~E2F_FLAG_SETJMP_OK ; +e2fsck_pass1_thread_join (ctx, thread_ctx); +return ; + } + thread_ctx->flags |=E2F_FLAG_SETJMP_OK ; +#endif + +e2fsck_pass1_thread (thread_ctx); + retval =e2fsck_pass1_thread_join (ctx, thread_ctx); +if (retval) { +com_err (ctx->program_name,0 , +_ ("while joining pass1 thread\n" )); + ctx->flags |=E2F_FLAG_ABORT ; +return ; + } +} + +void e2fsck_pass1 (e2fsck_t ctx) +{ +e2fsck_pass1_multithread (ctx); +} +#undef FINISH_INODE_LOOP/* @@ -2130,7 +2228,7 @@ static void process_inodes(e2fsck_t ctx, char *block_buf)ehandler_operation (buf);check_blocks (ctx, &pctx, block_buf, &inodes_to_process[i].ea_ibody_quota); -if (ctx->flags &E2F_FLAG_SIGNAL_MASK ) +if (e2fsck_should_abort (ctx))break ; } ctx->stashed_inode = old_stashed_inode;@@ -3300,7 +3398,7 @@ static void check_blocks(e2fsck_t ctx, struct problem_context *pctx, inlinedata_fs =ext2fs_has_feature_inline_data (ctx->fs->super);if (check_ext_attr (ctx, pctx, block_buf, &ea_block_quota)) { -if (ctx->flags &E2F_FLAG_SIGNAL_MASK ) +if (e2fsck_should_abort (ctx)) goto out; pb.num_blocks +=EXT2FS_B2C (ctx->fs, ea_block_quota.blocks); }@@ -3355,7 +3453,7 @@ static void check_blocks(e2fsck_t ctx, struct problem_context *pctx, }end_problem_latch (ctx,PR_LATCH_BLOCK );end_problem_latch (ctx,PR_LATCH_TOOBIG ); -if (ctx->flags &E2F_FLAG_SIGNAL_MASK ) +if (e2fsck_should_abort (ctx)) goto out;if (pctx->errcode)fix_problem (ctx,PR_1_BLOCK_ITERATE , pctx);@@ -3836,7 +3934,7 @@ static int process_bad_block(ext2_filsys fs, *block_nr =0 ;return BLOCK_CHANGED ; } -if (ctx->flags &E2F_FLAG_SIGNAL_MASK ) +if (e2fsck_should_abort (ctx))return BLOCK_ABORT ; }else mark_block_used (ctx, blk);@@ -3933,7 +4031,7 @@ static int process_bad_block(ext2_filsys fs, *block_nr =0 ;return BLOCK_CHANGED ; } -if (ctx->flags &E2F_FLAG_SIGNAL_MASK ) +if (e2fsck_should_abort (ctx))return BLOCK_ABORT ;return 0 ; }e2fsck/unix.c +1
-0@@ -1445,6 +1445,7 @@ int main (int argc, char *argv[]) }reserve_stdio_fds (); + ctx->global_ctx =NULL ;set_up_logging (ctx);if (ctx->logf) {int i;Signature · 1 line
2.29.2.299.gdc1121823c-goog
Re: [RFC PATCH v3 02/61] e2fsck: copy context when using multi-thread fsck
Quoted 78 lines · Darrick J. Wong
Quoted 76 lines · Saranya Muruganandam
From: Li Xi <[email protected]>
This patch only copy the context to a new one when -m is enabled. It doesn't actually start any thread. When pass1 test finishes, the new context is copied back to the original context.
Since the signal handler only changes the original context, so add
global_ctxin "structe2fsck_struct" and use that to check whether there is any signal of canceling.This patch handles the long jump properly so that all the existing tests can be passed even the context has been copied. Otherwise, test
f_expisize_ea_delwould fail when aborting.Signed-off-by Li Xi <[email protected]>Signed-off-by Wang Shilong <[email protected]>Signed-off-by Saranya Muruganandam <[email protected]>--- e2fsck/pass1.c | 114 +++++++++++++++++++++++++++++++++++++++++++++---- e2fsck/unix.c | 1 + 2 files changed, 107 insertions(+), 8 deletions(-)e2fsck/pass1.c +22
-3@@ -1144,7 +1144,22 @@ static int quota_inum_is_reserved(ext2_filsys fs, ext2_ino_t ino)return 0 ; } -void e2fsck_pass1 (e2fsck_t ctx) +static int e2fsck_should_abort (e2fsck_t ctx) +{ +e2fsck_t global_ctx; + +if (ctx->flags &E2F_FLAG_SIGNAL_MASK ) +return 1 ; + +if (ctx->global_ctx) { + global_ctx = ctx->global_ctx; +if (global_ctx->flags &E2F_FLAG_SIGNAL_MASK ) +return 1 ; + } +return 0 ; +} + +void e2fsck_pass1_thread (e2fsck_t ctx) {int i;__u64 max_sizes;@@ -1360,7 +1375,7 @@ void e2fsck_pass1(e2fsck_t ctx)if (ino > ino_threshold)pass1_readahead (ctx, &ra_group, &ino_threshold);ehandler_operation (old_op); -if (ctx->flags &E2F_FLAG_SIGNAL_MASK ) +if (e2fsck_should_abort (ctx)) goto endit;if (pctx.errcode ==EXT2_ET_BAD_BLOCK_IN_INODE_TABLE ) {/* @@ -1955,7 +1970,7 @@ void e2fsck_pass1(e2fsck_t ctx)if (process_inode_count >= ctx->process_inode_size) {process_inodes (ctx, block_buf); -if (ctx->flags &E2F_FLAG_SIGNAL_MASK ) +if (e2fsck_should_abort (ctx)) goto endit; } }@@ -2068,6 +2083,89 @@ endit:else ctx->invalid_bitmaps++; } + +static errcode_t e2fsck_pass1_thread_prepare (e2fsck_t global_ctx,e2fsck_t *thread_ctx) +{ +errcode_t retval;
Quoted 8 lines
e2fsck/pass1.c endit: · line 2092
Quoted 3 lines
e2fsck/pass1.c +3
-0+e2fsck_t thread_context; + + retval =ext2fs_get_mem (sizeof (struct e2fsck_struct ), &thread_context);Hm, so I guess the strategy here is that parallel e2fsck makes per-thread copies of the
ext2_filsysande2fsck_tglobal contexts? And then after the threaded parts complete, each thread merges its per-thread contexts back into the global one, right?
Yes.
Quoted 9 lines
This means that we have to be careful to track which fields in those cloned contexts have been updated by the thread so that we can copy them back and not lose any data.
I'm wondering if for future maintainability it would be better to track the per-thread data in a separate structure to make it very explicit which data (sub)structures are effectively per-thread and hence don't require locking?
Maybe use a per-thread structure is better maintained, but i am not sure we could remove locking completely.
Locking is mostly used for fix, because fixing is serialized now and for some global structure which could be used seldomly but could simplify codes.
Quoted 8 lines
(I ask that mostly because I'm having a hard time figuring out which fields are supposed to be shared and which ones aren't...)
--D
Quoted 2 lines
e2fsck/pass1.c +2
-0+if (retval) { +com_err (global_ctx->program_name, retval,"while allocating memory" );
Re: [RFC PATCH v3 02/61] e2fsck: copy context when using multi-thread fsck
Quoted 100 lines · Wang Shilong
Quoted 78 lines · Darrick J. Wong
Quoted 76 lines · Saranya Muruganandam
From: Li Xi <[email protected]>
This patch only copy the context to a new one when -m is enabled. It doesn't actually start any thread. When pass1 test finishes, the new context is copied back to the original context.
Since the signal handler only changes the original context, so add
global_ctxin "structe2fsck_struct" and use that to check whether there is any signal of canceling.This patch handles the long jump properly so that all the existing tests can be passed even the context has been copied. Otherwise, test
f_expisize_ea_delwould fail when aborting.Signed-off-by Li Xi <[email protected]>Signed-off-by Wang Shilong <[email protected]>Signed-off-by Saranya Muruganandam <[email protected]>--- e2fsck/pass1.c | 114 +++++++++++++++++++++++++++++++++++++++++++++---- e2fsck/unix.c | 1 + 2 files changed, 107 insertions(+), 8 deletions(-)e2fsck/pass1.c +22
-3@@ -1144,7 +1144,22 @@ static int quota_inum_is_reserved(ext2_filsys fs, ext2_ino_t ino)return 0 ; } -void e2fsck_pass1 (e2fsck_t ctx) +static int e2fsck_should_abort (e2fsck_t ctx) +{ +e2fsck_t global_ctx; + +if (ctx->flags &E2F_FLAG_SIGNAL_MASK ) +return 1 ; + +if (ctx->global_ctx) { + global_ctx = ctx->global_ctx; +if (global_ctx->flags &E2F_FLAG_SIGNAL_MASK ) +return 1 ; + } +return 0 ; +} + +void e2fsck_pass1_thread (e2fsck_t ctx) {int i;__u64 max_sizes;@@ -1360,7 +1375,7 @@ void e2fsck_pass1(e2fsck_t ctx)if (ino > ino_threshold)pass1_readahead (ctx, &ra_group, &ino_threshold);ehandler_operation (old_op); -if (ctx->flags &E2F_FLAG_SIGNAL_MASK ) +if (e2fsck_should_abort (ctx)) goto endit;if (pctx.errcode ==EXT2_ET_BAD_BLOCK_IN_INODE_TABLE ) {/* @@ -1955,7 +1970,7 @@ void e2fsck_pass1(e2fsck_t ctx)if (process_inode_count >= ctx->process_inode_size) {process_inodes (ctx, block_buf); -if (ctx->flags &E2F_FLAG_SIGNAL_MASK ) +if (e2fsck_should_abort (ctx)) goto endit; } }@@ -2068,6 +2083,89 @@ endit:else ctx->invalid_bitmaps++; } + +static errcode_t e2fsck_pass1_thread_prepare (e2fsck_t global_ctx,e2fsck_t *thread_ctx) +{ +errcode_t retval;Quoted 8 lines
e2fsck/pass1.c endit: · line 2092
Quoted 3 lines
e2fsck/pass1.c +3
-0+e2fsck_t thread_context; + + retval =ext2fs_get_mem (sizeof (struct e2fsck_struct ), &thread_context);Hm, so I guess the strategy here is that parallel e2fsck makes per-thread copies of the
ext2_filsysande2fsck_tglobal contexts? And then after the threaded parts complete, each thread merges its per-thread contexts back into the global one, right?Yes.
Quoted 9 lines
This means that we have to be careful to track which fields in those cloned contexts have been updated by the thread so that we can copy them back and not lose any data.
I'm wondering if for future maintainability it would be better to track the per-thread data in a separate structure to make it very explicit which data (sub)structures are effectively per-thread and hence don't require locking?
Quoted 6 lines
Maybe use a per-thread structure is better maintained, but i am not sure we could remove locking completely.
Locking is mostly used for fix, because fixing is serialized now and for some global structure which could be used seldomly but could simplify codes.
<nod> I was assuming that you'd still put a lock in the global structure and use it for data fields that aren't so frequently accessed.
--D
Quoted 8 lines
Quoted 8 lines
(I ask that mostly because I'm having a hard time figuring out which fields are supposed to be shared and which ones aren't...)
--D
Quoted 2 lines
e2fsck/pass1.c +2
-0+if (retval) { +com_err (global_ctx->program_name, retval,"while allocating memory" );
RFC PATCH v3 3/61 e2fsck: copy fs when using multi-thread fsck
In reply to Saranya MuruganandamFrom: Li Xi <[email protected]>
This patch only copy the fs to a new one when -m is enabled. It doesn't actually start any thread. When pass1 test finishes, the new fs is copied back to the original context.
This patch handles the fs fields in dblist, inode_map and block_map properly.
--- e2fsck/pass1.c | 41 +++++++++++++++++++++++++++++++++++++---- 1 file changed, 37 insertions(+), 4 deletions(-)
e2fsck/pass1.c +37 -4
@@ -49,6 +49,8 @@#include "e2fsck.h" #include <ext2fs/ext2_ext_attr.h> +/* todo remove this finally */ +#include <ext2fs/ext2fsP.h> #include <e2p/e2p.h> #include "problem.h"
@@ -2084,10 +2086,23 @@ endit: ctx->invalid_bitmaps++; } +static void e2fsck_pass1_copy_fs (ext2_filsys dest,ext2_filsys src) +{ +memcpy (dest, src,sizeof (struct struct_ext2_filsys )); +if (dest->dblist) + dest->dblist->fs = dest; +if (dest->inode_map) + dest->inode_map->fs = dest; +if (dest->block_map) + dest->block_map->fs = dest; +} +static errcode_t e2fsck_pass1_thread_prepare (e2fsck_t global_ctx,e2fsck_t *thread_ctx) {errcode_t retval;e2fsck_t thread_context; +ext2_filsys thread_fs; +ext2_filsys global_fs = global_ctx->fs; retval =ext2fs_get_mem (sizeof (struct e2fsck_struct ), &thread_context);if (retval) {
@@ -2095,18 +2110,32 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *threreturn retval; }memcpy (thread_context, global_ctx,sizeof (struct e2fsck_struct )); - thread_context->fs->priv_data = thread_context; thread_context->global_ctx = global_ctx; + retval =ext2fs_get_mem (sizeof (struct struct_ext2_filsys ), &thread_fs); +if (retval) { +com_err (global_ctx->program_name, retval,"while allocating memory" ); + goto out_context; + } + +e2fsck_pass1_copy_fs (thread_fs, global_fs); + thread_fs->priv_data = thread_context; + + thread_context->fs = thread_fs; *thread_ctx = thread_context;return 0 ; +out_context: +ext2fs_free_mem (&thread_context); +return retval; }static int e2fsck_pass1_thread_join (e2fsck_t global_ctx,e2fsck_t thread_ctx) { -int flags = global_ctx->flags; +int flags = global_ctx->flags; +ext2_filsys thread_fs = thread_ctx->fs; +ext2_filsys global_fs = global_ctx->fs;#ifdef HAVE_SETJMP_H -jmp_buf old_jmp; +jmp_buf old_jmp;memcpy (old_jmp, global_ctx->abort_loc,sizeof (jmp_buf));#endif
@@ -2118,7 +2147,11 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx) global_ctx->flags |= (flags &E2F_FLAG_SIGNAL_MASK ) | (global_ctx->flags &E2F_FLAG_SIGNAL_MASK ); - global_ctx->fs->priv_data = global_ctx; +e2fsck_pass1_copy_fs (global_fs, thread_fs); + global_fs->priv_data = global_ctx; + global_ctx->fs = global_fs; + +ext2fs_free_mem (&thread_ctx->fs);ext2fs_free_mem (&thread_ctx);return 0 ; }
Signature · 1 line
2.29.2.299.gdc1121823c-goog
Re: [RFC PATCH v3 03/61] e2fsck: copy fs when using multi-thread fsck
Quoted 6 lines · Saranya Muruganandam
From: Li Xi <[email protected]>
This patch only copy the fs to a new one when -m is enabled. It doesn't actually start any thread. When pass1 test finishes, the new fs is copied back to the original context.
Quoted 6 lines
This patch handles the fs fields in dblist,
inode_mapandblock_mapproperly.Signed-off-by Li Xi <[email protected]>Signed-off-by Wang Shilong <[email protected]>Signed-off-by Saranya Muruganandam <[email protected]>
I'm a bit surprised that we're not adding a ext2fs_clone_fs() function, but instead creating an e2fsck_pass1_copy_fs() function.
Again, what's going to happen when we need to copy the fs structure for other passes. Also, just simply copying the fs structure seems dangerous; how are we going to know which allocated substructures were created before the fs structure was clonsed (and hence can't be safely freed in a copy of the fs structure), and which substructures were allocated *after* the fs structure is cloned, in which case they need to be freed when the cloned fs structuers are released?
Perhaps the simplest thing to do is to assume that *everything* is cloned. If that results in too much memory consumed, maybe we can add a set of flags indicating which structures should *not* be freed for a particular fs structure, with the assumption that they will be freed when the top-level master fs structure is closed. There may be some potential traps with this, since in some cases a substructure can be released and re-allocated, in which case, if that substructure is shared by child fs structures, they could be left pointing at already freed copies of the data structures.
Basically, even if this works (and the fact that I saw a crash with a data structure being double-freed in the f_multithread_ok test strongly suggesting that it isn't), I'd be worried about making sure that the resulting architecture is one which is robust and maintainable, and not leaving traps for future developers when they don't realize what the assumptions are about shared substructures. Which strongly suggests that this should be a first-class aspect of libext2fs and it should be clearly documented how sharing does or doesn't work.
Cheers,
- Ted
RFC PATCH v3 4/61 e2fsck: clear icache when using multi-thread fsck
In reply to Saranya MuruganandamFrom: Wang Shilong <[email protected]>
icache of fs will be rebuilt when needed, so after copying fs, icache can be inited to NULL.
--- e2fsck/pass1.c | 37 ++++++++++++++++++++++++++++++++++--- 1 file changed, 34 insertions(+), 3 deletions(-)
e2fsck/pass1.c +34 -3
@@ -2086,8 +2086,10 @@ endit: ctx->invalid_bitmaps++; } -static void e2fsck_pass1_copy_fs (ext2_filsys dest,ext2_filsys src) +static errcode_t e2fsck_pass1_copy_fs (ext2_filsys dest,ext2_filsys src) { +errcode_t retval; +memcpy (dest, src,sizeof (struct struct_ext2_filsys ));if (dest->dblist) dest->dblist->fs = dest;
@@ -2095,6 +2097,29 @@ static void e2fsck_pass1_copy_fs(ext2_filsys dest, ext2_filsys src) dest->inode_map->fs = dest;if (dest->block_map) dest->block_map->fs = dest; + +/* icache will be rebuilt if needed, so do not copy from @src */ + src->icache =NULL ; +return 0 ; +} + +static void e2fsck_pass1_merge_fs (ext2_filsys dest,ext2_filsys src) +{ +struct ext2_inode_cache *icache = dest->icache; + +memcpy (dest, src,sizeof (struct struct_ext2_filsys )); +if (dest->dblist) + dest->dblist->fs = dest; +if (dest->inode_map) + dest->inode_map->fs = dest; +if (dest->block_map) + dest->block_map->fs = dest; + dest->icache = icache; + +if (src->icache) { +ext2fs_free_inode_cache (src->icache); + src->icache =NULL ; + } }static errcode_t e2fsck_pass1_thread_prepare (e2fsck_t global_ctx,e2fsck_t *thread_ctx)
@@ -2118,12 +2143,18 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thre goto out_context; } -e2fsck_pass1_copy_fs (thread_fs, global_fs); + retval =e2fsck_pass1_copy_fs (thread_fs, global_fs); +if (retval) { +com_err (global_ctx->program_name, retval,"while copying fs" ); + goto out_fs; + } thread_fs->priv_data = thread_context; thread_context->fs = thread_fs; *thread_ctx = thread_context;return 0 ; +out_fs: +ext2fs_free_mem (&thread_fs); out_context:ext2fs_free_mem (&thread_context);return retval;
@@ -2147,7 +2178,7 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx) global_ctx->flags |= (flags &E2F_FLAG_SIGNAL_MASK ) | (global_ctx->flags &E2F_FLAG_SIGNAL_MASK ); -e2fsck_pass1_copy_fs (global_fs, thread_fs); +e2fsck_pass1_merge_fs (global_fs, thread_fs); global_fs->priv_data = global_ctx; global_ctx->fs = global_fs;
Signature · 1 line
2.29.2.299.gdc1121823c-goog
Re: [RFC PATCH v3 04/61] e2fsck: clear icache when using multi-thread fsck
Quoted 8 lines · Saranya Muruganandam
From: Wang Shilong <[email protected]>
icache of fs will be rebuilt when needed, so after copying fs, icache can be inited to NULL.
Signed-off-by Li Xi <[email protected]>Signed-off-by Wang Shilong <[email protected]>Signed-off-by Saranya Muruganandam <[email protected]>
So this is part of the whole "how is the fs structure is shared question". It's going to be better if this is all in a single patch, instead of being spread out into separate patches. Certainly from a review perspective, it's going to be much simpler....
Quoted 8 lines
e2fsck/pass1.c +8
-0+static void e2fsck_pass1_merge_fs (ext2_filsys dest,ext2_filsys src) +{ +struct ext2_inode_cache *icache = dest->icache; + +memcpy (dest, src,sizeof (struct struct_ext2_filsys )); +if (dest->dblist) + dest->dblist->fs = dest; +if (dest->inode_map)
e2fsck/pass1.c static void e2fsck_pass1_copy_fs(ext2_filsys dest, ext2_filsys src) · line 2123
Quoted 10 lines
e2fsck/pass1.c +9
-0+ dest->inode_map->fs = dest; +if (dest->block_map) + dest->block_map->fs = dest; + dest->icache = icache; + +if (src->icache) { +ext2fs_free_inode_cache (src->icache); + src->icache =NULL ; + } }
I'm not sure how the merge_fs function is supposed to work. Right now it's not doing much in the way of merging; I'm guessing how it works is going to be done in a future patch, but this is going to make review a bit tricky.
.... peaking ahead in the patch series. Yeah, I think we need to think very carefully about how we do stuff here. It looks very ad hoc, with some aspects of the merge functionality in e2fsck/pass1.c, and some in libext2fs.
I wonder if we wouldn't be better off with the concept of a top-level fs structure, and child fs structures which point back to the top-level fs structure --- much like what is currently being done with the e2fsck_t structure. So if fs->parent_fs is non-NULL, we know that this is a child fs structure, and there are very clear rules about how things like io_managers are shared --- or not. Note that some io_managers, such as undo io manager, fundamentally *have* to be shared, which implies that we need to have some kind of flag indicating whether or not an io_manager is thread-aware or not. And if an io_manager is not thread-aware, then it wouldn't be safe to use it in multi-threading mode.
I'm going to stop doing a fine-grained review of this patch series, and it seems pretty clear to me that we need to have some fundamental architectural discussions about how to make multi-threading work. An ad-hoc approach is great for a prototype, as this surfaces all of these sorts of questions --- but I think we need to now stop, and think very carefully about how to make multi-threading support something which can be a long-term maintainable code base. And that means figuring out what the semantics should be about various shared data sructures, and then documenting how things are supposed to work very explicitly. (I note that many of these functions don't have much in the way of documentation, which is unfortunate from the prespective of some future developer who is wishing to further extend this work.)
- Ted
RFC PATCH v3 5/61 e2fsck: add assert when copying context
In reply to Saranya MuruganandamFrom: Li Xi <[email protected]>
Adding the assert would simplify the copying of context.
--- e2fsck/pass1.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
e2fsck/pass1.c +14 -0
@@ -46,6 +46,7 @@#ifdef HAVE_ERRNO_H #include <errno.h> #endif +#include <assert.h> #include "e2fsck.h" #include <ext2fs/ext2_ext_attr.h>
@@ -2129,6 +2130,19 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *threext2_filsys thread_fs;ext2_filsys global_fs = global_ctx->fs; +assert (global_ctx->inode_used_map ==NULL ); +assert (global_ctx->inode_dir_map ==NULL ); +assert (global_ctx->inode_bb_map ==NULL ); +assert (global_ctx->inode_imagic_map ==NULL ); +assert (global_ctx->inode_reg_map ==NULL ); +assert (global_ctx->inodes_to_rebuild ==NULL ); + +assert (global_ctx->block_found_map ==NULL ); +assert (global_ctx->block_dup_map ==NULL ); +assert (global_ctx->block_ea_map ==NULL ); +assert (global_ctx->block_metadata_map ==NULL ); +assert (global_ctx->fs->dblist ==NULL ); + retval =ext2fs_get_mem (sizeof (struct e2fsck_struct ), &thread_context);if (retval) {com_err (global_ctx->program_name, retval,"while allocating memory" );
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 6/61 e2fsck: copy bitmaps when copying context
In reply to Saranya MuruganandamFrom: Li Xi <[email protected]>
This patch copies bitmap when the copying context. In the multi-thread fsck, each thread use different bitmap that copied from the glboal bitmap. And Bitmaps from multiple threads will be merged into a global one after the pass1 finishes.
--- e2fsck/pass1.c | 151 +++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 139 insertions(+), 12 deletions(-)
e2fsck/pass1.c +139 -12
@@ -2087,6 +2087,22 @@ endit: ctx->invalid_bitmaps++; } +static errcode_t e2fsck_pass1_copy_bitmap (ext2_filsys fs,ext2fs_generic_bitmap *src, +ext2fs_generic_bitmap *dest) +{ +errcode_t ret; + + ret =ext2fs_copy_bitmap (*src, dest); +if (ret) +return ret; + + (*dest)->fs = fs; +ext2fs_free_generic_bmap (*src); + *src =NULL ; + +return 0 ; +} +static errcode_t e2fsck_pass1_copy_fs (ext2_filsys dest,ext2_filsys src) {errcode_t retval;
@@ -2094,33 +2110,52 @@ static errcode_t e2fsck_pass1_copy_fs(ext2_filsys dest, ext2_filsys src)memcpy (dest, src,sizeof (struct struct_ext2_filsys ));if (dest->dblist) dest->dblist->fs = dest; -if (dest->inode_map) - dest->inode_map->fs = dest; -if (dest->block_map) - dest->block_map->fs = dest; +if (src->block_map) { + retval =e2fsck_pass1_copy_bitmap (dest, &src->block_map, + &dest->block_map); +if (retval) +return retval; + } +if (src->inode_map) { + retval =e2fsck_pass1_copy_bitmap (dest, &src->inode_map, + &dest->inode_map); +if (retval) +return retval; + }/* icache will be rebuilt if needed, so do not copy from @src */ src->icache =NULL ;return 0 ; } -static void e2fsck_pass1_merge_fs (ext2_filsys dest,ext2_filsys src) +static int e2fsck_pass1_merge_fs (ext2_filsys dest,ext2_filsys src) {struct ext2_inode_cache *icache = dest->icache; +errcode_t retval =0 ;memcpy (dest, src,sizeof (struct struct_ext2_filsys ));if (dest->dblist) dest->dblist->fs = dest; -if (dest->inode_map) - dest->inode_map->fs = dest; -if (dest->block_map) - dest->block_map->fs = dest; +if (src->inode_map) { + retval =e2fsck_pass1_copy_bitmap (dest, &src->inode_map, + &dest->inode_map); +if (retval) +return retval; + } +if (src->block_map) { + retval =e2fsck_pass1_copy_bitmap (dest, &src->block_map, + &dest->block_map); +if (retval) +return retval; + } dest->icache = icache;if (src->icache) {ext2fs_free_inode_cache (src->icache); src->icache =NULL ; } + +return retval; }static errcode_t e2fsck_pass1_thread_prepare (e2fsck_t global_ctx,e2fsck_t *thread_ctx)
@@ -2174,8 +2209,9 @@ out_context:return retval; } -static int e2fsck_pass1_thread_join (e2fsck_t global_ctx,e2fsck_t thread_ctx) +static int e2fsck_pass1_thread_join_one (e2fsck_t global_ctx,e2fsck_t thread_ctx) { +errcode_t retval;int flags = global_ctx->flags;ext2_filsys thread_fs = thread_ctx->fs;ext2_filsys global_fs = global_ctx->fs;
@@ -2192,13 +2228,104 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx) global_ctx->flags |= (flags &E2F_FLAG_SIGNAL_MASK ) | (global_ctx->flags &E2F_FLAG_SIGNAL_MASK ); -e2fsck_pass1_merge_fs (global_fs, thread_fs); + retval =e2fsck_pass1_merge_fs (global_fs, thread_fs); +if (retval) { +com_err (global_ctx->program_name,0 ,_ ("while merging fs\n" )); +return retval; + } global_fs->priv_data = global_ctx; global_ctx->fs = global_fs; +if (thread_ctx->inode_used_map) { + retval =e2fsck_pass1_copy_bitmap (global_fs, + &thread_ctx->inode_used_map, + &global_ctx->inode_used_map); +if (retval) +return retval; + } +if (thread_ctx->inode_bad_map) { + retval =e2fsck_pass1_copy_bitmap (global_fs, + &thread_ctx->inode_bad_map, + &global_ctx->inode_bad_map); +if (retval) +return retval; + } +if (thread_ctx->inode_dir_map) { + retval =e2fsck_pass1_copy_bitmap (global_fs, + &thread_ctx->inode_dir_map, + &global_ctx->inode_dir_map); +if (retval) +return retval; + } +if (thread_ctx->inode_bb_map) { + retval =e2fsck_pass1_copy_bitmap (global_fs, + &thread_ctx->inode_bb_map, + &global_ctx->inode_bb_map); +if (retval) +return retval; + } +if (thread_ctx->inode_imagic_map) { + retval =e2fsck_pass1_copy_bitmap (global_fs, + &thread_ctx->inode_imagic_map, + &global_ctx->inode_imagic_map); +if (retval) +return retval; + } +if (thread_ctx->inode_reg_map) { + retval =e2fsck_pass1_copy_bitmap (global_fs, + &thread_ctx->inode_reg_map, + &global_ctx->inode_reg_map); +if (retval) +return retval; + } +if (thread_ctx->inodes_to_rebuild) { + retval =e2fsck_pass1_copy_bitmap (global_fs, + &thread_ctx->inodes_to_rebuild, + &global_ctx->inodes_to_rebuild); +if (retval) +return retval; + } +if (thread_ctx->block_found_map) { + retval =e2fsck_pass1_copy_bitmap (global_fs, + &thread_ctx->block_found_map, + &global_ctx->block_found_map); +if (retval) +return retval; + } +if (thread_ctx->block_dup_map) { + retval =e2fsck_pass1_copy_bitmap (global_fs, + &thread_ctx->block_dup_map, + &global_ctx->block_dup_map); +if (retval) +return retval; + } +if (thread_ctx->block_ea_map) { + retval =e2fsck_pass1_copy_bitmap (global_fs, + &thread_ctx->block_ea_map, + &global_ctx->block_ea_map); +if (retval) +return retval; + } +if (thread_ctx->block_metadata_map) { + retval =e2fsck_pass1_copy_bitmap (global_fs, + &thread_ctx->block_metadata_map, + &global_ctx->block_metadata_map); +if (retval) +return retval; + } + +return 0 ; +} + +static int e2fsck_pass1_thread_join (e2fsck_t global_ctx,e2fsck_t thread_ctx) +{ +errcode_t retval; + + retval =e2fsck_pass1_thread_join_one (global_ctx, thread_ctx);ext2fs_free_mem (&thread_ctx->fs);ext2fs_free_mem (&thread_ctx); -return 0 ; + +return retval; }void e2fsck_pass1_multithread (e2fsck_t ctx)
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 7/61 e2fsck: copy badblocks when copying fs
In reply to Saranya MuruganandamFrom: Wang Shilong <[email protected]>
This patch copies badblocks when the copying fs.
--- e2fsck/pass1.c | 13 +++++++++++++ 1 file changed, 13 insertions(+)
e2fsck/pass1.c +13 -0
@@ -2123,6 +2123,12 @@ static errcode_t e2fsck_pass1_copy_fs(ext2_filsys dest, ext2_filsys src)return retval; } +if (src->badblocks) { + retval =ext2fs_badblocks_copy (src->badblocks, &dest->badblocks); +if (retval) +return retval; + } +/* icache will be rebuilt if needed, so do not copy from @src */ src->icache =NULL ;return 0 ;
@@ -2155,6 +2161,13 @@ static int e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src) src->icache =NULL ; } +if (src->badblocks) { + retval =ext2fs_badblocks_copy (src->badblocks, &dest->badblocks); + +ext2fs_badblocks_list_free (src->badblocks); + src->badblocks =NULL ; + } +return retval; }
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 8/61 e2fsck: open io-channel when copying fs
In reply to Saranya MuruganandamFrom: Li Xi <[email protected]>
This patch also add writethrough flag to the thread io-channel. When multiple threads write the same disk, we don't want the data being saved in memory cache. This will be useful in the future, but even without that flag, the tests can be passed too.
This patch also cleanup the io channel cache of the global context. Otherwise, after pass1 step, the next steps would use old data saved in the cache. And the cached data might have already been overwritten in pass1.
--- e2fsck/e2fsck.h | 3 +++ e2fsck/pass1.c | 46 ++++++++++++++++++++++++++++++++++++++---- e2fsck/unix.c | 1 + lib/ext2fs/ext2_io.h | 2 ++ lib/ext2fs/ext2fs.h | 3 +++ lib/ext2fs/openfs.c | 48 +++++++++++++++++++++++++++++++------------- lib/ext2fs/undo_io.c | 19 ++++++++++++++++++ lib/ext2fs/unix_io.c | 24 +++++++++++++++++++--- 8 files changed, 125 insertions(+), 21 deletions(-)
e2fsck/e2fsck.h +3 -0
@@ -228,6 +228,8 @@ typedef struct e2fsck_struct *e2fsck_t;#define MAX_EXTENT_DEPTH_COUNT 5struct e2fsck_struct { +/* Global context to get the cancel flag */ +e2fsck_t global_ctx;ext2_filsys fs;const char *program_name;char *filesystem_name;
@@ -247,6 +249,7 @@ struct e2fsck_struct {ext2_ino_t free_inodes;int mount_flags;int openfs_flags; +io_manager io_manager;blkid_cache blkid;/* blkid cache */ #ifdef HAVE_SETJMP_H
e2fsck/pass1.c +42 -4
84 lines not on this page · raw message
e2fsck/unix.c +1 -0
@@ -1525,6 +1525,7 @@ restart: } ctx->openfs_flags = flags; + ctx->io_manager = io_ptr; retval =try_open_fs (ctx, flags, io_ptr, &fs); if (!ctx->superblock && !(ctx->options & E2F_OPT_PREEN) &&
lib/ext2fs/ext2_io.h +2 -0
@@ -81,6 +81,7 @@ struct struct_io_manager {errcode_t (*write_blk)(io_channel channel,unsigned long block,int count,const void *data);errcode_t (*flush)(io_channel channel); +errcode_t (*flush_cleanup)(io_channel channel);errcode_t (*write_byte)(io_channel channel,unsigned long offset,int count,const void *data); errcode_t (*set_option)(io_channel channel, const char *option,
@@ -113,6 +114,7 @@ struct struct_io_manager {#define io_channel_read_blk (c,b,n,d) ((c)->manager->read_blk((c),b,n,d))#define io_channel_write_blk (c,b,n,d) ((c)->manager->write_blk((c),b,n,d))#define io_channel_flush (c) ((c)->manager->flush((c))) +#define io_channel_flush_cleanup (c) ((c)->manager->flush_cleanup((c)))#define io_channel_bumpcount (c) ((c)->refcount++)/* io_manager.c */
lib/ext2fs/ext2fs.h +3 -0
@@ -1643,6 +1643,9 @@ extern errcode_t ext2fs_open2(const char *name, const char *io_options, int flags, int superblock, unsigned int block_size, io_manager manager, ext2_filsys *ret_fs); +errcode_t ext2fs_open_channel (ext2_filsys fs,const char *io_options, +io_manager manager,int flags, +int blocksize);/* * The dgrp_t argument to these two functions is not actually a group number * but a block number offset within a group table! Convert with the formula
lib/ext2fs/openfs.c +34 -14
68 lines not on this page · raw message
lib/ext2fs/undo_io.c +19 -0
@@ -1020,6 +1020,24 @@ static errcode_t undo_flush(io_channel channel)return retval; } +/* +* Flush data buffers to disk and cleanup the cache. +*/ +static errcode_t undo_flush_cleanup (io_channel channel) +{ +errcode_t retval =0 ; +struct undo_private_data *data; + +EXT2_CHECK_MAGIC (channel,EXT2_ET_MAGIC_IO_CHANNEL ); + data = (struct undo_private_data *) channel->private_data; +EXT2_CHECK_MAGIC (data,EXT2_ET_MAGIC_UNIX_IO_CHANNEL ); + +if (data->real) + retval =io_channel_flush_cleanup (data->real); + +return retval; +} +static errcode_t undo_set_option (io_channel channel,const char *option,const char *arg) {
@@ -1091,6 +1109,7 @@ static struct struct_io_manager struct_undo_manager = { .read_blk = undo_read_blk, .write_blk = undo_write_blk, .flush = undo_flush, + .flush_cleanup = undo_flush_cleanup, .write_byte = undo_write_byte, .set_option = undo_set_option, .get_stats = undo_get_stats,
lib/ext2fs/unix_io.c +21 -3
@@ -1030,9 +1030,9 @@ static errcode_t unix_write_byte(io_channel channel, unsigned long offset, }/* -* Flush data buffers to disk. +* Flush data buffers to disk and invalidate cache if needed */ -static errcode_t unix_flush (io_channel channel) +static errcode_t _unix_flush (io_channel channel,int invalidate) {struct unix_private_data *data;errcode_t retval =0 ;
@@ -1042,7 +1042,7 @@ static errcode_t unix_flush(io_channel channel)EXT2_CHECK_MAGIC (data,EXT2_ET_MAGIC_UNIX_IO_CHANNEL );#ifndef NO_IO_CACHE - retval =flush_cached_blocks (channel, data,0 ); + retval =flush_cached_blocks (channel, data, invalidate);#endif #ifdef HAVE_FSYNC if (!retval &&fsync (data->dev) !=0 )
@@ -1051,6 +1051,22 @@ static errcode_t unix_flush(io_channel channel)return retval; } +/* +* Flush data buffers to disk. +*/ +static errcode_t unix_flush (io_channel channel) +{ +return _unix_flush (channel,0 ); +} + +/* +* Flush data buffers to disk and invalidate cache. +*/ +static errcode_t unix_flush_cleanup (io_channel channel) +{ +return _unix_flush (channel,1 ); +} +static errcode_t unix_set_option (io_channel channel,const char *option,const char *arg) {
@@ -1225,6 +1241,7 @@ static struct struct_io_manager struct_unix_manager = { .discard = unix_discard, .cache_readahead = unix_cache_readahead, .zeroout = unix_zeroout, + .flush_cleanup = unix_flush_cleanup, };io_manager unix_io_manager = &struct_unix_manager;
@@ -1246,6 +1263,7 @@ static struct struct_io_manager struct_unixfd_manager = { .discard = unix_discard, .cache_readahead = unix_cache_readahead, .zeroout = unix_zeroout, + .flush_cleanup = unix_flush_cleanup, };io_manager unixfd_io_manager = &struct_unixfd_manager;
Signature · 1 line
2.29.2.299.gdc1121823c-goog
Re: [RFC PATCH v3 08/61] e2fsck: open io-channel when copying fs
Quoted 11 lines · Saranya Muruganandam
From: Li Xi <[email protected]>
This patch also add writethrough flag to the thread io-channel. When multiple threads write the same disk, we don't want the data being saved in memory cache. This will be useful in the future, but even without that flag, the tests can be passed too.
This patch also cleanup the io channel cache of the global context. Otherwise, after pass1 step, the next steps would use old data saved in the cache. And the cached data might have already been overwritten in pass1.
See my previous comments about why io_managers will almost certainly need to be thread-aware. This commit modifies undo_io.c, but it can't possibly work as-is, since you can't have multiple copies of the undo manager from different threads trying to update a single undo file. So these changes, by themselves, can't possibly be sufficient.
Instead of doing things incrementally, my suggestion is that we figure out how to make io_managers thread-safe and we get it right *first*, instead of making incremental changes throughout the patch series. I'd also suggest that we figure out some kind of test framework so we can test io_managers in isolation, so we can do stress tests, as well as functional correctness tests as unit tests.
Once we have that, let's merge these changes in incrementally into e2fsprogs, so we can have clean, well-designed and well-tested low-level infrastructure, which will be easier for us to review.
lib/ext2fs/ext2_io.h struct struct_io_manager { · line 87
Quoted 12 lines
lib/ext2fs/ext2_io.h +1
-0@@ -81,6 +81,7 @@ struct struct_io_manager {errcode_t (*write_blk)(io_channel channel,unsigned long block,int count,const void *data);errcode_t (*flush)(io_channel channel); +errcode_t (*flush_cleanup)(io_channel channel);errcode_t (*write_byte)(io_channel channel,unsigned long offset,int count,const void *data); errcode_t (*set_option)(io_channel channel, const char *option,
Please don't add new functions into the middle of struct_io_manager. There is a long reserved[14] to add padding to the structure for a reason. Add a new function pointer just before the reserved[] array, and then decrement the padding count.
The reason for this is that it's technically allowed for applications to provide their own io_manager to the library, which may be stacked on top of an some other io_manager (as is the case for undo iomanager). Or it might because the userspace application is providing their own io manager to interface with some other OS --- maybe Windows, or Fuschia in the future, who knows?
So if we add a new function pointer to the middle of struct_io_manager, this breaks the ABI, and that's a Bad Thing, as it may cause surprises in the future for applications which are using shared libraries and we update with a newer version of the shared library without doing a major version bump of the shared library.
- Ted
Re: [RFC PATCH v3 08/61] e2fsck: open io-channel when copying fs
Quoted 8 lines · Theodore Y. Ts'o
Quoted 7 lines · Saranya Muruganandam
From: Li Xi <[email protected]>
This patch also add writethrough flag to the thread io-channel. When multiple threads write the same disk, we don't want the data being saved in memory cache. This will be useful in the future, but even without that flag, the tests can be passed too.
Quoted 4 lines
Quoted 4 lines
This patch also cleanup the io channel cache of the global context. Otherwise, after pass1 step, the next steps would use old data saved in the cache. And the cached data might have already been overwritten in pass1.
Thinking about this so more, what I'd suggest is the following.
1) We define a new flag, IO_FLAG_THREADED, which instructs the io_manager that it should provide thread-safety. For all io managers defined in libext2fs, if they do not support the flag, they will return a newly defined error code, EXT2_ET_IO_CHANNEL_NO_SUPPORT_FLAG. Ideally, all of them should support IO_FLAG_THREADED, even undo_io, even if that means just throwing a pthread rwlock around all of its read/write operations.
2) We also define a two options which can be set via io_channel_set_option(), so that the "writethrough" and "cache" options can be set to on or off via "writethrough=on", "cache=off", "cache=on", etc. This allows parallel reads to the unix io manager to avoid needing any locks --- since pthread doesn't support spinlocks, and mutexes are going to be rather expensive for highly contended operations such as parallel read operations when reading bitmaps.
(In fact, some of the bugs that I saw when playing with the parallel bitmap reading was probably cacused by the fact that there were multiple threads trying to play with unix_io's cache without any locking protection; and there's no real benefit to having any kind of caching when reading the block bitmaps, and if we spin off separate io_channels for each thread while reading the block bitmaps, the cache will be *purely* useless. The cache also doesn't do us much good when we are reading the bitmaps, or during pass 1, even if we aren't multi-threading, so having a way to turn off the cache is probably a mild performance improvement even in the non-threaded case.)
And as I think I mentioned earlier, let's do some of these preparatory changes to support parallel fsck *first*, since it'll be a lot easier to review a bunch of shorter, patch series which are less than, say, 3-6 patches at a time, and once those get in, we can do with the next step --- as opposed to a monster set of 61 commits.
- Ted
P.S. Finally, if we use a single shared io_manager which is threading aware, I don't think we'll need the flush_cleanup() operation proposed in this patch, since the cache will always be consistent.
RFC PATCH v3 9/61 e2fsck: create logs for mult-threads
In reply to Saranya MuruganandamFrom: Li Xi <[email protected]>
When multi-threads are used, different logs should be created for different threads. Each thread has log files with suffix of ".$THREAD_INDEX".
And this patch adds f_multithread_logfile test case.
--- e2fsck/e2fsck.h | 6 ++++-- e2fsck/logfile.c | 10 ++++++++- e2fsck/pass1.c | 23 +++++++++++++++----- tests/f_multithread_logfile/expect.1 | 23 ++++++++++++++++++++ tests/f_multithread_logfile/image.gz | 1 + tests/f_multithread_logfile/name | 1 + tests/f_multithread_logfile/script | 32 ++++++++++++++++++++++++++++ 7 files changed, 88 insertions(+), 8 deletions(-) create mode 100644 tests/f_multithread_logfile/expect.1 create mode 120000 tests/f_multithread_logfile/image.gz create mode 100644 tests/f_multithread_logfile/name create mode 100644 tests/f_multithread_logfile/script
e2fsck/e2fsck.h +4 -2
@@ -308,8 +308,10 @@ struct e2fsck_struct {/* * For pass1_check_directory and pass1_get_blocks */ -ext2_ino_t stashed_ino; -struct ext2_inode *stashed_inode; +ext2_ino_t stashed_ino; +struct ext2_inode *stashed_inode; +/* Thread index, if global_ctx is null, this field is unused */ +int thread_index;/* * Location of the lost and found directory
e2fsck/logfile.c +9 -1
@@ -16,6 +16,7 @@#include <sys/types.h> #include <sys/stat.h> #include <fcntl.h> +#include <assert.h> #include "e2fsck.h" #include <pwd.h>
@@ -291,6 +292,8 @@ static FILE *set_up_log_file(e2fsck_t ctx, const char *key, const char *fn)struct string s, s1, s2;char *s0 =0 , *log_dir =0 , *log_fn =0 ;int log_dir_wait =0 ; +int string_size; +char string_index[10 ]; s.s = s1.s = s2.s =0 ;
@@ -307,6 +310,12 @@ static FILE *set_up_log_file(e2fsck_t ctx, const char *key, const char *fn) goto out;expand_logfn (ctx, log_fn, &s); +if (ctx->global_ctx) { +sprintf (string_index,"%d" , ctx->thread_index); +append_string (&s,"." ,1 ); +append_string (&s, string_index,0 ); + } +if ((log_fn[0 ] =='/' ) || !log_dir || !log_dir[0 ]) s0 = s.s;
@@ -325,7 +334,6 @@ static FILE *set_up_log_file(e2fsck_t ctx, const char *key, const char *fn)append_string (&s2, log_dir,0 );append_string (&s2,"/" ,1 );append_string (&s2, s.s,0 ); -printf ("%s\n" , s2.s); }if (s0)
e2fsck/pass1.c +18 -5
@@ -2250,6 +2250,9 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thre } thread_fs->priv_data = thread_context; + thread_context->thread_index =0 ; +set_up_logging (thread_context); + thread_context->fs = thread_fs; *thread_ctx = thread_context;return 0 ;
@@ -2262,12 +2265,14 @@ out_context:static int e2fsck_pass1_thread_join_one (e2fsck_t global_ctx,e2fsck_t thread_ctx) { -errcode_t retval; -int flags = global_ctx->flags; -ext2_filsys thread_fs = thread_ctx->fs; -ext2_filsys global_fs = global_ctx->fs; +errcode_t retval; +int flags = global_ctx->flags; +ext2_filsys thread_fs = thread_ctx->fs; +ext2_filsys global_fs = global_ctx->fs; +FILE *global_logf = global_ctx->logf; +FILE *global_problem_logf = global_ctx->problem_logf;#ifdef HAVE_SETJMP_H -jmp_buf old_jmp; +jmp_buf old_jmp;memcpy (old_jmp, global_ctx->abort_loc,sizeof (jmp_buf));#endif
@@ -2286,6 +2291,8 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx } global_fs->priv_data = global_ctx; global_ctx->fs = global_fs; + global_ctx->logf = global_logf; + global_ctx->problem_logf = global_problem_logf;if (thread_ctx->inode_used_map) { retval = e2fsck_pass1_copy_bitmap(global_fs,
@@ -2374,6 +2381,12 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx) retval =e2fsck_pass1_thread_join_one (global_ctx, thread_ctx);ext2fs_free_mem (&thread_ctx->fs); +if (thread_ctx->logf) +fclose (thread_ctx->logf); +if (thread_ctx->problem_logf) { +fputs ("</problem_log>\n" , thread_ctx->problem_logf); +fclose (thread_ctx->problem_logf); + }ext2fs_free_mem (&thread_ctx);return retval;
tests/f_multithread_logfile/expect.1 new file +23 -0
@@ -0,0 +1,23 @@ +ext2fs_open2: Bad magic number in super-block +../e2fsck/e2fsck: Superblock invalid, trying backup blocks... +Pass 1: Checking inodes, blocks, and sizes +Pass 2: Checking directory structure +Pass 3: Checking directory connectivity +Pass 4: Checking reference counts +Pass 5: Checking group summary information +Free blocks count wrong for group #0 (7987, counted=7982). +Fix? yes + +Free blocks count wrong (11602, counted=11597). +Fix? yes + +Free inodes count wrong for group #0 (1493, counted=1488). +Fix? yes + +Free inodes count wrong (2997, counted=2992). +Fix? yes + + +test_filesys: ***** FILE SYSTEM WAS MODIFIED ***** +test_filesys: 16/3008 files (0.0% non-contiguous), 403/12000 blocks +Exit status is 1
tests/f_multithread_logfile/image.gz new file +1 -0
@@ -0,0 +1 @@ +../f_zero_super/image.gz
tests/f_multithread_logfile/name new file +1 -0
@@ -0,0 +1 @@ +test "e2fsck -m" option works with "-E log_filename="
tests/f_multithread_logfile/script new file +32 -0
@@ -0,0 +1,32 @@ +LOG_FNAME="f_multithread_logfile_xxx" +FSCK_OPT="-fy -m -y -E log_filename=$LOG_FNAME" +SKIP_VERIFY="true" +ONE_PASS_ONLY="true" +SKIP_CLEANUP="true" + +rm -f $LOG_FNAME.* $LOG_FNAME + +. $cmd_dir/run_e2fsck + +rm -f $test_name.ok $test_name.failed +cmp -s $OUT1 $EXP1 +status1=$? + +if [ "$status1" -eq 0 ]; then + if [ ! -f $LOG_FNAME -o ! -f $LOG_FNAME.0 ]; then + echo "$LOG_FNAME or $LOG_FNAME.0 is not created" > $test_name.failed + echo "$test_name: $test_description: failed" + else + echo "$test_name: $test_description: ok" + touch $test_name.ok + fi +else + diff $DIFF_OPTS $test_dir/expect.1 \ + $test_name.1.log >> $test_name.failed + echo "$test_name: $test_description: failed" +fi + +unset IMAGE FSCK_OPT SECOND_FSCK_OPT OUT1 OUT2 EXP1 EXP2 +unset SKIP_VERIFY SKIP_CLEANUP SKIP_GUNZIP ONE_PASS_ONLY PREP_CMD +unset DESCRIPTION SKIP_UNLINK AFTER_CMD PASS_ZERO +unset LOG_FINAME
Signature · 1 line
2.29.2.299.gdc1121823c-goog
Re: [RFC PATCH v3 09/61] e2fsck: create logs for mult-threads
Quoted 7 lines · Saranya Muruganandam
From: Li Xi <[email protected]>
When multi-threads are used, different logs should be created for different threads. Each thread has log files with suffix of ".$THREAD_INDEX".
And this patch adds
f_multithread_logfiletest case.
I'm not convinced this is the best approach; why not add a (thread-aware) log abstraction? That way there can be a single logfile, which is going to be much more user/admin/SRE-friendly.
We also need to add some kind of mutex if we are multi-threading in e2fsck/message.c, since POSIX guarantees that the dio functions are thread-safe --- but only on a per function basis. So if we don't want the output to the terminal to potentially be scrambled, we need to take a mutex in print_e2fsck_message() if we are in multi-threading mode, so that only one thread is printing to the terminal at a time.
(And certainly if we are going to be asking a question of the user, we *definitely* need to be taking some kind of mutex first!)
- Ted
RFC PATCH v3 10/61 e2fsck: optionally configure one pfsck thread
In reply to Saranya MuruganandamFrom: Li Xi <[email protected]>
This patch creates only one thread to do pass1 check. The same codes can be used to create multiple threads, but other functions need to be modified to get ready for that.
pfsck support will be enabled with if configured with --enable-pfsck option.
--- configure | 81 +++++++++++++++++++----- configure.ac | 19 ++++++ e2fsck/e2fsck.h | 13 ++++ e2fsck/logfile.c | 2 + e2fsck/pass1.c | 153 ++++++++++++++++++++++++++++++++++++++++------ e2fsck/unix.c | 10 +++ lib/config.h.in | 3 + tests/test_one.in | 8 +++ 8 files changed, 256 insertions(+), 33 deletions(-)
configure +66 -15
@@ -755,6 +755,7 @@ SET_MAKE VERSION PACKAGE GETTEXT_PACKAGE +PTHREAD_LIB TDB_MAN_COMMENT TDB_CMT UUIDD_CMT
@@ -847,7 +848,6 @@ infodir docdir oldincludedir includedir -runstatedir localstatedir sharedstatedir sysconfdir
@@ -912,6 +912,7 @@ enable_mmp enable_tdb enable_bmap_stats enable_bmap_stats_ops +enable_pfsck enable_nls enable_threads with_gnu_ld
@@ -984,7 +985,6 @@ datadir='${datarootdir}' sysconfdir='${prefix}/etc' sharedstatedir='${prefix}/com' localstatedir='${prefix}/var' -runstatedir='${localstatedir}/run' includedir='${prefix}/include' oldincludedir='/usr/include' docdir='${datarootdir}/doc/${PACKAGE}'
@@ -1237,15 +1237,6 @@ do | -silent | --silent | --silen | --sile | --sil) silent=yes ;; - -runstatedir | --runstatedir | --runstatedi | --runstated \ - | --runstate | --runstat | --runsta | --runst | --runs \ - | --run | --ru | --r) - ac_prev=runstatedir ;; - -runstatedir=* | --runstatedir=* | --runstatedi=* | --runstated=* \ - | --runstate=* | --runstat=* | --runsta=* | --runst=* | --runs=* \ - | --run=* | --ru=* | --r=*) - runstatedir=$ac_optarg ;; - -sbindir | --sbindir | --sbindi | --sbind | --sbin | --sbi | --sb) ac_prev=sbindir ;; -sbindir=* | --sbindir=* | --sbindi=* | --sbind=* | --sbin=* \
@@ -1383,7 +1374,7 @@ fi for ac_var in exec_prefix prefix bindir sbindir libexecdir datarootdir \ datadir sysconfdir sharedstatedir localstatedir includedir \ oldincludedir docdir infodir htmldir dvidir pdfdir psdir \ - libdir localedir mandir runstatedir + libdir localedir mandir do eval ac_val=\$$ac_var # Remove trailing slashes.
@@ -1536,7 +1527,6 @@ Fine tuning of the installation directories: --sysconfdir=DIR read-only single-machine data [PREFIX/etc] --sharedstatedir=DIR modifiable architecture-independent data [PREFIX/com] --localstatedir=DIR modifiable single-machine data [PREFIX/var] - --runstatedir=DIR modifiable per-process data [LOCALSTATEDIR/run] --libdir=DIR object code libraries [EPREFIX/lib] --includedir=DIR C header files [PREFIX/include] --oldincludedir=DIR C header files for non-gcc [/usr/include]
@@ -1599,6 +1589,7 @@ Optional Features: --disable-tdb disable tdb support --disable-bmap-stats disable collection of bitmap stats. --enable-bmap-stats-ops enable collection of additional bitmap stats + --enable-pfsck enable parallel e2fsck --disable-nls do not use Native Language Support --enable-threads={posix|solaris|pth|windows} specify multithreading API
@@ -6121,6 +6112,68 @@ $as_echo "Disabling additional bitmap statistics by default" >&6; } fi +PTHREAD_LIB='' +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for pthread_join in -lpthread" >&5 +$as_echo_n "checking for pthread_join in -lpthread... " >&6; } +if ${ac_cv_lib_pthread_pthread_join+:} false; then : + $as_echo_n "(cached) " >&6 +else + ac_check_lib_save_LIBS=$LIBS +LIBS="-lpthread $LIBS" +cat confdefs.h - <<_ACEOF >conftest.$ac_ext +/* end confdefs.h. */ + +/* Override any GCC internal prototype to avoid an error. + Use char because int might match the return type of a GCC + builtin and then its argument prototype would still apply. */ +#ifdef __cplusplus +extern "C" +#endif +char pthread_join (); +int +main () +{ +return pthread_join (); + ; + return 0; +} +_ACEOF +if ac_fn_c_try_link "$LINENO"; then : + ac_cv_lib_pthread_pthread_join=yes +else + ac_cv_lib_pthread_pthread_join=no +fi +rm -f core conftest.err conftest.$ac_objext \ + conftest$ac_exeext conftest.$ac_ext +LIBS=$ac_check_lib_save_LIBS +fi +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_pthread_pthread_join" >&5 +$as_echo "$ac_cv_lib_pthread_pthread_join" >&6; } +if test "x$ac_cv_lib_pthread_pthread_join" = xyes; then : + PTHREAD_LIB=-pthread +fi + + +# Check whether --enable-pfsck was given. +if test "${enable_pfsck+set}" = set; then : + enableval=$enable_pfsck; if test "$enableval" = "no" || test -z "PTHREAD_LIB" +then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: Disabling parallel e2fsck" >&5 +$as_echo "Disabling parallel e2fsck" >&6; } +else + +$as_echo "#define CONFIG_PFSCK 1" >>confdefs.h + + { $as_echo "$as_me:${as_lineno-$LINENO}: result: Enabling parallel e2fsck" >&5 +$as_echo "Enabling parallel e2fsck" >&6; } +fi + +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: Disabling parallel e2fsck" >&5 +$as_echo "Disabling parallel e2fsck" >&6; } + +fi + MAKEFILE_LIBRARY=$srcdir/lib/Makefile.library GETTEXT_PACKAGE=e2fsprogs
@@ -7338,8 +7391,6 @@ main () if (*(data + i) != *(data3 + i)) return 14; close (fd); - free (data); - free (data3); return 0; } _ACEOF
configure.ac +19 -0
@@ -877,6 +877,25 @@ fi AC_MSG_RESULT([Disabling additional bitmap statistics by default]) ) dnl +dnl handle --enable-pfsck +dnl +PTHREAD_LIB='' +AC_CHECK_LIB(pthread,pthread_join,PTHREAD_LIB=-pthread) +AC_SUBST(PTHREAD_LIB) +AC_ARG_ENABLE([pfsck], +[ --enable-pfsck enable parallel e2fsck], +if test "$enableval" = "no" || test -z "PTHREAD_LIB" +then + AC_MSG_RESULT([Disabling parallel e2fsck]) +else + AC_DEFINE(CONFIG_PFSCK, 1, + [Define to 1 if parallel e2fsck is enabled]) + AC_MSG_RESULT([Enabling parallel e2fsck]) +fi +, +AC_MSG_RESULT([Disabling parallel e2fsck]) +) +dnl dnl dnl MAKEFILE_LIBRARY=$srcdir/lib/Makefile.library
e2fsck/e2fsck.h +13 -0
@@ -426,6 +426,19 @@ struct e2fsck_struct {char *undo_file; }; +#ifdef CONFIG_PFSCK +struct e2fsck_thread_info { +/* ID returned by pthread_create() */ +pthread_t eti_thread_id; +/* Application-defined thread index */ +int eti_thread_index; +/* Thread has been started */ +int eti_started; +/* Context used for this thread */ +e2fsck_t eti_thread_ctx; +}; +#endif +/* Data structures to evaluate whether an extent tree needs rebuilding. */ struct extent_tree_level {unsigned int num_extents;
e2fsck/logfile.c +2 -0
@@ -310,11 +310,13 @@ static FILE *set_up_log_file(e2fsck_t ctx, const char *key, const char *fn) goto out;expand_logfn (ctx, log_fn, &s); +#ifdef CONFIG_PFSCK if (ctx->global_ctx) {sprintf (string_index,"%d" , ctx->thread_index);append_string (&s,"." ,1 );append_string (&s, string_index,0 ); } +#endif if ((log_fn[0 ] =='/' ) || !log_dir || !log_dir[0 ]) s0 = s.s;
e2fsck/pass1.c +135 -18
197 lines not on this page · raw message
e2fsck/unix.c +10 -0
@@ -82,7 +82,9 @@ static void usage(e2fsck_t ctx) fprintf(stderr, "%s", _("\nEmergency help:\n" " -p Automatic repair (no questions)\n" +#ifdef CONFIG_PFSCK " -m multiple threads to speedup fsck\n" +#endif " -n Make no changes to the filesystem\n" " -y Assume \"yes\" to all questions\n" " -c Check for bad blocks and add them to the badblock list\n"
@@ -849,7 +851,11 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx) phys_mem_kb =get_memory_size () /1024 ; ctx->readahead_kb = ~0ULL ; +#ifdef CONFIG_PFSCK while ((c =getopt (argc, argv,"pamnyrcC:B:dE:fvtFVM:b:I:j:P:l:L:N:SsDkz:" )) !=EOF ) +#else + while ((c = getopt(argc, argv, "panyrcC:B:dE:fvtFVM:b:I:j:P:l:L:N:SsDkz:")) != EOF) +#endif switch (c) {case 'C' : ctx->progress = e2fsck_update_progress;
@@ -890,9 +896,11 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx) } ctx->options |=E2F_OPT_PREEN ;break ; +#ifdef CONFIG_PFSCK case 'm' : ctx->options |=E2F_OPT_MULTITHREAD ;break ; +#endif case 'n' :if (ctx->options & (E2F_OPT_YES |E2F_OPT_PREEN )) goto conflict_opt;
@@ -1011,6 +1019,7 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx) _("The -n and -l/-L options are incompatible."));fatal_error (ctx,0 ); } +#ifdef CONFIG_PFSCK if (ctx->options &E2F_OPT_MULTITHREAD ) {if ((ctx->options & (E2F_OPT_YES |E2F_OPT_NO |E2F_OPT_PREEN )) ==0 ) { com_err(ctx->program_name, 0, "%s",
@@ -1023,6 +1032,7 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx) fatal_error(ctx, 0); } } +#endif if (ctx->options &E2F_OPT_NO ) ctx->options |=E2F_OPT_READONLY ;
lib/config.h.in +3 -0
@@ -18,6 +18,9 @@ /* Define to 1 to enable mmp support */ #undef CONFIG_MMP +/* Define to 1 if parallel e2fsck is enabled */ +#undef CONFIG_PFSCK + /* Define to 1 to enable tdb support */ #undef CONFIG_TDB
tests/test_one.in +8 -0
@@ -27,6 +27,7 @@ esac test_dir=$1 cmd_dir=$SRCDIR +pfsck_enabled="no" if test "$TEST_CONFIG"x = x; then TEST_CONFIG=$SRCDIR/test_config
@@ -52,6 +53,13 @@ else test_description= fi +$FSCK --help 2>&1 | grep -q -w -- -m && pfsck_enabled=yes +if [ "$pfsck_enabled" != "yes" ] ; then + echo "$test_dir" | grep -q multithread && + echo "$test_name: $test_description: skipped (pfsck disabled)" && + exit 0 +fi + if [ -n "$SKIP_SLOW_TESTS" -a -f $test_dir/is_slow_test ]; then echo "$test_name: $test_description: skipped (slow test)" exit 0
Signature · 1 line
2.29.2.299.gdc1121823c-goog
Re: [RFC PATCH v3 10/61] e2fsck: optionally configure one pfsck thread
Quoted 8 lines · Saranya Muruganandam
From: Li Xi <[email protected]>
This patch creates only one thread to do pass1 check. The same codes can be used to create multiple threads, but other functions need to be modified to get ready for that.
pfsck support will be enabled with if configured with --enable-pfsck option.
This patch should probably be merged with patch #1 ("e2fsck: add -m option for multithread"). That will make it be *much* easier to review.
What I'd actually suggest is that we add some multi-threading support into libext2fs as separate commits, and put them *first* in the patch series. They should ideally have their own unit tests so they can be individually tested, but that way we can also review the library changes *first* to make sure they make sense, and that way the reviewer will understand what the library functions are before reviewing the e2fsck patches.
I'd then also follow that up with a commit which compiles e2fsck with the pthread library (and we need to make sure that is portable across different OS's, and that it works when linking both statically and dynamically), and then follow that up with e2fsck utility functions that make life easier for the multi-threading changes --- for example, a thread-safe logging abstraction.
Finally, I'd then add support to enable parallel fsck via a configure option, with the default controlled by whether or not pthread support is enabled. (My main concern will be that there may be some minimal C libraries which don't have threading support, when e2fsprogs is compiled in some embedded environment.)
I'm not convinced that it makes sense to have a command-line thread to enable multi-threading. What probably makes more sense is to have extended e2fsck option (using -E multithread, which takes an optional argument indicating how many threads should be used). In production, the default of whether or not multi-threading should be enabled would be via /etc/e2fsck.conf.
- Ted
RFC PATCH v3 11/61 e2fsck: add start/end group for thread
In reply to Saranya MuruganandamFrom: Li Xi <[email protected]>
When multi-threads are used for check, each thread needs to jump to different group in pass1 check. This patch adds the group jumping support. But still, only one thread is used to check.
--- e2fsck/e2fsck.h | 23 ++++++++++++++++-- e2fsck/logfile.c | 3 ++- e2fsck/pass1.c | 51 +++++++++++++++++++++++++++++++++------ e2fsck/problem.c | 5 ++++ e2fsck/problem.h | 3 +++ lib/ext2fs/ext2_err.et.in | 3 +++ 6 files changed, 78 insertions(+), 10 deletions(-)
e2fsck/e2fsck.h +21 -2
@@ -227,6 +227,22 @@ typedef struct e2fsck_struct *e2fsck_t;#define MAX_EXTENT_DEPTH_COUNT 5 +#ifdef CONFIG_PFSCK +/* +* Fields that used for multi-thread +*/ +struct e2fsck_thread { +/* Thread index */ +int et_thread_index; +/* The start group number for this thread */ +dgrp_t et_group_start; +/* The end (not included) group number for this thread*/ +dgrp_t et_group_end; +/* The next group number to check */ +dgrp_t et_group_next; +}; +#endif +struct e2fsck_struct {/* Global context to get the cancel flag */ e2fsck_t global_ctx;
@@ -310,8 +326,11 @@ struct e2fsck_struct {*/ ext2_ino_t stashed_ino;struct ext2_inode *stashed_inode; -/* Thread index, if global_ctx is null, this field is unused */ -int thread_index; + +/* if @global_ctx is null, this field is unused */ +#ifdef CONFIG_PFSCK +struct e2fsck_thread thread_info; +#endif /* * Location of the lost and found directory
e2fsck/logfile.c +2 -1
@@ -312,7 +312,8 @@ static FILE *set_up_log_file(e2fsck_t ctx, const char *key, const char *fn)expand_logfn (ctx, log_fn, &s);#ifdef CONFIG_PFSCK if (ctx->global_ctx) { -sprintf (string_index,"%d" , ctx->thread_index); +sprintf (string_index,"%d" , + ctx->thread_info.et_thread_index);append_string (&s,"." ,1 );append_string (&s, string_index,0 ); }
e2fsck/pass1.c +44 -7
96 lines not on this page · raw message
e2fsck/problem.c +5 -0
@@ -1269,6 +1269,11 @@ static struct e2fsck_problem problem_table[] = {N_ ("Encrypted @i %i has corrupt encryption @a.\n" ), PROMPT_CLEAR_INODE, 0, 0, 0, 0 }, +/* Failed to goto block group */ + { PR_1_SCAN_GOTO, +N_ ("failed to goto block group" ), + PROMPT_NONE, PR_FATAL, 0, 0, 0 }, +/* Pass 1b errors */ /* Pass 1B: Rescan for duplicate/bad blocks */
e2fsck/problem.h +3 -0
@@ -707,6 +707,9 @@ struct problem_context {/* Encrypted inode has corrupt encryption extended attribute */ #define PR_1_CORRUPT_ENCRYPTION_XATTR 0x01008B +/* Failed to goto block group */ +#define PR_1_SCAN_GOTO 0x0100A0 +/* * Pass 1b errors */
lib/ext2fs/ext2_err.et.in +3 -0
@@ -548,4 +548,7 @@ ec EXT2_ET_EA_INODE_CORRUPTED, ec EXT2_ET_NO_GDESC, "Group descriptors not loaded" +ec EXT2_ET_SCAN_FINISHED, + "Scanning finished" + end
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 12/61 e2fsck: split groups to different threads
In reply to Saranya MuruganandamFrom: Li Xi <[email protected]>
The start/end groups of a thread is calculated according to the thread number. But still, only one thread is used to check.
--- e2fsck/pass1.c | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-)
e2fsck/pass1.c +17 -6
@@ -2232,13 +2232,14 @@ static int e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src) }static errcode_t e2fsck_pass1_thread_prepare (e2fsck_t global_ctx,e2fsck_t *thread_ctx, -int thread_index) +int thread_index,int num_threads) {errcode_t retval;e2fsck_t thread_context;ext2_filsys thread_fs;ext2_filsys global_fs = global_ctx->fs;struct e2fsck_thread *tinfo; +dgrp_t average_group;assert (global_ctx->inode_used_map ==NULL );assert (global_ctx->inode_dir_map ==NULL );
@@ -2278,11 +2279,20 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thre thread_context->thread_info.et_thread_index = thread_index;set_up_logging (thread_context); -assert (thread_index ==0 ); +/* +* Distribute work to multiple threads: +* Each thread work on fs->group_desc_count / nthread groups. +*/ tinfo = &thread_context->thread_info; - tinfo->et_group_start =0 ; - tinfo->et_group_next =0 ; - tinfo->et_group_end = thread_fs->group_desc_count; + average_group = thread_fs->group_desc_count / num_threads; +if (average_group ==0 ) + average_group =1 ; + tinfo->et_group_start = average_group * thread_index; +if (thread_index == num_threads -1 ) + tinfo->et_group_end = thread_fs->group_desc_count; +else + tinfo->et_group_end = average_group * (thread_index +1 ); + tinfo->et_group_next = tinfo->et_group_start; thread_context->fs = thread_fs; *thread_ctx = thread_context;
@@ -2506,7 +2516,8 @@ static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo,for (i =0 ; i < num_threads; i++) { tmp_pinfo = &infos[i]; tmp_pinfo->eti_thread_index = i; - retval =e2fsck_pass1_thread_prepare (global_ctx, &thread_ctx, i); + retval =e2fsck_pass1_thread_prepare (global_ctx, &thread_ctx, + i, num_threads);if (retval) {com_err (global_ctx->program_name, retval,_ ("while preparing pass1 thread\n" ));
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 13/61 e2fsck: print thread log properly
In reply to Saranya MuruganandamFrom: Li Xi <[email protected]>
When multi-thread fsck is enabled, logs printed from multiple threads could overlap with each other. The overlap sometimes makes the logs unreadable because log_out() is used multiple times for a single line.
This patch adds leading [Thread XXX] to each logs if multi-thread is enabed by -m option.
This patch also adds message to show the group ranges and inode numbers for each thread, which is useful for debuging multi-thread check.
--- e2fsck/e2fsck.h | 4 ++ e2fsck/pass1.c | 22 +++++++++- e2fsck/problem.c | 6 +++ e2fsck/util.c | 61 ++++++++++++++++++++++++++-- tests/f_multithread/expect.1 | 4 +- tests/f_multithread_logfile/expect.1 | 4 +- tests/f_multithread_no/expect.1 | 4 +- 7 files changed, 98 insertions(+), 7 deletions(-)
e2fsck/e2fsck.h +4 -0
@@ -240,6 +240,10 @@ struct e2fsck_thread {dgrp_t et_group_end;/* The next group number to check */ dgrp_t et_group_next; +/* Scanned inode number */ +ext2_ino_t et_inode_number; +char et_log_buf[2048 ]; + char et_log_length; };#endif
e2fsck/pass1.c +21 -1
@@ -1441,6 +1441,10 @@ void e2fsck_pass1_run(e2fsck_t ctx) }if (!ino)break ; +#ifdef CONFIG_PFSCK +if (ctx->global_ctx) + ctx->thread_info.et_inode_number++; +#endif pctx.ino = ino; pctx.inode = inode; ctx->stashed_ino = ino;
@@ -2293,7 +2297,12 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *threelse tinfo->et_group_end = average_group * (thread_index +1 ); tinfo->et_group_next = tinfo->et_group_start; - + tinfo->et_inode_number =0 ; + tinfo->et_log_buf[0 ] ='\0' ; + tinfo->et_log_length =0 ; +if (thread_context->options &E2F_OPT_MULTITHREAD ) +log_out (thread_context,_ ("Scan group range [%d, %d)\n" ), + tinfo->et_group_start, tinfo->et_group_end); thread_context->fs = thread_fs; *thread_ctx = thread_context;return 0 ;
@@ -2334,6 +2343,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx global_ctx->fs = global_fs; global_ctx->logf = global_logf; global_ctx->problem_logf = global_problem_logf; + global_ctx->global_ctx =NULL ;if (thread_ctx->inode_used_map) { retval = e2fsck_pass1_copy_bitmap(global_fs,
@@ -2483,6 +2493,12 @@ static void *e2fsck_pass1_thread(void *arg)e2fsck_pass1_run (thread_ctx); out: +if (thread_ctx->options &E2F_OPT_MULTITHREAD ) +log_out (thread_ctx, +_ ("Scanned group range [%lu, %lu), inodes %lu\n" ), + thread_ctx->thread_info.et_group_start, + thread_ctx->thread_info.et_group_end, + thread_ctx->thread_info.et_inode_number);return NULL ; }
@@ -2618,6 +2634,10 @@ static errcode_t scan_callback(ext2_filsys fs,if (ctx->global_ctx) { tinfo = &ctx->thread_info; tinfo->et_group_next++; +if (ctx->options &E2F_OPT_DEBUG && + ctx->options &E2F_OPT_MULTITHREAD ) +log_out (ctx,_ ("group %d finished\n" ), + tinfo->et_group_next);if (tinfo->et_group_next >= tinfo->et_group_end)return EXT2_ET_SCAN_FINISHED ; }
e2fsck/problem.c +6 -0
@@ -2481,6 +2481,12 @@ int fix_problem(e2fsck_t ctx, problem_t code, struct problem_context *pctx)if (*message) message =_ (message);if (!suppress) { +#ifdef CONFIG_PFSCK +if ((ctx->options &E2F_OPT_MULTITHREAD ) && ctx->global_ctx) +printf ("[Thread %d] " , + ctx->thread_info.et_thread_index); +#endif +if ((ctx->options &E2F_OPT_PREEN ) && !(ptr->flags &PR_PREEN_NOHDR )) { printf("%s: ", ctx->device_name ?
e2fsck/util.c +58 -3
77 lines not on this page · raw message
tests/f_multithread/expect.1 +3 -1
@@ -1,6 +1,8 @@ ext2fs_open2: Bad magic number in super-block ../e2fsck/e2fsck: Superblock invalid, trying backup blocks... -Pass 1: Checking inodes, blocks, and sizes +[Thread 0] Scan group range [0, 2) +[Thread 0] Pass 1: Checking inodes, blocks, and sizes +[Thread 0] Scanned group range [0, 2), inodes 3008 Pass 2: Checking directory structure Pass 3: Checking directory connectivity Pass 4: Checking reference counts
tests/f_multithread_logfile/expect.1 +3 -1
@@ -1,6 +1,8 @@ ext2fs_open2: Bad magic number in super-block ../e2fsck/e2fsck: Superblock invalid, trying backup blocks... -Pass 1: Checking inodes, blocks, and sizes +[Thread 0] Scan group range [0, 2) +[Thread 0] Pass 1: Checking inodes, blocks, and sizes +[Thread 0] Scanned group range [0, 2), inodes 3008 Pass 2: Checking directory structure Pass 3: Checking directory connectivity Pass 4: Checking reference counts
tests/f_multithread_no/expect.1 +3 -1
@@ -1,6 +1,8 @@ ext2fs_open2: Bad magic number in super-block ../e2fsck/e2fsck: Superblock invalid, trying backup blocks... -Pass 1: Checking inodes, blocks, and sizes +[Thread 0] Scan group range [0, 2) +[Thread 0] Pass 1: Checking inodes, blocks, and sizes +[Thread 0] Scanned group range [0, 2), inodes 3008 Pass 2: Checking directory structure Pass 3: Checking directory connectivity Pass 4: Checking reference counts
Signature · 1 line
2.29.2.299.gdc1121823c-goog
Re: [RFC PATCH v3 13/61] e2fsck: print thread log properly
Quoted 9 lines · Saranya Muruganandam
From: Li Xi <[email protected]>
When multi-thread fsck is enabled, logs printed from multiple threads could overlap with each other. The overlap sometimes makes the logs unreadable because
log_out()is used multiple times for a single line.This patch adds leading [Thread XXX] to each logs if multi-thread is enabed by -m option.
Given that the leading "[Thread XXX]" is output using a separate printf, this is still going to result in potential overlapped messages:
e2fsck/util.c out: · line 100
Quoted 9 lines
e2fsck/util.c +9
-0+#ifdef CONFIG_PFSCK +static void thread_log_out (struct e2fsck_thread *tinfo) +{ +printf ("[Thread %d] %s" , tinfo->et_thread_index, + tinfo->et_log_buf); + tinfo->et_log_length =0 ; + tinfo->et_log_buf[0 ] ='\0' ; +} +#endif
Instead of using a separate 2k buffer for each thread, why not just have a mutex so only one thread can be printing to the terminal and/or log files at one time?
- Ted
RFC PATCH v3 14/61 e2fsck: merge bitmaps after thread completes
In reply to Saranya MuruganandamFrom: Wang Shilong <[email protected]>
A new method merge_bmap has been added to bitmap operations. But only red-black bitmap has that operation now.
--- e2fsck/pass1.c | 239 +++++++++++++++++++++++--------------- lib/ext2fs/bitmaps.c | 10 ++ lib/ext2fs/blkmap64_rb.c | 65 +++++++++++ lib/ext2fs/bmap64.h | 4 + lib/ext2fs/ext2fs.h | 8 ++ lib/ext2fs/gen_bitmap64.c | 29 +++++ 6 files changed, 264 insertions(+), 91 deletions(-)
e2fsck/pass1.c +148 -91
@@ -2124,12 +2124,38 @@ static errcode_t e2fsck_pass1_copy_bitmap(ext2_filsys fs, ext2fs_generic_bitmapreturn ret; (*dest)->fs = fs; -ext2fs_free_generic_bmap (*src); - *src =NULL ;return 0 ; } +static void e2fsck_pass1_free_bitmap (ext2fs_generic_bitmap *bitmap) +{ +if (*bitmap) { +ext2fs_free_generic_bmap (*bitmap); + *bitmap =NULL ; + } + +} + +static errcode_t e2fsck_pass1_merge_bitmap (ext2_filsys fs,ext2fs_generic_bitmap *src, +ext2fs_generic_bitmap *dest) +{ +errcode_t ret =0 ; + +if (*src) { +if (*dest ==NULL ) { + *dest = *src; + *src =NULL ; + }else { + ret =ext2fs_merge_bitmap (*src, *dest,NULL ,NULL ); +if (ret) +return ret; + } + (*dest)->fs = fs; + } + +return 0 ; +}static errcode_t e2fsck_pass1_copy_fs (ext2_filsys dest,e2fsck_t src_context,ext2_filsys src)
@@ -2137,6 +2163,8 @@ static errcode_t e2fsck_pass1_copy_fs(ext2_filsys dest, e2fsck_t src_context,errcode_t retval;memcpy (dest, src,sizeof (struct struct_ext2_filsys )); + dest->inode_map =NULL ; + dest->block_map =NULL ;if (dest->dblist) dest->dblist->fs = dest;if (src->block_map) {
@@ -2196,42 +2224,50 @@ static int e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)errcode_t retval =0 ;io_channel dest_io;io_channel dest_image_io; +ext2fs_inode_bitmap inode_map; +ext2fs_block_bitmap block_map; dest_io = dest->io; dest_image_io = dest->image_io; + inode_map = dest->inode_map; + block_map = dest->block_map;memcpy (dest, src,sizeof (struct struct_ext2_filsys )); dest->io = dest_io; dest->image_io = dest_image_io; dest->icache = icache; + dest->inode_map = inode_map; + dest->block_map = block_map;if (dest->dblist) dest->dblist->fs = dest; -if (src->inode_map) { - retval =e2fsck_pass1_copy_bitmap (dest, &src->inode_map, - &dest->inode_map); -if (retval) -return retval; - } -if (src->block_map) { - retval =e2fsck_pass1_copy_bitmap (dest, &src->block_map, - &dest->block_map); -if (retval) -return retval; - }if (src->icache) {ext2fs_free_inode_cache (src->icache); src->icache =NULL ; } + retval =e2fsck_pass1_merge_bitmap (dest, &src->inode_map, + &dest->inode_map); +if (retval) + goto out; + + retval =e2fsck_pass1_merge_bitmap (dest, &src->block_map, + &dest->block_map); +if (retval) + goto out; +if (src->badblocks) { retval =ext2fs_badblocks_copy (src->badblocks, &dest->badblocks);ext2fs_badblocks_list_free (src->badblocks); src->badblocks =NULL ; } - +out:io_channel_close (src->io); +if (src->inode_map) +ext2fs_free_generic_bmap (src->inode_map); +if (src->block_map) +ext2fs_free_generic_bmap (src->block_map);return retval; }
@@ -2321,6 +2357,18 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctxext2_filsys global_fs = global_ctx->fs;FILE *global_logf = global_ctx->logf;FILE *global_problem_logf = global_ctx->problem_logf; +ext2fs_inode_bitmap inode_bad_map = global_ctx->inode_bad_map; +ext2fs_inode_bitmap inode_used_map = global_ctx->inode_used_map; +ext2fs_inode_bitmap inode_dir_map = global_ctx->inode_dir_map; +ext2fs_inode_bitmap inode_bb_map = global_ctx->inode_bb_map; +ext2fs_inode_bitmap inode_imagic_map = global_ctx->inode_imagic_map; +ext2fs_inode_bitmap inode_reg_map = global_ctx->inode_reg_map; +ext2fs_block_bitmap block_found_map = global_ctx->block_found_map; +ext2fs_block_bitmap block_dup_map = global_ctx->block_dup_map; +ext2fs_block_bitmap block_ea_map = global_ctx->block_ea_map; +ext2fs_block_bitmap block_metadata_map = global_ctx->block_metadata_map; +ext2fs_block_bitmap inodes_to_rebuild = global_ctx->inodes_to_rebuild; +#ifdef HAVE_SETJMP_H jmp_buf old_jmp;
@@ -2330,6 +2378,19 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx#ifdef HAVE_SETJMP_H memcpy (global_ctx->abort_loc, old_jmp,sizeof (jmp_buf));#endif + + global_ctx->inode_used_map = inode_used_map; + global_ctx->inode_bad_map = inode_bad_map; + global_ctx->inode_dir_map = inode_dir_map; + global_ctx->inode_bb_map = inode_bb_map; + global_ctx->inode_imagic_map = inode_imagic_map; + global_ctx->inodes_to_rebuild = inodes_to_rebuild; + global_ctx->inode_reg_map = inode_reg_map; + global_ctx->block_found_map = block_found_map; + global_ctx->block_dup_map = block_dup_map; + global_ctx->block_ea_map = block_ea_map; + global_ctx->block_metadata_map = block_metadata_map; +/* Keep the global singal flags*/ global_ctx->flags |= (flags &E2F_FLAG_SIGNAL_MASK ) | (global_ctx->flags &E2F_FLAG_SIGNAL_MASK );
@@ -2345,83 +2406,62 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx global_ctx->problem_logf = global_problem_logf; global_ctx->global_ctx =NULL ; -if (thread_ctx->inode_used_map) { - retval =e2fsck_pass1_copy_bitmap (global_fs, - &thread_ctx->inode_used_map, - &global_ctx->inode_used_map); -if (retval) -return retval; - } -if (thread_ctx->inode_bad_map) { - retval =e2fsck_pass1_copy_bitmap (global_fs, - &thread_ctx->inode_bad_map, - &global_ctx->inode_bad_map); -if (retval) -return retval; - } -if (thread_ctx->inode_dir_map) { - retval =e2fsck_pass1_copy_bitmap (global_fs, + retval =e2fsck_pass1_merge_bitmap (global_fs, + &thread_ctx->inode_used_map, + &global_ctx->inode_used_map); +if (retval) +return retval; + + retval =e2fsck_pass1_merge_bitmap (global_fs, + &thread_ctx->inode_bad_map, + &global_ctx->inode_bad_map); +if (retval) +return retval; + retval =e2fsck_pass1_merge_bitmap (global_fs, &thread_ctx->inode_dir_map, &global_ctx->inode_dir_map); -if (retval) -return retval; - } -if (thread_ctx->inode_bb_map) { - retval =e2fsck_pass1_copy_bitmap (global_fs, - &thread_ctx->inode_bb_map, - &global_ctx->inode_bb_map); -if (retval) -return retval; - } -if (thread_ctx->inode_imagic_map) { - retval =e2fsck_pass1_copy_bitmap (global_fs, - &thread_ctx->inode_imagic_map, - &global_ctx->inode_imagic_map); -if (retval) -return retval; - } -if (thread_ctx->inode_reg_map) { - retval =e2fsck_pass1_copy_bitmap (global_fs, - &thread_ctx->inode_reg_map, - &global_ctx->inode_reg_map); -if (retval) -return retval; - } -if (thread_ctx->inodes_to_rebuild) { - retval =e2fsck_pass1_copy_bitmap (global_fs, - &thread_ctx->inodes_to_rebuild, - &global_ctx->inodes_to_rebuild); -if (retval) -return retval; - } -if (thread_ctx->block_found_map) { - retval =e2fsck_pass1_copy_bitmap (global_fs, - &thread_ctx->block_found_map, - &global_ctx->block_found_map); -if (retval) -return retval; - } -if (thread_ctx->block_dup_map) { - retval =e2fsck_pass1_copy_bitmap (global_fs, - &thread_ctx->block_dup_map, - &global_ctx->block_dup_map); -if (retval) -return retval; - } -if (thread_ctx->block_ea_map) { - retval =e2fsck_pass1_copy_bitmap (global_fs, - &thread_ctx->block_ea_map, - &global_ctx->block_ea_map); -if (retval) -return retval; - } -if (thread_ctx->block_metadata_map) { - retval =e2fsck_pass1_copy_bitmap (global_fs, - &thread_ctx->block_metadata_map, - &global_ctx->block_metadata_map); -if (retval) -return retval; - } +if (retval) +return retval; + retval =e2fsck_pass1_merge_bitmap (global_fs, + &thread_ctx->inode_bb_map, + &global_ctx->inode_bb_map); +if (retval) +return retval; + retval =e2fsck_pass1_merge_bitmap (global_fs, + &thread_ctx->inode_imagic_map, + &global_ctx->inode_imagic_map); +if (retval) +return retval; + retval =e2fsck_pass1_merge_bitmap (global_fs, + &thread_ctx->inode_reg_map, + &global_ctx->inode_reg_map); +if (retval) +return retval; + retval =e2fsck_pass1_merge_bitmap (global_fs, + &thread_ctx->inodes_to_rebuild, + &global_ctx->inodes_to_rebuild); +if (retval) +return retval; + retval =e2fsck_pass1_merge_bitmap (global_fs, + &thread_ctx->block_found_map, + &global_ctx->block_found_map); +if (retval) +return retval; + retval =e2fsck_pass1_merge_bitmap (global_fs, + &thread_ctx->block_dup_map, + &global_ctx->block_dup_map); +if (retval) +return retval; + retval =e2fsck_pass1_merge_bitmap (global_fs, + &thread_ctx->block_ea_map, + &global_ctx->block_ea_map); +if (retval) +return retval; + retval =e2fsck_pass1_merge_bitmap (global_fs, + &thread_ctx->block_metadata_map, + &global_ctx->block_metadata_map); +if (retval) +return retval;return 0 ; }
@@ -2438,6 +2478,17 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)fputs ("</problem_log>\n" , thread_ctx->problem_logf);fclose (thread_ctx->problem_logf); } +e2fsck_pass1_free_bitmap (&thread_ctx->inode_used_map); +e2fsck_pass1_free_bitmap (&thread_ctx->inode_bad_map); +e2fsck_pass1_free_bitmap (&thread_ctx->inode_dir_map); +e2fsck_pass1_free_bitmap (&thread_ctx->inode_bb_map); +e2fsck_pass1_free_bitmap (&thread_ctx->inode_imagic_map); +e2fsck_pass1_free_bitmap (&thread_ctx->inode_reg_map); +e2fsck_pass1_free_bitmap (&thread_ctx->inodes_to_rebuild); +e2fsck_pass1_free_bitmap (&thread_ctx->block_found_map); +e2fsck_pass1_free_bitmap (&thread_ctx->block_dup_map); +e2fsck_pass1_free_bitmap (&thread_ctx->block_ea_map); +e2fsck_pass1_free_bitmap (&thread_ctx->block_metadata_map);ext2fs_free_mem (&thread_ctx);return retval;
@@ -2464,7 +2515,13 @@ static int e2fsck_pass1_threads_join(struct e2fsck_thread_info *infos,if (ret ==0 ) ret = rc; } -e2fsck_pass1_thread_join (global_ctx, infos[i].eti_thread_ctx); + rc =e2fsck_pass1_thread_join (global_ctx, infos[i].eti_thread_ctx); +if (rc) { +com_err (global_ctx->program_name, rc, +_ ("while joining pass1 thread\n" )); +if (ret ==0 ) + ret = rc; + } }free (infos);
lib/ext2fs/bitmaps.c +10 -0
@@ -45,6 +45,16 @@ errcode_t ext2fs_copy_bitmap(ext2fs_generic_bitmap src, {return (ext2fs_copy_generic_bmap (src, dest)); } + +errcode_t ext2fs_merge_bitmap (ext2fs_generic_bitmap src, +ext2fs_generic_bitmap dest, +ext2fs_generic_bitmap dup, +ext2fs_generic_bitmap dup_allowed) +{ +return ext2fs_merge_generic_bmap (src, dest, dup, + dup_allowed); +} +void ext2fs_set_bitmap_padding (ext2fs_generic_bitmap map) {ext2fs_set_generic_bmap_padding (map);
lib/ext2fs/blkmap64_rb.c +65 -0
76 lines not on this page · raw message
lib/ext2fs/bmap64.h +4 -0
@@ -72,6 +72,10 @@ struct ext2_bitmap_ops {void (*free_bmap)(ext2fs_generic_bitmap_64 bitmap);errcode_t (*copy_bmap)(ext2fs_generic_bitmap_64 src,ext2fs_generic_bitmap_64 dest); +errcode_t (*merge_bmap)(ext2fs_generic_bitmap_64 src, +ext2fs_generic_bitmap_64 dest, +ext2fs_generic_bitmap_64 dup, +ext2fs_generic_bitmap_64 dup_allowed);errcode_t (*resize_bmap)(ext2fs_generic_bitmap_64 bitmap,__u64 new_end,__u64 new_real_end);
lib/ext2fs/ext2fs.h +8 -0
@@ -837,6 +837,10 @@ extern void ext2fs_free_block_bitmap(ext2fs_block_bitmap bitmap);extern void ext2fs_free_inode_bitmap (ext2fs_inode_bitmap bitmap);extern errcode_t ext2fs_copy_bitmap (ext2fs_generic_bitmap src,ext2fs_generic_bitmap *dest); +errcode_t ext2fs_merge_bitmap (ext2fs_generic_bitmap src, +ext2fs_generic_bitmap dest, +ext2fs_generic_bitmap dup, +ext2fs_generic_bitmap dup_allowed);extern errcode_t ext2fs_write_inode_bitmap (ext2_filsys fs);extern errcode_t ext2fs_write_block_bitmap (ext2_filsys fs);extern errcode_t ext2fs_read_inode_bitmap (ext2_filsys fs);
@@ -1433,6 +1437,10 @@ void ext2fs_set_generic_bmap_padding(ext2fs_generic_bitmap bmap);errcode_t ext2fs_resize_generic_bmap (ext2fs_generic_bitmap bmap,__u64 new_end,__u64 new_real_end); +errcode_t ext2fs_merge_generic_bmap (ext2fs_generic_bitmap gen_src, +ext2fs_generic_bitmap gen_dest, +ext2fs_generic_bitmap gen_dup, +ext2fs_generic_bitmap dup_allowed);errcode_t ext2fs_compare_generic_bmap (errcode_t neq,ext2fs_generic_bitmap bm1,ext2fs_generic_bitmap bm2);
lib/ext2fs/gen_bitmap64.c +29 -0
@@ -344,6 +344,35 @@ errcode_t ext2fs_copy_generic_bmap(ext2fs_generic_bitmap gen_src,return 0 ; } +errcode_t ext2fs_merge_generic_bmap (ext2fs_generic_bitmap gen_src, +ext2fs_generic_bitmap gen_dest, +ext2fs_generic_bitmap gen_dup, +ext2fs_generic_bitmap gen_dup_allowed) +{ +ext2fs_generic_bitmap_64 src = (ext2fs_generic_bitmap_64 )gen_src; +ext2fs_generic_bitmap_64 dest = (ext2fs_generic_bitmap_64 )gen_dest; +ext2fs_generic_bitmap_64 dup = (ext2fs_generic_bitmap_64 )gen_dup; +ext2fs_generic_bitmap_64 dup_allowed = (ext2fs_generic_bitmap_64 )gen_dup_allowed; + +if (!src || !dest) +return EINVAL ; + +if (!EXT2FS_IS_64_BITMAP (src) || !EXT2FS_IS_64_BITMAP (dest) || + (dup && !EXT2FS_IS_64_BITMAP (dup)) || + (dup_allowed && !EXT2FS_IS_64_BITMAP (dup_allowed))) +return EINVAL ; + +if (src->bitmap_ops != dest->bitmap_ops || + (dup && src->bitmap_ops != dup->bitmap_ops) || + (dup_allowed && src->bitmap_ops != dup_allowed->bitmap_ops)) +return EINVAL ; + +if (src->bitmap_ops->merge_bmap ==NULL ) +return EOPNOTSUPP ; + +return src->bitmap_ops->merge_bmap (src, dest, dup, dup_allowed); +} +errcode_t ext2fs_resize_generic_bmap (ext2fs_generic_bitmap gen_bmap,__u64 new_end,__u64 new_real_end)
Signature · 1 line
2.29.2.299.gdc1121823c-goog
Re: [RFC PATCH v3 14/61] e2fsck: merge bitmaps after thread completes
Quoted 8 lines · Saranya Muruganandam
From: Wang Shilong <[email protected]>
A new method
merge_bmaphas been added to bitmap operations. But only red-black bitmap has that operation now.Signed-off-by Li Xi <[email protected]>Signed-off-by Wang Shilong <[email protected]>Signed-off-by Saranya Muruganandam <[email protected]>
There are hacks to deal with the fact that only the read-block bitmap backend supports merge_bmap in this patch series. Why not implement that for *all* of the back ends, along with some unit tests; it's going to be less work than working around failures due to partial implementation of merge_bmap --- and the workarounds weren't complete in any case; you can force the use of a particular back-end via /etc/e2fsck.conf --- see the function e2fsck_allocate_{block,inode}_bmap() in e2fsck/util.c.
This was mainly way of forcing additional testing of the backend implementations, and also so we could test/tune whether certain backends were preferable for certain file systems for different ways in which block/inode bitmaps are used by e2fsck --- but it's possible for an sysadmin / SRE to explicitly set them on a production e2fsck, and it would be preferable if things didn't blow up because we took a shortcut and didn't implement merge_bmap for all of the bitmap
backends. (It's *really* not that hard, anyway....)
e2fsck/pass1.c static errcode_t e2fsck_pass1_copy_bitmap(ext2_filsys fs, ext2fs_generic_bitmap · line 2138
Quoted 8 lines
e2fsck/pass1.c +8
-0+static void e2fsck_pass1_free_bitmap (ext2fs_generic_bitmap *bitmap) +{ +if (*bitmap) { +ext2fs_free_generic_bmap (*bitmap); + *bitmap =NULL ; + } + +}
I'm not sure why this is needed? ext2fs_free_Generic_bmap() will already return if NULL is passed into it.
Also, by the end of the patch series, there are no callers of this function....
And again, please separate out the libext2fs changes from the e2fsck changes, do the libext2fs changes first, and there should really be some unit tests of merge_bmap so we can validate that it works correctly before we drop it into use with e2fsck.
There should also be better documentation of what dup and dup_allowed in merge_bmap().
- Ted
RFC PATCH v3 15/61 e2fsck: do not change global variables
In reply to Saranya MuruganandamFrom: Li Xi <[email protected]>
Global variables used in pass1 check are changed to local variables in this patch. This will avoid conflict between threads.
--- e2fsck/pass1.c | 75 +++++++++++++++++++++++++++++++------------------- 1 file changed, 47 insertions(+), 28 deletions(-)
e2fsck/pass1.c +47 -28
@@ -86,7 +86,6 @@ static void alloc_bb_map(e2fsck_t ctx);static void alloc_imagic_map (e2fsck_t ctx);static void mark_inode_bad (e2fsck_t ctx,ino_t ino);static void handle_fs_bad_blocks (e2fsck_t ctx); -static void process_inodes (e2fsck_t ctx,char *block_buf);static EXT2_QSORT_TYPE process_inode_cmp (const void *a,const void *b);static errcode_t scan_callback (ext2_filsys fs,ext2_inode_scan scan,dgrp_t group,void * priv_data);
@@ -121,15 +120,15 @@ struct process_inode_block { };struct scan_callback_struct { -e2fsck_t ctx; -char *block_buf; +e2fsck_t ctx; +char *block_buf; +struct process_inode_block *inodes_to_process; +int *process_inode_count; }; -/* -* For the inodes to process list. -*/ -static struct process_inode_block *inodes_to_process; -static int process_inode_count; +static void process_inodes (e2fsck_t ctx,char *block_buf, +struct process_inode_block *inodes_to_process, +int *process_inode_count);static __u64 ext2_max_sizes[EXT2_MAX_BLOCK_LOG_SIZE -EXT2_MIN_BLOCK_LOG_SIZE +1 ];
@@ -1168,7 +1167,6 @@ static int e2fsck_should_abort(e2fsck_t ctx)void e2fsck_pass1_run (e2fsck_t ctx) {int i; -__u64 max_sizes;ext2_filsys fs = ctx->fs;ext2_ino_t ino =0 ;struct ext2_inode *inode =NULL ;
@@ -1191,6 +1189,8 @@ void e2fsck_pass1_run(e2fsck_t ctx)ext2_ino_t ino_threshold =0 ;dgrp_t ra_group =0 ;struct ea_quota ea_ibody_quota; +struct process_inode_block *inodes_to_process; +int process_inode_count;init_resource_track (&rtrack, ctx->fs->io);clear_problem_context (&pctx);
@@ -1215,17 +1215,6 @@ void e2fsck_pass1_run(e2fsck_t ctx)mtrace_print ("Pass 1" );#endif -#define EXT2_BPP (bits) (1ULL << ((bits) - 2)) - -for (i =EXT2_MIN_BLOCK_LOG_SIZE ; i <=EXT2_MAX_BLOCK_LOG_SIZE ; i++) { - max_sizes =EXT2_NDIR_BLOCKS +EXT2_BPP (i); - max_sizes = max_sizes +EXT2_BPP (i) *EXT2_BPP (i); - max_sizes = max_sizes +EXT2_BPP (i) *EXT2_BPP (i) *EXT2_BPP (i); - max_sizes = (max_sizes * (1UL << i)); - ext2_max_sizes[i -EXT2_MIN_BLOCK_LOG_SIZE ] = max_sizes; - } -#undef EXT2_BPP - imagic_fs =ext2fs_has_feature_imagic_inodes (sb); extent_fs =ext2fs_has_feature_extents (sb); inlinedata_fs =ext2fs_has_feature_inline_data (sb);
@@ -1349,6 +1338,8 @@ void e2fsck_pass1_run(e2fsck_t ctx) ctx->stashed_inode = inode; scan_struct.ctx = ctx; scan_struct.block_buf = block_buf; + scan_struct.inodes_to_process = inodes_to_process; + scan_struct.process_inode_count = &process_inode_count;ext2fs_set_inode_callback (scan, scan_callback, &scan_struct);if (ctx->progress && ((ctx->progress)(ctx,1 ,0 , ctx->fs->group_desc_count)))
@@ -1997,13 +1988,15 @@ void e2fsck_pass1_run(e2fsck_t ctx) goto endit;if (process_inode_count >= ctx->process_inode_size) { -process_inodes (ctx, block_buf); +process_inodes (ctx, block_buf, inodes_to_process, + &process_inode_count);if (e2fsck_should_abort (ctx)) goto endit; } } -process_inodes (ctx, block_buf); +process_inodes (ctx, block_buf, inodes_to_process, + &process_inode_count);ext2fs_close_inode_scan (scan); scan =NULL ;
@@ -2113,6 +2106,27 @@ endit: ctx->invalid_bitmaps++; } +static void init_ext2_max_sizes () +{ +int i; +__u64 max_sizes; + +/* +* Init ext2_max_sizes which will be immutable and shared between +* threads +*/ +#define EXT2_BPP (bits) (1ULL << ((bits) - 2)) + +for (i =EXT2_MIN_BLOCK_LOG_SIZE ; i <=EXT2_MAX_BLOCK_LOG_SIZE ; i++) { + max_sizes =EXT2_NDIR_BLOCKS +EXT2_BPP (i); + max_sizes = max_sizes +EXT2_BPP (i) *EXT2_BPP (i); + max_sizes = max_sizes +EXT2_BPP (i) *EXT2_BPP (i) *EXT2_BPP (i); + max_sizes = (max_sizes * (1UL << i)); + ext2_max_sizes[i -EXT2_MIN_BLOCK_LOG_SIZE ] = max_sizes; + } +#undef EXT2_BPP +} +#ifdef CONFIG_PFSCK static errcode_t e2fsck_pass1_copy_bitmap (ext2_filsys fs,ext2fs_generic_bitmap *src,ext2fs_generic_bitmap *dest)
@@ -2656,6 +2670,7 @@ out_abort:void e2fsck_pass1 (e2fsck_t ctx) { +init_ext2_max_sizes ();#ifdef CONFIG_PFSCK e2fsck_pass1_multithread (ctx);#else
@@ -2680,7 +2695,9 @@ static errcode_t scan_callback(ext2_filsys fs, scan_struct = (struct scan_callback_struct *) priv_data; ctx = scan_struct->ctx; -process_inodes ((e2fsck_t ) fs->priv_data, scan_struct->block_buf); +process_inodes ((e2fsck_t ) fs->priv_data, scan_struct->block_buf, + scan_struct->inodes_to_process, + scan_struct->process_inode_count);if (ctx->progress) if ((ctx->progress)(ctx, 1, group+1,
@@ -2706,7 +2723,9 @@ static errcode_t scan_callback(ext2_filsys fs,/* * Process the inodes in the "inodes to process" list. */ -static void process_inodes (e2fsck_t ctx,char *block_buf) +static void process_inodes (e2fsck_t ctx,char *block_buf, +struct process_inode_block *inodes_to_process, +int *process_inode_count) {int i;struct ext2_inode *old_stashed_inode;
@@ -2718,15 +2737,15 @@ static void process_inodes(e2fsck_t ctx, char *block_buf)#if 0 printf ("begin process_inodes: " );#endif -if (process_inode_count ==0 ) +if (*process_inode_count ==0 )return ; old_operation =ehandler_operation (0 ); old_stashed_inode = ctx->stashed_inode; old_stashed_ino = ctx->stashed_ino; -qsort (inodes_to_process, process_inode_count, +qsort (inodes_to_process, *process_inode_count,sizeof (struct process_inode_block ), process_inode_cmp);clear_problem_context (&pctx); -for (i=0 ; i < process_inode_count; i++) { +for (i=0 ; i < *process_inode_count; i++) { pctx.inode = ctx->stashed_inode = (struct ext2_inode *) &inodes_to_process[i].inode; pctx.ino = ctx->stashed_ino = inodes_to_process[i].ino;
@@ -2744,7 +2763,7 @@ static void process_inodes(e2fsck_t ctx, char *block_buf) } ctx->stashed_inode = old_stashed_inode; ctx->stashed_ino = old_stashed_ino; - process_inode_count =0 ; + *process_inode_count =0 ;#if 0 printf ("end process inodes\n" );#endif
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 16/61 e2fsck: optimize the inserting of dir_info_db
In reply to Saranya MuruganandamFrom: Li Xi <[email protected]>
Binary search is now used when inserting an dir info to the array. Memmove is now used when moving array. Both of them improves the performance of inserting.
This patch is also a prepartion for the merging of two dir db arrays.
--- e2fsck/dirinfo.c | 172 ++++++++++++++++++++++++++++++----------------- 1 file changed, 112 insertions(+), 60 deletions(-)
e2fsck/dirinfo.c +112 -60
@@ -7,6 +7,7 @@#undef DIRINFO_DEBUG +#include <assert.h> #include "config.h" #include "e2fsck.h" #include <sys/stat.h>
@@ -122,6 +123,104 @@ static void setup_db(e2fsck_t ctx) "directory map"); } +/* +* Return the min index that has ino larger or equal to @ino +* If not found, return -ENOENT +*/ +static int +e2fsck_dir_info_min_larger_equal (struct dir_info_db *dir_info, +ext2_ino_t ino,ext2_ino_t *index) +{ +ext2_ino_t low =0 ; +ext2_ino_t mid, high; +ext2_ino_t tmp_ino; +int found =0 ; + +if (dir_info->count ==0 ) +return -ENOENT ; + + high = dir_info->count -1 ; +while (low <= high) { +/* sum may overflow, but result will fit into mid again */ + mid = (unsigned long long )(low + high) /2 ; + tmp_ino = dir_info->array[mid].ino; +if (ino == tmp_ino) { + *index = mid; + found =1 ; +return 0 ; + }else if (ino < tmp_ino) { +/* +* The mid ino is larger than @ino, remember the index +* here so we won't miss this ino +*/ + *index = mid; + found =1 ; +if (mid ==0 ) +break ; + high = mid -1 ; + }else { + low = mid +1 ; + } + } + +if (found) +return 0 ; + +return -ENOENT ; +} + +/* +* +* Insert an inode into the sorted array. The array should have at least one +* free slot. +* +* Normally, add_dir_info is called with each inode in +* sequential order; but once in a while (like when pass 3 +* needs to recreate the root directory or lost+found +* directory) it is called out of order. In those cases, we +* need to move the dir_info entries down to make room, since +* the dir_info array needs to be sorted by inode number for +* get_dir_info()'s sake. +*/ +static void e2fsck_insert_dir_info (struct dir_info_db *dir_info,ext2_ino_t ino,ext2_ino_t parent) +{ +ext2_ino_t index; +struct dir_info *dir; +size_t dir_size =sizeof (*dir); +struct dir_info *array = dir_info->array; +ext2_ino_t array_count = dir_info->count; +int err; + +/* +* Removing this check won't break anything. But since seqential ino +* inserting happens a lot, this check avoids binary search. +*/ +if (array_count ==0 || array[array_count -1 ].ino < ino) { + dir = &array[array_count]; + dir_info->count++; + goto out; + } + + err =e2fsck_dir_info_min_larger_equal (dir_info, ino, &index); +if (err >=0 && array[index].ino == ino) { + dir = &array[index]; + goto out; + } +if (err <0 ) { + dir = &array[array_count]; + dir_info->count++; + goto out; + } + + dir = &array[index]; +memmove ((char *)dir + dir_size, dir, dir_size * (array_count - index)); + dir_info->count++; +out: + dir->ino = ino; + dir->dotdot = parent; + dir->parent = parent; +} +/* * This subroutine is called during pass1 to create a directory info * entry. During pass1, the passed-in parent is 0; it will get filled
@@ -171,30 +270,7 @@ void e2fsck_add_dir_info(e2fsck_t ctx, ext2_ino_t ino, ext2_ino_t parent) }#endif -/* -* Normally, add_dir_info is called with each inode in -* sequential order; but once in a while (like when pass 3 -* needs to recreate the root directory or lost+found -* directory) it is called out of order. In those cases, we -* need to move the dir_info entries down to make room, since -* the dir_info array needs to be sorted by inode number for -* get_dir_info()'s sake. -*/ -if (ctx->dir_info->count && - ctx->dir_info->array[ctx->dir_info->count-1 ].ino >= ino) { -for (i = ctx->dir_info->count-1 ; i >0 ; i--) -if (ctx->dir_info->array[i-1 ].ino < ino) -break ; - dir = &ctx->dir_info->array[i]; -if (dir->ino != ino) -for (j = ctx->dir_info->count++; j > i; j--) - ctx->dir_info->array[j] = ctx->dir_info->array[j-1 ]; - }else - dir = &ctx->dir_info->array[ctx->dir_info->count++]; - - dir->ino = ino; - dir->dotdot = parent; - dir->parent = parent; +e2fsck_insert_dir_info (ctx->dir_info, ino, parent); }/*
@@ -204,7 +280,8 @@ void e2fsck_add_dir_info(e2fsck_t ctx, ext2_ino_t ino, ext2_ino_t parent)static struct dir_info *e2fsck_get_dir_info (e2fsck_t ctx,ext2_ino_t ino) {struct dir_info_db *db = ctx->dir_info; -ext2_ino_t low, high, mid; +ext2_ino_t index; +int err;if (!db)return 0 ;
@@ -245,44 +322,19 @@ static struct dir_info *e2fsck_get_dir_info(e2fsck_t ctx, ext2_ino_t ino)if (db->last_lookup && db->last_lookup->ino == ino)return db->last_lookup; - low =0 ; - high = ctx->dir_info->count -1 ; -if (ino == ctx->dir_info->array[low].ino) { + err =e2fsck_dir_info_min_larger_equal (ctx->dir_info, ino, &index); +if (err <0 ) +return NULL ; +assert (ino <= ctx->dir_info->array[index].ino); +if (ino == ctx->dir_info->array[index].ino) {#ifdef DIRINFO_DEBUG -printf ("(%u,%u,%u)\n" , ino, - ctx->dir_info->array[low].dotdot, - ctx->dir_info->array[low].parent); +printf ("(%d,%d,%d)\n" , ino, + ctx->dir_info->array[index].dotdot, + ctx->dir_info->array[index].parent);#endif -return &ctx->dir_info->array[low]; +return &ctx->dir_info->array[index]; } -if (ino == ctx->dir_info->array[high].ino) { -#ifdef DIRINFO_DEBUG -printf ("(%u,%u,%u)\n" , ino, - ctx->dir_info->array[high].dotdot, - ctx->dir_info->array[high].parent); -#endif -return &ctx->dir_info->array[high]; - } - -while (low < high) { -/* sum may overflow, but result will fit into mid again */ - mid = (unsigned long long )(low + high) /2 ; -if (mid == low || mid == high) -break ; -if (ino == ctx->dir_info->array[mid].ino) { -#ifdef DIRINFO_DEBUG -printf ("(%u,%u,%u)\n" , ino, - ctx->dir_info->array[mid].dotdot, - ctx->dir_info->array[mid].parent); -#endif -return &ctx->dir_info->array[mid]; - } -if (ino < ctx->dir_info->array[mid].ino) - high = mid; -else - low = mid; - } -return 0 ; +return NULL ; } static void e2fsck_put_dir_info(e2fsck_t ctx EXT2FS_NO_TDB_UNUSED,
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 17/61 e2fsck: merge dir_info after thread finishes
In reply to Saranya MuruganandamFrom: Li Xi <[email protected]>
dir_info need be merged after thread finish.
--- e2fsck/dirinfo.c | 66 ++++++++++++++++++++++++++++++++++++++++++++++++ e2fsck/e2fsck.h | 2 ++ e2fsck/pass1.c | 58 +++++++++++++++++++++++++++++++++++++++--- 3 files changed, 123 insertions(+), 3 deletions(-)
e2fsck/dirinfo.c +66 -0
@@ -169,6 +169,72 @@ e2fsck_dir_info_min_larger_equal(struct dir_info_db *dir_info,return -ENOENT ; } +/* +* Merge two sorted dir info to @dest +*/ +void e2fsck_merge_dir_info (e2fsck_t ctx,struct dir_info_db *src, +struct dir_info_db *dest) +{ +size_t size_dir_info =sizeof (struct dir_info ); +ext2_ino_t size = dest->size; +struct dir_info *src_array = src->array; +struct dir_info *dest_array = dest->array; +ext2_ino_t src_count = src->count; +ext2_ino_t dest_count = dest->count; +ext2_ino_t total_count = src_count + dest_count; +struct dir_info *tmp_array; +struct dir_info *array_ptr; +ext2_ino_t src_index =0 ; +ext2_ino_t dest_index =0 ; + +if (src->count ==0 ) +return ; + +if (size < total_count) + size = total_count; + +if (size < src->size) + size = src->size; + + tmp_array =e2fsck_allocate_memory (ctx, size * size_dir_info, +"directory map" ); + array_ptr = tmp_array; +/* +* This can be improved by binary search and memcpy, but codes +* would be more complex. And if the groups distributed to each +* thread are strided, this implementation won't be too bad +* comparing to the optimiztion. +*/ +while (src_index < src_count || dest_index < dest_count) { +if (src_index >= src_count) { +memcpy (array_ptr, &dest_array[dest_index], + (dest_count - dest_index) * size_dir_info); +break ; + } +if (dest_index >= dest_count) { +memcpy (array_ptr, &src_array[src_index], + (src_count - src_index) * size_dir_info); +break ; + } +if (src_array[src_index].ino < dest_array[dest_index].ino) { + *array_ptr = src_array[src_index]; + src_index++; + }else { +assert (src_array[src_index].ino > + dest_array[dest_index].ino); + *array_ptr = dest_array[dest_index]; + dest_index++; + } + array_ptr++; + } + +if (dest->array) +ext2fs_free_mem (&dest->array); + dest->array = tmp_array; + dest->size = size; + dest->count = total_count; +} +/* * * Insert an inode into the sorted array. The array should have at least one
e2fsck/e2fsck.h +2 -0
@@ -507,6 +507,8 @@ extern void read_bad_blocks_file(e2fsck_t ctx, const char *bad_blocks_file,/* dirinfo.c */ extern void e2fsck_add_dir_info (e2fsck_t ctx,ext2_ino_t ino,ext2_ino_t parent); +void e2fsck_merge_dir_info (e2fsck_t ctx,struct dir_info_db *src, +struct dir_info_db *dest);extern void e2fsck_free_dir_info (e2fsck_t ctx);extern int e2fsck_get_num_dirinfo (e2fsck_t ctx);extern struct dir_info_iter *e2fsck_dir_info_iter_begin (e2fsck_t ctx);
e2fsck/pass1.c +55 -3
94 lines not on this page · raw message
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 18/61 e2fsck: rbtree bitmap for dir
In reply to Saranya MuruganandamFrom: Wang Shilong <[email protected]>
Only rbtree support merge operation now, use it for bitmaps.
--- e2fsck/pass1.c | 1 + 1 file changed, 1 insertion(+)
e2fsck/pass1.c +1 -0
@@ -1235,6 +1235,7 @@ void e2fsck_pass1_run(e2fsck_t ctx) } pctx.errcode =e2fsck_allocate_inode_bitmap (fs,_ ("directory inode map" ), + ctx->global_ctx ?EXT2FS_BMAP64_RBTREE :EXT2FS_BMAP64_AUTODIR ,"inode_dir_map" , &ctx->inode_dir_map);if (pctx.errcode) {
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 19/61 e2fsck: merge badblocks after thread finishes
In reply to Saranya MuruganandamFrom: Wang Shilong <[email protected]>
Badblocks should be merged properly after threads finish.
--- e2fsck/pass1.c | 19 +++++++--- lib/ext2fs/badblocks.c | 85 ++++++++++++++++++++++++++++++++++++++---- lib/ext2fs/ext2fs.h | 2 + lib/ext2fs/ext2fsP.h | 1 - 4 files changed, 94 insertions(+), 13 deletions(-)
e2fsck/pass1.c +14 -5
@@ -2180,6 +2180,7 @@ static errcode_t e2fsck_pass1_copy_fs(ext2_filsys dest, e2fsck_t src_context,memcpy (dest, src,sizeof (struct struct_ext2_filsys )); dest->inode_map =NULL ; dest->block_map =NULL ; + dest->badblocks =NULL ;if (dest->dblist) dest->dblist->fs = dest;if (src->block_map) {
@@ -2196,7 +2197,8 @@ static errcode_t e2fsck_pass1_copy_fs(ext2_filsys dest, e2fsck_t src_context, }if (src->badblocks) { - retval =ext2fs_badblocks_copy (src->badblocks, &dest->badblocks); + retval =ext2fs_badblocks_copy (src->badblocks, + &dest->badblocks);if (retval)return retval; }
@@ -2241,11 +2243,13 @@ static int e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)io_channel dest_image_io;ext2fs_inode_bitmap inode_map;ext2fs_block_bitmap block_map; +ext2_badblocks_list badblocks; dest_io = dest->io; dest_image_io = dest->image_io; inode_map = dest->inode_map; block_map = dest->block_map; + badblocks = dest->badblocks;memcpy (dest, src,sizeof (struct struct_ext2_filsys )); dest->io = dest_io;
@@ -2253,6 +2257,7 @@ static int e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src) dest->icache = icache; dest->inode_map = inode_map; dest->block_map = block_map; + dest->badblocks = badblocks;if (dest->dblist) dest->dblist->fs = dest;
@@ -2272,10 +2277,12 @@ static int e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src) goto out;if (src->badblocks) { - retval =ext2fs_badblocks_copy (src->badblocks, &dest->badblocks); - -ext2fs_badblocks_list_free (src->badblocks); - src->badblocks =NULL ; +if (dest->badblocks ==NULL ) + retval =ext2fs_badblocks_copy (src->badblocks, + &dest->badblocks); +else + retval =ext2fs_badblocks_merge (src->badblocks, + dest->badblocks); } out:io_channel_close (src->io);
@@ -2283,6 +2290,8 @@ out:ext2fs_free_generic_bmap (src->inode_map);if (src->block_map)ext2fs_free_generic_bmap (src->block_map); +if (src->badblocks) +ext2fs_badblocks_list_free (src->badblocks);return retval; }
lib/ext2fs/badblocks.c +78 -7
109 lines not on this page · raw message
lib/ext2fs/ext2fs.h +2 -0
@@ -814,6 +814,8 @@ extern int ext2fs_badblocks_list_iterate(ext2_badblocks_iterate iter,extern void ext2fs_badblocks_list_iterate_end (ext2_badblocks_iterate iter);extern errcode_t ext2fs_badblocks_copy (ext2_badblocks_list src,ext2_badblocks_list *dest); +extern errcode_t ext2fs_badblocks_merge (ext2_badblocks_list src, +ext2_badblocks_list dest);extern int ext2fs_badblocks_equal (ext2_badblocks_list bb1,ext2_badblocks_list bb2);extern int ext2fs_u32_list_count (ext2_u32_list bb);
lib/ext2fs/ext2fsP.h +0 -1
@@ -34,7 +34,6 @@ struct ext2_struct_u32_list {int num;int size;__u32 *list; -int badblocks_flags; };struct ext2_struct_u32_iterate {
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 20/61 e2fsck: merge icounts after thread finishes
In reply to Saranya MuruganandamFrom: Li Xi <[email protected]>
Merge inode_count and inode_link_info properly after threads finish.
--- e2fsck/pass1.c | 47 ++++++++++++++++++++ lib/ext2fs/ext2fs.h | 1 + lib/ext2fs/icount.c | 103 ++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 151 insertions(+)
e2fsck/pass1.c +47 -0
@@ -2388,6 +2388,41 @@ static void e2fsck_pass1_merge_dir_info(e2fsck_t global_ctx, e2fsck_t thread_ctx global_ctx->dir_info); } +static inline errcode_t +e2fsck_pass1_merge_icount (ext2_icount_t *dest_icount, +ext2_icount_t *src_icount) +{ +if (*src_icount) { +if (*dest_icount ==NULL ) { + *dest_icount = *src_icount; + *src_icount =NULL ; + }else { +errcode_t ret; + + ret =ext2fs_icount_merge (*src_icount, + *dest_icount); +if (ret) +return ret; + } + } + +return 0 ; +} + +static errcode_t e2fsck_pass1_merge_icounts (e2fsck_t global_ctx,e2fsck_t thread_ctx) +{ +errcode_t ret; + + ret =e2fsck_pass1_merge_icount (&global_ctx->inode_count, + &thread_ctx->inode_count); +if (ret) +return ret; + ret =e2fsck_pass1_merge_icount (&global_ctx->inode_link_info, + &thread_ctx->inode_link_info); + +return ret; +} +static int e2fsck_pass1_thread_join_one (e2fsck_t global_ctx,e2fsck_t thread_ctx) {errcode_t retval;
@@ -2408,6 +2443,8 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctxext2fs_block_bitmap block_ea_map = global_ctx->block_ea_map;ext2fs_block_bitmap block_metadata_map = global_ctx->block_metadata_map;ext2fs_block_bitmap inodes_to_rebuild = global_ctx->inodes_to_rebuild; +ext2_icount_t inode_count = global_ctx->inode_count; +ext2_icount_t inode_link_info = global_ctx->inode_link_info;#ifdef HAVE_SETJMP_H jmp_buf old_jmp;
@@ -2432,6 +2469,8 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx global_ctx->block_metadata_map = block_metadata_map; global_ctx->dir_info = dir_info;e2fsck_pass1_merge_dir_info (global_ctx, thread_ctx); + global_ctx->inode_count = inode_count; + global_ctx->inode_link_info = inode_link_info;/* Keep the global singal flags*/ global_ctx->flags |= (flags & E2F_FLAG_SIGNAL_MASK) |
@@ -2447,6 +2486,12 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx global_ctx->logf = global_logf; global_ctx->problem_logf = global_problem_logf; global_ctx->global_ctx =NULL ; + retval =e2fsck_pass1_merge_icounts (global_ctx, thread_ctx); +if (retval) { +com_err (global_ctx->program_name,0 , +_ ("while merging icounts\n" )); +return retval; + } retval = e2fsck_pass1_merge_bitmap(global_fs, &thread_ctx->inode_used_map,
@@ -2532,6 +2577,8 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)e2fsck_pass1_free_bitmap (&thread_ctx->block_ea_map);e2fsck_pass1_free_bitmap (&thread_ctx->block_metadata_map);e2fsck_free_dir_info (thread_ctx); +ext2fs_free_icount (thread_ctx->inode_count); +ext2fs_free_icount (thread_ctx->inode_link_info);ext2fs_free_mem (&thread_ctx);return retval;
lib/ext2fs/ext2fs.h +1 -0
@@ -1513,6 +1513,7 @@ extern errcode_t ext2fs_icount_decrement(ext2_icount_t icount, ext2_ino_t ino, __u16 *ret);extern errcode_t ext2fs_icount_store (ext2_icount_t icount,ext2_ino_t ino,__u16 count); +extern errcode_t ext2fs_icount_merge (ext2_icount_t src,ext2_icount_t dest);extern ext2_ino_t ext2fs_get_icount_size (ext2_icount_t icount);errcode_t ext2fs_icount_validate (ext2_icount_t icount,FILE *);
lib/ext2fs/icount.c +103 -0
115 lines not on this page · raw message
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 21/61 e2fsck: merge dblist after thread finishes
In reply to Saranya MuruganandamFrom: Li Xi <[email protected]>
Merge dblist properly.
--- e2fsck/pass1.c | 19 +++++++++++++++++++ lib/ext2fs/dblist.c | 36 ++++++++++++++++++++++++++++++++++++ lib/ext2fs/ext2fs.h | 1 + 3 files changed, 56 insertions(+)
e2fsck/pass1.c +19 -0
@@ -2244,12 +2244,14 @@ static int e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)ext2fs_inode_bitmap inode_map;ext2fs_block_bitmap block_map;ext2_badblocks_list badblocks; +ext2_dblist dblist; dest_io = dest->io; dest_image_io = dest->image_io; inode_map = dest->inode_map; block_map = dest->block_map; badblocks = dest->badblocks; + dblist = dest->dblist;memcpy (dest, src,sizeof (struct struct_ext2_filsys )); dest->io = dest_io;
@@ -2258,6 +2260,7 @@ static int e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src) dest->inode_map = inode_map; dest->block_map = block_map; dest->badblocks = badblocks; + dest->dblist = dblist;if (dest->dblist) dest->dblist->fs = dest;
@@ -2276,6 +2279,19 @@ static int e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)if (retval) goto out; +if (src->dblist) { +if (dest->dblist) { + retval =ext2fs_merge_dblist (src->dblist, + dest->dblist); +if (retval) + goto out; + }else { + dest->dblist = src->dblist; + dest->dblist->fs = dest; + src->dblist =NULL ; + } + } +if (src->badblocks) {if (dest->badblocks ==NULL ) retval = ext2fs_badblocks_copy(src->badblocks,
@@ -2292,6 +2308,9 @@ out:ext2fs_free_generic_bmap (src->block_map);if (src->badblocks)ext2fs_badblocks_list_free (src->badblocks); +if (src->dblist) +ext2fs_free_dblist (src->dblist); +return retval; }
lib/ext2fs/dblist.c +36 -0
@@ -119,6 +119,42 @@ errcode_t ext2fs_copy_dblist(ext2_dblist src, ext2_dblist *dest)return 0 ; } +/* +* Merge a directory block list @src to @dest +*/ +errcode_t ext2fs_merge_dblist (ext2_dblist src,ext2_dblist dest) +{ +unsigned long long src_count = src->count; +unsigned long long dest_count = dest->count; +unsigned long long size = src_count + dest_count; +size_t size_entry =sizeof (struct ext2_db_entry2 ); +struct ext2_db_entry2 *array, *array2; +errcode_t retval; + +if (src_count ==0 ) +return 0 ; + +if (src->sorted || (dest->sorted && dest_count !=0 )) +return EINVAL ; + + retval =ext2fs_get_array (size, size_entry, &array); +if (retval) +return retval; + + array2 = array; +memcpy (array, src->list, src_count * size_entry); + array += src_count; +memcpy (array, dest->list, dest_count * size_entry); +ext2fs_free_mem (&dest->list); + + dest->list = array2; + dest->count = src_count + dest_count; + dest->size = size; + dest->sorted =0 ; + +return 0 ; +} +/* * Close a directory block list *
lib/ext2fs/ext2fs.h +1 -0
@@ -1122,6 +1122,7 @@ extern errcode_t ext2fs_add_dir_block(ext2_dblist dblist, ext2_ino_t ino, blk_t blk, int blockcnt);extern errcode_t ext2fs_add_dir_block2 (ext2_dblist dblist,ext2_ino_t ino,blk64_t blk,e2_blkcnt_t blockcnt); +extern errcode_t ext2fs_merge_dblist (ext2_dblist src,ext2_dblist dest);extern void ext2fs_dblist_sort (ext2_dblist dblist,EXT2_QSORT_TYPE (*sortfunc)(const void *,const void *));
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 22/61 e2fsck: add debug codes for multiple threads
In reply to Saranya MuruganandamFrom: Li Xi <[email protected]>
These debug codes are added to run the multiple pass1 check thread one by one in order. If all the codes are correct, fsck of multiple threads should have exactly the same outcome with single thread.
--- e2fsck/e2fsck.h | 16 ++++++++++++++++ e2fsck/pass1.c | 29 +++++++++++++++++++++++++++++ 2 files changed, 45 insertions(+)
e2fsck/e2fsck.h +16 -0
@@ -450,6 +450,18 @@ struct e2fsck_struct { }; #ifdef CONFIG_PFSCK +#ifdef DEBUG_THREADS +/* + * Enabling DEBUG_THREADS would cause the parallel + * fsck threads run sequentially. + */ +struct e2fsck_thread_debug { + pthread_mutex_t etd_mutex; + pthread_cond_t etd_cond; + int etd_finished_threads; +}; +#endif + struct e2fsck_thread_info { /* ID returned by pthread_create() */ pthread_t eti_thread_id;
@@ -459,7 +471,11 @@ struct e2fsck_thread_info {int eti_started;/* Context used for this thread */ e2fsck_t eti_thread_ctx; +#ifdef DEBUG_THREADS +struct e2fsck_thread_debug *eti_debug; +#endif }; +#endif /* Data structures to evaluate whether an extent tree needs rebuilding. */
e2fsck/pass1.c +29 -0
@@ -2641,6 +2641,18 @@ static void *e2fsck_pass1_thread(void *arg) {struct e2fsck_thread_info *info = arg;e2fsck_t thread_ctx = info->eti_thread_ctx; +#ifdef DEBUG_THREADS +struct e2fsck_thread_debug *thread_debug = info->eti_debug; +#endif + +#ifdef DEBUG_THREADS +pthread_mutex_lock (&thread_debug->etd_mutex); +while (info->eti_thread_index > thread_debug->etd_finished_threads) { +pthread_cond_wait (&thread_debug->etd_cond, + &thread_debug->etd_mutex); + } +pthread_mutex_unlock (&thread_debug->etd_mutex); +#endif #ifdef HAVE_SETJMP_H /*
@@ -2665,6 +2677,14 @@ out: thread_ctx->thread_info.et_group_start, thread_ctx->thread_info.et_group_end, thread_ctx->thread_info.et_inode_number); + +#ifdef DEBUG_THREADS +pthread_mutex_lock (&thread_debug->etd_mutex); + thread_debug->etd_finished_threads++; +pthread_cond_broadcast (&thread_debug->etd_cond); +pthread_mutex_unlock (&thread_debug->etd_mutex); +#endif +return NULL ; }
@@ -2678,6 +2698,12 @@ static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo,struct e2fsck_thread_info *tmp_pinfo;int i;e2fsck_t thread_ctx; +#ifdef DEBUG_THREADS +struct e2fsck_thread_debug thread_debug = + {PTHREAD_MUTEX_INITIALIZER ,PTHREAD_COND_INITIALIZER ,0 }; + + thread_debug.etd_finished_threads =0 ; +#endif retval =pthread_attr_init (&attr);if (retval) {
@@ -2698,6 +2724,9 @@ static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo,for (i =0 ; i < num_threads; i++) { tmp_pinfo = &infos[i]; tmp_pinfo->eti_thread_index = i; +#ifdef DEBUG_THREADS + tmp_pinfo->eti_debug = &thread_debug; +#endif retval =e2fsck_pass1_thread_prepare (global_ctx, &thread_ctx, i, num_threads);if (retval) {
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 23/61 e2fsck: merge counts after threads finish
In reply to Saranya MuruganandamFrom: Wang Shilong <[email protected]>
Merge counts properly.
--- e2fsck/pass1.c | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+)
e2fsck/pass1.c +34 -0
@@ -2464,6 +2464,23 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctxext2fs_block_bitmap inodes_to_rebuild = global_ctx->inodes_to_rebuild;ext2_icount_t inode_count = global_ctx->inode_count;ext2_icount_t inode_link_info = global_ctx->inode_link_info; +__u32 fs_directory_count = global_ctx->fs_directory_count; +__u32 fs_regular_count = global_ctx->fs_regular_count; +__u32 fs_blockdev_count = global_ctx->fs_blockdev_count; +__u32 fs_chardev_count = global_ctx->fs_chardev_count; +__u32 fs_links_count = global_ctx->fs_links_count; +__u32 fs_symlinks_count = global_ctx->fs_symlinks_count; +__u32 fs_fast_symlinks_count = global_ctx->fs_fast_symlinks_count; +__u32 fs_fifo_count = global_ctx->fs_fifo_count; +__u32 fs_total_count = global_ctx->fs_total_count; +__u32 fs_badblocks_count = global_ctx->fs_badblocks_count; +__u32 fs_sockets_count = global_ctx->fs_sockets_count; +__u32 fs_ind_count = global_ctx->fs_ind_count; +__u32 fs_dind_count = global_ctx->fs_dind_count; +__u32 fs_tind_count = global_ctx->fs_tind_count; +__u32 fs_fragmented = global_ctx->fs_fragmented; +__u32 fs_fragmented_dir = global_ctx->fs_fragmented_dir; +__u32 large_files = global_ctx->large_files;#ifdef HAVE_SETJMP_H jmp_buf old_jmp;
@@ -2490,6 +2507,23 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctxe2fsck_pass1_merge_dir_info (global_ctx, thread_ctx); global_ctx->inode_count = inode_count; global_ctx->inode_link_info = inode_link_info; + global_ctx->fs_directory_count += fs_directory_count; + global_ctx->fs_regular_count += fs_regular_count; + global_ctx->fs_blockdev_count += fs_blockdev_count; + global_ctx->fs_chardev_count += fs_chardev_count; + global_ctx->fs_links_count += fs_links_count; + global_ctx->fs_symlinks_count += fs_symlinks_count; + global_ctx->fs_fast_symlinks_count += fs_fast_symlinks_count; + global_ctx->fs_fifo_count += fs_fifo_count; + global_ctx->fs_total_count += fs_total_count; + global_ctx->fs_badblocks_count += fs_badblocks_count; + global_ctx->fs_sockets_count += fs_sockets_count; + global_ctx->fs_ind_count += fs_ind_count; + global_ctx->fs_dind_count += fs_dind_count; + global_ctx->fs_tind_count += fs_tind_count; + global_ctx->fs_fragmented += fs_fragmented; + global_ctx->fs_fragmented_dir += fs_fragmented_dir; + global_ctx->large_files += large_files;/* Keep the global singal flags*/ global_ctx->flags |= (flags & E2F_FLAG_SIGNAL_MASK) |
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 24/61 e2fsck: merge fs flags when threads finish
In reply to Saranya MuruganandamFrom: Li Xi <[email protected]>
merge fs flags properly.
--- e2fsck/pass1.c | 5 +++++ 1 file changed, 5 insertions(+)
e2fsck/pass1.c +5 -0
@@ -2245,6 +2245,7 @@ static int e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)ext2fs_block_bitmap block_map;ext2_badblocks_list badblocks;ext2_dblist dblist; +int flags; dest_io = dest->io; dest_image_io = dest->image_io;
@@ -2252,6 +2253,7 @@ static int e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src) block_map = dest->block_map; badblocks = dest->badblocks; dblist = dest->dblist; + flags = dest->flags;memcpy (dest, src,sizeof (struct struct_ext2_filsys )); dest->io = dest_io;
@@ -2263,6 +2265,9 @@ static int e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src) dest->dblist = dblist;if (dest->dblist) dest->dblist->fs = dest; + dest->flags = src->flags | flags; +if (!(src->flags &EXT2_FLAG_VALID ) || !(flags &EXT2_FLAG_VALID )) +ext2fs_unmark_valid (dest);if (src->icache) {ext2fs_free_inode_cache (src->icache);
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 25/61 e2fsck: merge dx_dir_info after threads finish
In reply to Saranya MuruganandamFrom: Wang Shilong <[email protected]>
Merge properly.
--- e2fsck/dx_dirinfo.c | 64 +++++++++++++++++++++++++++++++++++++++++++++ e2fsck/e2fsck.h | 1 + e2fsck/pass1.c | 23 ++++++++++++++++ 3 files changed, 88 insertions(+)
e2fsck/dx_dirinfo.c +64 -0
@@ -5,6 +5,7 @@* under the terms of the GNU Public License. */ +#include <assert.h> #include "config.h" #include "e2fsck.h"
@@ -79,6 +80,69 @@ void e2fsck_add_dx_dir(e2fsck_t ctx, ext2_ino_t ino, struct ext2_inode *inode, "dx_block info array"); } +/* +* Merge two sorted dir info to @dest +*/ +void e2fsck_merge_dx_dir (e2fsck_t global_ctx,e2fsck_t thread_ctx) +{ +struct dx_dir_info *src_array = thread_ctx->dx_dir_info; +struct dx_dir_info *dest_array = global_ctx->dx_dir_info; +size_t size_dx_info =sizeof (struct dx_dir_info ); +ext2_ino_t size = global_ctx->dx_dir_info_size; +ext2_ino_t src_count = thread_ctx->dx_dir_info_count; +ext2_ino_t dest_count = global_ctx->dx_dir_info_count; +ext2_ino_t total_count = src_count + dest_count; +struct dx_dir_info *array; +struct dx_dir_info *array_ptr; +ext2_ino_t src_index =0 , dest_index =0 ; + +if (thread_ctx->dx_dir_info_count ==0 ) +return ; + +if (size < total_count) + size = total_count; + + array =e2fsck_allocate_memory (global_ctx, size * size_dx_info, +"directory map" ); + array_ptr = array; +/* +* This can be improved by binary search and memcpy, but codes +* would be more complex. And if the groups distributed to each +* thread are strided, this implementation won't be too bad +* comparing to the optimiztion. +*/ +while (src_index < src_count || dest_index < dest_count) { +if (src_index >= src_count) { +memcpy (array_ptr, &dest_array[dest_index], + (dest_count - dest_index) * size_dx_info); +break ; + } +if (dest_index >= dest_count) { +memcpy (array_ptr, &src_array[src_index], + (src_count - src_index) * size_dx_info); +break ; + } +if (src_array[src_index].ino < dest_array[dest_index].ino) { + *array_ptr = src_array[src_index]; + src_index++; + }else { +assert (src_array[src_index].ino > + dest_array[dest_index].ino); + *array_ptr = dest_array[dest_index]; + dest_index++; + } + array_ptr++; + } + +if (global_ctx->dx_dir_info) +ext2fs_free_mem (&global_ctx->dx_dir_info); +if (thread_ctx->dx_dir_info) +ext2fs_free_mem (&thread_ctx->dx_dir_info); + global_ctx->dx_dir_info = array; + global_ctx->dx_dir_info_size = size; + global_ctx->dx_dir_info_count = total_count; +} +/* * get_dx_dir_info() --- given an inode number, try to find the directory * information entry for it.
e2fsck/e2fsck.h +1 -0
@@ -539,6 +539,7 @@ extern int e2fsck_dir_info_get_parent(e2fsck_t ctx, ext2_ino_t ino, ext2_ino_t *parent);extern int e2fsck_dir_info_get_dotdot (e2fsck_t ctx,ext2_ino_t ino,ext2_ino_t *dotdot); +extern void e2fsck_merge_dx_dir (e2fsck_t global_ctx,e2fsck_t thread_ctx);/* dx_dirinfo.c */ extern void e2fsck_add_dx_dir(e2fsck_t ctx, ext2_ino_t ino,
e2fsck/pass1.c +23 -0
@@ -2412,6 +2412,22 @@ static void e2fsck_pass1_merge_dir_info(e2fsck_t global_ctx, e2fsck_t thread_ctx global_ctx->dir_info); } +static void e2fsck_pass1_merge_dx_dir (e2fsck_t global_ctx,e2fsck_t thread_ctx) +{ +if (thread_ctx->dx_dir_info ==NULL ) +return ; + +if (global_ctx->dx_dir_info ==NULL ) { + global_ctx->dx_dir_info = thread_ctx->dx_dir_info; + global_ctx->dx_dir_info_size = thread_ctx->dx_dir_info_size; + global_ctx->dx_dir_info_count = thread_ctx->dx_dir_info_count; + thread_ctx->dx_dir_info =NULL ; +return ; + } + +e2fsck_merge_dx_dir (global_ctx, thread_ctx); +} +static inline errcode_t e2fsck_pass1_merge_icount (ext2_icount_t *dest_icount,ext2_icount_t *src_icount)
@@ -2457,6 +2473,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctxFILE *global_problem_logf = global_ctx->problem_logf;ext2fs_inode_bitmap inode_bad_map = global_ctx->inode_bad_map;struct dir_info_db *dir_info = global_ctx->dir_info; +struct dx_dir_info *dx_dir_info = global_ctx->dx_dir_info;ext2fs_inode_bitmap inode_used_map = global_ctx->inode_used_map;ext2fs_inode_bitmap inode_dir_map = global_ctx->inode_dir_map;ext2fs_inode_bitmap inode_bb_map = global_ctx->inode_bb_map;
@@ -2486,6 +2503,8 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx__u32 fs_fragmented = global_ctx->fs_fragmented;__u32 fs_fragmented_dir = global_ctx->fs_fragmented_dir;__u32 large_files = global_ctx->large_files; +ext2_ino_t dx_dir_info_size = global_ctx->dx_dir_info_size; +ext2_ino_t dx_dir_info_count = global_ctx->dx_dir_info_count;#ifdef HAVE_SETJMP_H jmp_buf old_jmp;
@@ -2510,6 +2529,10 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx global_ctx->block_metadata_map = block_metadata_map; global_ctx->dir_info = dir_info;e2fsck_pass1_merge_dir_info (global_ctx, thread_ctx); + global_ctx->dx_dir_info = dx_dir_info; + global_ctx->dx_dir_info_count = dx_dir_info_count; + global_ctx->dx_dir_info_size = dx_dir_info_size; +e2fsck_pass1_merge_dx_dir (global_ctx, thread_ctx); global_ctx->inode_count = inode_count; global_ctx->inode_link_info = inode_link_info; global_ctx->fs_directory_count += fs_directory_count;
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 26/61 e2fsck: merge dirs_to_hash when threads finish
In reply to Saranya MuruganandamFrom: Wang Shilong <[email protected]>
@dirs_to_hash list need be merged after threads finish, test covered by t_dangerous.
--- e2fsck/pass1.c | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+)
e2fsck/pass1.c +30 -0
@@ -2463,6 +2463,25 @@ static errcode_t e2fsck_pass1_merge_icounts(e2fsck_t global_ctx, e2fsck_t threadreturn ret; } +static errcode_t e2fsck_pass1_merge_dirs_to_hash (e2fsck_t global_ctx, +e2fsck_t thread_ctx) +{ +errcode_t retval =0 ; + +if (!thread_ctx->dirs_to_hash) +return 0 ; + +if (!global_ctx->dirs_to_hash) + retval =ext2fs_badblocks_copy (thread_ctx->dirs_to_hash, + &global_ctx->dirs_to_hash); +else + retval =ext2fs_badblocks_merge (thread_ctx->dirs_to_hash, + global_ctx->dirs_to_hash); + +return retval; +} + +static int e2fsck_pass1_thread_join_one (e2fsck_t global_ctx,e2fsck_t thread_ctx) {errcode_t retval;
@@ -2505,6 +2524,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx__u32 large_files = global_ctx->large_files;ext2_ino_t dx_dir_info_size = global_ctx->dx_dir_info_size;ext2_ino_t dx_dir_info_count = global_ctx->dx_dir_info_count; +ext2_u32_list dirs_to_hash = global_ctx->dirs_to_hash;#ifdef HAVE_SETJMP_H jmp_buf old_jmp;
@@ -2574,6 +2594,14 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctxreturn retval; } + global_ctx->dirs_to_hash = dirs_to_hash; + retval =e2fsck_pass1_merge_dirs_to_hash (global_ctx, thread_ctx); +if (retval) { +com_err (global_ctx->program_name,0 , +_ ("while merging dirs to hash\n" )); +return retval; + } + retval =e2fsck_pass1_merge_bitmap (global_fs, &thread_ctx->inode_used_map, &global_ctx->inode_used_map);
@@ -2660,6 +2688,8 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)e2fsck_free_dir_info (thread_ctx);ext2fs_free_icount (thread_ctx->inode_count);ext2fs_free_icount (thread_ctx->inode_link_info); +if (thread_ctx->dirs_to_hash) +ext2fs_badblocks_list_free (thread_ctx->dirs_to_hash);ext2fs_free_mem (&thread_ctx);return retval;
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 27/61 e2fsck: merge context flags properly
In reply to Saranya MuruganandamFrom: Wang Shilong <[email protected]>
e2fsck might restart after pass1, so we should keep flags if possible, this patch try to fix f_illitable_flexbg failure
--- e2fsck/pass1.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)
e2fsck/pass1.c +1 -3
@@ -2573,9 +2573,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx global_ctx->fs_fragmented_dir += fs_fragmented_dir; global_ctx->large_files += large_files; -/* Keep the global singal flags*/ - global_ctx->flags |= (flags &E2F_FLAG_SIGNAL_MASK ) | - (global_ctx->flags &E2F_FLAG_SIGNAL_MASK ); + global_ctx->flags |= flags; retval =e2fsck_pass1_merge_fs (global_fs, thread_fs);if (retval) {
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 28/61 e2fsck: merge quota context after threads finish
In reply to Saranya MuruganandamFrom: Wang Shilong <[email protected]>
Every threads calculate its own quota accounting,
merge them after threads finish.
--- e2fsck/pass1.c | 15 ++++++++++++++- lib/support/mkquota.c | 39 +++++++++++++++++++++++++++++++++++++++ lib/support/quotaio.h | 3 +++ 3 files changed, 56 insertions(+), 1 deletion(-)
e2fsck/pass1.c +14 -1
@@ -2388,6 +2388,12 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *threlog_out (thread_context,_ ("Scan group range [%d, %d)\n" ), tinfo->et_group_start, tinfo->et_group_end); thread_context->fs = thread_fs; + retval =quota_init_context (&thread_context->qctx, thread_fs,0 ); +if (retval) { +com_err (global_ctx->program_name, retval, +"while init quota context" ); + goto out_fs; + } *thread_ctx = thread_context;return 0 ; out_fs:
@@ -2481,7 +2487,6 @@ static errcode_t e2fsck_pass1_merge_dirs_to_hash(e2fsck_t global_ctx,return retval; } -static int e2fsck_pass1_thread_join_one (e2fsck_t global_ctx,e2fsck_t thread_ctx) {errcode_t retval;
@@ -2525,6 +2530,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctxext2_ino_t dx_dir_info_size = global_ctx->dx_dir_info_size;ext2_ino_t dx_dir_info_count = global_ctx->dx_dir_info_count;ext2_u32_list dirs_to_hash = global_ctx->dirs_to_hash; +quota_ctx_t qctx = global_ctx->qctx;#ifdef HAVE_SETJMP_H jmp_buf old_jmp;
@@ -2600,6 +2606,12 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctxreturn retval; } + global_ctx->qctx = qctx; + retval =quota_merge_and_update_usage (global_ctx->qctx, + thread_ctx->qctx); +if (retval) +return retval; + retval =e2fsck_pass1_merge_bitmap (global_fs, &thread_ctx->inode_used_map, &global_ctx->inode_used_map);
@@ -2688,6 +2700,7 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)ext2fs_free_icount (thread_ctx->inode_link_info);if (thread_ctx->dirs_to_hash)ext2fs_badblocks_list_free (thread_ctx->dirs_to_hash); +quota_release_context (&thread_ctx->qctx);ext2fs_free_mem (&thread_ctx);return retval;
lib/support/mkquota.c +39 -0
@@ -639,6 +639,45 @@ out:return err; } +static errcode_t merge_usage (dict_t *dest,dict_t *src) +{ +dnode_t *n; +struct dquot *src_dq, *dest_dq; + +for (n =dict_first (src); n; n =dict_next (src, n)) { + src_dq =dnode_get (n); +if (!src_dq) +continue ; + dest_dq =get_dq (dest, src_dq->dq_id); +if (dest_dq ==NULL ) +return -ENOMEM ; + dest_dq->dq_dqb.dqb_curspace += src_dq->dq_dqb.dqb_curspace; + dest_dq->dq_dqb.dqb_curinodes += src_dq->dq_dqb.dqb_curinodes; + } + +return 0 ; +} + + +errcode_t quota_merge_and_update_usage (quota_ctx_t dest_qctx, +quota_ctx_t src_qctx) +{ +dict_t *dict; +enum quota_type qtype; +errcode_t retval =0 ; + +for (qtype =0 ; qtype <MAXQUOTAS ; qtype++) { + dict = src_qctx->quota_dict[qtype]; +if (!dict) +continue ; + retval =merge_usage (dest_qctx->quota_dict[qtype], dict); +if (retval) +break ; + } + +return retval; +} +/* * Compares the measured quota in qctx->quota_dict with that in the quota inode * on disk and updates the limits in qctx->quota_dict. 'usage_inconsistent' is
lib/support/quotaio.h +3 -0
@@ -40,6 +40,7 @@#include "ext2fs/ext2_fs.h" #include "ext2fs/ext2fs.h" #include "dqblk_v2.h" +#include "support/dict.h" typedef int64_t qsize_t ;/* Type in which we store size limitations */
@@ -233,6 +234,8 @@ int quota_file_exists(ext2_filsys fs, enum quota_type qtype);void quota_set_sb_inum (ext2_filsys fs,ext2_ino_t ino,enum quota_type qtype);errcode_t quota_compare_and_update (quota_ctx_t qctx,enum quota_type qtype,int *usage_inconsistent); +errcode_t quota_merge_and_update_usage (quota_ctx_t dest_qctx, +quota_ctx_t src_qctx);int parse_quota_opts (const char *opts,int (*func)(char *));/* parse_qtype.c */
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 29/61 e2fsck: serialize fix operations
In reply to Saranya MuruganandamFrom: Wang Shilong <[email protected]>
Allow different threads to fix at the same time could be dangerous and error-prone now, and most of time parallel scanning and checking is important.
So this patch adds a mutex to serialize
fix operations during pass1.
And the good benefit of this, we don't need block allocations and free, superblock updates protection any more, since only fix operations during pass1 could touch them.
--- e2fsck/e2fsck.h | 6 +++++ e2fsck/pass1.c | 63 +++++++++++++++++++++++++++++++++++++++++++------ e2fsck/util.c | 38 +++++++++++++++++++++++++++++ 3 files changed, 100 insertions(+), 7 deletions(-)
e2fsck/e2fsck.h +6 -0
@@ -447,6 +447,10 @@ struct e2fsck_struct {/* Undo file */ char *undo_file; +#ifdef CONFIG_PFSCK +/* serialize fix operation for multiple threads */ +pthread_mutex_t fs_fix_mutex; +#endif };#ifdef CONFIG_PFSCK
@@ -744,6 +748,8 @@ extern errcode_t e2fsck_allocate_subcluster_bitmap(ext2_filsys fs,const char *profile_name, ext2fs_block_bitmap *ret);unsigned long long get_memory_size (void ); +extern void e2fsck_pass1_fix_lock (e2fsck_t ctx); +extern void e2fsck_pass1_fix_unlock (e2fsck_t ctx);/* unix.c */ extern void e2fsck_clear_progbar (e2fsck_t ctx);
e2fsck/pass1.c +56 -7
215 lines not on this page · raw message
e2fsck/util.c +38 -0
@@ -42,6 +42,10 @@#include <sys/sysctl.h> #endif +#ifdef CONFIG_PFSCK +#include <pthread.h> +#endif +#include "e2fsck.h" extern e2fsck_t e2fsck_global_ctx;/* Try your very best not to use this! */
@@ -565,13 +569,45 @@ void e2fsck_read_inode_full(e2fsck_t ctx, unsigned long ino, } } +#ifdef CONFIG_PFSCK +void e2fsck_pass1_fix_lock (e2fsck_t ctx) +{ +e2fsck_t global_ctx = ctx->global_ctx; +if (!global_ctx) + global_ctx = ctx; + +pthread_mutex_lock (&global_ctx->fs_fix_mutex); +} + +void e2fsck_pass1_fix_unlock (e2fsck_t ctx) +{ +e2fsck_t global_ctx = ctx->global_ctx; +if (!global_ctx) + global_ctx = ctx; + +pthread_mutex_unlock (&global_ctx->fs_fix_mutex); +} +#else +void e2fsck_pass1_fix_lock (e2fsck_t ctx) +{ + +} + +void e2fsck_pass1_fix_unlock (e2fsck_t ctx) +{ + +} +#endif +void e2fsck_write_inode_full (e2fsck_t ctx,unsigned long ino,struct ext2_inode * inode,int bufsize,const char *proc) {errcode_t retval; +e2fsck_pass1_fix_lock (ctx); retval =ext2fs_write_inode_full (ctx->fs, ino, inode, bufsize); +e2fsck_pass1_fix_unlock (ctx);if (retval) {com_err ("ext2fs_write_inode" , retval,_ ("while writing inode %lu in %s" ), ino, proc);
@@ -584,7 +620,9 @@ void e2fsck_write_inode(e2fsck_t ctx, unsigned long ino, {errcode_t retval; +e2fsck_pass1_fix_lock (ctx); retval =ext2fs_write_inode (ctx->fs, ino, inode); +e2fsck_pass1_fix_unlock (ctx);if (retval) {com_err ("ext2fs_write_inode" , retval,_ ("while writing inode %lu in %s" ), ino, proc);
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 30/61 e2fsck: move some fixes out of parallel pthreads
In reply to Saranya MuruganandamFrom: Wang Shilong <[email protected]>
We could only use @found_map_block to find free blocks after we have collectd all used blocks, so something like handle_fs_bad_blocks(), ext2fs_create_resize_inode(), e2fsck_pass1_dupblocks() really should be handled after all threads has been finished.
--- e2fsck/e2fsck.h | 4 + e2fsck/pass1.c | 306 ++++++++++++++++----------- e2fsck/util.c | 36 +++- tests/f_multithread/expect.1 | 2 +- tests/f_multithread_logfile/expect.1 | 2 +- tests/f_multithread_no/expect.1 | 2 +- 6 files changed, 216 insertions(+), 136 deletions(-)
e2fsck/e2fsck.h +4 -0
@@ -450,6 +450,8 @@ struct e2fsck_struct {#ifdef CONFIG_PFSCK /* serialize fix operation for multiple threads */ pthread_mutex_t fs_fix_mutex; +/* protect block_found_map, block_dup_map */ +pthread_mutex_t fs_block_map_mutex;#endif };
@@ -750,6 +752,8 @@ extern errcode_t e2fsck_allocate_subcluster_bitmap(ext2_filsys fs,unsigned long long get_memory_size (void );extern void e2fsck_pass1_fix_lock (e2fsck_t ctx);extern void e2fsck_pass1_fix_unlock (e2fsck_t ctx); +extern void e2fsck_pass1_block_map_lock (e2fsck_t ctx); +extern void e2fsck_pass1_block_map_unlock (e2fsck_t ctx);/* unix.c */ extern void e2fsck_clear_progbar (e2fsck_t ctx);
e2fsck/pass1.c +181 -125
543 lines not on this page · raw message
e2fsck/util.c +28 -8
@@ -570,23 +570,34 @@ void e2fsck_read_inode_full(e2fsck_t ctx, unsigned long ino, }#ifdef CONFIG_PFSCK +#define e2fsck_get_lock_context (ctx) \ + e2fsck_t global_ctx = ctx->global_ctx; \ + if (!global_ctx) \ + global_ctx = ctx; \ +void e2fsck_pass1_fix_lock (e2fsck_t ctx) { -e2fsck_t global_ctx = ctx->global_ctx; -if (!global_ctx) - global_ctx = ctx; - +e2fsck_get_lock_context (ctx);pthread_mutex_lock (&global_ctx->fs_fix_mutex); }void e2fsck_pass1_fix_unlock (e2fsck_t ctx) { -e2fsck_t global_ctx = ctx->global_ctx; -if (!global_ctx) - global_ctx = ctx; - +e2fsck_get_lock_context (ctx);pthread_mutex_unlock (&global_ctx->fs_fix_mutex); } + +void e2fsck_pass1_block_map_lock (e2fsck_t ctx) +{ +e2fsck_get_lock_context (ctx); +pthread_mutex_lock (&global_ctx->fs_block_map_mutex); +} + +void e2fsck_pass1_block_map_unlock (e2fsck_t ctx) +{ +e2fsck_get_lock_context (ctx); +pthread_mutex_unlock (&global_ctx->fs_block_map_mutex); +}#else void e2fsck_pass1_fix_lock (e2fsck_t ctx) {
@@ -596,6 +607,15 @@ void e2fsck_pass1_fix_lock(e2fsck_t ctx)void e2fsck_pass1_fix_unlock (e2fsck_t ctx) { +} + +void e2fsck_pass1_block_map_lock (e2fsck_t ctx) +{ + +} + +void e2fsck_pass1_block_map_unlock (e2fsck_t ctx) +{ }#endif
tests/f_multithread/expect.1 +1 -1
@@ -1,7 +1,7 @@ ext2fs_open2: Bad magic number in super-block ../e2fsck/e2fsck: Superblock invalid, trying backup blocks... +Pass 1: Checking inodes, blocks, and sizes [Thread 0] Scan group range [0, 2) -[Thread 0] Pass 1: Checking inodes, blocks, and sizes [Thread 0] Scanned group range [0, 2), inodes 3008 Pass 2: Checking directory structure Pass 3: Checking directory connectivity
tests/f_multithread_logfile/expect.1 +1 -1
@@ -1,7 +1,7 @@ ext2fs_open2: Bad magic number in super-block ../e2fsck/e2fsck: Superblock invalid, trying backup blocks... +Pass 1: Checking inodes, blocks, and sizes [Thread 0] Scan group range [0, 2) -[Thread 0] Pass 1: Checking inodes, blocks, and sizes [Thread 0] Scanned group range [0, 2), inodes 3008 Pass 2: Checking directory structure Pass 3: Checking directory connectivity
tests/f_multithread_no/expect.1 +1 -1
@@ -1,7 +1,7 @@ ext2fs_open2: Bad magic number in super-block ../e2fsck/e2fsck: Superblock invalid, trying backup blocks... +Pass 1: Checking inodes, blocks, and sizes [Thread 0] Scan group range [0, 2) -[Thread 0] Pass 1: Checking inodes, blocks, and sizes [Thread 0] Scanned group range [0, 2), inodes 3008 Pass 2: Checking directory structure Pass 3: Checking directory connectivity
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 31/61 e2fsck: split and merge invalid bitmaps
In reply to Saranya MuruganandamFrom: Wang Shilong <[email protected]>
Invalid bitmaps are splitted per thread, and we
should merge them after thread finish.
--- e2fsck/pass1.c | 71 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 71 insertions(+)
e2fsck/pass1.c +71 -0
@@ -2379,6 +2379,62 @@ out:return retval; } +static void e2fsck_pass1_copy_invalid_bitmaps (e2fsck_t global_ctx, +e2fsck_t thread_ctx) +{ +dgrp_t i, j; +dgrp_t grp_start = thread_ctx->thread_info.et_group_start; +dgrp_t grp_end = thread_ctx->thread_info.et_group_end; +dgrp_t total = grp_end - grp_start; + + thread_ctx->invalid_inode_bitmap_flag = +e2fsck_allocate_memory (global_ctx,sizeof (int ) * total, +"invalid_inode_bitmap" ); + thread_ctx->invalid_block_bitmap_flag = +e2fsck_allocate_memory (global_ctx,sizeof (int ) * total, +"invalid_block_bitmap" ); + thread_ctx->invalid_inode_table_flag = +e2fsck_allocate_memory (global_ctx,sizeof (int ) * total, +"invalid_inode_table" ); + +memcpy (thread_ctx->invalid_block_bitmap_flag, + &global_ctx->invalid_block_bitmap_flag[grp_start], + total *sizeof (int )); +memcpy (thread_ctx->invalid_inode_bitmap_flag, + &global_ctx->invalid_inode_bitmap_flag[grp_start], + total *sizeof (int )); +memcpy (thread_ctx->invalid_inode_table_flag, + &global_ctx->invalid_inode_table_flag[grp_start], + total *sizeof (int )); + + thread_ctx->invalid_bitmaps =0 ; +for (i = grp_start, j =0 ; i < grp_end; i++, j++) { +if (thread_ctx->invalid_block_bitmap_flag[j]) + thread_ctx->invalid_bitmaps++; +if (thread_ctx->invalid_inode_bitmap_flag[j]) + thread_ctx->invalid_bitmaps++; +if (thread_ctx->invalid_inode_table_flag[j]) + thread_ctx->invalid_bitmaps++; + } +} + +static void e2fsck_pass1_merge_invalid_bitmaps (e2fsck_t global_ctx, +e2fsck_t thread_ctx) +{ +dgrp_t i, j; +dgrp_t grp_start = thread_ctx->thread_info.et_group_start; +dgrp_t grp_end = thread_ctx->thread_info.et_group_end; +dgrp_t total = grp_end - grp_start; + +memcpy (&global_ctx->invalid_block_bitmap_flag[grp_start], + thread_ctx->invalid_block_bitmap_flag, total *sizeof (int )); +memcpy (&global_ctx->invalid_inode_bitmap_flag[grp_start], + thread_ctx->invalid_inode_bitmap_flag, total *sizeof (int )); +memcpy (&global_ctx->invalid_inode_table_flag[grp_start], + thread_ctx->invalid_inode_table_flag, total *sizeof (int )); + global_ctx->invalid_bitmaps += thread_ctx->invalid_bitmaps; +} +static errcode_t e2fsck_pass1_thread_prepare (e2fsck_t global_ctx,e2fsck_t *thread_ctx,int thread_index,int num_threads) {
@@ -2455,6 +2511,7 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thre goto out_fs; } *thread_ctx = thread_context; +e2fsck_pass1_copy_invalid_bitmaps (global_ctx, thread_context);return 0 ; out_fs:ext2fs_free_mem (&thread_fs);
@@ -2589,6 +2646,10 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctxext2_ino_t dx_dir_info_count = global_ctx->dx_dir_info_count;ext2_u32_list dirs_to_hash = global_ctx->dirs_to_hash;quota_ctx_t qctx = global_ctx->qctx; +int *invalid_block_bitmap_flag = global_ctx->invalid_block_bitmap_flag; +int *invalid_inode_bitmap_flag = global_ctx->invalid_inode_bitmap_flag; +int *invalid_inode_table_flag = global_ctx->invalid_inode_table_flag; +int invalid_bitmaps = global_ctx->invalid_bitmaps;#ifdef HAVE_SETJMP_H jmp_buf old_jmp;
@@ -2667,6 +2728,11 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx thread_ctx->qctx);if (retval)return retval; + global_ctx->invalid_block_bitmap_flag = invalid_block_bitmap_flag; + global_ctx->invalid_inode_bitmap_flag = invalid_inode_bitmap_flag; + global_ctx->invalid_inode_table_flag = invalid_inode_table_flag; + global_ctx->invalid_bitmaps = invalid_bitmaps; +e2fsck_pass1_merge_invalid_bitmaps (global_ctx, thread_ctx); retval = e2fsck_pass1_merge_bitmap(global_fs, &thread_ctx->inode_used_map,
@@ -2739,6 +2805,9 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)if (thread_ctx->dirs_to_hash)ext2fs_badblocks_list_free (thread_ctx->dirs_to_hash);quota_release_context (&thread_ctx->qctx); +ext2fs_free_mem (&thread_ctx->invalid_block_bitmap_flag); +ext2fs_free_mem (&thread_ctx->invalid_inode_bitmap_flag); +ext2fs_free_mem (&thread_ctx->invalid_inode_table_flag);ext2fs_free_mem (&thread_ctx);return retval;
@@ -2752,6 +2821,8 @@ static int e2fsck_pass1_threads_join(struct e2fsck_thread_info *infos,int i;struct e2fsck_thread_info *pinfo; +/* merge invalid bitmaps will recalculate it */ + global_ctx->invalid_bitmaps =0 ;for (i =0 ; i < num_threads; i++) { pinfo = &infos[i];
Signature · 1 line
2.29.2.299.gdc1121823c-goog
Re: [RFC PATCH v3 31/61] e2fsck: split and merge invalid bitmaps
Quoted 4 lines · Saranya Muruganandam
From: Wang Shilong <[email protected]>
Invalid bitmaps are splitted per thread, and we
should merge them after thread finish.
For a large filesystem, would it make more sense to merge results periodically to reduce the peak memory consumption? That might not be all that high of a peak though since the end merges could be deleting records from the per-thread data structure after each succesful insertion merge.
--D
Quoted 126 lines
Signed-off-by Wang Shilong <[email protected]>Signed-off-by Saranya Muruganandam <[email protected]>--- e2fsck/pass1.c | 71 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 71 insertions(+)e2fsck/pass1.c +71
-0@@ -2379,6 +2379,62 @@ out:return retval; } +static void e2fsck_pass1_copy_invalid_bitmaps (e2fsck_t global_ctx, +e2fsck_t thread_ctx) +{ +dgrp_t i, j; +dgrp_t grp_start = thread_ctx->thread_info.et_group_start; +dgrp_t grp_end = thread_ctx->thread_info.et_group_end; +dgrp_t total = grp_end - grp_start; + + thread_ctx->invalid_inode_bitmap_flag = +e2fsck_allocate_memory (global_ctx,sizeof (int ) * total, +"invalid_inode_bitmap" ); + thread_ctx->invalid_block_bitmap_flag = +e2fsck_allocate_memory (global_ctx,sizeof (int ) * total, +"invalid_block_bitmap" ); + thread_ctx->invalid_inode_table_flag = +e2fsck_allocate_memory (global_ctx,sizeof (int ) * total, +"invalid_inode_table" ); + +memcpy (thread_ctx->invalid_block_bitmap_flag, + &global_ctx->invalid_block_bitmap_flag[grp_start], + total *sizeof (int )); +memcpy (thread_ctx->invalid_inode_bitmap_flag, + &global_ctx->invalid_inode_bitmap_flag[grp_start], + total *sizeof (int )); +memcpy (thread_ctx->invalid_inode_table_flag, + &global_ctx->invalid_inode_table_flag[grp_start], + total *sizeof (int )); + + thread_ctx->invalid_bitmaps =0 ; +for (i = grp_start, j =0 ; i < grp_end; i++, j++) { +if (thread_ctx->invalid_block_bitmap_flag[j]) + thread_ctx->invalid_bitmaps++; +if (thread_ctx->invalid_inode_bitmap_flag[j]) + thread_ctx->invalid_bitmaps++; +if (thread_ctx->invalid_inode_table_flag[j]) + thread_ctx->invalid_bitmaps++; + } +} + +static void e2fsck_pass1_merge_invalid_bitmaps (e2fsck_t global_ctx, +e2fsck_t thread_ctx) +{ +dgrp_t i, j; +dgrp_t grp_start = thread_ctx->thread_info.et_group_start; +dgrp_t grp_end = thread_ctx->thread_info.et_group_end; +dgrp_t total = grp_end - grp_start; + +memcpy (&global_ctx->invalid_block_bitmap_flag[grp_start], + thread_ctx->invalid_block_bitmap_flag, total *sizeof (int )); +memcpy (&global_ctx->invalid_inode_bitmap_flag[grp_start], + thread_ctx->invalid_inode_bitmap_flag, total *sizeof (int )); +memcpy (&global_ctx->invalid_inode_table_flag[grp_start], + thread_ctx->invalid_inode_table_flag, total *sizeof (int )); + global_ctx->invalid_bitmaps += thread_ctx->invalid_bitmaps; +} +static errcode_t e2fsck_pass1_thread_prepare (e2fsck_t global_ctx,e2fsck_t *thread_ctx,int thread_index,int num_threads) {@@ -2455,6 +2511,7 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thre goto out_fs; } *thread_ctx = thread_context; +e2fsck_pass1_copy_invalid_bitmaps (global_ctx, thread_context);return 0 ; out_fs:ext2fs_free_mem (&thread_fs);@@ -2589,6 +2646,10 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctxext2_ino_t dx_dir_info_count = global_ctx->dx_dir_info_count;ext2_u32_list dirs_to_hash = global_ctx->dirs_to_hash;quota_ctx_t qctx = global_ctx->qctx; +int *invalid_block_bitmap_flag = global_ctx->invalid_block_bitmap_flag; +int *invalid_inode_bitmap_flag = global_ctx->invalid_inode_bitmap_flag; +int *invalid_inode_table_flag = global_ctx->invalid_inode_table_flag; +int invalid_bitmaps = global_ctx->invalid_bitmaps;#ifdef HAVE_SETJMP_H jmp_buf old_jmp;@@ -2667,6 +2728,11 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx thread_ctx->qctx);if (retval)return retval; + global_ctx->invalid_block_bitmap_flag = invalid_block_bitmap_flag; + global_ctx->invalid_inode_bitmap_flag = invalid_inode_bitmap_flag; + global_ctx->invalid_inode_table_flag = invalid_inode_table_flag; + global_ctx->invalid_bitmaps = invalid_bitmaps; +e2fsck_pass1_merge_invalid_bitmaps (global_ctx, thread_ctx); retval = e2fsck_pass1_merge_bitmap(global_fs, &thread_ctx->inode_used_map,@@ -2739,6 +2805,9 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)if (thread_ctx->dirs_to_hash)ext2fs_badblocks_list_free (thread_ctx->dirs_to_hash);quota_release_context (&thread_ctx->qctx); +ext2fs_free_mem (&thread_ctx->invalid_block_bitmap_flag); +ext2fs_free_mem (&thread_ctx->invalid_inode_bitmap_flag); +ext2fs_free_mem (&thread_ctx->invalid_inode_table_flag);ext2fs_free_mem (&thread_ctx);return retval;@@ -2752,6 +2821,8 @@ static int e2fsck_pass1_threads_join(struct e2fsck_thread_info *infos,int i;struct e2fsck_thread_info *pinfo; +/* merge invalid bitmaps will recalculate it */ + global_ctx->invalid_bitmaps =0 ;for (i =0 ; i < num_threads; i++) { pinfo = &infos[i];Signature · 1 line
2.29.2.299.gdc1121823c-goog
Re: [RFC PATCH v3 31/61] e2fsck: split and merge invalid bitmaps
Quoted 4 lines · Darrick J. Wong
Quoted 2 lines · Saranya Muruganandam
From: Wang Shilong <[email protected]>
Quoted 8 lines
Quoted 2 lines
Invalid bitmaps are splitted per thread, and we
should merge them after thread finish.For a large filesystem, would it make more sense to merge results periodically to reduce the peak memory consumption? That might not be all that high of a peak though since the end merges could be deleting records from the per-thread data structure after each succesful insertion merge.
That could be possible optimization, comparing to original memory usage, pfsck did not increase memory usage too much, it just try to add some extra structure overhead X number of threads.
And from our benchmarking on PiB filesystem, we saw similar memory peak numbers for pfsck, we might post them in the changelog next time.
Quoted 129 lines
--D
Quoted 126 lines
Signed-off-by Wang Shilong <[email protected]>Signed-off-by Saranya Muruganandam <[email protected]>--- e2fsck/pass1.c | 71 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 71 insertions(+)e2fsck/pass1.c +71
-1@@ -2379,6 +2379,62 @@ out:return retval; } +static void e2fsck_pass1_copy_invalid_bitmaps (e2fsck_t global_ctx, +e2fsck_t thread_ctx) +{ +dgrp_t i, j; +dgrp_t grp_start = thread_ctx->thread_info.et_group_start; +dgrp_t grp_end = thread_ctx->thread_info.et_group_end; +dgrp_t total = grp_end - grp_start; + + thread_ctx->invalid_inode_bitmap_flag = +e2fsck_allocate_memory (global_ctx,sizeof (int ) * total, +"invalid_inode_bitmap" ); + thread_ctx->invalid_block_bitmap_flag = +e2fsck_allocate_memory (global_ctx,sizeof (int ) * total, +"invalid_block_bitmap" ); + thread_ctx->invalid_inode_table_flag = +e2fsck_allocate_memory (global_ctx,sizeof (int ) * total, +"invalid_inode_table" ); + +memcpy (thread_ctx->invalid_block_bitmap_flag, + &global_ctx->invalid_block_bitmap_flag[grp_start], + total *sizeof (int )); +memcpy (thread_ctx->invalid_inode_bitmap_flag, + &global_ctx->invalid_inode_bitmap_flag[grp_start], + total *sizeof (int )); +memcpy (thread_ctx->invalid_inode_table_flag, + &global_ctx->invalid_inode_table_flag[grp_start], + total *sizeof (int )); + + thread_ctx->invalid_bitmaps =0 ; +for (i = grp_start, j =0 ; i < grp_end; i++, j++) { +if (thread_ctx->invalid_block_bitmap_flag[j]) + thread_ctx->invalid_bitmaps++; +if (thread_ctx->invalid_inode_bitmap_flag[j]) + thread_ctx->invalid_bitmaps++; +if (thread_ctx->invalid_inode_table_flag[j]) + thread_ctx->invalid_bitmaps++; + } +} + +static void e2fsck_pass1_merge_invalid_bitmaps (e2fsck_t global_ctx, +e2fsck_t thread_ctx) +{ +dgrp_t i, j; +dgrp_t grp_start = thread_ctx->thread_info.et_group_start; +dgrp_t grp_end = thread_ctx->thread_info.et_group_end; +dgrp_t total = grp_end - grp_start; + +memcpy (&global_ctx->invalid_block_bitmap_flag[grp_start], + thread_ctx->invalid_block_bitmap_flag, total *sizeof (int )); +memcpy (&global_ctx->invalid_inode_bitmap_flag[grp_start], + thread_ctx->invalid_inode_bitmap_flag, total *sizeof (int )); +memcpy (&global_ctx->invalid_inode_table_flag[grp_start], + thread_ctx->invalid_inode_table_flag, total *sizeof (int )); + global_ctx->invalid_bitmaps += thread_ctx->invalid_bitmaps; +} +static errcode_t e2fsck_pass1_thread_prepare (e2fsck_t global_ctx,e2fsck_t *thread_ctx,int thread_index,int num_threads) {@@ -2455,6 +2511,7 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thre goto out_fs; } *thread_ctx = thread_context; +e2fsck_pass1_copy_invalid_bitmaps (global_ctx, thread_context);return 0 ; out_fs:ext2fs_free_mem (&thread_fs);@@ -2589,6 +2646,10 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctxext2_ino_t dx_dir_info_count = global_ctx->dx_dir_info_count;ext2_u32_list dirs_to_hash = global_ctx->dirs_to_hash;quota_ctx_t qctx = global_ctx->qctx; +int *invalid_block_bitmap_flag = global_ctx->invalid_block_bitmap_flag; +int *invalid_inode_bitmap_flag = global_ctx->invalid_inode_bitmap_flag; +int *invalid_inode_table_flag = global_ctx->invalid_inode_table_flag; +int invalid_bitmaps = global_ctx->invalid_bitmaps;#ifdef HAVE_SETJMP_H jmp_buf old_jmp;@@ -2667,6 +2728,11 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx thread_ctx->qctx);if (retval)return retval; + global_ctx->invalid_block_bitmap_flag = invalid_block_bitmap_flag; + global_ctx->invalid_inode_bitmap_flag = invalid_inode_bitmap_flag; + global_ctx->invalid_inode_table_flag = invalid_inode_table_flag; + global_ctx->invalid_bitmaps = invalid_bitmaps; +e2fsck_pass1_merge_invalid_bitmaps (global_ctx, thread_ctx); retval = e2fsck_pass1_merge_bitmap(global_fs, &thread_ctx->inode_used_map,@@ -2739,6 +2805,9 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)if (thread_ctx->dirs_to_hash)ext2fs_badblocks_list_free (thread_ctx->dirs_to_hash);quota_release_context (&thread_ctx->qctx); +ext2fs_free_mem (&thread_ctx->invalid_block_bitmap_flag); +ext2fs_free_mem (&thread_ctx->invalid_inode_bitmap_flag); +ext2fs_free_mem (&thread_ctx->invalid_inode_table_flag);ext2fs_free_mem (&thread_ctx);return retval;@@ -2752,6 +2821,8 @@ static int e2fsck_pass1_threads_join(struct e2fsck_thread_info *infos,int i;struct e2fsck_thread_info *pinfo; +/* merge invalid bitmaps will recalculate it */ + global_ctx->invalid_bitmaps =0 ;for (i =0 ; i < num_threads; i++) { pinfo = &infos[i]; --2.29.2.299.gdc1121823c-goog
RFC PATCH v3 32/61 e2fsck: merge EA blocks properly
In reply to Saranya MuruganandamFrom: Wang Shilong <[email protected]>
EA blocks might be shared, merge them carefully.
--- e2fsck/e2fsck.h | 1 + e2fsck/pass1.c | 245 +++++++++++++++++++++++++++++++++++++++++------- 2 files changed, 213 insertions(+), 33 deletions(-)
e2fsck/e2fsck.h +1 -0
@@ -299,6 +299,7 @@ struct e2fsck_struct {ext2_refcount_t refcount;ext2_refcount_t refcount_extra; +ext2_refcount_t refcount_orig;/* * Quota blocks and inodes to be charged for each ea block.
e2fsck/pass1.c +212 -33
342 lines not on this page · raw message
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 33/61 e2fsck: kickoff mutex lock for block found map
In reply to Saranya MuruganandamFrom: Wang Shilong <[email protected]>
Now @block_found_map is no longer shared by multiple threads, and @block_dup_map need be checked again after threads finish.
--- e2fsck/e2fsck.h | 9 +- e2fsck/pass1.c | 172 ++++++++++++++++++------------ e2fsck/util.c | 34 ++++-- lib/ext2fs/bitops.h | 2 + lib/ext2fs/gen_bitmap64.c | 33 ++++++ tests/f_itable_collision/expect.1 | 3 - 6 files changed, 172 insertions(+), 81 deletions(-)
e2fsck/e2fsck.h +6 -3
@@ -202,6 +202,7 @@ struct resource_track {#define E2F_FLAG_TIME_INSANE 0x2000/* Time is insane */ #define E2F_FLAG_PROBLEMS_FIXED 0x4000/* At least one problem was fixed */ #define E2F_FLAG_ALLOC_OK 0x8000/* Can we allocate blocks? */ +#define E2F_FLAG_DUP_BLOCK 0x20000/* dup block found during pass1 */ #define E2F_RESET_FLAGS (E2F_FLAG_TIME_INSANE | E2F_FLAG_PROBLEMS_FIXED)
@@ -452,7 +453,7 @@ struct e2fsck_struct {/* serialize fix operation for multiple threads */ pthread_mutex_t fs_fix_mutex;/* protect block_found_map, block_dup_map */ -pthread_mutex_t fs_block_map_mutex; +pthread_rwlock_t fs_block_map_rwlock;#endif };
@@ -753,8 +754,10 @@ extern errcode_t e2fsck_allocate_subcluster_bitmap(ext2_filsys fs,unsigned long long get_memory_size (void );extern void e2fsck_pass1_fix_lock (e2fsck_t ctx);extern void e2fsck_pass1_fix_unlock (e2fsck_t ctx); -extern void e2fsck_pass1_block_map_lock (e2fsck_t ctx); -extern void e2fsck_pass1_block_map_unlock (e2fsck_t ctx); +extern void e2fsck_pass1_block_map_w_lock (e2fsck_t ctx); +extern void e2fsck_pass1_block_map_w_unlock (e2fsck_t ctx); +extern void e2fsck_pass1_block_map_r_lock (e2fsck_t ctx); +extern void e2fsck_pass1_block_map_r_unlock (e2fsck_t ctx);/* unix.c */ extern void e2fsck_clear_progbar (e2fsck_t ctx);
e2fsck/pass1.c +103 -69
352 lines not on this page · raw message
e2fsck/util.c +28 -6
@@ -587,17 +587,29 @@ void e2fsck_pass1_fix_unlock(e2fsck_t ctx) pthread_mutex_unlock(&global_ctx->fs_fix_mutex); } -void e2fsck_pass1_block_map_lock (e2fsck_t ctx) +void e2fsck_pass1_block_map_w_lock (e2fsck_t ctx) {e2fsck_get_lock_context (ctx); -pthread_mutex_lock (&global_ctx->fs_block_map_mutex); +pthread_rwlock_wrlock (&global_ctx->fs_block_map_rwlock); } -void e2fsck_pass1_block_map_unlock (e2fsck_t ctx) +void e2fsck_pass1_block_map_w_unlock (e2fsck_t ctx) {e2fsck_get_lock_context (ctx); -pthread_mutex_unlock (&global_ctx->fs_block_map_mutex); +pthread_rwlock_unlock (&global_ctx->fs_block_map_rwlock); } + +void e2fsck_pass1_block_map_r_lock (e2fsck_t ctx) +{ +e2fsck_get_lock_context (ctx); +pthread_rwlock_rdlock (&global_ctx->fs_block_map_rwlock); +} + +void e2fsck_pass1_block_map_r_unlock (e2fsck_t ctx) +{ +e2fsck_get_lock_context (ctx); +pthread_rwlock_unlock (&global_ctx->fs_block_map_rwlock); + }#else void e2fsck_pass1_fix_lock (e2fsck_t ctx) {
@@ -608,14 +620,24 @@ void e2fsck_pass1_fix_unlock(e2fsck_t ctx) { } +void e2fsck_pass1_block_map_w_lock (e2fsck_t ctx) +{ -void e2fsck_pass1_block_map_lock (e2fsck_t ctx) +} + +void e2fsck_pass1_block_map_w_unlock (e2fsck_t ctx) { } -void e2fsck_pass1_block_map_unlock (e2fsck_t ctx) +void e2fsck_pass1_block_map_r_lock (e2fsck_t ctx) { + +} + +void e2fsck_pass1_block_map_r_unlock (e2fsck_t ctx) +{ + }#endif
lib/ext2fs/bitops.h +2 -0
@@ -120,6 +120,8 @@ extern int ext2fs_test_block_bitmap_range(ext2fs_block_bitmap bitmap,extern void ext2fs_set_bitmap_padding (ext2fs_generic_bitmap map);extern __u32 ext2fs_get_generic_bitmap_start (ext2fs_generic_bitmap bitmap);extern __u32 ext2fs_get_generic_bitmap_end (ext2fs_generic_bitmap bitmap); +extern int ext2fs_test_block_bitmap_range2_valid (ext2fs_block_bitmap bitmap, +blk64_t block,unsigned int num);/* 64-bit versions */
lib/ext2fs/gen_bitmap64.c +33 -0
@@ -725,6 +725,39 @@ int ext2fs_test_block_bitmap_range2(ext2fs_block_bitmap gen_bmap,return bmap->bitmap_ops->test_clear_bmap_extent (bmap, block, num); } +int ext2fs_test_block_bitmap_range2_valid (ext2fs_block_bitmap bitmap, +blk64_t block,unsigned int num) +{ +ext2fs_generic_bitmap_64 bmap = (ext2fs_generic_bitmap_64 )bitmap; +__u64 end = block + num; + +if (!bmap) +return 0 ; + +if (EXT2FS_IS_32_BITMAP (bmap)) { +if ((block & ~0xffffffffULL ) || + ((block+num-1 ) & ~0xffffffffULL )) { +return 0 ; + } + } + +if (!EXT2FS_IS_64_BITMAP (bmap)) +return 0 ; + +/* convert to clusters if necessary */ + block >>= bmap->cluster_bits; + end += (1 << bmap->cluster_bits) -1 ; + end >>= bmap->cluster_bits; + num = end - block; + +if ((block < bmap->start) || (block > bmap->end) || + (block+num-1 > bmap->end)) +return 0 ; + +return 1 ; +} + +void ext2fs_mark_block_bitmap_range2 (ext2fs_block_bitmap gen_bmap,blk64_t block,unsigned int num) {
tests/f_itable_collision/expect.1 +0 -3
@@ -1,6 +1,5 @@ Pass 1: Checking inodes, blocks, and sizes Inode 12 block 37 conflicts with critical metadata, skipping block checks. -Illegal block number passed to ext2fs_test_block_bitmap #268435455 for in-use block map Illegal block number passed to ext2fs_mark_block_bitmap #268435455 for in-use block map Inode 12, i_blocks is 48, should be 56. Fix? yes
@@ -27,9 +26,7 @@ Clear inode? yes Restarting e2fsck from the beginning... Pass 1: Checking inodes, blocks, and sizes Inode 12 block 37 conflicts with critical metadata, skipping block checks. -Illegal block number passed to ext2fs_test_block_bitmap #4294967294 for in-use block map Illegal block number passed to ext2fs_mark_block_bitmap #4294967294 for in-use block map -Illegal block number passed to ext2fs_test_block_bitmap #268435455 for in-use block map Illegal block number passed to ext2fs_mark_block_bitmap #268435455 for in-use block map Running additional passes to resolve blocks claimed by more than one inode...
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 34/61 e2fsck: allow admin specify number of threads
In reply to Saranya MuruganandamFrom: Wang Shilong <[email protected]>
-m option is introduced to specify number of threads for pfsck.
--- e2fsck/e2fsck.h | 2 + e2fsck/pass1.c | 165 ++++++++++++++++---------- e2fsck/unix.c | 17 ++- tests/f_multithread/script | 2 +- tests/f_multithread_completion/script | 2 +- tests/f_multithread_logfile/script | 2 +- tests/f_multithread_no/script | 2 +- tests/f_multithread_preen/script | 2 +- tests/f_multithread_yes/script | 2 +- 9 files changed, 128 insertions(+), 68 deletions(-)
e2fsck/e2fsck.h +2 -0
@@ -450,6 +450,7 @@ struct e2fsck_struct {/* Undo file */ char *undo_file;#ifdef CONFIG_PFSCK +__u32 fs_num_threads;/* serialize fix operation for multiple threads */ pthread_mutex_t fs_fix_mutex;/* protect block_found_map, block_dup_map */
@@ -684,6 +685,7 @@ int check_backup_super_block(e2fsck_t ctx);void check_resize_inode (e2fsck_t ctx);/* util.c */ +#define E2FSCK_MAX_THREADS (65536)extern void *e2fsck_allocate_memory (e2fsck_t ctx,unsigned long size,const char *description);extern int ask (e2fsck_t ctx,const char * string,int def);
e2fsck/pass1.c +104 -61
206 lines not on this page · raw message
e2fsck/unix.c +16 -1
@@ -819,6 +819,10 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx)int res;/* result of sscanf */ #ifdef CONFIG_JBD_DEBUG char *jbd_debug; +#endif +#ifdef CONFIG_PFSCK +char *pm; +unsigned long thread_num;#endif unsigned long long phys_mem_kb;
@@ -852,7 +856,7 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx) ctx->readahead_kb = ~0ULL ;#ifdef CONFIG_PFSCK -while ((c =getopt (argc, argv,"pamnyrcC:B:dE:fvtFVM:b:I:j:P:l:L:N:SsDkz:" )) !=EOF ) +while ((c =getopt (argc, argv,"pam:nyrcC:B:dE:fvtFVM:b:I:j:P:l:L:N:SsDkz:" )) !=EOF ) #else while ((c = getopt(argc, argv, "panyrcC:B:dE:fvtFVM:b:I:j:P:l:L:N:SsDkz:")) != EOF)#endif
@@ -898,7 +902,18 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx)break ;#ifdef CONFIG_PFSCK case 'm' : + thread_num =strtoul (optarg, &pm,0 ); +if (*pm) +fatal_error (ctx, +_ ("Invalid multiple thread num.\n" )); +if (thread_num >E2FSCK_MAX_THREADS ) { +fprintf (stderr, +_ ("threads %lu too large (max %lu)\n" ), + thread_num,E2FSCK_MAX_THREADS ); +fatal_error (ctx,0 ); + } ctx->options |=E2F_OPT_MULTITHREAD ; + ctx->fs_num_threads = thread_num;break ;#endif case 'n' :
tests/f_multithread/script +1 -1
@@ -1,4 +1,4 @@ -FSCK_OPT="-fy -m" +FSCK_OPT="-fy -m1" SECOND_FSCK_OPT=-yf . $cmd_dir/run_e2fsck
tests/f_multithread_completion/script +1 -1
@@ -1,4 +1,4 @@ -FSCK_OPT="-fy -m -C 1" +FSCK_OPT="-fy -m1 -C 1" SECOND_FSCK_OPT=-yf . $cmd_dir/run_e2fsck
tests/f_multithread_logfile/script +1 -1
@@ -1,5 +1,5 @@ LOG_FNAME="f_multithread_logfile_xxx" -FSCK_OPT="-fy -m -y -E log_filename=$LOG_FNAME" +FSCK_OPT="-fy -m1 -y -E log_filename=$LOG_FNAME" SKIP_VERIFY="true" ONE_PASS_ONLY="true" SKIP_CLEANUP="true"
tests/f_multithread_no/script +1 -1
@@ -1,4 +1,4 @@ -FSCK_OPT="-fn -m" +FSCK_OPT="-fn -m1" SECOND_FSCK_OPT=-yf . $cmd_dir/run_e2fsck
tests/f_multithread_preen/script +1 -1
@@ -1,4 +1,4 @@ -FSCK_OPT="-fp -m" +FSCK_OPT="-fp -m1" SECOND_FSCK_OPT=-yf . $cmd_dir/run_e2fsck
tests/f_multithread_yes/script +1 -1
@@ -1,4 +1,4 @@ -FSCK_OPT="-f -m" +FSCK_OPT="-f -m1" SECOND_FSCK_OPT=-yf . $cmd_dir/run_e2fsck
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 35/61 e2fsck: adjust number of threads
In reply to Saranya MuruganandamFrom: Wang Shilong <[email protected]>
number of threads should not exceed flex bg numbers, and output messages if we adjust threads number.
--- e2fsck/pass1.c | 32 +++++++++++++++----------------- lib/ext2fs/ext2fs.h | 32 +++++++++++++++++++++++++++++++- 2 files changed, 46 insertions(+), 18 deletions(-)
e2fsck/pass1.c +15 -17
@@ -1282,6 +1282,7 @@ static void e2fsck_pass1_set_thread_num(e2fsck_t ctx) } out: ctx->fs_num_threads = num_threads; + ctx->fs->fs_num_threads = num_threads; }#endif
@@ -2551,14 +2552,14 @@ static void e2fsck_pass1_merge_invalid_bitmaps(e2fsck_t global_ctx, }static errcode_t e2fsck_pass1_thread_prepare (e2fsck_t global_ctx,e2fsck_t *thread_ctx, -int thread_index,int num_threads) +int thread_index,int num_threads, +dgrp_t average_group) {errcode_t retval;e2fsck_t thread_context;ext2_filsys thread_fs;ext2_filsys global_fs = global_ctx->fs;struct e2fsck_thread *tinfo; -dgrp_t average_group;assert (global_ctx->inode_used_map ==NULL );assert (global_ctx->inode_dir_map ==NULL );
@@ -2605,16 +2606,9 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thre thread_context->thread_info.et_thread_index = thread_index;set_up_logging (thread_context); -/* -* Distribute work to multiple threads: -* Each thread work on fs->group_desc_count / nthread groups. -*/ tinfo = &thread_context->thread_info; - average_group = thread_fs->group_desc_count / num_threads; -if (average_group ==0 ) - average_group =1 ; tinfo->et_group_start = average_group * thread_index; -if (thread_index == num_threads -1 ) +if (thread_index == global_fs->fs_num_threads -1 ) tinfo->et_group_end = thread_fs->group_desc_count;else tinfo->et_group_end = average_group * (thread_index +1 );
@@ -3130,12 +3124,13 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx) }static int e2fsck_pass1_threads_join (struct e2fsck_thread_info *infos, -int num_threads,e2fsck_t global_ctx) +e2fsck_t global_ctx) {errcode_t rc;errcode_t ret =0 ;int i;struct e2fsck_thread_info *pinfo; +int num_threads = global_ctx->fs_num_threads;/* merge invalid bitmaps will recalculate it */ global_ctx->invalid_bitmaps =0 ;
@@ -3217,7 +3212,7 @@ out: }static int e2fsck_pass1_threads_start (struct e2fsck_thread_info **pinfo, -int num_threads,e2fsck_t global_ctx) +e2fsck_t global_ctx) {struct e2fsck_thread_info *infos;pthread_attr_t attr;
@@ -3226,6 +3221,8 @@ static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo,struct e2fsck_thread_info *tmp_pinfo;int i;e2fsck_t thread_ctx; +dgrp_t average_group; +int num_threads = global_ctx->fs_num_threads;#ifdef DEBUG_THREADS struct e2fsck_thread_debug thread_debug = {PTHREAD_MUTEX_INITIALIZER ,PTHREAD_COND_INITIALIZER ,0 };
@@ -3249,6 +3246,7 @@ static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo,return retval; } + average_group =ext2fs_get_avg_group (global_ctx->fs);for (i =0 ; i < num_threads; i++) { tmp_pinfo = &infos[i]; tmp_pinfo->eti_thread_index = i;
@@ -3256,7 +3254,8 @@ static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo, tmp_pinfo->eti_debug = &thread_debug;#endif retval =e2fsck_pass1_thread_prepare (global_ctx, &thread_ctx, - i, num_threads); + i, num_threads, + average_group);if (retval) {com_err (global_ctx->program_name, retval,_ ("while preparing pass1 thread\n" ));
@@ -3286,7 +3285,7 @@ static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo, }if (retval) { -e2fsck_pass1_threads_join (infos, num_threads, global_ctx); +e2fsck_pass1_threads_join (infos, global_ctx);return retval; } *pinfo = infos;
@@ -3296,17 +3295,16 @@ static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo,static void e2fsck_pass1_multithread (e2fsck_t global_ctx) {struct e2fsck_thread_info *infos =NULL ; -int num_threads = global_ctx->fs_num_threads;errcode_t retval; - retval =e2fsck_pass1_threads_start (&infos, num_threads, global_ctx); + retval =e2fsck_pass1_threads_start (&infos, global_ctx);if (retval) {com_err (global_ctx->program_name, retval,_ ("while starting pass1 threads\n" )); goto out_abort; } - retval =e2fsck_pass1_threads_join (infos, num_threads, global_ctx); + retval =e2fsck_pass1_threads_join (infos, global_ctx);if (retval) {com_err (global_ctx->program_name, retval,_ ("while joining pass1 threads\n" ));
lib/ext2fs/ext2fs.h +31 -1
@@ -255,10 +255,11 @@ struct struct_ext2_filsys {int cluster_ratio_bits;__u16 default_bitmap_type;__u16 pad; +__u32 fs_num_threads;/* * Reserved for future expansion */ -__u32 reserved[5 ]; +__u32 reserved[4 ];/* * Reserved for the use of the calling application.
@@ -2121,6 +2122,35 @@ ext2fs_const_inode(const struct ext2_inode_large * large_inode) return (const struct ext2_inode *) large_inode; } +static dgrp_t ext2fs_get_avg_group (ext2_filsys fs) +{ +#ifdef CONFIG_PFSCK +dgrp_t average_group; +unsigned flexbg_size; + +if (fs->fs_num_threads <=1 ) +return fs->group_desc_count; + + average_group = fs->group_desc_count / fs->fs_num_threads; +if (average_group <=1 ) +return 1 ; + +if (ext2fs_has_feature_flex_bg (fs->super)) { +int times =1 ; + + flexbg_size =1 << fs->super->s_log_groups_per_flex; +if (average_group % flexbg_size) { + times = average_group / flexbg_size; + average_group = times * flexbg_size; + } + } + +return average_group; +#else +return fs->group_desc_count; +#endif +} +#undef _INLINE_#endif
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 36/61 e2fsck: fix readahead for pfsck of pass1
In reply to Saranya MuruganandamFrom: Wang Shilong <[email protected]>
Several improvments for this patch:
1) move readahead_kb detection to preparing phase.
2) inode readahead should be aware of thread block group boundary.
3) make readahead_kb aware of multiple threads.
--- e2fsck/pass1.c | 28 +++++++++++++++++++--------- 1 file changed, 19 insertions(+), 9 deletions(-)
e2fsck/pass1.c +19 -9
@@ -1093,16 +1093,19 @@ out:static void pass1_readahead (e2fsck_t ctx,dgrp_t *group,ext2_ino_t *next_ino) {ext2_ino_t inodes_in_group =0 , inodes_per_block, inodes_per_buffer; -dgrp_t start = *group, grp; +dgrp_t start = *group, grp, grp_end = ctx->fs->group_desc_count;blk64_t blocks_to_read =0 ;errcode_t err =EXT2_ET_INVALID_ARGUMENT ; +#ifdef CONFIG_PFSCK + grp_end = ctx->thread_info.et_group_end; +#endif if (ctx->readahead_kb ==0 ) goto out;/* Keep iterating groups until we have enough to readahead */ inodes_per_block =EXT2_INODES_PER_BLOCK (ctx->fs->super); -for (grp = start; grp < ctx->fs->group_desc_count; grp++) { +for (grp = start; grp < grp_end; grp++) {if (ext2fs_bg_flags_test (ctx->fs, grp,EXT2_BG_INODE_UNINIT ))continue ; inodes_in_group = ctx->fs->super->s_inodes_per_group -
@@ -1295,12 +1298,25 @@ static errcode_t e2fsck_pass1_prepare(e2fsck_t ctx) {struct problem_context pctx;ext2_filsys fs = ctx->fs; +unsigned long long readahead_kb;init_ext2_max_sizes (); -#ifdef CONFIG_PFSCK +#ifdef CONFIG_PFSCK e2fsck_pass1_set_thread_num (ctx);#endif +/* If we can do readahead, figure out how many groups to pull in. */ +if (!e2fsck_can_readahead (ctx->fs)) + ctx->readahead_kb =0 ; +else if (ctx->readahead_kb == ~0ULL ) + ctx->readahead_kb =e2fsck_guess_readahead (ctx->fs); +#ifdef CONFIG_PFSCK +/* don't use more than 1/10 of memory for threads checking */ + readahead_kb =get_memory_size () / (10 * ctx->fs_num_threads); +/* maybe better disable RA if this is too small? */ +if (ctx->readahead_kb > readahead_kb) + ctx->readahead_kb = readahead_kb; +#endif clear_problem_context (&pctx);if (!(ctx->options &E2F_OPT_PREEN ))fix_problem (ctx,PR_1_PASS_HEADER , &pctx);
@@ -1477,13 +1493,7 @@ void e2fsck_pass1_run(e2fsck_t ctx)init_resource_track (&rtrack, ctx->fs->io);clear_problem_context (&pctx); -/* If we can do readahead, figure out how many groups to pull in. */ -if (!e2fsck_can_readahead (ctx->fs)) - ctx->readahead_kb =0 ; -else if (ctx->readahead_kb == ~0ULL ) - ctx->readahead_kb =e2fsck_guess_readahead (ctx->fs);pass1_readahead (ctx, &ra_group, &ino_threshold); -if (ext2fs_has_feature_dir_index (fs->super) && !(ctx->options &E2F_OPT_NO )) {if (ext2fs_u32_list_create (&ctx->dirs_to_hash,50 ))
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 37/61 e2fsck: merge options after threads finish
In reply to Saranya MuruganandamFrom: Wang Shilong <[email protected]>
It will be possible that threads might append E2F_OPT_YES, so we need merge options to global, test f_yesall cover this.
--- e2fsck/pass1.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-)
e2fsck/pass1.c +4 -5
@@ -2935,6 +2935,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctxext2_refcount_t ea_inode_refs = global_ctx->ea_inode_refs;ext2fs_block_bitmap block_found_map = global_ctx->block_found_map;ext2fs_block_bitmap block_dup_map = global_ctx->block_dup_map; +int options = global_ctx->options;#ifdef HAVE_SETJMP_H jmp_buf old_jmp;
@@ -2987,7 +2988,8 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx global_ctx->fs_fragmented += fs_fragmented; global_ctx->fs_fragmented_dir += fs_fragmented_dir; global_ctx->large_files += large_files; - +/* threads might enable E2F_OPT_YES */ + global_ctx->options |= options; global_ctx->flags |= flags; retval =e2fsck_pass1_merge_fs (global_fs, thread_fs);
@@ -3022,10 +3024,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx thread_ctx->qctx);if (retval)return retval; - global_ctx->invalid_block_bitmap_flag = invalid_block_bitmap_flag; - global_ctx->invalid_inode_bitmap_flag = invalid_inode_bitmap_flag; - global_ctx->invalid_inode_table_flag = invalid_inode_table_flag; - global_ctx->invalid_bitmaps = invalid_bitmaps; +e2fsck_pass1_merge_invalid_bitmaps (global_ctx, thread_ctx); retval = e2fsck_pass1_merge_bitmap(global_fs,
Signature · 1 line
2.29.2.299.gdc1121823c-goog
Re: [RFC PATCH v3 37/61] e2fsck: merge options after threads finish
Quoted 4 lines · Saranya Muruganandam
From: Wang Shilong <[email protected]>
It will be possible that threads might append
E2F_OPT_YES, so we need merge options to global, testf_yesallcover this.
I don't get it -- presumably fix_problem already has to take some sort of a lock to complain the console and (possibly) wait for input, right? Why not just set E2F_OPT_YES right then and there and have it take effect across all threads immediately? I guess you'd have to add another lock to protect the options bitflags.
--D
Quoted 43 lines
Signed-off-by Wang Shilong <[email protected]>Signed-off-by Saranya Muruganandam <[email protected]>--- e2fsck/pass1.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-)e2fsck/pass1.c +4
-5@@ -2935,6 +2935,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctxext2_refcount_t ea_inode_refs = global_ctx->ea_inode_refs;ext2fs_block_bitmap block_found_map = global_ctx->block_found_map;ext2fs_block_bitmap block_dup_map = global_ctx->block_dup_map; +int options = global_ctx->options;#ifdef HAVE_SETJMP_H jmp_buf old_jmp;@@ -2987,7 +2988,8 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx global_ctx->fs_fragmented += fs_fragmented; global_ctx->fs_fragmented_dir += fs_fragmented_dir; global_ctx->large_files += large_files; - +/* threads might enable E2F_OPT_YES */ + global_ctx->options |= options; global_ctx->flags |= flags; retval =e2fsck_pass1_merge_fs (global_fs, thread_fs);@@ -3022,10 +3024,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx thread_ctx->qctx);if (retval)return retval; - global_ctx->invalid_block_bitmap_flag = invalid_block_bitmap_flag; - global_ctx->invalid_inode_bitmap_flag = invalid_inode_bitmap_flag; - global_ctx->invalid_inode_table_flag = invalid_inode_table_flag; - global_ctx->invalid_bitmaps = invalid_bitmaps; +e2fsck_pass1_merge_invalid_bitmaps (global_ctx, thread_ctx); retval = e2fsck_pass1_merge_bitmap(global_fs,Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 38/61 e2fsck: reset lost_and_found after threads finish
In reply to Saranya MuruganandamFrom: Wang Shilong <[email protected]>
This should not be kept, the reaons is similar to what e2fsck_pass1 has done before.
--- e2fsck/pass1.c | 5 +++++ 1 file changed, 5 insertions(+)
e2fsck/pass1.c +5 -0
@@ -2991,6 +2991,11 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx/* threads might enable E2F_OPT_YES */ global_ctx->options |= options; global_ctx->flags |= flags; +/* +* The l+f inode may have been cleared, so zap it now and +* later passes will recalculate it if necessary +*/ + global_ctx->lost_and_found =0 ; retval =e2fsck_pass1_merge_fs (global_fs, thread_fs);if (retval) {
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 39/61 e2fsck: merge extent depth count after threads finish
In reply to Saranya MuruganandamFrom: Wang Shilong <[email protected]>
tests covered by f_extent_htree.
--- e2fsck/pass1.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-)
e2fsck/pass1.c +10 -1
@@ -2935,8 +2935,11 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctxext2_refcount_t ea_inode_refs = global_ctx->ea_inode_refs;ext2fs_block_bitmap block_found_map = global_ctx->block_found_map;ext2fs_block_bitmap block_dup_map = global_ctx->block_dup_map; -int options = global_ctx->options; +int options = global_ctx->options, i; +__u32 extent_depth_count[MAX_EXTENT_DEPTH_COUNT ]; +memcpy (extent_depth_count, global_ctx->extent_depth_count, +sizeof (extent_depth_count));#ifdef HAVE_SETJMP_H jmp_buf old_jmp;
@@ -2996,6 +2999,12 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx* later passes will recalculate it if necessary */ global_ctx->lost_and_found =0 ; +memcpy (global_ctx->extent_depth_count, extent_depth_count, +sizeof (extent_depth_count)); +/* merge extent depth count */ +for (i =0 ; i <MAX_EXTENT_DEPTH_COUNT ; i++) + global_ctx->extent_depth_count[i] += + thread_ctx->extent_depth_count[i]; retval =e2fsck_pass1_merge_fs (global_fs, thread_fs);if (retval) {
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 40/61 e2fsck: simplify e2fsck context merging codes
In reply to Saranya MuruganandamFrom: Wang Shilong <[email protected]>
We tried to copy thread context to global context directly and then copy back some saved variables before merging.
Since we have finished almost all necessary variables in the e2fsck context, we could simplify codes, and this could help us understand what is missing rather than hide problems.
--- e2fsck/pass1.c | 151 ++++++++++--------------------------------------- 1 file changed, 31 insertions(+), 120 deletions(-)
e2fsck/pass1.c +31 -120
@@ -2432,6 +2432,7 @@ static int e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)ext2_badblocks_list badblocks;ext2_dblist dblist;int flags; +e2fsck_t dest_ctx = dest->priv_data; dest_io = dest->io; dest_image_io = dest->image_io;
@@ -2449,6 +2450,7 @@ static int e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src) dest->block_map = block_map; dest->badblocks = badblocks; dest->dblist = dblist; + dest->priv_data = dest_ctx;if (dest->dblist) dest->dblist->fs = dest; dest->flags = src->flags | flags;
@@ -2882,140 +2884,51 @@ static errcode_t e2fsck_pass1_merge_ea_refcount(e2fsck_t global_ctx,return retval; } -static int e2fsck_pass1_thread_join_one (e2fsck_t global_ctx,e2fsck_t thread_ctx) +static errcode_t e2fsck_pass1_merge_context (e2fsck_t global_ctx, +e2fsck_t thread_ctx) { -errcode_t retval; -int flags = global_ctx->flags; -ext2_filsys thread_fs = thread_ctx->fs; -ext2_filsys global_fs = global_ctx->fs; -FILE *global_logf = global_ctx->logf; -FILE *global_problem_logf = global_ctx->problem_logf; -ext2fs_inode_bitmap inode_bad_map = global_ctx->inode_bad_map; -struct dir_info_db *dir_info = global_ctx->dir_info; -struct dx_dir_info *dx_dir_info = global_ctx->dx_dir_info; -ext2fs_inode_bitmap inode_used_map = global_ctx->inode_used_map; -ext2fs_inode_bitmap inode_dir_map = global_ctx->inode_dir_map; -ext2fs_inode_bitmap inode_bb_map = global_ctx->inode_bb_map; -ext2fs_inode_bitmap inode_imagic_map = global_ctx->inode_imagic_map; -ext2fs_inode_bitmap inode_reg_map = global_ctx->inode_reg_map; -ext2fs_block_bitmap inodes_to_rebuild = global_ctx->inodes_to_rebuild; -ext2_icount_t inode_count = global_ctx->inode_count; -ext2_icount_t inode_link_info = global_ctx->inode_link_info; -__u32 fs_directory_count = global_ctx->fs_directory_count; -__u32 fs_regular_count = global_ctx->fs_regular_count; -__u32 fs_blockdev_count = global_ctx->fs_blockdev_count; -__u32 fs_chardev_count = global_ctx->fs_chardev_count; -__u32 fs_links_count = global_ctx->fs_links_count; -__u32 fs_symlinks_count = global_ctx->fs_symlinks_count; -__u32 fs_fast_symlinks_count = global_ctx->fs_fast_symlinks_count; -__u32 fs_fifo_count = global_ctx->fs_fifo_count; -__u32 fs_total_count = global_ctx->fs_total_count; -__u32 fs_badblocks_count = global_ctx->fs_badblocks_count; -__u32 fs_sockets_count = global_ctx->fs_sockets_count; -__u32 fs_ind_count = global_ctx->fs_ind_count; -__u32 fs_dind_count = global_ctx->fs_dind_count; -__u32 fs_tind_count = global_ctx->fs_tind_count; -__u32 fs_fragmented = global_ctx->fs_fragmented; -__u32 fs_fragmented_dir = global_ctx->fs_fragmented_dir; -__u32 large_files = global_ctx->large_files; -ext2_ino_t dx_dir_info_size = global_ctx->dx_dir_info_size; -ext2_ino_t dx_dir_info_count = global_ctx->dx_dir_info_count; -ext2_u32_list dirs_to_hash = global_ctx->dirs_to_hash; -quota_ctx_t qctx = global_ctx->qctx; -int *invalid_block_bitmap_flag = global_ctx->invalid_block_bitmap_flag; -int *invalid_inode_bitmap_flag = global_ctx->invalid_inode_bitmap_flag; -int *invalid_inode_table_flag = global_ctx->invalid_inode_table_flag; -int invalid_bitmaps = global_ctx->invalid_bitmaps; -ext2_refcount_t refcount = global_ctx->refcount; -ext2_refcount_t refcount_extra = global_ctx->refcount_extra; -ext2_refcount_t refcount_orig = global_ctx->refcount_orig; -ext2_refcount_t ea_block_quota_blocks = global_ctx->ea_block_quota_blocks; -ext2_refcount_t ea_block_quota_inodes = global_ctx->ea_block_quota_inodes; -ext2fs_block_bitmap block_ea_map = global_ctx->block_ea_map; -ext2_refcount_t ea_inode_refs = global_ctx->ea_inode_refs; -ext2fs_block_bitmap block_found_map = global_ctx->block_found_map; -ext2fs_block_bitmap block_dup_map = global_ctx->block_dup_map; -int options = global_ctx->options, i; -__u32 extent_depth_count[MAX_EXTENT_DEPTH_COUNT ]; - -memcpy (extent_depth_count, global_ctx->extent_depth_count, -sizeof (extent_depth_count)); -#ifdef HAVE_SETJMP_H -jmp_buf old_jmp; - -memcpy (old_jmp, global_ctx->abort_loc,sizeof (jmp_buf)); -#endif -memcpy (global_ctx, thread_ctx,sizeof (struct e2fsck_struct )); -#ifdef HAVE_SETJMP_H -memcpy (global_ctx->abort_loc, old_jmp,sizeof (jmp_buf)); -#endif +ext2_filsys global_fs = global_ctx->fs; +errcode_t retval; +int i; - global_ctx->inode_used_map = inode_used_map; - global_ctx->inode_bad_map = inode_bad_map; - global_ctx->inode_dir_map = inode_dir_map; - global_ctx->inode_bb_map = inode_bb_map; - global_ctx->inode_imagic_map = inode_imagic_map; - global_ctx->inodes_to_rebuild = inodes_to_rebuild; - global_ctx->inode_reg_map = inode_reg_map; - global_ctx->block_dup_map = block_dup_map; - global_ctx->block_found_map = block_found_map; - global_ctx->dir_info = dir_info; -e2fsck_pass1_merge_dir_info (global_ctx, thread_ctx); - global_ctx->dx_dir_info = dx_dir_info; - global_ctx->dx_dir_info_count = dx_dir_info_count; - global_ctx->dx_dir_info_size = dx_dir_info_size; -e2fsck_pass1_merge_dx_dir (global_ctx, thread_ctx); - global_ctx->inode_count = inode_count; - global_ctx->inode_link_info = inode_link_info; - global_ctx->refcount = refcount; - global_ctx->refcount_extra = refcount_extra; - global_ctx->refcount_orig = refcount_orig; - global_ctx->ea_block_quota_blocks = ea_block_quota_blocks; - global_ctx->ea_block_quota_inodes = ea_block_quota_inodes; - global_ctx->block_ea_map = block_ea_map; - global_ctx->ea_inode_refs = ea_inode_refs; - global_ctx->fs_directory_count += fs_directory_count; - global_ctx->fs_regular_count += fs_regular_count; - global_ctx->fs_blockdev_count += fs_blockdev_count; - global_ctx->fs_chardev_count += fs_chardev_count; - global_ctx->fs_links_count += fs_links_count; - global_ctx->fs_symlinks_count += fs_symlinks_count; - global_ctx->fs_fast_symlinks_count += fs_fast_symlinks_count; - global_ctx->fs_fifo_count += fs_fifo_count; - global_ctx->fs_total_count += fs_total_count; - global_ctx->fs_badblocks_count += fs_badblocks_count; - global_ctx->fs_sockets_count += fs_sockets_count; - global_ctx->fs_ind_count += fs_ind_count; - global_ctx->fs_dind_count += fs_dind_count; - global_ctx->fs_tind_count += fs_tind_count; - global_ctx->fs_fragmented += fs_fragmented; - global_ctx->fs_fragmented_dir += fs_fragmented_dir; - global_ctx->large_files += large_files; + global_ctx->fs_directory_count += thread_ctx->fs_directory_count; + global_ctx->fs_regular_count += thread_ctx->fs_regular_count; + global_ctx->fs_blockdev_count += thread_ctx->fs_blockdev_count; + global_ctx->fs_chardev_count += thread_ctx->fs_chardev_count; + global_ctx->fs_links_count += thread_ctx->fs_links_count; + global_ctx->fs_symlinks_count += thread_ctx->fs_symlinks_count; + global_ctx->fs_fast_symlinks_count += thread_ctx->fs_fast_symlinks_count; + global_ctx->fs_fifo_count += thread_ctx->fs_fifo_count; + global_ctx->fs_total_count += thread_ctx->fs_total_count; + global_ctx->fs_badblocks_count += thread_ctx->fs_badblocks_count; + global_ctx->fs_sockets_count += thread_ctx->fs_sockets_count; + global_ctx->fs_ind_count += thread_ctx->fs_ind_count; + global_ctx->fs_dind_count += thread_ctx->fs_dind_count; + global_ctx->fs_tind_count += thread_ctx->fs_tind_count; + global_ctx->fs_fragmented += thread_ctx->fs_fragmented; + global_ctx->fs_fragmented_dir += thread_ctx->fs_fragmented_dir; + global_ctx->large_files += thread_ctx->large_files;/* threads might enable E2F_OPT_YES */ - global_ctx->options |= options; - global_ctx->flags |= flags; + global_ctx->options |= thread_ctx->options; + global_ctx->flags |= thread_ctx->flags;/* * The l+f inode may have been cleared, so zap it now and * later passes will recalculate it if necessary */ global_ctx->lost_and_found =0 ; -memcpy (global_ctx->extent_depth_count, extent_depth_count, -sizeof (extent_depth_count));/* merge extent depth count */ for (i =0 ; i <MAX_EXTENT_DEPTH_COUNT ; i++) global_ctx->extent_depth_count[i] += thread_ctx->extent_depth_count[i]; - retval =e2fsck_pass1_merge_fs (global_fs, thread_fs); +e2fsck_pass1_merge_dir_info (global_ctx, thread_ctx); +e2fsck_pass1_merge_dx_dir (global_ctx, thread_ctx); + + retval =e2fsck_pass1_merge_fs (global_ctx->fs, thread_ctx->fs);if (retval) {com_err (global_ctx->program_name,0 ,_ ("while merging fs\n" ));return retval; } - global_fs->priv_data = global_ctx; - global_ctx->fs = global_fs; - global_ctx->logf = global_logf; - global_ctx->problem_logf = global_problem_logf; - global_ctx->global_ctx =NULL ; retval =e2fsck_pass1_merge_icounts (global_ctx, thread_ctx);if (retval) { com_err(global_ctx->program_name, 0,
@@ -3023,7 +2936,6 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctxreturn retval; } - global_ctx->dirs_to_hash = dirs_to_hash; retval =e2fsck_pass1_merge_dirs_to_hash (global_ctx, thread_ctx);if (retval) { com_err(global_ctx->program_name, 0,
@@ -3033,7 +2945,6 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctxe2fsck_pass1_merge_ea_inode_refs (global_ctx, thread_ctx);e2fsck_pass1_merge_ea_refcount (global_ctx, thread_ctx); - global_ctx->qctx = qctx; retval =quota_merge_and_update_usage (global_ctx->qctx, thread_ctx->qctx);if (retval)
@@ -3107,7 +3018,7 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx) {errcode_t retval; - retval =e2fsck_pass1_thread_join_one (global_ctx, thread_ctx); + retval =e2fsck_pass1_merge_context (global_ctx, thread_ctx);ext2fs_free_mem (&thread_ctx->fs);if (thread_ctx->logf)fclose (thread_ctx->logf);
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 41/61 e2fsck: set E2F_FLAG_ALLOC_OK after threads
In reply to Saranya MuruganandamFrom: Wang Shilong <[email protected]>
Only flag ALLOC OK after all threads finished without problem.
--- e2fsck/pass1.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)
e2fsck/pass1.c +8 -2
@@ -1379,9 +1379,12 @@ static void e2fsck_pass1_post(e2fsck_t ctx) {struct problem_context pctx;ext2_filsys fs = ctx->fs; +char *block_buf; -char *block_buf = - (char *)e2fsck_allocate_memory (ctx, ctx->fs->blocksize *3 , +if (e2fsck_should_abort (ctx)) +return ; + + block_buf = (char *)e2fsck_allocate_memory (ctx, ctx->fs->blocksize *3 ,"block interate buffer" );reserve_block_for_root_repair (ctx);reserve_block_for_lnf_repair (ctx);
@@ -1459,6 +1462,8 @@ static void e2fsck_pass1_post(e2fsck_t ctx)ext2fs_free_mem (&block_buf); ctx->flags &= ~E2F_FLAG_DUP_BLOCK ; } + + ctx->flags |=E2F_FLAG_ALLOC_OK ; }
@@ -2290,6 +2295,7 @@ void e2fsck_pass1_run(e2fsck_t ctx) } ctx->flags |=E2F_FLAG_ALLOC_OK ; +ext2fs_free_mem (&inodes_to_process); endit:e2fsck_use_inode_shortcuts (ctx,0 );ext2fs_free_mem (&inodes_to_process);
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 42/61 e2fsck: wait fix thread finish before checking
In reply to Saranya MuruganandamFrom: Wang Shilong <[email protected]>
Before proceeding next inodes, waitting existed
fixing finished.
--- e2fsck/e2fsck.c | 3 +++ e2fsck/e2fsck.h | 5 +++- e2fsck/pass1.c | 65 +++++++++++++++++++++++++++++++++++++++++++------ e2fsck/util.c | 56 +++++++++++++++++++++++++++++++++++++++--- 4 files changed, 117 insertions(+), 12 deletions(-)
e2fsck/e2fsck.c +3 -0
@@ -178,6 +178,9 @@ errcode_t e2fsck_reset_context(e2fsck_t ctx) ctx->fs_fragmented =0 ; ctx->fs_fragmented_dir =0 ; ctx->large_files =0 ; +#ifdef CONFIG_PFSCK + ctx->fs_need_locking =0 ; +#endif for (i=0 ; i <MAX_EXTENT_DEPTH_COUNT ; i++) ctx->extent_depth_count[i] =0 ;
e2fsck/e2fsck.h +4 -1
@@ -451,8 +451,9 @@ struct e2fsck_struct {char *undo_file;#ifdef CONFIG_PFSCK __u32 fs_num_threads; +int fs_need_locking;/* serialize fix operation for multiple threads */ -pthread_mutex_t fs_fix_mutex; +pthread_rwlock_t fs_fix_rwlock;/* protect block_found_map, block_dup_map */ pthread_rwlock_t fs_block_map_rwlock;#endif
@@ -514,6 +515,8 @@ extern int e2fsck_strnlen(const char * s, int count);extern void e2fsck_pass1 (e2fsck_t ctx);extern void e2fsck_pass1_dupblocks (e2fsck_t ctx,char *block_buf); +extern void e2fsck_pass1_check_lock (e2fsck_t ctx); +extern void e2fsck_pass1_check_unlock (e2fsck_t ctx);extern void e2fsck_pass2 (e2fsck_t ctx);extern void e2fsck_pass3 (e2fsck_t ctx);extern void e2fsck_pass4 (e2fsck_t ctx);
e2fsck/pass1.c +57 -8
235 lines not on this page · raw message
e2fsck/util.c +53 -3
104 lines not on this page · raw message
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 43/61 e2fsck: cleanup e2fsck_pass1_thread_join()
In reply to Saranya MuruganandamFrom: Wang Shilong <[email protected]>
Use e2fsck_reset_context() to free memory to simpify codes.
--- e2fsck/pass1.c | 34 ++++++++-------------------------- 1 file changed, 8 insertions(+), 26 deletions(-)
e2fsck/pass1.c +8 -26
@@ -3079,32 +3079,14 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)fputs ("</problem_log>\n" , thread_ctx->problem_logf);fclose (thread_ctx->problem_logf); } -e2fsck_pass1_free_bitmap (&thread_ctx->inode_used_map); -e2fsck_pass1_free_bitmap (&thread_ctx->inode_bad_map); -e2fsck_pass1_free_bitmap (&thread_ctx->inode_dir_map); -e2fsck_pass1_free_bitmap (&thread_ctx->inode_bb_map); -e2fsck_pass1_free_bitmap (&thread_ctx->inode_imagic_map); -e2fsck_pass1_free_bitmap (&thread_ctx->inode_reg_map); -e2fsck_pass1_free_bitmap (&thread_ctx->inodes_to_rebuild); -e2fsck_pass1_free_bitmap (&thread_ctx->block_found_map); -e2fsck_pass1_free_bitmap (&thread_ctx->block_ea_map); -if (thread_ctx->refcount) -ea_refcount_free (thread_ctx->refcount); -if (thread_ctx->refcount_extra) -ea_refcount_free (thread_ctx->refcount_extra); -if (thread_ctx->ea_inode_refs) -ea_refcount_free (thread_ctx->ea_inode_refs); -if (thread_ctx->refcount_orig) -ea_refcount_free (thread_ctx->refcount_orig); -e2fsck_free_dir_info (thread_ctx); -ext2fs_free_icount (thread_ctx->inode_count); -ext2fs_free_icount (thread_ctx->inode_link_info); -if (thread_ctx->dirs_to_hash) -ext2fs_badblocks_list_free (thread_ctx->dirs_to_hash); -quota_release_context (&thread_ctx->qctx); -ext2fs_free_mem (&thread_ctx->invalid_block_bitmap_flag); -ext2fs_free_mem (&thread_ctx->invalid_inode_bitmap_flag); -ext2fs_free_mem (&thread_ctx->invalid_inode_table_flag); + +/* +* @block_metadata_map and @block_dup_map are +* shared, so we don't free them. +*/ + thread_ctx->block_metadata_map =NULL ; + thread_ctx->block_dup_map =NULL ; +e2fsck_reset_context (thread_ctx);ext2fs_free_mem (&thread_ctx);return retval;
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 44/61 e2fsck: avoid too much memory allocation for pfsck
In reply to Saranya MuruganandamFrom: Wang Shilong <[email protected]>
e2fsck init memory according to filesystem inodes/dir numbers recorded in the superblock, this should be aware of filesystem number of threads, otherwise, oom happen.
--- lib/ext2fs/dblist.c | 2 ++ lib/ext2fs/icount.c | 4 ++++ 2 files changed, 6 insertions(+)
lib/ext2fs/dblist.c +2 -0
@@ -58,6 +58,8 @@ static errcode_t make_dblist(ext2_filsys fs, ext2_ino_t size,if (retval) goto cleanup; dblist->size = (num_dirs *2 ) +12 ; +if (fs->fs_num_threads) + dblist->size /= fs->fs_num_threads; } len = (size_t )sizeof (struct ext2_db_entry2 ) * dblist->size; dblist->count = count;
lib/ext2fs/icount.c +4 -0
@@ -237,6 +237,8 @@ errcode_t ext2fs_create_icount_tdb(ext2_filsys fs EXT2FS_NO_TDB_UNUSED,* value. */ num_inodes = fs->super->s_inodes_count - fs->super->s_free_inodes_count; +if (fs->fs_num_threads) + num_inodes /= fs->fs_num_threads; icount->tdb =tdb_open (fn, num_inodes,TDB_NOLOCK |TDB_NOSYNC ,O_RDWR |O_CREAT |O_TRUNC ,0600 );
@@ -288,6 +290,8 @@ errcode_t ext2fs_create_icount2(ext2_filsys fs, int flags, unsigned int size,if (retval) goto errout; icount->size += fs->super->s_inodes_count /50 ; +if (fs->fs_num_threads) + icount->size /= fs->fs_num_threads; } bytes = (size_t ) (icount->size *sizeof (struct ext2_icount_el ));
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 45/61 e2fsck: make default smallest RA size to 1M
In reply to Saranya MuruganandamFrom: Wang Shilong <[email protected]>
If we have a smaller inodes per group, default ra size could be very small(etc 128KiB), this hurts performances.
Tune above 128K to 1M, i see pass1 time drop down from 677.12 seconds to 246 secons with 32 threads.
--- e2fsck/readahead.c | 4 ++++ 1 file changed, 4 insertions(+)
e2fsck/readahead.c +4 -0
@@ -234,6 +234,8 @@ int e2fsck_can_readahead(ext2_filsys fs)return err !=EXT2_ET_OP_NOT_SUPPORTED ; } +#define MIN_DEFAULT_RA (1024 * 1024) +unsigned long long e2fsck_guess_readahead (ext2_filsys fs) {unsigned long long guess;
@@ -245,6 +247,8 @@ unsigned long long e2fsck_guess_readahead(ext2_filsys fs)* in e2fsck runtime. */ guess =2ULL * fs->blocksize * fs->inode_blocks_per_group; +if (guess <MIN_DEFAULT_RA ) + guess =MIN_DEFAULT_RA ;/* Disable RA if it'd use more 1/50th of RAM. */ if (get_memory_size () > (guess *50 ))
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 46/61 ext2fs: parallel bitmap loading
In reply to Saranya MuruganandamFrom: Wang Shilong <[email protected]>
In our benchmarking for PiB size filesystem, pass5 takes 10446s to finish and 99.5% of time takes on reading bitmaps.
It makes sense to reading bitmaps using multiple threads, a quickly benchmark show 10446s to 626s with 64 threads.
--- lib/ext2fs/rw_bitmaps.c | 289 ++++++++++++++++++++++++++++++++++------ 1 file changed, 250 insertions(+), 39 deletions(-)
lib/ext2fs/rw_bitmaps.c +250 -39
@@ -23,6 +23,7 @@#ifdef HAVE_SYS_TYPES_H #include <sys/types.h> #endif +#include <pthread.h> #include "ext2_fs.h" #include "ext2fs.h"
@@ -205,22 +206,12 @@ static int bitmap_tail_verify(unsigned char *bitmap, int first, int last)return 1 ; } -static errcode_t read_bitmaps (ext2_filsys fs,int do_inode,int do_block) +static errcode_t read_bitmaps_range_prepare (ext2_filsys fs,int do_inode,int do_block) { -dgrp_t i; -char *block_bitmap =0 , *inode_bitmap =0 ; -char *buf;errcode_t retval;int block_nbytes =EXT2_CLUSTERS_PER_GROUP (fs->super) /8 ;int inode_nbytes =EXT2_INODES_PER_GROUP (fs->super) /8 ; -int tail_flags =0 ; -int csum_flag; -unsigned int cnt; -blk64_t blk; -blk64_t blk_itr =EXT2FS_B2C (fs, fs->super->s_first_data_block); -blk64_t blk_cnt; -ext2_ino_t ino_itr =1 ; -ext2_ino_t ino_cnt; +char *buf;EXT2_CHECK_MAGIC (fs,EXT2_ET_MAGIC_EXT2FS_FILSYS );
@@ -230,11 +221,10 @@ static errcode_t read_bitmaps(ext2_filsys fs, int do_inode, int do_block) fs->write_bitmaps = ext2fs_write_bitmaps; - csum_flag =ext2fs_has_group_desc_csum (fs); - retval =ext2fs_get_mem (strlen (fs->device_name) +80 , &buf);if (retval)return retval; +if (do_block) {if (fs->block_map)ext2fs_free_block_bitmap (fs->block_map);
@@ -243,11 +233,8 @@ static errcode_t read_bitmaps(ext2_filsys fs, int do_inode, int do_block) retval =ext2fs_allocate_block_bitmap (fs, buf, &fs->block_map);if (retval) goto cleanup; - retval =io_channel_alloc_buf (fs->io,0 , &block_bitmap); -if (retval) - goto cleanup; - }else - block_nbytes =0 ; + } +if (do_inode) {if (fs->inode_map)ext2fs_free_inode_bitmap (fs->inode_map);
@@ -256,13 +243,69 @@ static errcode_t read_bitmaps(ext2_filsys fs, int do_inode, int do_block) retval =ext2fs_allocate_inode_bitmap (fs, buf, &fs->inode_map);if (retval) goto cleanup; - retval =io_channel_alloc_buf (fs->io,0 , &inode_bitmap); + } +ext2fs_free_mem (&buf); + +return retval; + +cleanup: +if (do_block) { +ext2fs_free_block_bitmap (fs->block_map); + fs->block_map =0 ; + } +if (do_inode) { +ext2fs_free_inode_bitmap (fs->inode_map); + fs->inode_map =0 ; + } +if (buf) +ext2fs_free_mem (&buf); +return retval; +} + +static errcode_t read_bitmaps_range_start (ext2_filsys fs,int do_inode,int do_block, +dgrp_t start,dgrp_t end,pthread_mutex_t *mutex, +io_channel io) +{ +dgrp_t i; +char *block_bitmap =0 , *inode_bitmap =0 ; +char *buf; +errcode_t retval; +int block_nbytes =EXT2_CLUSTERS_PER_GROUP (fs->super) /8 ; +int inode_nbytes =EXT2_INODES_PER_GROUP (fs->super) /8 ; +int tail_flags =0 ; +int csum_flag; +unsigned int cnt; +blk64_t blk; +blk64_t blk_itr =EXT2FS_B2C (fs, fs->super->s_first_data_block); +blk64_t blk_cnt; +ext2_ino_t ino_itr =1 ; +ext2_ino_t ino_cnt; +io_channel this_io; + +if (!io) + this_io = fs->io; +else + this_io = io; + + csum_flag =ext2fs_has_group_desc_csum (fs); + +if (do_block) { + retval =io_channel_alloc_buf (this_io,0 , &block_bitmap); +if (retval) + goto cleanup; + }else { + block_nbytes =0 ; + } + +if (do_inode) { + retval =io_channel_alloc_buf (this_io,0 , &inode_bitmap);if (retval) goto cleanup; - }else + }else { inode_nbytes =0 ; -ext2fs_free_mem (&buf); + } +/* io should be null */ if (fs->flags &EXT2_FLAG_IMAGE_FILE ) { blk = (ext2fs_le32_to_cpu (fs->image_header->offset_inodemap) / fs->blocksize); ino_cnt = fs->super->s_inodes_count;
@@ -303,7 +346,9 @@ static errcode_t read_bitmaps(ext2_filsys fs, int do_inode, int do_block) goto success_cleanup; } -for (i =0 ; i < fs->group_desc_count; i++) { + blk_itr += ((blk64_t )start * (block_nbytes <<3 )); + ino_itr += ((blk64_t )start * (inode_nbytes <<3 )); +for (i = start; i <= end; i++) {if (block_bitmap) { blk =ext2fs_block_bitmap_loc (fs, i); if ((csum_flag &&
@@ -312,7 +357,7 @@ static errcode_t read_bitmaps(ext2_filsys fs, int do_inode, int do_block) (blk >= ext2fs_blocks_count(fs->super))) blk =0 ;if (blk) { - retval =io_channel_read_blk64 (fs->io, blk, + retval =io_channel_read_blk64 (this_io, blk,1 , block_bitmap);if (retval) { retval =EXT2_ET_BLOCK_BITMAP_READ ;
@@ -333,8 +378,12 @@ static errcode_t read_bitmaps(ext2_filsys fs, int do_inode, int do_block) }else memset (block_bitmap,0 , block_nbytes); cnt = block_nbytes <<3 ; +if (mutex) +pthread_mutex_lock (mutex); retval =ext2fs_set_block_bitmap_range2 (fs->block_map, blk_itr, cnt, block_bitmap); +if (mutex) +pthread_mutex_unlock (mutex);if (retval) goto cleanup; blk_itr += block_nbytes <<3 ;
@@ -347,7 +396,7 @@ static errcode_t read_bitmaps(ext2_filsys fs, int do_inode, int do_block) (blk >= ext2fs_blocks_count(fs->super))) blk =0 ;if (blk) { - retval =io_channel_read_blk64 (fs->io, blk, + retval =io_channel_read_blk64 (this_io, blk,1 , inode_bitmap);if (retval) { retval =EXT2_ET_INODE_BITMAP_READ ;
@@ -369,29 +418,28 @@ static errcode_t read_bitmaps(ext2_filsys fs, int do_inode, int do_block) }else memset (inode_bitmap,0 , inode_nbytes); cnt = inode_nbytes <<3 ; +if (mutex) +pthread_mutex_lock (mutex); retval =ext2fs_set_inode_bitmap_range2 (fs->inode_map, ino_itr, cnt, inode_bitmap); +if (mutex) +pthread_mutex_unlock (mutex);if (retval) goto cleanup; ino_itr += inode_nbytes <<3 ; } } -/* Mark group blocks for any BLOCK_UNINIT groups */ -if (do_block) { - retval =mark_uninit_bg_group_blocks (fs); -if (retval) - goto cleanup; - } - success_cleanup: -if (inode_bitmap) { -ext2fs_free_mem (&inode_bitmap); - fs->flags &= ~EXT2_FLAG_IBITMAP_TAIL_PROBLEM ; - } -if (block_bitmap) { -ext2fs_free_mem (&block_bitmap); - fs->flags &= ~EXT2_FLAG_BBITMAP_TAIL_PROBLEM ; +if (start ==0 && end == fs->group_desc_count -1 ) { +if (inode_bitmap) { +ext2fs_free_mem (&inode_bitmap); + fs->flags &= ~EXT2_FLAG_IBITMAP_TAIL_PROBLEM ; + } +if (block_bitmap) { +ext2fs_free_mem (&block_bitmap); + fs->flags &= ~EXT2_FLAG_BBITMAP_TAIL_PROBLEM ; + } } fs->flags |= tail_flags;return 0 ;
@@ -412,6 +460,169 @@ cleanup:if (buf)ext2fs_free_mem (&buf);return retval; + +} + +static errcode_t read_bitmaps_range_end (ext2_filsys fs,int do_inode,int do_block) +{ +errcode_t retval =0 ; + +/* Mark group blocks for any BLOCK_UNINIT groups */ +if (do_block) { + retval =mark_uninit_bg_group_blocks (fs); +if (retval) + goto cleanup; + } + +return retval; +cleanup: +if (do_block) { +ext2fs_free_block_bitmap (fs->block_map); + fs->block_map =0 ; + } +if (do_inode) { +ext2fs_free_inode_bitmap (fs->inode_map); + fs->inode_map =0 ; + } +return retval; +} + +static errcode_t read_bitmaps_range (ext2_filsys fs,int do_inode,int do_block, +dgrp_t start,dgrp_t end) +{ +errcode_t retval; + + retval =read_bitmaps_range_prepare (fs, do_inode, do_block); +if (retval) +return retval; + + retval =read_bitmaps_range_start (fs, do_inode, do_block, start, end,NULL ,NULL ); +if (retval) +return retval; + +return read_bitmaps_range_end (fs, do_inode, do_block); +} + +#ifdef CONFIG_PFSCK +struct read_bitmaps_thread_info { +ext2_filsys rbt_fs; +int rbt_do_inode; +int rbt_do_block; +dgrp_t rbt_grp_start; +dgrp_t rbt_grp_end; +errcode_t rbt_retval; +pthread_mutex_t *rbt_mutex; +io_channel rbt_io; +}; + +static void *read_bitmaps_thread (void *data) +{ +struct read_bitmaps_thread_info *rbt = data; + + rbt->rbt_retval =read_bitmaps_range_start (rbt->rbt_fs, + rbt->rbt_do_inode, rbt->rbt_do_block, + rbt->rbt_grp_start, rbt->rbt_grp_end, + rbt->rbt_mutex, rbt->rbt_io); +return NULL ; +} +#endif + +static errcode_t read_bitmaps (ext2_filsys fs,int do_inode,int do_block) +{ +#ifdef CONFIG_PFSCK +pthread_attr_t attr; +int num_threads = fs->fs_num_threads; +pthread_t *thread_ids =NULL ; +struct read_bitmaps_thread_info *thread_infos =NULL ; +pthread_mutex_t rbt_mutex =PTHREAD_MUTEX_INITIALIZER ; +errcode_t retval; +errcode_t rc; +dgrp_t average_group; +int i; +io_manager manager = unix_io_manager; +#else +int num_threads =1 ; +#endif + +if (num_threads <=1 || (fs->flags &EXT2_FLAG_IMAGE_FILE )) +return read_bitmaps_range (fs, do_inode, do_block,0 , fs->group_desc_count -1 ); + +#ifdef CONFIG_PFSCK + retval =pthread_attr_init (&attr); +if (retval) +return retval; + + thread_ids =calloc (sizeof (pthread_t), num_threads); +if (!thread_ids) +return -ENOMEM ; + + thread_infos =calloc (sizeof (struct read_bitmaps_thread_info ), + num_threads); +if (!thread_infos) + goto out; + + average_group =ext2fs_get_avg_group (fs); + retval =read_bitmaps_range_prepare (fs, do_inode, do_block); +if (retval) + goto out; + +fprintf (stdout,"Multiple threads triggered to read bitmaps\n" ); +for (i =0 ; i < num_threads; i++) { + thread_infos[i].rbt_fs = fs; + thread_infos[i].rbt_do_inode = do_inode; + thread_infos[i].rbt_do_block = do_block; + thread_infos[i].rbt_mutex = &rbt_mutex; +if (i ==0 ) + thread_infos[i].rbt_grp_start =0 ; +else + thread_infos[i].rbt_grp_start = average_group * i +1 ; + +if (i == num_threads -1 ) + thread_infos[i].rbt_grp_end = fs->group_desc_count -1 ; +else + thread_infos[i].rbt_grp_end = average_group * (i +1 ); + retval = manager->open (fs->device_name,IO_FLAG_RW , + &thread_infos[i].rbt_io); +if (retval) +break ; +io_channel_set_blksize (thread_infos[i].rbt_io, fs->io->block_size); + retval =pthread_create (&thread_ids[i], &attr, + &read_bitmaps_thread, &thread_infos[i]); +if (retval) { +io_channel_close (thread_infos[i].rbt_io); +break ; + } + } +for (i =0 ; i < num_threads; i++) { +if (!thread_ids[i]) +break ; + rc =pthread_join (thread_ids[i],NULL ); +if (rc && !retval) + retval = rc; + rc = thread_infos[i].rbt_retval; +if (rc && !retval) + retval = rc; +io_channel_close (thread_infos[i].rbt_io); + } +out: + rc =pthread_attr_destroy (&attr); +if (rc && !retval) + retval = rc; +free (thread_infos); +free (thread_ids); + +if (!retval) + retval =read_bitmaps_range_end (fs, do_inode, do_block); + +if (!retval) { +if (do_inode) + fs->flags &= ~EXT2_FLAG_IBITMAP_TAIL_PROBLEM ; +if (do_block) + fs->flags &= ~EXT2_FLAG_BBITMAP_TAIL_PROBLEM ; + } + +return retval; +#endif }errcode_t ext2fs_read_inode_bitmap (ext2_filsys fs)
Signature · 1 line
2.29.2.299.gdc1121823c-goog
Re: [RFC PATCH v3 46/61] ext2fs: parallel bitmap loading
Quoted 10 lines · Saranya Muruganandam
From: Wang Shilong <[email protected]>
In our benchmarking for PiB size filesystem, pass5 takes 10446s to finish and 99.5% of time takes on reading bitmaps.
It makes sense to reading bitmaps using multiple threads, a quickly benchmark show 10446s to 626s with 64 threads.
Signed-off-by Wang Shilong <[email protected]>Signed-off-by Saranya Muruganandam <[email protected]>
Note: This patch will *explode* with much hilarity if num_threads is greater than the number of block groups. That's because the ext2fs_get_avg_group() will return 1 if fs_num_threads is greater than group_desc_count.
This will result in the group start and end limits to go beyond the array boundaries.... and then *boom*. So there will probably need to be some kind of safety checks if the caller has set fs_num_threads to a value which is much larger than is appropriate for a given file system.
Speaking of which, relying on fs_num_threads being set by e2fsck means that we won't get the benefits of the parallel block bitmap reads for debugfs and dumpe2fs. So we should think about how the other tools should trigger read bitmaps. And this might be something that we want to do independent of whether we are doing parallel fsck.
Suggested approach:
1) Create create ext2fs_is_device_rotational() which returns whether or not a particular device is a HDD, or a non-rotational device (e.g., SSD, GCE PD, AWS EBS, etc.), using rotational as a proxy for "reading using multiple threads is a good thing".
2) Create an ext2fs_get_num_procs() which calls sysconf(_SC_NPROCESSOR_ONLN) if sysconf and _SC_NPROESSOR_ONLN is available. If not, there may be other OS-specific ways of determining the number of CPU's available.
3) If HAVE_PTHREADS and the number of block groups is greater than the number of CPU's * 2, a function that (for now) we drop in libsupport will set fs->fs_num_threads to the number of processors as the default. There may not be a reason to change the default for debugfs and dumpe2fs, but for e2fsck, this would be used for the default, but it could be over-ridden via "-E multithread=<number of threads>".
- Ted
RFC PATCH v3 47/61 e2fsck: update mmp block in one thread
In reply to Saranya MuruganandamFrom: Wang Shilong <[email protected]>
For multiple threads, different threads will try to update mmp block at the same time, only allow one thread to update it.
--- e2fsck/e2fsck.h | 1 + e2fsck/pass1.c | 36 ++++++++++++++++++++++++++++++++++-- 2 files changed, 35 insertions(+), 2 deletions(-)
e2fsck/e2fsck.h +1 -0
@@ -451,6 +451,7 @@ struct e2fsck_struct {char *undo_file;#ifdef CONFIG_PFSCK __u32 fs_num_threads; +__u32 mmp_update_thread;int fs_need_locking;/* serialize fix operation for multiple threads */ pthread_rwlock_t fs_fix_rwlock;
e2fsck/pass1.c +34 -2
@@ -1497,7 +1497,7 @@ void e2fsck_pass1_run(e2fsck_t ctx)dgrp_t ra_group =0 ;struct ea_quota ea_ibody_quota;struct process_inode_block *inodes_to_process; -int process_inode_count; +int process_inode_count, check_mmp;init_resource_track (&rtrack, ctx->fs->io);clear_problem_context (&pctx);
@@ -1646,8 +1646,33 @@ void e2fsck_pass1_run(e2fsck_t ctx)#endif while (1 ) { + check_mmp =0 ;e2fsck_pass1_check_lock (ctx); -if (ino % (fs->super->s_inodes_per_group *4 ) ==1 ) { +#ifdef CONFIG_PFSCK +if (!ctx->mmp_update_thread) { +e2fsck_pass1_block_map_w_lock (ctx); +if (!ctx->mmp_update_thread) { +if (ctx->global_ctx) + ctx->mmp_update_thread = + ctx->thread_info.et_thread_index +1 ; +else + ctx->mmp_update_thread =1 ; + check_mmp =1 ; + } +e2fsck_pass1_block_map_w_unlock (ctx); + } + +/* only one active thread could update mmp block. */ +e2fsck_pass1_block_map_r_lock (ctx); +if (!ctx->global_ctx || ctx->mmp_update_thread == + (ctx->thread_info.et_thread_index +1 )) + check_mmp =1 ; +e2fsck_pass1_block_map_r_unlock (ctx); +#else + check_mmp =1 ; +#endif + +if (check_mmp && (ino % (fs->super->s_inodes_per_group *4 ) ==1 )) {if (e2fsck_mmp_update (fs))fatal_error (ctx,0 ); }
@@ -2365,6 +2390,13 @@ endit:print_resource_track (ctx,_ ("Pass 1" ), &rtrack, ctx->fs->io); else ctx->invalid_bitmaps++; +#ifdef CONFIG_PFSCK +/* reset update_thread after this thread exit */ +e2fsck_pass1_block_map_w_lock (ctx); +if (ctx->mmp_update_thread) + ctx->mmp_update_thread =0 ; +e2fsck_pass1_block_map_w_unlock (ctx); +#endif }#ifdef CONFIG_PFSCK
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 48/61 e2fsck: reset @inodes_to_rebuild if restart
In reply to Saranya MuruganandamFrom: Wang Shilong <[email protected]>
Verify multiple thread on a corrupted images hit following bug:
pass1.c:2902: e2fsck_pass1_thread_prepare:
Assertion `global_ctx->inodes_to_rebuild == NULL' failed. Signal (6) SIGABRT si_code=SI_TKILL ./e2fsck/e2fsck[0x43829e] /lib64/libpthread.so.0(+0x14b20)[0x7f3b45135b20] /lib64/libc.so.6(gsignal+0x145)[0x7f3b44f2c625] /lib64/libc.so.6(abort+0x12b)[0x7f3b44f158d9] /lib64/libc.so.6(+0x257a9)[0x7f3b44f157a9] /lib64/libc.so.6(+0x34a66)[0x7f3b44f24a66] ./e2fsck/e2fsck(e2fsck_pass1+0x1662)[0x423572] ./e2fsck/e2fsck(e2fsck_run+0x5a)[0x41611a] ./e2fsck/e2fsck(main+0x1608)[0x4121b8] /lib64/libc.so.6(__libc_start_main+0xf3)[0x7f3b44f171a3] ./e2fsck/e2fsck(_start+0x2e)[0x413dde]
@inodes_to_rebuild could be not NULL after we restart pass1
--- e2fsck/e2fsck.c | 4 ++++ 1 file changed, 4 insertions(+)
e2fsck/e2fsck.c +4 -0
@@ -75,6 +75,10 @@ errcode_t e2fsck_reset_context(e2fsck_t ctx)ext2fs_free_block_bitmap (ctx->block_found_map); ctx->block_found_map =0 ; } +if (ctx->inodes_to_rebuild) { +ext2fs_free_inode_bitmap (ctx->inodes_to_rebuild); + ctx->inodes_to_rebuild =0 ; + }if (ctx->inode_link_info) {ext2fs_free_icount (ctx->inode_link_info); ctx->inode_link_info =0 ;
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 49/61 e2fsck: fix build for make rpm
In reply to Saranya MuruganandamFrom: Wang Shilong <[email protected]>
link e2fsck with -lpthread properly to make rpm pass.
--- MCONFIG.in | 1 + e2fsck/Makefile.in | 9 +++++---- 2 files changed, 6 insertions(+), 4 deletions(-)
MCONFIG.in +1 -0
@@ -146,6 +146,7 @@ DEPLIBUUID = @DEPLIBUUID@ DEPLIBSUPPORT = $(LIB)/libsupport@STATIC_LIB_EXT@ DEPLIBBLKID = @DEPLIBBLKID@ @PRIVATE_LIBS_CMT@ $(DEPLIBUUID) TESTENV = LD_LIBRARY_PATH="$(LIB):$${LD_LIBRARY_PATH}" DYLD_LIBRARY_PATH="$(LIB):$${DYLD_LIBRARY_PATH}" +SEM_INIT_LIB = @SEM_INIT_LIB@ STATIC_LIBSS = $(LIB)/libss@STATIC_LIB_EXT@ @DLOPEN_LIB@ STATIC_LIBCOM_ERR = $(LIB)/libcom_err@STATIC_LIB_EXT@ @SEM_INIT_LIB@
e2fsck/Makefile.in +5 -4
@@ -16,20 +16,21 @@ MANPAGES= e2fsck.8 FMANPAGES= e2fsck.conf.5 LIBS= $(LIBSUPPORT) $(LIBEXT2FS) $(LIBCOM_ERR) $(LIBBLKID) $(LIBUUID) \ - $(LIBINTL) $(LIBE2P) $(LIBMAGIC) $(SYSLIBS) + $(LIBINTL) $(LIBE2P) $(LIBMAGIC) $(SYSLIBS) $(SEM_INIT_LIB) DEPLIBS= $(DEPLIBSUPPORT) $(LIBEXT2FS) $(DEPLIBCOM_ERR) $(DEPLIBBLKID) \ $(DEPLIBUUID) $(DEPLIBE2P) STATIC_LIBS= $(STATIC_LIBSUPPORT) $(STATIC_LIBEXT2FS) $(STATIC_LIBCOM_ERR) \ $(STATIC_LIBBLKID) $(STATIC_LIBUUID) $(LIBINTL) $(STATIC_LIBE2P) \ - $(LIBMAGIC) $(SYSLIBS) + $(LIBMAGIC) $(SYSLIBS) $(SEM_INIT_LIB) STATIC_DEPLIBS= $(DEPSTATIC_LIBSUPPORT) $(STATIC_LIBEXT2FS) \ $(DEPSTATIC_LIBCOM_ERR) $(DEPSTATIC_LIBBLKID) \ $(DEPSTATIC_LIBUUID) $(DEPSTATIC_LIBE2P) PROFILED_LIBS= $(PROFILED_LIBSUPPORT) $(PROFILED_LIBEXT2FS) \ $(PROFILED_LIBCOM_ERR) $(PROFILED_LIBBLKID) $(PROFILED_LIBUUID) \ - $(PROFILED_LIBE2P) $(LIBINTL) $(LIBMAGIC) $(SYSLIBS) + $(PROFILED_LIBE2P) $(LIBINTL) $(LIBMAGIC) $(SYSLIBS) \ + $(SEM_INIT_LIB) PROFILED_DEPLIBS= $(DEPPROFILED_LIBSUPPORT) $(PROFILED_LIBEXT2FS) \ $(DEPPROFILED_LIBCOM_ERR) $(DEPPROFILED_LIBBLKID) \ $(DEPPROFILED_LIBUUID) $(DEPPROFILED_LIBE2P)
@@ -115,7 +116,7 @@ all-static:: e2fsck.static e2fsck: $(OBJS) $(DEPLIBS) $(E) " LD $@" - $(Q) $(LD) $(ALL_LDFLAGS) $(RDYNAMIC) -o e2fsck $(OBJS) $(LIBS) + $(Q) $(LD) $(ALL_LDFLAGS) $(RDYNAMIC) -o e2fsck $(OBJS) $(LIBS) e2fsck.static: $(OBJS) $(STATIC_DEPLIBS) $(E) " LD $@"
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 50/61 e2fsck: move ext2fs_get_avg_group to rw_bitmaps.c
In reply to Saranya MuruganandamFrom: Wang Shilong <[email protected]>
A bounch of ext2fs_get_avg_group() unused warning messages are annoying, move it to rw_bitmaps.c to make gcc happy.
Reviewed-on: https://review.whamcloud.com/40060
--- lib/ext2fs/ext2fs.h | 29 +---------------------------- lib/ext2fs/rw_bitmaps.c | 25 +++++++++++++++++++++++++ 2 files changed, 26 insertions(+), 28 deletions(-)
lib/ext2fs/ext2fs.h +1 -28
@@ -2122,34 +2122,7 @@ ext2fs_const_inode(const struct ext2_inode_large * large_inode) return (const struct ext2_inode *) large_inode; } -static dgrp_t ext2fs_get_avg_group (ext2_filsys fs) -{ -#ifdef CONFIG_PFSCK -dgrp_t average_group; -unsigned flexbg_size; - -if (fs->fs_num_threads <=1 ) -return fs->group_desc_count; - - average_group = fs->group_desc_count / fs->fs_num_threads; -if (average_group <=1 ) -return 1 ; - -if (ext2fs_has_feature_flex_bg (fs->super)) { -int times =1 ; - - flexbg_size =1 << fs->super->s_log_groups_per_flex; -if (average_group % flexbg_size) { - times = average_group / flexbg_size; - average_group = times * flexbg_size; - } - } - -return average_group; -#else -return fs->group_desc_count; -#endif -} +dgrp_t ext2fs_get_avg_group (ext2_filsys fs);#undef _INLINE_#endif
lib/ext2fs/rw_bitmaps.c +25 -0
@@ -515,6 +515,31 @@ struct read_bitmaps_thread_info {io_channel rbt_io; }; +dgrp_t ext2fs_get_avg_group (ext2_filsys fs) +{ +dgrp_t average_group; +unsigned flexbg_size; + +if (fs->fs_num_threads <=1 ) +return fs->group_desc_count; + + average_group = fs->group_desc_count / fs->fs_num_threads; +if (average_group <=1 ) +return 1 ; + +if (ext2fs_has_feature_flex_bg (fs->super)) { +int times =1 ; + + flexbg_size =1 << fs->super->s_log_groups_per_flex; +if (average_group % flexbg_size) { + times = average_group / flexbg_size; + average_group = times * flexbg_size; + } + } + +return average_group; +} +static void *read_bitmaps_thread (void *data) {struct read_bitmaps_thread_info *rbt = data;
Signature · 1 line
2.29.2.299.gdc1121823c-goog
Re: [RFC PATCH v3 50/61] e2fsck: move ext2fs_get_avg_group to rw_bitmaps.c
Quoted 4 lines · Saranya Muruganandam
From: Wang Shilong <[email protected]>
A bounch of
ext2fs_get_avg_group()unused warning messages are annoying, move it torw_bitmaps.cto make gcc happy.
This should be merged with the commit that initially defined ext2fs_get_avg_group() in the first place.
More generally, *all* of the "fix *" commits should be merged with the original commits. This will make it **so** much easier to review the commits.
- Ted
RFC PATCH v3 51/61 configure: enable pfsck by default
In reply to Saranya MuruganandamFrom: Wang Shilong <[email protected]>
Since most of work has been done, compile e2fsprogs with pfsck enabled by default.
So it could testing widely now.
--- configure | 13 +++++++++++-- configure.ac | 13 ++++++++++--- 2 files changed, 21 insertions(+), 5 deletions(-)
configure +11 -2
@@ -1589,7 +1589,7 @@ Optional Features: --disable-tdb disable tdb support --disable-bmap-stats disable collection of bitmap stats. --enable-bmap-stats-ops enable collection of additional bitmap stats - --enable-pfsck enable parallel e2fsck + --disable-pfsck disable parallel e2fsck --disable-nls do not use Native Language Support --enable-threads={posix|solaris|pth|windows} specify multithreading API
@@ -6169,8 +6169,17 @@ $as_echo "Enabling parallel e2fsck" >&6; } fi else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: Disabling parallel e2fsck" >&5 + if test -z "PTHREAD_LIB" +then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: Disabling parallel e2fsck" >&5 $as_echo "Disabling parallel e2fsck" >&6; } +else + +$as_echo "#define CONFIG_PFSCK 1" >>confdefs.h + + { $as_echo "$as_me:${as_lineno-$LINENO}: result: Enabling parallel e2fsck by default" >&5 +$as_echo "Enabling parallel e2fsck by default" >&6; } +fi fi
configure.ac +10 -3
@@ -877,13 +877,13 @@ fi AC_MSG_RESULT([Disabling additional bitmap statistics by default]) ) dnl -dnl handle --enable-pfsck +dnl handle --disable-pfsck dnl PTHREAD_LIB='' AC_CHECK_LIB(pthread,pthread_join,PTHREAD_LIB=-pthread) AC_SUBST(PTHREAD_LIB) AC_ARG_ENABLE([pfsck], -[ --enable-pfsck enable parallel e2fsck], +[ --disable-pfsck disable parallel e2fsck], if test "$enableval" = "no" || test -z "PTHREAD_LIB" then AC_MSG_RESULT([Disabling parallel e2fsck])
@@ -893,7 +893,14 @@ else AC_MSG_RESULT([Enabling parallel e2fsck]) fi , -AC_MSG_RESULT([Disabling parallel e2fsck]) +if test -z "PTHREAD_LIB" +then + AC_MSG_RESULT([Disabling parallel e2fsck]) +else + AC_DEFINE(CONFIG_PFSCK, 1, + [Define to 1 if parallel e2fsck is enabled]) + AC_MSG_RESULT([Enabling parallel e2fsck by default]) +fi ) dnl dnl
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 53/61 e2fsck: fix f_multithread_ok test
In reply to Saranya MuruganandamFrom: Andreas Dilger <[email protected]>
Don't use $OUT for both the input amd output of a pipeline, as the output file is truncated befor the input is read.
Fix the handling in the failure case to generate the *.failed file, and print the actual $test_name instead of "test_name".
--- tests/f_multithread_ok/script | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
tests/f_multithread_ok/script +4 -4
@@ -5,15 +5,15 @@ SKIP_CLEANUP="true" . $cmd_dir/run_e2fsck -cat $OUT1 | grep -v Thread > $OUT1 -rm -f $test_name.ok $test_name.failed -cmp -s $OUT1 $EXP1 +grep -v Thread $OUT1 > $OUT1.tmp +cmp -s $OUT1.tmp $EXP1 status1=$? if [ "$status1" -eq 0 ]; then echo "$test_name: $test_description: ok" touch $test_name.ok else - echo "test_name: $test_description: failed" + echo "$test_name: $test_description: failed" + cmp $OUT1.tmp $EXP1 > $test_name.failed fi unset IMAGE FSCK_OPT SECOND_FSCK_OPT OUT1 OUT2 EXP1 EXP2
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 54/61 e2fsck: fix race in ext2fs_read_bitmaps()
In reply to Saranya MuruganandamFrom: Wang Shilong <[email protected]>
During corruption testing hiting following segfault:
Multiple threads triggered to read bitmaps Signal (11) SIGSEGV si_code=SEGV_MAPERR fault addr=0x200 ./e2fsck[0x4382ae] /lib64/libpthread.so.0(+0x14b20)[0x7f5854d2fb20] ./e2fsck(ext2fs_rb_insert_color+0xc)[0x46ac0c] ./e2fsck[0x467bb4]
./e2fsck[0x467e6d]
./e2fsck[0x45ba95]
./e2fsck[0x45c124] /lib64/libpthread.so.0(+0x94e2)[0x7f5854d244e2] /lib64/libc.so.6(clone+0x43)[0x7f5854beb6c3]
Problem is @block_map might be set NULL if one of thread exit, move such kind of cleanup operation to main thread after all threads exit.
Another potential problem is e2fsck_read_bitmap() could be called during pass1, this need be serialized, serialize it in the pass1.
--- e2fsck/pass1.c | 2 +- lib/ext2fs/rw_bitmaps.c | 25 ++++++++----------------- 2 files changed, 9 insertions(+), 18 deletions(-)
e2fsck/pass1.c +1 -1
@@ -4183,9 +4183,9 @@ report_problem: }continue ; } +e2fsck_pass1_fix_lock (ctx);e2fsck_read_bitmaps (ctx); pb->inode_modified =1 ; -e2fsck_pass1_fix_lock (ctx); pctx->errcode =ext2fs_extent_delete (ehandle,0 );e2fsck_pass1_fix_unlock (ctx);
lib/ext2fs/rw_bitmaps.c +8 -17
@@ -445,14 +445,6 @@ success_cleanup:return 0 ; cleanup: -if (do_block) { -ext2fs_free_block_bitmap (fs->block_map); - fs->block_map =0 ; - } -if (do_inode) { -ext2fs_free_inode_bitmap (fs->inode_map); - fs->inode_map =0 ; - }if (inode_bitmap)ext2fs_free_mem (&inode_bitmap);if (block_bitmap)
@@ -463,9 +455,12 @@ cleanup: } -static errcode_t read_bitmaps_range_end (ext2_filsys fs,int do_inode,int do_block) +static errcode_t read_bitmaps_range_end (ext2_filsys fs,int do_inode,int do_block, +errcode_t retval) { -errcode_t retval =0 ; + +if (retval) + goto cleanup;/* Mark group blocks for any BLOCK_UNINIT groups */ if (do_block) {
@@ -474,7 +469,7 @@ static errcode_t read_bitmaps_range_end(ext2_filsys fs, int do_inode, int do_blo goto cleanup; } -return retval; +return 0 ; cleanup:if (do_block) {ext2fs_free_block_bitmap (fs->block_map);
@@ -497,10 +492,8 @@ static errcode_t read_bitmaps_range(ext2_filsys fs, int do_inode, int do_block,return retval; retval =read_bitmaps_range_start (fs, do_inode, do_block, start, end,NULL ,NULL ); -if (retval) -return retval; -return read_bitmaps_range_end (fs, do_inode, do_block); +return read_bitmaps_range_end (fs, do_inode, do_block, retval); }#ifdef CONFIG_PFSCK
@@ -636,9 +629,7 @@ out:free (thread_infos);free (thread_ids); -if (!retval) - retval =read_bitmaps_range_end (fs, do_inode, do_block); - + retval =read_bitmaps_range_end (fs, do_inode, do_block, retval);if (!retval) {if (do_inode) fs->flags &= ~EXT2_FLAG_IBITMAP_TAIL_PROBLEM ;
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 55/61 e2fsck: fix readahead for pass1 without pfsck
In reply to Saranya MuruganandamFrom: Wang Shilong <[email protected]>
If admin try fsck without -m option, codes try old behavior, thread information won't be inited either. @et_group_end is 0 thus readahead for pass1 will be totally disabled.
With the patch applied, we could get same performance number without pfsck as before.
--- e2fsck/pass1.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
e2fsck/pass1.c +2 -1
@@ -1100,7 +1100,8 @@ static void pass1_readahead(e2fsck_t ctx, dgrp_t *group, ext2_ino_t *next_ino)errcode_t err =EXT2_ET_INVALID_ARGUMENT ;#ifdef CONFIG_PFSCK - grp_end = ctx->thread_info.et_group_end; +if (ctx->fs->fs_num_threads >1 ) + grp_end = ctx->thread_info.et_group_end;#endif if (ctx->readahead_kb ==0 ) goto out;
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 56/61 e2fsck: fix memory leaks with pfsck enabled
In reply to Saranya MuruganandamFrom: Wang Shilong <[email protected]>
valgrind detected few memory leaks:
1) quota context is not released after merging.
2) three block bufs are not freed in read_bitmaps_range_start()
3) @refcount_orig should be released
--- e2fsck/e2fsck.c | 4 ++++ e2fsck/pass1.c | 1 + lib/ext2fs/rw_bitmaps.c | 14 ++++---------- 3 files changed, 9 insertions(+), 10 deletions(-)
e2fsck/e2fsck.c +4 -0
@@ -102,6 +102,10 @@ errcode_t e2fsck_reset_context(e2fsck_t ctx)ea_refcount_free (ctx->refcount_extra); ctx->refcount_extra =0 ; } +if (ctx->refcount_orig) { +ea_refcount_free (ctx->refcount_orig); + ctx->refcount_orig =0 ; + }if (ctx->ea_block_quota_blocks) {ea_refcount_free (ctx->ea_block_quota_blocks); ctx->ea_block_quota_blocks =0 ;
e2fsck/pass1.c +1 -0
@@ -3113,6 +3113,7 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)fclose (thread_ctx->problem_logf); } +quota_release_context (&thread_ctx->qctx);/* * @block_metadata_map and @block_dup_map are * shared, so we don't free them.
lib/ext2fs/rw_bitmaps.c +4 -10
@@ -269,7 +269,7 @@ static errcode_t read_bitmaps_range_start(ext2_filsys fs, int do_inode, int do_bdgrp_t i;char *block_bitmap =0 , *inode_bitmap =0 ;char *buf; -errcode_t retval; +errcode_t retval =0 ;int block_nbytes =EXT2_CLUSTERS_PER_GROUP (fs->super) /8 ;int inode_nbytes =EXT2_INODES_PER_GROUP (fs->super) /8 ;int tail_flags =0 ;
@@ -432,18 +432,12 @@ static errcode_t read_bitmaps_range_start(ext2_filsys fs, int do_inode, int do_b success_cleanup:if (start ==0 && end == fs->group_desc_count -1 ) { -if (inode_bitmap) { -ext2fs_free_mem (&inode_bitmap); +if (inode_bitmap) fs->flags &= ~EXT2_FLAG_IBITMAP_TAIL_PROBLEM ; - } -if (block_bitmap) { -ext2fs_free_mem (&block_bitmap); +if (block_bitmap) fs->flags &= ~EXT2_FLAG_BBITMAP_TAIL_PROBLEM ; - } } fs->flags |= tail_flags; -return 0 ; - cleanup:if (inode_bitmap)ext2fs_free_mem (&inode_bitmap);
@@ -451,8 +445,8 @@ cleanup:ext2fs_free_mem (&block_bitmap);if (buf)ext2fs_free_mem (&buf); -return retval; +return retval; } static errcode_t read_bitmaps_range_end(ext2_filsys fs, int do_inode, int do_block,
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 57/61 ext2fs: fix to set tail flags with pfsck enabled
In reply to Saranya MuruganandamFrom: Wang Shilong <[email protected]>
If any of block/inode bitmap block checksum error happen, tail flag should be set properly.
However, we firstly set tail flags in each thread, after threads finish we clear those tail problem wrongly.
This will make fsck miss bitmap checksum erors later, patch try to fix the problem by move all this kind of logic in read_bitmaps_range_end()
--- lib/ext2fs/rw_bitmaps.c | 46 +++++++++++++++++------------------------ 1 file changed, 19 insertions(+), 27 deletions(-)
lib/ext2fs/rw_bitmaps.c +19 -27
@@ -264,7 +264,7 @@ cleanup:static errcode_t read_bitmaps_range_start (ext2_filsys fs,int do_inode,int do_block,dgrp_t start,dgrp_t end,pthread_mutex_t *mutex, -io_channel io) +io_channel io,int *tail_flags) {dgrp_t i;char *block_bitmap =0 , *inode_bitmap =0 ;
@@ -272,7 +272,6 @@ static errcode_t read_bitmaps_range_start(ext2_filsys fs, int do_inode, int do_berrcode_t retval =0 ;int block_nbytes =EXT2_CLUSTERS_PER_GROUP (fs->super) /8 ;int inode_nbytes =EXT2_INODES_PER_GROUP (fs->super) /8 ; -int tail_flags =0 ;int csum_flag;unsigned int cnt;blk64_t blk;
@@ -343,7 +342,7 @@ static errcode_t read_bitmaps_range_start(ext2_filsys fs, int do_inode, int do_b blk_itr += cnt; blk_cnt -= cnt; } - goto success_cleanup; + goto cleanup; } blk_itr += ((blk64_t )start * (block_nbytes <<3 ));
@@ -374,7 +373,7 @@ static errcode_t read_bitmaps_range_start(ext2_filsys fs, int do_inode, int do_b }if (!bitmap_tail_verify ((unsigned char *) block_bitmap, block_nbytes, fs->blocksize -1 )) - tail_flags |=EXT2_FLAG_BBITMAP_TAIL_PROBLEM ; + *tail_flags |=EXT2_FLAG_BBITMAP_TAIL_PROBLEM ; }else memset (block_bitmap,0 , block_nbytes); cnt = block_nbytes <<3 ;
@@ -414,7 +413,7 @@ static errcode_t read_bitmaps_range_start(ext2_filsys fs, int do_inode, int do_b }if (!bitmap_tail_verify ((unsigned char *) inode_bitmap, inode_nbytes, fs->blocksize -1 )) - tail_flags |=EXT2_FLAG_IBITMAP_TAIL_PROBLEM ; + *tail_flags |=EXT2_FLAG_IBITMAP_TAIL_PROBLEM ; }else memset (inode_bitmap,0 , inode_nbytes); cnt = inode_nbytes <<3 ;
@@ -430,14 +429,6 @@ static errcode_t read_bitmaps_range_start(ext2_filsys fs, int do_inode, int do_b } } -success_cleanup: -if (start ==0 && end == fs->group_desc_count -1 ) { -if (inode_bitmap) - fs->flags &= ~EXT2_FLAG_IBITMAP_TAIL_PROBLEM ; -if (block_bitmap) - fs->flags &= ~EXT2_FLAG_BBITMAP_TAIL_PROBLEM ; - } - fs->flags |= tail_flags; cleanup:if (inode_bitmap)ext2fs_free_mem (&inode_bitmap);
@@ -450,7 +441,7 @@ cleanup: }static errcode_t read_bitmaps_range_end (ext2_filsys fs,int do_inode,int do_block, -errcode_t retval) +errcode_t retval,int tail_flags) {if (retval)
@@ -461,7 +452,11 @@ static errcode_t read_bitmaps_range_end(ext2_filsys fs, int do_inode, int do_blo retval =mark_uninit_bg_group_blocks (fs);if (retval) goto cleanup; + fs->flags &= ~EXT2_FLAG_BBITMAP_TAIL_PROBLEM ; } +if (do_inode) + fs->flags &= ~EXT2_FLAG_IBITMAP_TAIL_PROBLEM ; + fs->flags |= tail_flags;return 0 ; cleanup:
@@ -480,14 +475,16 @@ static errcode_t read_bitmaps_range(ext2_filsys fs, int do_inode, int do_block, dgrp_t start, dgrp_t end) {errcode_t retval; +int tail_flags =0 ; retval =read_bitmaps_range_prepare (fs, do_inode, do_block);if (retval)return retval; - retval =read_bitmaps_range_start (fs, do_inode, do_block, start, end,NULL ,NULL ); + retval =read_bitmaps_range_start (fs, do_inode, do_block, start, end,NULL , +NULL , &tail_flags); -return read_bitmaps_range_end (fs, do_inode, do_block, retval); +return read_bitmaps_range_end (fs, do_inode, do_block, retval, tail_flags); }#ifdef CONFIG_PFSCK
@@ -499,6 +496,7 @@ struct read_bitmaps_thread_info {dgrp_t rbt_grp_end;errcode_t rbt_retval;pthread_mutex_t *rbt_mutex; +int rbt_tail_flags;io_channel rbt_io; };
@@ -534,7 +532,7 @@ static void* read_bitmaps_thread(void *data) rbt->rbt_retval =read_bitmaps_range_start (rbt->rbt_fs, rbt->rbt_do_inode, rbt->rbt_do_block, rbt->rbt_grp_start, rbt->rbt_grp_end, - rbt->rbt_mutex, rbt->rbt_io); + rbt->rbt_mutex, rbt->rbt_io, &rbt->rbt_tail_flags); return NULL; }#endif
@@ -550,7 +548,7 @@ static errcode_t read_bitmaps(ext2_filsys fs, int do_inode, int do_block)errcode_t retval;errcode_t rc;dgrp_t average_group; -int i; +int i, tail_flags =0 ;io_manager manager = unix_io_manager;#else int num_threads =1 ;
@@ -584,6 +582,7 @@ static errcode_t read_bitmaps(ext2_filsys fs, int do_inode, int do_block) thread_infos[i].rbt_do_inode = do_inode; thread_infos[i].rbt_do_block = do_block; thread_infos[i].rbt_mutex = &rbt_mutex; + thread_infos[i].rbt_tail_flags =0 ;if (i ==0 ) thread_infos[i].rbt_grp_start =0 ;else
@@ -614,6 +613,7 @@ static errcode_t read_bitmaps(ext2_filsys fs, int do_inode, int do_block) rc = thread_infos[i].rbt_retval;if (rc && !retval) retval = rc; + tail_flags |= thread_infos[i].rbt_tail_flags;io_channel_close (thread_infos[i].rbt_io); } out:
@@ -623,15 +623,7 @@ out:free (thread_infos);free (thread_ids); - retval =read_bitmaps_range_end (fs, do_inode, do_block, retval); -if (!retval) { -if (do_inode) - fs->flags &= ~EXT2_FLAG_IBITMAP_TAIL_PROBLEM ; -if (do_block) - fs->flags &= ~EXT2_FLAG_BBITMAP_TAIL_PROBLEM ; - } - -return retval; +return read_bitmaps_range_end (fs, do_inode, do_block, retval, tail_flags);#endif }
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 58/61 e2fsck: misc cleanups for pfsck
In reply to Saranya MuruganandamFrom: Andreas Dilger <[email protected]>
Add -m option description to e2fsck.8 man page.
Rename e2fsck_struct fs_num_threads to pfs_num_threads to avoid confusion with the ext2_filsys fs_num_threads field, and move thread_info to be together with the other CONFIG_PFSCK fields.
Move ext2_filsys fs_num_threads to fit into the __u16 "pad" field to avoid consuming one of the few remaining __u32 reserved fields.
Fix a few print format warnings.
--- e2fsck/e2fsck.8.in | 8 +++++++- e2fsck/e2fsck.h | 13 +++++-------- e2fsck/pass1.c | 29 +++++++++++++++-------------- e2fsck/unix.c | 4 ++-- lib/ext2fs/ext2fs.h | 5 ++--- 5 files changed, 31 insertions(+), 28 deletions(-)
e2fsck/e2fsck.8.in +7 -1
@@ -8,7 +8,7 @@ e2fsck \- check a Linux ext2/ext3/ext4 file system .SH SYNOPSIS .B e2fsck [ -.B \-pacnyrdfkvtDFV +.B \-pacnyrdfkmvtDFV ] [ .B \-b
@@ -329,6 +329,12 @@ Set the bad blocks list to be the list of blocks specified by option, except the bad blocks list is cleared before the blocks listed in the file are added to the bad blocks list.) .TP +.B \-m " threads" +Run e2fsck with up to the specified number of +.IR threads . +The actual number of threads may be lower, if the filesystem does not +have enough block groups to effectively parallelize the workload. +.TP .B \-n Open the filesystem read-only, and assume an answer of `no' to all questions. Allows
e2fsck/e2fsck.h +5 -8
@@ -243,8 +243,8 @@ struct e2fsck_thread {dgrp_t et_group_next;/* Scanned inode number */ ext2_ino_t et_inode_number; -char et_log_buf[2048 ];char et_log_length; + char et_log_buf[2048]; };#endif
@@ -333,11 +333,6 @@ struct e2fsck_struct {ext2_ino_t stashed_ino;struct ext2_inode *stashed_inode; -/* if @global_ctx is null, this field is unused */ -#ifdef CONFIG_PFSCK -struct e2fsck_thread thread_info; -#endif -/* * Location of the lost and found directory */
@@ -450,7 +445,9 @@ struct e2fsck_struct {/* Undo file */ char *undo_file;#ifdef CONFIG_PFSCK -__u32 fs_num_threads; +/* if @global_ctx is null, this field is unused */ +struct e2fsck_thread thread_info; +__u32 pfs_num_threads;__u32 mmp_update_thread;int fs_need_locking;/* serialize fix operation for multiple threads */
@@ -689,7 +686,7 @@ int check_backup_super_block(e2fsck_t ctx);void check_resize_inode (e2fsck_t ctx);/* util.c */ -#define E2FSCK_MAX_THREADS (65536) +#define E2FSCK_MAX_THREADS (65535)extern void *e2fsck_allocate_memory (e2fsck_t ctx,unsigned long size,const char *description);extern int ask (e2fsck_t ctx,const char * string,int def);
e2fsck/pass1.c +15 -14
@@ -1260,7 +1260,7 @@ static void e2fsck_pass1_set_thread_num(e2fsck_t ctx) {unsigned flexbg_size =1 ;ext2_filsys fs = ctx->fs; -int num_threads = ctx->fs_num_threads; +int num_threads = ctx->pfs_num_threads;int max_threads;if (num_threads <1 ) {
@@ -1280,6 +1280,8 @@ static void e2fsck_pass1_set_thread_num(e2fsck_t ctx) max_threads = fs->group_desc_count / flexbg_size;if (max_threads ==0 ) max_threads =1 ; +if (max_threads >E2FSCK_MAX_THREADS ) + max_threads =E2FSCK_MAX_THREADS ;if (num_threads > max_threads) { fprintf(stderr, "Use max possible thread num: %d instead\n",
@@ -1287,7 +1289,7 @@ static void e2fsck_pass1_set_thread_num(e2fsck_t ctx) num_threads = max_threads; } out: - ctx->fs_num_threads = num_threads; + ctx->pfs_num_threads = num_threads; ctx->fs->fs_num_threads = num_threads; }#endif
@@ -1315,7 +1317,7 @@ static errcode_t e2fsck_pass1_prepare(e2fsck_t ctx)#ifdef CONFIG_PFSCK /* don't use more than 1/10 of memory for threads checking */ - readahead_kb =get_memory_size () / (10 * ctx->fs_num_threads); + readahead_kb =get_memory_size () / (10 * ctx->pfs_num_threads);/* maybe better disable RA if this is too small? */ if (ctx->readahead_kb > readahead_kb) ctx->readahead_kb = readahead_kb;
@@ -1373,7 +1375,7 @@ static errcode_t e2fsck_pass1_prepare(e2fsck_t ctx)#ifdef CONFIG_PFSCK pthread_rwlock_init (&ctx->fs_fix_rwlock,NULL );pthread_rwlock_init (&ctx->fs_block_map_rwlock,NULL ); -if (ctx->fs_num_threads >1 ) +if (ctx->pfs_num_threads >1 ) ctx->fs_need_locking =1 ;#endif
@@ -1633,7 +1635,7 @@ void e2fsck_pass1_run(e2fsck_t ctx)if (ctx->global_ctx) {if (ctx->options &E2F_OPT_DEBUG && ctx->options &E2F_OPT_MULTITHREAD ) -fprintf (stderr,"thread %d jumping to group %d\n" , +fprintf (stderr,"thread %d jumping to group %u\n" , ctx->thread_info.et_thread_index, ctx->thread_info.et_group_start); pctx.errcode = ext2fs_inode_scan_goto_blockgroup(scan,
@@ -3129,11 +3131,11 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)static int e2fsck_pass1_threads_join (struct e2fsck_thread_info *infos,e2fsck_t global_ctx) { -errcode_t rc; -errcode_t ret =0 ; -int i; -struct e2fsck_thread_info *pinfo; -int num_threads = global_ctx->fs_num_threads; +errcode_t rc; +errcode_t ret =0 ; +struct e2fsck_thread_info *pinfo; +int num_threads = global_ctx->pfs_num_threads; +int i;/* merge invalid bitmaps will recalculate it */ global_ctx->invalid_bitmaps =0 ;
@@ -3199,7 +3201,7 @@ static void *e2fsck_pass1_thread(void *arg) out:if (thread_ctx->options &E2F_OPT_MULTITHREAD )log_out (thread_ctx, -_ ("Scanned group range [%lu, %lu), inodes %lu\n" ), +_ ("Scanned group range [%u, %u), inodes %u\n" ), thread_ctx->thread_info.et_group_start, thread_ctx->thread_info.et_group_end, thread_ctx->thread_info.et_inode_number);
@@ -3225,7 +3227,7 @@ static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo,int i;e2fsck_t thread_ctx;dgrp_t average_group; -int num_threads = global_ctx->fs_num_threads; +int num_threads = global_ctx->pfs_num_threads;#ifdef DEBUG_THREADS struct e2fsck_thread_debug thread_debug = {PTHREAD_MUTEX_INITIALIZER ,PTHREAD_COND_INITIALIZER ,0 };
@@ -3329,8 +3331,7 @@ void e2fsck_pass1(e2fsck_t ctx)if (retval)return ;#ifdef CONFIG_PFSCK -if (ctx->fs_num_threads >1 || - ctx->options &E2F_OPT_MULTITHREAD ) { +if (ctx->pfs_num_threads >1 || ctx->options &E2F_OPT_MULTITHREAD ) { need_single =0 ;e2fsck_pass1_multithread (ctx); }
e2fsck/unix.c +2 -2
@@ -908,12 +908,12 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx) _("Invalid multiple thread num.\n"));if (thread_num >E2FSCK_MAX_THREADS ) {fprintf (stderr, -_ ("threads %lu too large (max %lu)\n" ), +_ ("threads %lu too large (max %u)\n" ), thread_num,E2FSCK_MAX_THREADS );fatal_error (ctx,0 ); } ctx->options |=E2F_OPT_MULTITHREAD ; - ctx->fs_num_threads = thread_num; + ctx->pfs_num_threads = thread_num;break ;#endif case 'n' :
lib/ext2fs/ext2fs.h +2 -3
@@ -254,12 +254,11 @@ struct struct_ext2_filsys {time_t now;int cluster_ratio_bits;__u16 default_bitmap_type; -__u16 pad; -__u32 fs_num_threads; +__u16 fs_num_threads;/* * Reserved for future expansion */ -__u32 reserved[4 ]; +__u32 reserved[5 ];/* * Reserved for the use of the calling application.
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 59/61 e2fsck: update mmp block race
In reply to Saranya MuruganandamFrom: Wang Shilong <[email protected]>
Update mmp block is only expected in one thread, @mmp_update_thread is used to get/set active thread number, however this should be set globally shared with different threads rather than be private per thread.
--- e2fsck/pass1.c | 20 +++++++++----------- 1 file changed, 9 insertions(+), 11 deletions(-)
e2fsck/pass1.c +9 -11
@@ -1501,6 +1501,7 @@ void e2fsck_pass1_run(e2fsck_t ctx)struct ea_quota ea_ibody_quota;struct process_inode_block *inodes_to_process;int process_inode_count, check_mmp; +e2fsck_t global_ctx = ctx->global_ctx ? ctx->global_ctx : ctx;init_resource_track (&rtrack, ctx->fs->io);clear_problem_context (&pctx);
@@ -1652,14 +1653,11 @@ void e2fsck_pass1_run(e2fsck_t ctx) check_mmp =0 ;e2fsck_pass1_check_lock (ctx);#ifdef CONFIG_PFSCK -if (!ctx->mmp_update_thread) { +if (!global_ctx->mmp_update_thread) {e2fsck_pass1_block_map_w_lock (ctx); -if (!ctx->mmp_update_thread) { -if (ctx->global_ctx) - ctx->mmp_update_thread = - ctx->thread_info.et_thread_index +1 ; -else - ctx->mmp_update_thread =1 ; +if (!global_ctx->mmp_update_thread) { + global_ctx->mmp_update_thread = + ctx->thread_info.et_thread_index +1 ; check_mmp =1 ; }e2fsck_pass1_block_map_w_unlock (ctx);
@@ -1667,8 +1665,8 @@ void e2fsck_pass1_run(e2fsck_t ctx)/* only one active thread could update mmp block. */ e2fsck_pass1_block_map_r_lock (ctx); -if (!ctx->global_ctx || ctx->mmp_update_thread == - (ctx->thread_info.et_thread_index +1 )) +if (global_ctx->mmp_update_thread == + ctx->thread_info.et_thread_index +1 ) check_mmp =1 ;e2fsck_pass1_block_map_r_unlock (ctx);#else
@@ -2396,8 +2394,8 @@ endit:#ifdef CONFIG_PFSCK /* reset update_thread after this thread exit */ e2fsck_pass1_block_map_w_lock (ctx); -if (ctx->mmp_update_thread) - ctx->mmp_update_thread =0 ; +if (check_mmp) + global_ctx->mmp_update_thread =0 ;e2fsck_pass1_block_map_w_unlock (ctx);#endif }
Signature · 1 line
2.29.2.299.gdc1121823c-goog
RFC PATCH v3 60/61 e2fsck: propagate number of threads
In reply to Saranya MuruganandamSometimes, such as in orphan_inode case, e2fsck_pass1 is called after reading the block bitmaps. This results in reading the block bitmap sequentially and multithreading only gets kicked in later. Fix the thread count earlier while setting up the file system.
--- e2fsck/unix.c | 3 +++ 1 file changed, 3 insertions(+)
e2fsck/unix.c +3 -0
@@ -1729,6 +1729,9 @@ failure: ctx->fs = fs; fs->now = ctx->now; +#ifdef CONFIG_PFSCK + fs->fs_num_threads = ctx->pfs_num_threads; +#endif sb = fs->super;if (sb->s_rev_level >E2FSCK_CURRENT_REV ) {
Signature · 1 line
2.29.2.299.gdc1121823c-goog
Re: [RFC PATCH v3 60/61] e2fsck: propagate number of threads
Quoted 5 lines · Saranya Muruganandam
Sometimes, such as in
orphan_inodecase,e2fsck_pass1is called after reading the block bitmaps. This results in reading the block bitmap sequentially and multithreading only gets kicked in later. Fix the thread count earlier while setting up the file system.
I wonder if read_bitmaps() should be using a different way of determining how many threads compared to how many threads are used by e2fsck.
Also, looking more closely, it's not at all clear we should fs_num_threads should be in the fs structure at all. It's actually used in three places. (a) to influence the default size of a dblist, if the size is not provided, (b) to influence the default size of an icount structure, if the size is not provided, (c) to figure out how many threads are needed in rw_bitmaps.c
(a) and (b) can be dealt with in e2fsck by having e2fsck pass in an explicit size. (In fact, for the dblist, we can do a better job by summing the number of directories in the block group range for each thread, instead of taking the total number of threads in the group, and dividing by "the averge number of block groups").
As far as (c) is concerned, perhaps it would be better to make it be explicit, via a new interface:
errcode_t ext2fs_rw_bitmaps(ext2_filsys fs, int flags, int num_threads);
Where flags is an OR of the following flags:
#define EXT2FS_BITMAPS_WRITE 0x0001 #define EXT2FS_BITMAPS_BLOCK 0x0002 #define EXT2FS_BITMAPS_INODE 0x0004
... and where num_threads is a hint that can be ignored if the pthreads interface is not available (via a HAVE_PTHREADS autoconf test).
If we do that, then there's no need to have fs->fs_num_threads at all, and that can purely be something which is local matter by the application (e.g., e2fsck, and maybe later other programs like debugfs's ncheck/icheck commands).
See? It's really, really important that we have well-defined interfaces in libext2fs, because that's how we keep tech debt in e2fsprogs down to a manageable level.
So it might be a good idea to separate out the rw_bitmaps patch() from the rest of the series, and create a much smaller patch series that has the autoconf test to see if pthreads are available, and if so, how it should be linked in (this would also subsume the "fix build for rpm" patch, except we would make sure that it works for all OS's, instead of serendiptously when Linux shared libraries were in use, and then adding a hack when it was noticed that the build configuration that rpm happened to use wasn't working).
By breaking down the patch series, we can make sure each change is done cleanly, and portably, and then we can use the individual pieces to to parallelize e2fsck. This will probably be better than trying to deal with this whole patch series as a monolithic thing.
What do folks think?
- Ted
RFC PATCH v3 61/61 e2fsck: Annotating fields in e2fsck_struct
In reply to Saranya MuruganandamAdding information on fields in e2fsck_struct
on how they are used when running parallel fsck.
--- e2fsck/e2fsck.h | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-)
e2fsck/e2fsck.h +14 -6
@@ -251,7 +251,7 @@ struct e2fsck_thread {struct e2fsck_struct {/* Global context to get the cancel flag */ e2fsck_t global_ctx; -ext2_filsys fs; +ext2_filsys fs;/* [fs_fix_rwlock] */ const char *program_name;char *filesystem_name;char *device_name;
@@ -260,7 +260,9 @@ struct e2fsck_struct {char *log_fn;FILE *problem_logf;char *problem_log_fn; -int flags;/* E2fsck internal flags */ +/* E2fsck internal flags. +* shared by different threads for pass1 [fs_fix_rwlock] */ +int flags;int options;unsigned blocksize;/* blocksize */ blk64_t use_superblock;/* sb requested by user */
@@ -281,6 +283,7 @@ struct e2fsck_struct {int (*progress)(e2fsck_t ctx,int pass,unsigned long cur,unsigned long max); +/* The following inode bitmaps are separately used in thread_ctx Pass1*/ ext2fs_inode_bitmap inode_used_map;/* Inodes which are in use */ ext2fs_inode_bitmap inode_bad_map;/* Inodes which are bad somehow */ ext2fs_inode_bitmap inode_dir_map;/* Inodes which are directories */
@@ -288,12 +291,14 @@ struct e2fsck_struct {ext2fs_inode_bitmap inode_imagic_map;/* AFS inodes */ ext2fs_inode_bitmap inode_reg_map;/* Inodes which are regular files*/ +/* Following 3 protected by [fs_block_map_rwlock] */ ext2fs_block_bitmap block_found_map;/* Blocks which are in use */ ext2fs_block_bitmap block_dup_map;/* Blks referenced more than once */ ext2fs_block_bitmap block_ea_map;/* Blocks which are used by EA's */ /* -* Inode count arrays +* Inode count arrays. +* Separately used in thread_ctx, pass1 */ ext2_icount_t inode_count;ext2_icount_t inode_link_info;
@@ -315,7 +320,8 @@ struct e2fsck_struct {/* * Array of flags indicating whether an inode bitmap, block -* bitmap, or inode table is invalid +* bitmap, or inode table is invalid. +* Separately used in thread_ctx, pass1 */ int *invalid_inode_bitmap_flag;int *invalid_block_bitmap_flag;
@@ -328,7 +334,8 @@ struct e2fsck_struct {char *block_buf;/* -* For pass1_check_directory and pass1_get_blocks +* For pass1_check_directory and pass1_get_blocks. +* Separately used in thread_ctx in pass1 */ ext2_ino_t stashed_ino;struct ext2_inode *stashed_inode;
@@ -387,6 +394,7 @@ struct e2fsck_struct {/* * How we display the progress update (for unix) +* shared by different threads for pass1 [fs_fix_rwlock] */ int progress_fd;int progress_pos;
@@ -395,7 +403,7 @@ struct e2fsck_struct {int interactive;/* Are we connected directly to a tty? */ char start_meta[2 ], stop_meta[2 ]; -/* File counts */ +/* File counts. Separately used in thread_ctx, pass1 */ __u32 fs_directory_count;__u32 fs_regular_count;__u32 fs_blockdev_count;
Signature · 1 line
2.29.2.299.gdc1121823c-goog
Quoted 28 lines · Saranya Muruganandam
Currently it has been popular that single disk could be more than TiB, etc 16Tib with only one single disk, with this trend, one single filesystem could be larger and larger and easily reach PiB with LUN system.
The journal filesystem like ext4 need be offline to do regular check and repair from time to time, however the problem is e2fsck still do this using single thread, this could be challenging at scale for two reasons:
1) even with readahead, IO speed still limits several tens MiB per second.
2) could not utilize CPU cores.It could be challenging to try multh-threads for all phase of e2fsck, but as first step, we might try this for most time-consuming pass1, according to our benchmarking it cost of 80% time for whole e2fck phase.
Pass1 is trying to scanning all valid inode of filesystem and check it one by one, and the patchset idea is trying to split these to different threads and trying to do this at the same time, we try to merge these inodes and corresponding inode's extent information after threads finish.
To simplify complexity and make it less error-prone, the fix is still serialized, since most of time there will be only minor errors for filesystem, what's important for us is parallel reading and checking.
Here is a benchmarking on our Lustre filesystem with 1.2 PiB OSD ext4 based filesystem:
Quoted 7 lines
DDN SFA18KE StorageServer
DCR(DeClustering RAID) with 162 x HGST 10TB NL-SAS Tested Server
A Virtual Machine running on SFA18KE
8 x CPU cores (Xeon(R) Gold 6140)
150GB memory
CentoOS7.7 (Lustre patched kernel)
This introductory patch presumably came from the original patch series; hence "our Lustre file system". Just to make it clearer, it's probably better to make it clear who did which benchmarks. And Saranya, you might want to include your benchmark results since it will be easier for people to replicate.
Quoted 8 lines
I've tested the whole patch series using 'make test' of e2fsck itself, and i manually set default threads to 4 which still pass almost of test suite, failure cases are below:
f_h_badrootf_multithreadf_multithread_logfilef_multithread_nof_multithread_ok
h_h_badrootfailed because out of order checking output, and others are because of extra multiple threads log output.
And this "I" is Saranya, yes?
Quoted 15 lines
Andreas Dilger (2):
e2fsck: fixf_multithread_oktest
e2fsck: misc cleanups for pfsckLi Xi (18):
e2fsck: add -m option for multithread
e2fsck: copy context when using multi-thread fsck
e2fsck: copy fs when using multi-thread fsck
e2fsck: add assert when copying context
e2fsck: copy bitmaps when copying context
e2fsck: open io-channel when copying fs
e2fsck: create logs for mult-threads
e2fsck: optionally configure one pfsck thread
e2fsck: add start/end group for thread
e2fsck: split groups to different threads
Quoted 8 lines
e2fsck: print thread log properly
e2fsck: do not change global variables
e2fsck: optimize the inserting ofdir_info_db
e2fsck: mergedir_infoafter thread finishes
e2fsck: merge icounts after thread finishes
e2fsck: merge dblist after thread finishes
e2fsck: add debug codes for multiple threads
e2fsck: merge fs flags when threads finish
The fact that all of these patches are prefixed with e2fsck: hides the fact that some of these changes include changes to libext2fs. It's probably better to separate out the changes to libext2fs so we can pay special attention to issues of presering the ABI.
I'll talk more about this in the individual patches.
- Ted
Quoted 8 lines · Saranya Muruganandam
I've tested the whole patch series using 'make test' of e2fsck itself, and i manually set default threads to 4 which still pass almost of test suite, failure cases are below:
f_h_badrootf_multithreadf_multithread_logfilef_multithread_nof_multithread_ok
h_h_badrootfailed because out of order checking output, and others are because of extra multiple threads log output.
I just tried the full series, and I'm only seeing one test failure. Unfortunately, it's f_multithread, and it's double free crash:
...
Pass 5: Checking group summary information
Multiple threads triggered to read bitmaps
double free or corruption (!prev)
Signal (6) SIGABRT si_code=SI_TKILL ../e2fsck/e2fsck(+0x45fab)[0x556589911fab] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140)[0x7fe52ec34140] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x141)[0x7fe52ea96c41] /lib/x86_64-linux-gnu/libc.so.6(abort+0x123)[0x7fe52ea80537] /lib/x86_64-linux-gnu/libc.so.6(+0x7e6c8)[0x7fe52ead96c8] /lib/x86_64-linux-gnu/libc.so.6(+0x859ba)[0x7fe52eae09ba] /lib/x86_64-linux-gnu/libc.so.6(+0x86ffc)[0x7fe52eae1ffc] ../lib/libext2fs.so.2(ext2fs_free_mem+0x23)[0x7fe52ed1091a] ../lib/libext2fs.so.2(+0x47329)[0x7fe52ed1f329] ../lib/libext2fs.so.2(+0x475b4)[0x7fe52ed1f5b4] /lib/x86_64-linux-gnu/libpthread.so.0(+0x8ea7)[0x7fe52ec28ea7] /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7fe52eb58d4f] Exit status is 8
Here's the contents of f_multithread_ok.1.log after running "./test_script --valgrind f_multithread_ok" in the tests directory:
Pass 1: Checking inodes, blocks, and sizes
[Thread 0] Scan group range [0, 1)
[Thread 1] Scan group range [1, 2)
[Thread 2] Scan group range [2, 3)
[Thread 3] Scan group range [3, 4)
[Thread 2] Scanned group range [2, 3), inodes 8192
[Thread 1] Scanned group range [1, 2), inodes 8192
[Thread 3] Scanned group range [3, 4), inodes 8192
[Thread 0] Scanned group range [0, 1), inodes 8192
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
Multiple threads triggered to read bitmaps
==182288== Thread 2:
==182288== Conditional jump or move depends on uninitialised value(s)
==182288== at 0x488E31B: read_bitmaps_range_start (rw_bitmaps.c:437) ==182288== by 0x488E5B3: read_bitmaps_thread (rw_bitmaps.c:532) ==182288== by 0x4965EA6: start_thread (pthread_create.c:477) ==182288== by 0x4A7CD4E: clone (clone.S:95) ==182288== ==182288== Conditional jump or move depends on uninitialised value(s) ==182288== at 0x4839961: free (vg_replace_malloc.c:538) ==182288== by 0x487F919: ext2fs_free_mem (ext2fs.h:1891) ==182288== by 0x488E328: read_bitmaps_range_start (rw_bitmaps.c:438) ==182288== by 0x488E5B3: read_bitmaps_thread (rw_bitmaps.c:532) ==182288== by 0x4965EA6: start_thread (pthread_create.c:477) ==182288== by 0x4A7CD4E: clone (clone.S:95) ==182288== ==182288== Invalid free() / delete / delete[] / realloc() ==182288== at 0x48399AB: free (vg_replace_malloc.c:538) ==182288== by 0x487F919: ext2fs_free_mem (ext2fs.h:1891) ==182288== by 0x488E328: read_bitmaps_range_start (rw_bitmaps.c:438) ==182288== by 0x488E5B3: read_bitmaps_thread (rw_bitmaps.c:532) ==182288== by 0x4965EA6: start_thread (pthread_create.c:477) ==182288== by 0x4A7CD4E: clone (clone.S:95) ==182288== Address 0x4b46100 is 0 bytes inside a block of size 3,144 free'd ==182288== at 0x48399AB: free (vg_replace_malloc.c:538) ==182288== by 0x487F919: ext2fs_free_mem (ext2fs.h:1891) ==182288== by 0x488E328: read_bitmaps_range_start (rw_bitmaps.c:438) ==182288== by 0x488E5B3: read_bitmaps_thread (rw_bitmaps.c:532) ==182288== by 0x4965EA6: start_thread (pthread_create.c:477) ==182288== by 0x4A7CD4E: clone (clone.S:95) ==182288== Block was alloc'd at ==182288== at 0x483877F: malloc (vg_replace_malloc.c:307) ==182288== by 0x487F7C9: ext2fs_get_mem (ext2fs.h:1847) ==182288== by 0x11EE47: e2fsck_allocate_context (e2fsck.c:27) ==182288== by 0x11B228: PRS (unix.c:829) ==182288== by 0x11CD55: main (unix.c:1465)
...
- Ted