Skip to contentSkip to conversation

Introduce parallel fsck to e2fsck pass1

RFC PATCH In review Saranya Muruganandam 5 participants 84 messages Nov 18, 2020 – Dec 18, 2020 Library code

lore
Share & tools
lore
#0 lore raw

v3 cover Introduce parallel fsck to e2fsck pass1

Currently it has been popular that single disk could be more than TiB, etc 16Tib with only one single disk, with this trend, one single filesystem could be larger and larger and easily reach PiB with LUN system.

The journal filesystem like ext4 need be offline to do regular check and repair from time to time, however the problem is e2fsck still do this using single thread, this could be challenging at scale for two reasons:

1) even with readahead, IO speed still limits several tens MiB per second.
2) could not utilize CPU cores.

It could be challenging to try multh-threads for all phase of e2fsck, but as first step, we might try this for most time-consuming pass1, according to our benchmarking it cost of 80% time for whole e2fck phase.

Pass1 is trying to scanning all valid inode of filesystem and check it one by one, and the patchset idea is trying to split these to different threads and trying to do this at the same time, we try to merge these inodes and corresponding inode's extent information after threads finish.

To simplify complexity and make it less error-prone, the fix is still serialized, since most of time there will be only minor errors for filesystem, what's important for us is parallel reading and checking.

Here is a benchmarking on our Lustre filesystem with 1.2 PiB OSD ext4 based filesystem:

DDN SFA18KE StorageServer
DCR(DeClustering RAID) with 162 x HGST 10TB NL-SAS Tested Server
A Virtual Machine running on SFA18KE
8 x CPU cores (Xeon(R) Gold 6140)
150GB memory
CentoOS7.7 (Lustre patched kernel)

Created 600 Million x 32K byte files.

Without Patch With Patch thr=64
pass1: 13079.66 488.57 seconds
Total: 15673.33 3188.42

We have 5x total time reduction of total time which is very inspiring.

I've tested the whole patch series using 'make test' of e2fsck itself, and i manually set default threads to 4 which still pass almost of test suite, failure cases are below:

f_h_badroot f_multithread f_multithread_logfile f_multithread_no f_multithread_ok

h_h_badroot failed because out of order checking output, and others are because of extra multiple threads log output.

The other test i've done to verify is:
1) filled a filesystem with different features enabled(encryption, shared xattr)
2) copied the above image out using e2image.
3) using e2fuzz to randomly corrupt above image, and then have another copy of corrupted image.
4) running e2fsck with single thread(without patches) and multiple threads with corrupted image respectively.
5) using valgrind to check any memory leak.
6) compare single thread and multiple threads fix results by mounting two images back and compare the directores and files.(size and name match)

Any review or comments are welcomed!

Thanks you very much!

changelog v2->v3:
1) Parallelize rw_bitmaps
2) Configuration to turn off parallel fsck.(default=on)
3) Add f_multithread_ok
4) Add annotations to e2fsck_struct instead of reshuffle
5) Fix memory leaks
6) Some more extra fixes

Andreas Dilger (2):
e2fsck: fix f_multithread_ok test
e2fsck: misc cleanups for pfsck

Li Xi (18):
e2fsck: add -m option for multithread
e2fsck: copy context when using multi-thread fsck
e2fsck: copy fs when using multi-thread fsck
e2fsck: add assert when copying context
e2fsck: copy bitmaps when copying context
e2fsck: open io-channel when copying fs
e2fsck: create logs for mult-threads
e2fsck: optionally configure one pfsck thread
e2fsck: add start/end group for thread
e2fsck: split groups to different threads
e2fsck: print thread log properly
e2fsck: do not change global variables
e2fsck: optimize the inserting of dir_info_db
e2fsck: merge dir_info after thread finishes
e2fsck: merge icounts after thread finishes
e2fsck: merge dblist after thread finishes
e2fsck: add debug codes for multiple threads
e2fsck: merge fs flags when threads finish

Saranya Muruganandam (2):
e2fsck: propagate number of threads
e2fsck: Annotating fields in e2fsck_struct

Wang Shilong (39):
e2fsck: clear icache when using multi-thread fsck
e2fsck: copy badblocks when copying fs
e2fsck: merge bitmaps after thread completes
e2fsck: rbtree bitmap for dir
e2fsck: merge badblocks after thread finishes
e2fsck: merge counts after threads finish
e2fsck: merge dx_dir_info after threads finish
e2fsck: merge dirs_to_hash when threads finish
e2fsck: merge context flags properly
e2fsck: merge quota context after threads finish
e2fsck: serialize fix operations
e2fsck: move some fixes out of parallel pthreads
e2fsck: split and merge invalid bitmaps
e2fsck: merge EA blocks properly
e2fsck: kickoff mutex lock for block found map
e2fsck: allow admin specify number of threads
e2fsck: adjust number of threads
e2fsck: fix readahead for pfsck of pass1
e2fsck: merge options after threads finish
e2fsck: reset lost_and_found after threads finish
e2fsck: merge extent depth count after threads finish
e2fsck: simplify e2fsck context merging codes
e2fsck: set E2F_FLAG_ALLOC_OK after threads
e2fsck: wait fix thread finish before checking
e2fsck: cleanup e2fsck_pass1_thread_join()
e2fsck: avoid too much memory allocation for pfsck
e2fsck: make default smallest RA size to 1M
ext2fs: parallel bitmap loading
e2fsck: update mmp block in one thread
e2fsck: reset @inodes_to_rebuild if restart
e2fsck: fix build for make rpm
e2fsck: move ext2fs_get_avg_group to rw_bitmaps.c
configure: enable pfsck by default
test: add pfsck test
e2fsck: fix race in ext2fs_read_bitmaps()
e2fsck: fix readahead for pass1 without pfsck
e2fsck: fix memory leaks with pfsck enabled
ext2fs: fix to set tail flags with pfsck enabled
e2fsck: update mmp block race

 MCONFIG.in                              |    1 +
 configure                               |   90 +-
 configure.ac                            |   26 +
 e2fsck/Makefile.in                      |    9 +-
 e2fsck/dirinfo.c                        |  238 ++-
 e2fsck/dx_dirinfo.c                     |   64 +
 e2fsck/e2fsck.8.in                      |    8 +-
 e2fsck/e2fsck.c                         |   11 +
 e2fsck/e2fsck.h                         |  102 +-
 e2fsck/logfile.c                        |   13 +-
 e2fsck/pass1.c                          | 1766 ++++++++++++++++++++---
 e2fsck/problem.c                        |   11 +
 e2fsck/problem.h                        |    3 +
 e2fsck/readahead.c                      |    4 +
 e2fsck/unix.c                           |   59 +-
 e2fsck/util.c                           |  193 ++-
 lib/config.h.in                         |    3 +
 lib/ext2fs/badblocks.c                  |   85 +-
 lib/ext2fs/bitmaps.c                    |   10 +
 lib/ext2fs/bitops.h                     |    2 +
 lib/ext2fs/blkmap64_rb.c                |   65 +
 lib/ext2fs/bmap64.h                     |    4 +
 lib/ext2fs/dblist.c                     |   38 +
 lib/ext2fs/ext2_err.et.in               |    3 +
 lib/ext2fs/ext2_io.h                    |    2 +
 lib/ext2fs/ext2fs.h                     |   19 +-
 lib/ext2fs/ext2fsP.h                    |    1 -
 lib/ext2fs/gen_bitmap64.c               |   62 +
 lib/ext2fs/icount.c                     |  107 ++
 lib/ext2fs/openfs.c                     |   48 +-
 lib/ext2fs/rw_bitmaps.c                 |  301 +++-
 lib/ext2fs/undo_io.c                    |   19 +
 lib/ext2fs/unix_io.c                    |   24 +-
 lib/support/mkquota.c                   |   39 +
 lib/support/quotaio.h                   |    3 +
 tests/f_itable_collision/expect.1       |    3 -
 tests/f_multithread/expect.1            |   25 +
 tests/f_multithread/expect.2            |    7 +
 tests/f_multithread/image.gz            |    1 +
 tests/f_multithread/name                |    1 +
 tests/f_multithread/script              |    4 +
 tests/f_multithread_completion/expect.1 |    2 +
 tests/f_multithread_completion/expect.2 |   23 +
 tests/f_multithread_completion/image.gz |    1 +
 tests/f_multithread_completion/name     |    1 +
 tests/f_multithread_completion/script   |    4 +
 tests/f_multithread_logfile/expect.1    |   25 +
 tests/f_multithread_logfile/image.gz    |    1 +
 tests/f_multithread_logfile/name        |    1 +
 tests/f_multithread_logfile/script      |   32 +
 tests/f_multithread_no/expect.1         |   26 +
 tests/f_multithread_no/expect.2         |   23 +
 tests/f_multithread_no/image.gz         |    1 +
 tests/f_multithread_no/name             |    1 +
 tests/f_multithread_no/script           |    4 +
 tests/f_multithread_ok/expect.1         |    8 +
 tests/f_multithread_ok/image.gz         |  Bin 0 -> 796311 bytes
 tests/f_multithread_ok/name             |    1 +
 tests/f_multithread_ok/script           |   21 +
 tests/f_multithread_preen/expect.1      |   11 +
 tests/f_multithread_preen/expect.2      |   23 +
 tests/f_multithread_preen/image.gz      |    1 +
 tests/f_multithread_preen/name          |    1 +
 tests/f_multithread_preen/script        |    4 +
 tests/f_multithread_yes/expect.1        |    2 +
 tests/f_multithread_yes/expect.2        |   23 +
 tests/f_multithread_yes/image.gz        |    1 +
 tests/f_multithread_yes/name            |    1 +
 tests/f_multithread_yes/script          |    4 +
 tests/test_one.in                       |    8 +
 70 files changed, 3335 insertions(+), 393 deletions(-)
 create mode 100644 tests/f_multithread/expect.1
 create mode 100644 tests/f_multithread/expect.2
 create mode 120000 tests/f_multithread/image.gz
 create mode 100644 tests/f_multithread/name
 create mode 100644 tests/f_multithread/script
 create mode 100644 tests/f_multithread_completion/expect.1
 create mode 100644 tests/f_multithread_completion/expect.2
 create mode 120000 tests/f_multithread_completion/image.gz
 create mode 100644 tests/f_multithread_completion/name
 create mode 100644 tests/f_multithread_completion/script
 create mode 100644 tests/f_multithread_logfile/expect.1
 create mode 120000 tests/f_multithread_logfile/image.gz
 create mode 100644 tests/f_multithread_logfile/name
 create mode 100644 tests/f_multithread_logfile/script
 create mode 100644 tests/f_multithread_no/expect.1
 create mode 100644 tests/f_multithread_no/expect.2
 create mode 120000 tests/f_multithread_no/image.gz
 create mode 100644 tests/f_multithread_no/name
 create mode 100644 tests/f_multithread_no/script
 create mode 100644 tests/f_multithread_ok/expect.1
 create mode 100644 tests/f_multithread_ok/image.gz
 create mode 100644 tests/f_multithread_ok/name
 create mode 100644 tests/f_multithread_ok/script
 create mode 100644 tests/f_multithread_preen/expect.1
 create mode 100644 tests/f_multithread_preen/expect.2
 create mode 120000 tests/f_multithread_preen/image.gz
 create mode 100644 tests/f_multithread_preen/name
 create mode 100644 tests/f_multithread_preen/script
 create mode 100644 tests/f_multithread_yes/expect.1
 create mode 100644 tests/f_multithread_yes/expect.2
 create mode 120000 tests/f_multithread_yes/image.gz
 create mode 100644 tests/f_multithread_yes/name
 create mode 100644 tests/f_multithread_yes/script
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#1 lore raw

RFC PATCH v3 1/61 e2fsck: add -m option for multithread

From: Li Xi <[email protected]>

-m option is added but no actual functionality is added. This patch only adds the logic that when -m is specified, one of -p/-y/-n options should be specified. And when -m is specified, -C shouldn't be specified and the completion progress report won't be triggered by sending SIGUSR1/SIGUSR2 signals. This simplifies the implementation of multi-thread fsck in the future.

Completion progress support with multi-thread fsck will be added back after multi-thread fsck implementation is finished. Right now, disable it to simplify the implementation of multi-thread fsck.

---
 e2fsck/e2fsck.h                         |  1 +
 e2fsck/unix.c                           | 31 ++++++++++++++++++++-----
 tests/f_multithread/expect.1            | 23 ++++++++++++++++++
 tests/f_multithread/expect.2            |  7 ++++++
 tests/f_multithread/image.gz            |  1 +
 tests/f_multithread/name                |  1 +
 tests/f_multithread/script              |  4 ++++
 tests/f_multithread_completion/expect.1 |  2 ++
 tests/f_multithread_completion/expect.2 | 23 ++++++++++++++++++
 tests/f_multithread_completion/image.gz |  1 +
 tests/f_multithread_completion/name     |  1 +
 tests/f_multithread_completion/script   |  4 ++++
 tests/f_multithread_no/expect.1         | 24 +++++++++++++++++++
 tests/f_multithread_no/expect.2         | 23 ++++++++++++++++++
 tests/f_multithread_no/image.gz         |  1 +
 tests/f_multithread_no/name             |  1 +
 tests/f_multithread_no/script           |  4 ++++
 tests/f_multithread_preen/expect.1      | 11 +++++++++
 tests/f_multithread_preen/expect.2      | 23 ++++++++++++++++++
 tests/f_multithread_preen/image.gz      |  1 +
 tests/f_multithread_preen/name          |  1 +
 tests/f_multithread_preen/script        |  4 ++++
 tests/f_multithread_yes/expect.1        |  2 ++
 tests/f_multithread_yes/expect.2        | 23 ++++++++++++++++++
 tests/f_multithread_yes/image.gz        |  1 +
 tests/f_multithread_yes/name            |  1 +
 tests/f_multithread_yes/script          |  4 ++++
 27 files changed, 217 insertions(+), 6 deletions(-)
 create mode 100644 tests/f_multithread/expect.1
 create mode 100644 tests/f_multithread/expect.2
 create mode 120000 tests/f_multithread/image.gz
 create mode 100644 tests/f_multithread/name
 create mode 100644 tests/f_multithread/script
 create mode 100644 tests/f_multithread_completion/expect.1
 create mode 100644 tests/f_multithread_completion/expect.2
 create mode 120000 tests/f_multithread_completion/image.gz
 create mode 100644 tests/f_multithread_completion/name
 create mode 100644 tests/f_multithread_completion/script
 create mode 100644 tests/f_multithread_no/expect.1
 create mode 100644 tests/f_multithread_no/expect.2
 create mode 120000 tests/f_multithread_no/image.gz
 create mode 100644 tests/f_multithread_no/name
 create mode 100644 tests/f_multithread_no/script
 create mode 100644 tests/f_multithread_preen/expect.1
 create mode 100644 tests/f_multithread_preen/expect.2
 create mode 120000 tests/f_multithread_preen/image.gz
 create mode 100644 tests/f_multithread_preen/name
 create mode 100644 tests/f_multithread_preen/script
 create mode 100644 tests/f_multithread_yes/expect.1
 create mode 100644 tests/f_multithread_yes/expect.2
 create mode 120000 tests/f_multithread_yes/image.gz
 create mode 100644 tests/f_multithread_yes/name
 create mode 100644 tests/f_multithread_yes/script
e2fsck/e2fsck.h +1 -0
index 85f953b2..e2784248 100644
@@ -177,6 +177,7 @@ struct resource_track {
 #define E2F_OPT_ICOUNT_FULLMAP	0x20000 /* use an array for inode counts */
 #define E2F_OPT_UNSHARE_BLOCKS  0x40000
 #define E2F_OPT_CLEAR_UNINIT	0x80000 /* Hack to clear the uninit bit */
+#define E2F_OPT_MULTITHREAD	0x100000 /* Use multiple threads to speedup */
 
 /*
  * E2fsck flags
e2fsck/unix.c +25 -6
index 1cb51672..051b31a5 100644
@@ -75,13 +75,14 @@ int journal_enable_debug = -1;
 static void usage(e2fsck_t ctx)
 {
 	fprintf(stderr,
-		_("Usage: %s [-panyrcdfktvDFV] [-b superblock] [-B blocksize]\n"
+		_("Usage: %s [-pamnyrcdfktvDFV] [-b superblock] [-B blocksize]\n"
 		"\t\t[-l|-L bad_blocks_file] [-C fd] [-j external_journal]\n"
 		"\t\t[-E extended-options] [-z undo_file] device\n"),
 		ctx->program_name);
 
 	fprintf(stderr, "%s", _("\nEmergency help:\n"
 		" -p                   Automatic repair (no questions)\n"
+		" -m                   multiple threads to speedup fsck\n"
 		" -n                   Make no changes to the filesystem\n"
 		" -y                   Assume \"yes\" to all questions\n"
 		" -c                   Check for bad blocks and add them to the badblock list\n"
@@ -847,7 +848,8 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx)
 
 	phys_mem_kb = get_memory_size() / 1024;
 	ctx->readahead_kb = ~0ULL;
-	while ((c = getopt(argc, argv, "panyrcC:B:dE:fvtFVM:b:I:j:P:l:L:N:SsDkz:")) != EOF)
+
+	while ((c = getopt(argc, argv, "pamnyrcC:B:dE:fvtFVM:b:I:j:P:l:L:N:SsDkz:")) != EOF)
 		switch (c) {
 		case 'C':
 			ctx->progress = e2fsck_update_progress;
@@ -888,6 +890,9 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx)
 			}
 			ctx->options |= E2F_OPT_PREEN;
 			break;
+		case 'm':
+			ctx->options |= E2F_OPT_MULTITHREAD;
+			break;
 		case 'n':
 			if (ctx->options & (E2F_OPT_YES|E2F_OPT_PREEN))
 				goto conflict_opt;
@@ -1006,6 +1011,18 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx)
 			_("The -n and -l/-L options are incompatible."));
 		fatal_error(ctx, 0);
 	}
+	if (ctx->options & E2F_OPT_MULTITHREAD) {
+		if ((ctx->options & (E2F_OPT_YES|E2F_OPT_NO|E2F_OPT_PREEN)) == 0) {
+			com_err(ctx->program_name, 0, "%s",
+				_("The -m option should be used together with one of -p/-y/-n options."));
+			fatal_error(ctx, 0);
+		}
+		if (ctx->progress) {
+			com_err(ctx->program_name, 0, "%s",
+				_("Only one of the options -C or -m may be specified."));
+			fatal_error(ctx, 0);
+		}
+	}
 	if (ctx->options & E2F_OPT_NO)
 		ctx->options |= E2F_OPT_READONLY;
 
@@ -1112,10 +1129,12 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx)
 #ifdef SA_RESTART
 	sa.sa_flags = SA_RESTART;
 #endif
-	sa.sa_handler = signal_progress_on;
-	sigaction(SIGUSR1, &sa, 0);
-	sa.sa_handler = signal_progress_off;
-	sigaction(SIGUSR2, &sa, 0);
+	if ((ctx->options & E2F_OPT_MULTITHREAD) == 0) {
+		sa.sa_handler = signal_progress_on;
+		sigaction(SIGUSR1, &sa, 0);
+		sa.sa_handler = signal_progress_off;
+		sigaction(SIGUSR2, &sa, 0);
+	}
 #endif
 
 	/* Update our PATH to include /sbin if we need to run badblocks  */
tests/f_multithread/expect.1 new file +23 -0
new file mode 100644
index 00000000..e2b954d0
@@ -0,0 +1,23 @@
+ext2fs_open2: Bad magic number in super-block
+../e2fsck/e2fsck: Superblock invalid, trying backup blocks...
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+Free blocks count wrong for group #0 (7987, counted=7982).
+Fix? yes
+
+Free blocks count wrong (11602, counted=11597).
+Fix? yes
+
+Free inodes count wrong for group #0 (1493, counted=1488).
+Fix? yes
+
+Free inodes count wrong (2997, counted=2992).
+Fix? yes
+
+
+test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
+test_filesys: 16/3008 files (0.0% non-contiguous), 403/12000 blocks
+Exit status is 1
tests/f_multithread/expect.2 new file +7 -0
new file mode 100644
index 00000000..a833aefc
@@ -0,0 +1,7 @@
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+test_filesys: 16/3008 files (0.0% non-contiguous), 403/12000 blocks
+Exit status is 0
tests/f_multithread/image.gz new file +1 -0
new file mode 120000
index 00000000..0fd40018
@@ -0,0 +1 @@
+../f_zero_super/image.gz
\ No newline at end of file
tests/f_multithread/name new file +1 -0
new file mode 100644
index 00000000..df838ea6
@@ -0,0 +1 @@
+test "e2fsck -m" option
\ No newline at end of file
tests/f_multithread/script new file +4 -0
new file mode 100644
index 00000000..0fe96cd0
@@ -0,0 +1,4 @@
+FSCK_OPT="-fy -m"
+SECOND_FSCK_OPT=-yf
+
+. $cmd_dir/run_e2fsck
tests/f_multithread_completion/expect.1 new file +2 -0
new file mode 100644
index 00000000..61cac9bb
@@ -0,0 +1,2 @@
+../e2fsck/e2fsck: Only one of the options -C or -m may be specified.
+Exit status is 8
tests/f_multithread_completion/expect.2 new file +23 -0
new file mode 100644
index 00000000..e2b954d0
@@ -0,0 +1,23 @@
+ext2fs_open2: Bad magic number in super-block
+../e2fsck/e2fsck: Superblock invalid, trying backup blocks...
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+Free blocks count wrong for group #0 (7987, counted=7982).
+Fix? yes
+
+Free blocks count wrong (11602, counted=11597).
+Fix? yes
+
+Free inodes count wrong for group #0 (1493, counted=1488).
+Fix? yes
+
+Free inodes count wrong (2997, counted=2992).
+Fix? yes
+
+
+test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
+test_filesys: 16/3008 files (0.0% non-contiguous), 403/12000 blocks
+Exit status is 1
tests/f_multithread_completion/image.gz new file +1 -0
new file mode 120000
index 00000000..0fd40018
@@ -0,0 +1 @@
+../f_zero_super/image.gz
\ No newline at end of file
tests/f_multithread_completion/name new file +1 -0
new file mode 100644
index 00000000..a959045d
@@ -0,0 +1 @@
+test "e2fsck -m" option conflicts with "-C"
\ No newline at end of file
tests/f_multithread_completion/script new file +4 -0
new file mode 100644
index 00000000..bf23cd61
@@ -0,0 +1,4 @@
+FSCK_OPT="-fy -m -C 1"
+SECOND_FSCK_OPT=-yf
+
+. $cmd_dir/run_e2fsck
tests/f_multithread_no/expect.1 new file +24 -0
new file mode 100644
index 00000000..d14c4083
@@ -0,0 +1,24 @@
+ext2fs_open2: Bad magic number in super-block
+../e2fsck/e2fsck: Superblock invalid, trying backup blocks...
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+Free blocks count wrong for group #0 (7987, counted=7982).
+Fix? no
+
+Free blocks count wrong (11602, counted=11597).
+Fix? no
+
+Free inodes count wrong for group #0 (1493, counted=1488).
+Fix? no
+
+Free inodes count wrong (2997, counted=2992).
+Fix? no
+
+
+test_filesys: ********** WARNING: Filesystem still has errors **********
+
+test_filesys: 11/3008 files (0.0% non-contiguous), 398/12000 blocks
+Exit status is 4
tests/f_multithread_no/expect.2 new file +23 -0
new file mode 100644
index 00000000..e2b954d0
@@ -0,0 +1,23 @@
+ext2fs_open2: Bad magic number in super-block
+../e2fsck/e2fsck: Superblock invalid, trying backup blocks...
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+Free blocks count wrong for group #0 (7987, counted=7982).
+Fix? yes
+
+Free blocks count wrong (11602, counted=11597).
+Fix? yes
+
+Free inodes count wrong for group #0 (1493, counted=1488).
+Fix? yes
+
+Free inodes count wrong (2997, counted=2992).
+Fix? yes
+
+
+test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
+test_filesys: 16/3008 files (0.0% non-contiguous), 403/12000 blocks
+Exit status is 1
tests/f_multithread_no/image.gz new file +1 -0
new file mode 120000
index 00000000..0fd40018
@@ -0,0 +1 @@
+../f_zero_super/image.gz
\ No newline at end of file
tests/f_multithread_no/name new file +1 -0
new file mode 100644
index 00000000..fa49692e
@@ -0,0 +1 @@
+test "e2fsck -m" option works with "-n"
\ No newline at end of file
tests/f_multithread_no/script new file +4 -0
new file mode 100644
index 00000000..b93deb3a
@@ -0,0 +1,4 @@
+FSCK_OPT="-fn -m"
+SECOND_FSCK_OPT=-yf
+
+. $cmd_dir/run_e2fsck
tests/f_multithread_preen/expect.1 new file +11 -0
new file mode 100644
index 00000000..b4b0cd9a
@@ -0,0 +1,11 @@
+../e2fsck/e2fsck: Bad magic number in super-block while trying to open test.img
+test_filesys: 
+The superblock could not be read or does not describe a valid ext2/ext3/ext4
+filesystem.  If the device is valid and it really contains an ext2/ext3/ext4
+filesystem (and not swap or ufs or something else), then the superblock
+is corrupt, and you might try running e2fsck with an alternate superblock:
+    e2fsck -b 8193 <device>
+ or
+    e2fsck -b 32768 <device>
+
+Exit status is 8
tests/f_multithread_preen/expect.2 new file +23 -0
new file mode 100644
index 00000000..e2b954d0
@@ -0,0 +1,23 @@
+ext2fs_open2: Bad magic number in super-block
+../e2fsck/e2fsck: Superblock invalid, trying backup blocks...
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+Free blocks count wrong for group #0 (7987, counted=7982).
+Fix? yes
+
+Free blocks count wrong (11602, counted=11597).
+Fix? yes
+
+Free inodes count wrong for group #0 (1493, counted=1488).
+Fix? yes
+
+Free inodes count wrong (2997, counted=2992).
+Fix? yes
+
+
+test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
+test_filesys: 16/3008 files (0.0% non-contiguous), 403/12000 blocks
+Exit status is 1
tests/f_multithread_preen/image.gz new file +1 -0
new file mode 120000
index 00000000..0fd40018
@@ -0,0 +1 @@
+../f_zero_super/image.gz
\ No newline at end of file
tests/f_multithread_preen/name new file +1 -0
new file mode 100644
index 00000000..90d199df
@@ -0,0 +1 @@
+test "e2fsck -m" option works with "-p"
\ No newline at end of file
tests/f_multithread_preen/script new file +4 -0
new file mode 100644
index 00000000..ecb79cd6
@@ -0,0 +1,4 @@
+FSCK_OPT="-fp -m"
+SECOND_FSCK_OPT=-yf
+
+. $cmd_dir/run_e2fsck
tests/f_multithread_yes/expect.1 new file +2 -0
new file mode 100644
index 00000000..8b780ecf
@@ -0,0 +1,2 @@
+../e2fsck/e2fsck: The -m option should be used together with one of -p/-y/-n options.
+Exit status is 8
tests/f_multithread_yes/expect.2 new file +23 -0
new file mode 100644
index 00000000..e2b954d0
@@ -0,0 +1,23 @@
+ext2fs_open2: Bad magic number in super-block
+../e2fsck/e2fsck: Superblock invalid, trying backup blocks...
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+Free blocks count wrong for group #0 (7987, counted=7982).
+Fix? yes
+
+Free blocks count wrong (11602, counted=11597).
+Fix? yes
+
+Free inodes count wrong for group #0 (1493, counted=1488).
+Fix? yes
+
+Free inodes count wrong (2997, counted=2992).
+Fix? yes
+
+
+test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
+test_filesys: 16/3008 files (0.0% non-contiguous), 403/12000 blocks
+Exit status is 1
tests/f_multithread_yes/image.gz new file +1 -0
new file mode 120000
index 00000000..0fd40018
@@ -0,0 +1 @@
+../f_zero_super/image.gz
\ No newline at end of file
tests/f_multithread_yes/name new file +1 -0
new file mode 100644
index 00000000..3a703195
@@ -0,0 +1 @@
+test "e2fsck -m" option works with "-y"
\ No newline at end of file
tests/f_multithread_yes/script new file +4 -0
new file mode 100644
index 00000000..38891f6a
@@ -0,0 +1,4 @@
+FSCK_OPT="-f -m"
+SECOND_FSCK_OPT=-yf
+
+. $cmd_dir/run_e2fsck
Signature · 1 line
2.29.2.299.gdc1121823c-goog
Reviewed-by on v3 1/61
#2 lore raw

Re: [RFC PATCH v3 01/61] e2fsck: add -m option for multithread

This patch looks good to me. You can add -

Quoted 514 lines · Saranya Muruganandam

From: Li Xi <[email protected]>

-m option is added but no actual functionality is added. This patch only adds the logic that when -m is specified, one of

(quoted diff) +0 -2
-p/-y/-n options should be specified. And when -m is specified,
-C shouldn't be specified and the completion progress report won't

be triggered by sending SIGUSR1/SIGUSR2 signals. This simplifies the implementation of multi-thread fsck in the future.

Completion progress support with multi-thread fsck will be added back after multi-thread fsck implementation is finished. Right now, disable it to simplify the implementation of multi-thread fsck.

---
 e2fsck/e2fsck.h                         |  1 +
 e2fsck/unix.c                           | 31 ++++++++++++++++++++-----
 tests/f_multithread/expect.1            | 23 ++++++++++++++++++
 tests/f_multithread/expect.2            |  7 ++++++
 tests/f_multithread/image.gz            |  1 +
 tests/f_multithread/name                |  1 +
 tests/f_multithread/script              |  4 ++++
 tests/f_multithread_completion/expect.1 |  2 ++
 tests/f_multithread_completion/expect.2 | 23 ++++++++++++++++++
 tests/f_multithread_completion/image.gz |  1 +
 tests/f_multithread_completion/name     |  1 +
 tests/f_multithread_completion/script   |  4 ++++
 tests/f_multithread_no/expect.1         | 24 +++++++++++++++++++
 tests/f_multithread_no/expect.2         | 23 ++++++++++++++++++
 tests/f_multithread_no/image.gz         |  1 +
 tests/f_multithread_no/name             |  1 +
 tests/f_multithread_no/script           |  4 ++++
 tests/f_multithread_preen/expect.1      | 11 +++++++++
 tests/f_multithread_preen/expect.2      | 23 ++++++++++++++++++
 tests/f_multithread_preen/image.gz      |  1 +
 tests/f_multithread_preen/name          |  1 +
 tests/f_multithread_preen/script        |  4 ++++
 tests/f_multithread_yes/expect.1        |  2 ++
 tests/f_multithread_yes/expect.2        | 23 ++++++++++++++++++
 tests/f_multithread_yes/image.gz        |  1 +
 tests/f_multithread_yes/name            |  1 +
 tests/f_multithread_yes/script          |  4 ++++
 27 files changed, 217 insertions(+), 6 deletions(-)
 create mode 100644 tests/f_multithread/expect.1
 create mode 100644 tests/f_multithread/expect.2
 create mode 120000 tests/f_multithread/image.gz
 create mode 100644 tests/f_multithread/name
 create mode 100644 tests/f_multithread/script
 create mode 100644 tests/f_multithread_completion/expect.1
 create mode 100644 tests/f_multithread_completion/expect.2
 create mode 120000 tests/f_multithread_completion/image.gz
 create mode 100644 tests/f_multithread_completion/name
 create mode 100644 tests/f_multithread_completion/script
 create mode 100644 tests/f_multithread_no/expect.1
 create mode 100644 tests/f_multithread_no/expect.2
 create mode 120000 tests/f_multithread_no/image.gz
 create mode 100644 tests/f_multithread_no/name
 create mode 100644 tests/f_multithread_no/script
 create mode 100644 tests/f_multithread_preen/expect.1
 create mode 100644 tests/f_multithread_preen/expect.2
 create mode 120000 tests/f_multithread_preen/image.gz
 create mode 100644 tests/f_multithread_preen/name
 create mode 100644 tests/f_multithread_preen/script
 create mode 100644 tests/f_multithread_yes/expect.1
 create mode 100644 tests/f_multithread_yes/expect.2
 create mode 120000 tests/f_multithread_yes/image.gz
 create mode 100644 tests/f_multithread_yes/name
 create mode 100644 tests/f_multithread_yes/script
e2fsck/e2fsck.h +1 -0
index 85f953b2..e2784248 100644
@@ -177,6 +177,7 @@ struct resource_track {
 #define E2F_OPT_ICOUNT_FULLMAP 0x20000 /* use an array for inode counts */
 #define E2F_OPT_UNSHARE_BLOCKS  0x40000
 #define E2F_OPT_CLEAR_UNINIT   0x80000 /* Hack to clear the uninit bit */
+#define E2F_OPT_MULTITHREAD    0x100000 /* Use multiple threads to speedup */
 
 /*
  * E2fsck flags
e2fsck/unix.c +25 -6
index 1cb51672..051b31a5 100644
@@ -75,13 +75,14 @@ int journal_enable_debug = -1;
 static void usage(e2fsck_t ctx)
 {
        fprintf(stderr,
-               _("Usage: %s [-panyrcdfktvDFV] [-b superblock] [-B blocksize]\n"
+               _("Usage: %s [-pamnyrcdfktvDFV] [-b superblock] [-B blocksize]\n"
                "\t\t[-l|-L bad_blocks_file] [-C fd] [-j external_journal]\n"
                "\t\t[-E extended-options] [-z undo_file] device\n"),
                ctx->program_name);
 
        fprintf(stderr, "%s", _("\nEmergency help:\n"
                " -p                   Automatic repair (no questions)\n"
+               " -m                   multiple threads to speedup fsck\n"
                " -n                   Make no changes to the filesystem\n"
                " -y                   Assume \"yes\" to all questions\n"
                " -c                   Check for bad blocks and add them to the badblock list\n"
@@ -847,7 +848,8 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx)
 
        phys_mem_kb = get_memory_size() / 1024;
        ctx->readahead_kb = ~0ULL;
-       while ((c = getopt(argc, argv, "panyrcC:B:dE:fvtFVM:b:I:j:P:l:L:N:SsDkz:")) != EOF)
+
+       while ((c = getopt(argc, argv, "pamnyrcC:B:dE:fvtFVM:b:I:j:P:l:L:N:SsDkz:")) != EOF)
                switch (c) {
                case 'C':
                        ctx->progress = e2fsck_update_progress;
@@ -888,6 +890,9 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx)
                        }
                        ctx->options |= E2F_OPT_PREEN;
                        break;
+               case 'm':
+                       ctx->options |= E2F_OPT_MULTITHREAD;
+                       break;
                case 'n':
                        if (ctx->options & (E2F_OPT_YES|E2F_OPT_PREEN))
                                goto conflict_opt;
@@ -1006,6 +1011,18 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx)
                        _("The -n and -l/-L options are incompatible."));
                fatal_error(ctx, 0);
        }
+       if (ctx->options & E2F_OPT_MULTITHREAD) {
+               if ((ctx->options & (E2F_OPT_YES|E2F_OPT_NO|E2F_OPT_PREEN)) == 0) {
+                       com_err(ctx->program_name, 0, "%s",
+                               _("The -m option should be used together with one of -p/-y/-n options."));
+                       fatal_error(ctx, 0);
+               }
+               if (ctx->progress) {
+                       com_err(ctx->program_name, 0, "%s",
+                               _("Only one of the options -C or -m may be specified."));
+                       fatal_error(ctx, 0);
+               }
+       }
        if (ctx->options & E2F_OPT_NO)
                ctx->options |= E2F_OPT_READONLY;
 
@@ -1112,10 +1129,12 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx)
 #ifdef SA_RESTART
        sa.sa_flags = SA_RESTART;
 #endif
-       sa.sa_handler = signal_progress_on;
-       sigaction(SIGUSR1, &sa, 0);
-       sa.sa_handler = signal_progress_off;
-       sigaction(SIGUSR2, &sa, 0);
+       if ((ctx->options & E2F_OPT_MULTITHREAD) == 0) {
+               sa.sa_handler = signal_progress_on;
+               sigaction(SIGUSR1, &sa, 0);
+               sa.sa_handler = signal_progress_off;
+               sigaction(SIGUSR2, &sa, 0);
+       }
 #endif
 
        /* Update our PATH to include /sbin if we need to run badblocks  */
tests/f_multithread/expect.1 new file +23 -0
new file mode 100644
index 00000000..e2b954d0
@@ -0,0 +1,23 @@
+ext2fs_open2: Bad magic number in super-block
+../e2fsck/e2fsck: Superblock invalid, trying backup blocks...
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+Free blocks count wrong for group #0 (7987, counted=7982).
+Fix? yes
+
+Free blocks count wrong (11602, counted=11597).
+Fix? yes
+
+Free inodes count wrong for group #0 (1493, counted=1488).
+Fix? yes
+
+Free inodes count wrong (2997, counted=2992).
+Fix? yes
+
+
+test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
+test_filesys: 16/3008 files (0.0% non-contiguous), 403/12000 blocks
+Exit status is 1
tests/f_multithread/expect.2 new file +7 -0
new file mode 100644
index 00000000..a833aefc
@@ -0,0 +1,7 @@
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+test_filesys: 16/3008 files (0.0% non-contiguous), 403/12000 blocks
+Exit status is 0
tests/f_multithread/image.gz new file +1 -0
new file mode 120000
index 00000000..0fd40018
@@ -0,0 +1 @@
+../f_zero_super/image.gz
\ No newline at end of file
tests/f_multithread/name new file +1 -0
new file mode 100644
index 00000000..df838ea6
@@ -0,0 +1 @@
+test "e2fsck -m" option
\ No newline at end of file
tests/f_multithread/script new file +4 -0
new file mode 100644
index 00000000..0fe96cd0
@@ -0,0 +1,4 @@
+FSCK_OPT="-fy -m"
+SECOND_FSCK_OPT=-yf
+
+. $cmd_dir/run_e2fsck
tests/f_multithread_completion/expect.1 new file +2 -0
new file mode 100644
index 00000000..61cac9bb
@@ -0,0 +1,2 @@
+../e2fsck/e2fsck: Only one of the options -C or -m may be specified.
+Exit status is 8
tests/f_multithread_completion/expect.2 new file +23 -0
new file mode 100644
index 00000000..e2b954d0
@@ -0,0 +1,23 @@
+ext2fs_open2: Bad magic number in super-block
+../e2fsck/e2fsck: Superblock invalid, trying backup blocks...
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+Free blocks count wrong for group #0 (7987, counted=7982).
+Fix? yes
+
+Free blocks count wrong (11602, counted=11597).
+Fix? yes
+
+Free inodes count wrong for group #0 (1493, counted=1488).
+Fix? yes
+
+Free inodes count wrong (2997, counted=2992).
+Fix? yes
+
+
+test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
+test_filesys: 16/3008 files (0.0% non-contiguous), 403/12000 blocks
+Exit status is 1
tests/f_multithread_completion/image.gz new file +1 -0
new file mode 120000
index 00000000..0fd40018
@@ -0,0 +1 @@
+../f_zero_super/image.gz
\ No newline at end of file
tests/f_multithread_completion/name new file +1 -0
new file mode 100644
index 00000000..a959045d
@@ -0,0 +1 @@
+test "e2fsck -m" option conflicts with "-C"
\ No newline at end of file
tests/f_multithread_completion/script new file +4 -0
new file mode 100644
index 00000000..bf23cd61
@@ -0,0 +1,4 @@
+FSCK_OPT="-fy -m -C 1"
+SECOND_FSCK_OPT=-yf
+
+. $cmd_dir/run_e2fsck
tests/f_multithread_no/expect.1 new file +24 -0
new file mode 100644
index 00000000..d14c4083
@@ -0,0 +1,24 @@
+ext2fs_open2: Bad magic number in super-block
+../e2fsck/e2fsck: Superblock invalid, trying backup blocks...
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+Free blocks count wrong for group #0 (7987, counted=7982).
+Fix? no
+
+Free blocks count wrong (11602, counted=11597).
+Fix? no
+
+Free inodes count wrong for group #0 (1493, counted=1488).
+Fix? no
+
+Free inodes count wrong (2997, counted=2992).
+Fix? no
+
+
+test_filesys: ********** WARNING: Filesystem still has errors **********
+
+test_filesys: 11/3008 files (0.0% non-contiguous), 398/12000 blocks
+Exit status is 4
tests/f_multithread_no/expect.2 new file +23 -0
new file mode 100644
index 00000000..e2b954d0
@@ -0,0 +1,23 @@
+ext2fs_open2: Bad magic number in super-block
+../e2fsck/e2fsck: Superblock invalid, trying backup blocks...
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+Free blocks count wrong for group #0 (7987, counted=7982).
+Fix? yes
+
+Free blocks count wrong (11602, counted=11597).
+Fix? yes
+
+Free inodes count wrong for group #0 (1493, counted=1488).
+Fix? yes
+
+Free inodes count wrong (2997, counted=2992).
+Fix? yes
+
+
+test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
+test_filesys: 16/3008 files (0.0% non-contiguous), 403/12000 blocks
+Exit status is 1
tests/f_multithread_no/image.gz new file +1 -0
new file mode 120000
index 00000000..0fd40018
@@ -0,0 +1 @@
+../f_zero_super/image.gz
\ No newline at end of file
tests/f_multithread_no/name new file +1 -0
new file mode 100644
index 00000000..fa49692e
@@ -0,0 +1 @@
+test "e2fsck -m" option works with "-n"
\ No newline at end of file
tests/f_multithread_no/script new file +4 -0
new file mode 100644
index 00000000..b93deb3a
@@ -0,0 +1,4 @@
+FSCK_OPT="-fn -m"
+SECOND_FSCK_OPT=-yf
+
+. $cmd_dir/run_e2fsck
tests/f_multithread_preen/expect.1 new file +11 -0
new file mode 100644
index 00000000..b4b0cd9a
@@ -0,0 +1,11 @@
+../e2fsck/e2fsck: Bad magic number in super-block while trying to open test.img
+test_filesys:
+The superblock could not be read or does not describe a valid ext2/ext3/ext4
+filesystem.  If the device is valid and it really contains an ext2/ext3/ext4
+filesystem (and not swap or ufs or something else), then the superblock
+is corrupt, and you might try running e2fsck with an alternate superblock:
+    e2fsck -b 8193 <device>
+ or
+    e2fsck -b 32768 <device>
+
+Exit status is 8
tests/f_multithread_preen/expect.2 new file +23 -0
new file mode 100644
index 00000000..e2b954d0
@@ -0,0 +1,23 @@
+ext2fs_open2: Bad magic number in super-block
+../e2fsck/e2fsck: Superblock invalid, trying backup blocks...
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+Free blocks count wrong for group #0 (7987, counted=7982).
+Fix? yes
+
+Free blocks count wrong (11602, counted=11597).
+Fix? yes
+
+Free inodes count wrong for group #0 (1493, counted=1488).
+Fix? yes
+
+Free inodes count wrong (2997, counted=2992).
+Fix? yes
+
+
+test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
+test_filesys: 16/3008 files (0.0% non-contiguous), 403/12000 blocks
+Exit status is 1
tests/f_multithread_preen/image.gz new file +1 -0
new file mode 120000
index 00000000..0fd40018
@@ -0,0 +1 @@
+../f_zero_super/image.gz
\ No newline at end of file
tests/f_multithread_preen/name new file +1 -0
new file mode 100644
index 00000000..90d199df
@@ -0,0 +1 @@
+test "e2fsck -m" option works with "-p"
\ No newline at end of file
tests/f_multithread_preen/script new file +4 -0
new file mode 100644
index 00000000..ecb79cd6
@@ -0,0 +1,4 @@
+FSCK_OPT="-fp -m"
+SECOND_FSCK_OPT=-yf
+
+. $cmd_dir/run_e2fsck
tests/f_multithread_yes/expect.1 new file +2 -0
new file mode 100644
index 00000000..8b780ecf
@@ -0,0 +1,2 @@
+../e2fsck/e2fsck: The -m option should be used together with one of -p/-y/-n options.
+Exit status is 8
tests/f_multithread_yes/expect.2 new file +23 -0
new file mode 100644
index 00000000..e2b954d0
@@ -0,0 +1,23 @@
+ext2fs_open2: Bad magic number in super-block
+../e2fsck/e2fsck: Superblock invalid, trying backup blocks...
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+Free blocks count wrong for group #0 (7987, counted=7982).
+Fix? yes
+
+Free blocks count wrong (11602, counted=11597).
+Fix? yes
+
+Free inodes count wrong for group #0 (1493, counted=1488).
+Fix? yes
+
+Free inodes count wrong (2997, counted=2992).
+Fix? yes
+
+
+test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
+test_filesys: 16/3008 files (0.0% non-contiguous), 403/12000 blocks
+Exit status is 1
tests/f_multithread_yes/image.gz new file +1 -0
new file mode 120000
index 00000000..0fd40018
@@ -0,0 +1 @@
+../f_zero_super/image.gz
\ No newline at end of file
tests/f_multithread_yes/name new file +1 -0
new file mode 100644
index 00000000..3a703195
@@ -0,0 +1 @@
+test "e2fsck -m" option works with "-y"
\ No newline at end of file
tests/f_multithread_yes/script new file +4 -1
new file mode 100644
index 00000000..38891f6a
@@ -0,0 +1,4 @@
+FSCK_OPT="-f -m"
+SECOND_FSCK_OPT=-yf
+
+. $cmd_dir/run_e2fsck
--

2.29.2.299.gdc1121823c-goog

#3 lore raw

Re: [RFC PATCH v3 01/61] e2fsck: add -m option for multithread

In reply to Saranya Muruganandam
Quoted 9 lines · Saranya Muruganandam

From: Li Xi <[email protected]>

-m option is added but no actual functionality is added. This patch only adds the logic that when -m is specified, one of

(quoted diff) +0 -2
-p/-y/-n options should be specified. And when -m is specified,
-C shouldn't be specified and the completion progress report won't

be triggered by sending SIGUSR1/SIGUSR2 signals. This simplifies the implementation of multi-thread fsck in the future.

Quoted 7 lines

Completion progress support with multi-thread fsck will be added back after multi-thread fsck implementation is finished. Right now, disable it to simplify the implementation of multi-thread fsck.

I'm a bit surprised the changes to the e2fsck man page aren't inclulded in this patch. I see the man page changes were added in "e2fsck: misc cleanups for pfsck", which I would have merged into other patches, but that's a bit of a nit-pick. (The way I normally do these sorts of changes is to expert the patches using "git format-patch", and then I'll use an editor to move patch hunks around, and then apply them all using "git am". Or I'll using something like the "guilt" program, which is essentially quilt for git patch series.)

(Don't worry about making changes for this, unless you're going to be making lots of other changes to the patch series.)

- Ted

#4 lore raw

RFC PATCH v3 2/61 e2fsck: copy context when using multi-thread fsck

In reply to Saranya Muruganandam

From: Li Xi <[email protected]>

This patch only copy the context to a new one when -m is enabled. It doesn't actually start any thread. When pass1 test finishes, the new context is copied back to the original context.

Since the signal handler only changes the original context, so add global_ctx in "struct e2fsck_struct" and use that to check whether there is any signal of canceling.

This patch handles the long jump properly so that all the existing tests can be passed even the context has been copied. Otherwise, test f_expisize_ea_del would fail when aborting.

---
 e2fsck/pass1.c | 114 +++++++++++++++++++++++++++++++++++++++++++++----
 e2fsck/unix.c  |   1 +
 2 files changed, 107 insertions(+), 8 deletions(-)
e2fsck/pass1.c +106 -8
index 8eecd958..64d237d3 100644
@@ -1144,7 +1144,22 @@ static int quota_inum_is_reserved(ext2_filsys fs, ext2_ino_t ino)
 	return 0;
 }
 
-void e2fsck_pass1(e2fsck_t ctx)
+static int e2fsck_should_abort(e2fsck_t ctx)
+{
+	e2fsck_t global_ctx;
+
+	if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+		return 1;
+
+	if (ctx->global_ctx) {
+		global_ctx = ctx->global_ctx;
+		if (global_ctx->flags & E2F_FLAG_SIGNAL_MASK)
+			return 1;
+	}
+	return 0;
+}
+
+void e2fsck_pass1_thread(e2fsck_t ctx)
 {
 	int	i;
 	__u64	max_sizes;
@@ -1360,7 +1375,7 @@ void e2fsck_pass1(e2fsck_t ctx)
 		if (ino > ino_threshold)
 			pass1_readahead(ctx, &ra_group, &ino_threshold);
 		ehandler_operation(old_op);
-		if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+		if (e2fsck_should_abort(ctx))
 			goto endit;
 		if (pctx.errcode == EXT2_ET_BAD_BLOCK_IN_INODE_TABLE) {
 			/*
@@ -1955,7 +1970,7 @@ void e2fsck_pass1(e2fsck_t ctx)
 		if (process_inode_count >= ctx->process_inode_size) {
 			process_inodes(ctx, block_buf);
 
-			if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+			if (e2fsck_should_abort(ctx))
 				goto endit;
 		}
 	}
@@ -2068,6 +2083,89 @@ endit:
 	else
 		ctx->invalid_bitmaps++;
 }
+
+static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thread_ctx)
+{
+	errcode_t	retval;
+	e2fsck_t	thread_context;
+
+	retval = ext2fs_get_mem(sizeof(struct e2fsck_struct), &thread_context);
+	if (retval) {
+		com_err(global_ctx->program_name, retval, "while allocating memory");
+		return retval;
+	}
+	memcpy(thread_context, global_ctx, sizeof(struct e2fsck_struct));
+	thread_context->fs->priv_data = thread_context;
+	thread_context->global_ctx = global_ctx;
+
+	*thread_ctx = thread_context;
+	return 0;
+}
+
+static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
+{
+	int	flags = global_ctx->flags;
+#ifdef HAVE_SETJMP_H
+	jmp_buf	old_jmp;
+
+	memcpy(old_jmp, global_ctx->abort_loc, sizeof(jmp_buf));
+#endif
+	memcpy(global_ctx, thread_ctx, sizeof(struct e2fsck_struct));
+#ifdef HAVE_SETJMP_H
+	memcpy(global_ctx->abort_loc, old_jmp, sizeof(jmp_buf));
+#endif
+	/* Keep the global singal flags*/
+	global_ctx->flags |= (flags & E2F_FLAG_SIGNAL_MASK) |
+			     (global_ctx->flags & E2F_FLAG_SIGNAL_MASK);
+
+	global_ctx->fs->priv_data = global_ctx;
+	ext2fs_free_mem(&thread_ctx);
+	return 0;
+}
+
+void e2fsck_pass1_multithread(e2fsck_t ctx)
+{
+	errcode_t	retval;
+	e2fsck_t	thread_ctx;
+
+	retval = e2fsck_pass1_thread_prepare(ctx, &thread_ctx);
+	if (retval) {
+		com_err(ctx->program_name, 0,
+			_("while preparing pass1 thread\n"));
+		ctx->flags |= E2F_FLAG_ABORT;
+		return;
+	}
+
+#ifdef HAVE_SETJMP_H
+	/*
+	 * When fatal_error() happens, jump to here. The thread
+	 * context's flags will be saved, but its abort_loc will
+	 * be overwritten by original jump buffer for the later
+	 * tests.
+	 */
+	if (setjmp(thread_ctx->abort_loc)) {
+		thread_ctx->flags &= ~E2F_FLAG_SETJMP_OK;
+		e2fsck_pass1_thread_join(ctx, thread_ctx);
+		return;
+	}
+	thread_ctx->flags |= E2F_FLAG_SETJMP_OK;
+#endif
+
+	e2fsck_pass1_thread(thread_ctx);
+	retval = e2fsck_pass1_thread_join(ctx, thread_ctx);
+	if (retval) {
+		com_err(ctx->program_name, 0,
+			_("while joining pass1 thread\n"));
+		ctx->flags |= E2F_FLAG_ABORT;
+		return;
+	}
+}
+
+void e2fsck_pass1(e2fsck_t ctx)
+{
+	e2fsck_pass1_multithread(ctx);
+}
+
 #undef FINISH_INODE_LOOP
 
 /*
@@ -2130,7 +2228,7 @@ static void process_inodes(e2fsck_t ctx, char *block_buf)
 		ehandler_operation(buf);
 		check_blocks(ctx, &pctx, block_buf,
 			     &inodes_to_process[i].ea_ibody_quota);
-		if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+		if (e2fsck_should_abort(ctx))
 			break;
 	}
 	ctx->stashed_inode = old_stashed_inode;
@@ -3300,7 +3398,7 @@ static void check_blocks(e2fsck_t ctx, struct problem_context *pctx,
 	inlinedata_fs = ext2fs_has_feature_inline_data(ctx->fs->super);
 
 	if (check_ext_attr(ctx, pctx, block_buf, &ea_block_quota)) {
-		if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+		if (e2fsck_should_abort(ctx))
 			goto out;
 		pb.num_blocks += EXT2FS_B2C(ctx->fs, ea_block_quota.blocks);
 	}
@@ -3355,7 +3453,7 @@ static void check_blocks(e2fsck_t ctx, struct problem_context *pctx,
 	}
 	end_problem_latch(ctx, PR_LATCH_BLOCK);
 	end_problem_latch(ctx, PR_LATCH_TOOBIG);
-	if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+	if (e2fsck_should_abort(ctx))
 		goto out;
 	if (pctx->errcode)
 		fix_problem(ctx, PR_1_BLOCK_ITERATE, pctx);
@@ -3836,7 +3934,7 @@ static int process_bad_block(ext2_filsys fs,
 				*block_nr = 0;
 				return BLOCK_CHANGED;
 			}
-			if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+			if (e2fsck_should_abort(ctx))
 				return BLOCK_ABORT;
 		} else
 			mark_block_used(ctx, blk);
@@ -3933,7 +4031,7 @@ static int process_bad_block(ext2_filsys fs,
 			*block_nr = 0;
 			return BLOCK_CHANGED;
 		}
-		if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+		if (e2fsck_should_abort(ctx))
 			return BLOCK_ABORT;
 		return 0;
 	}
e2fsck/unix.c +1 -0
index 051b31a5..42f616e2 100644
@@ -1445,6 +1445,7 @@ int main (int argc, char *argv[])
 	}
 	reserve_stdio_fds();
 
+	ctx->global_ctx = NULL;
 	set_up_logging(ctx);
 	if (ctx->logf) {
 		int i;
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#5 lore raw

Re: [RFC PATCH v3 02/61] e2fsck: copy context when using multi-thread fsck

Thanks for the patch. I noticed that this patch doesn't compile. It's because it needs "global_ctx" to be present in the e2fsck_struct but it gets added later in the patch series in the patch 1e1b0216 ("LU-8465 e2fsck: open io-channel when copying fs"). Given that this and also a couple of other patches need global_ctx, should we split 1e1b0216 ("LU-8465 e2fsck: open io-channel when copying fs") should we add global_ctx in this patch or as a separate patch before this?

Quoted 219 lines · Saranya Muruganandam

From: Li Xi <[email protected]>

This patch only copy the context to a new one when -m is enabled. It doesn't actually start any thread. When pass1 test finishes, the new context is copied back to the original context.

Since the signal handler only changes the original context, so add global_ctx in "struct e2fsck_struct" and use that to check whether there is any signal of canceling.

This patch handles the long jump properly so that all the existing tests can be passed even the context has been copied. Otherwise, test f_expisize_ea_del would fail when aborting.

---
 e2fsck/pass1.c | 114 +++++++++++++++++++++++++++++++++++++++++++++----
 e2fsck/unix.c  |   1 +
 2 files changed, 107 insertions(+), 8 deletions(-)
e2fsck/pass1.c +106 -8
index 8eecd958..64d237d3 100644
@@ -1144,7 +1144,22 @@ static int quota_inum_is_reserved(ext2_filsys fs, ext2_ino_t ino)
        return 0;
 }
 
-void e2fsck_pass1(e2fsck_t ctx)
+static int e2fsck_should_abort(e2fsck_t ctx)
+{
+       e2fsck_t global_ctx;
+
+       if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+               return 1;
+
+       if (ctx->global_ctx) {
+               global_ctx = ctx->global_ctx;
+               if (global_ctx->flags & E2F_FLAG_SIGNAL_MASK)
+                       return 1;
+       }
+       return 0;
+}
+
+void e2fsck_pass1_thread(e2fsck_t ctx)
 {
        int     i;
        __u64   max_sizes;
@@ -1360,7 +1375,7 @@ void e2fsck_pass1(e2fsck_t ctx)
                if (ino > ino_threshold)
                        pass1_readahead(ctx, &ra_group, &ino_threshold);
                ehandler_operation(old_op);
-               if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+               if (e2fsck_should_abort(ctx))
                        goto endit;
                if (pctx.errcode == EXT2_ET_BAD_BLOCK_IN_INODE_TABLE) {
                        /*
@@ -1955,7 +1970,7 @@ void e2fsck_pass1(e2fsck_t ctx)
                if (process_inode_count >= ctx->process_inode_size) {
                        process_inodes(ctx, block_buf);
 
-                       if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+                       if (e2fsck_should_abort(ctx))
                                goto endit;
                }
        }
@@ -2068,6 +2083,89 @@ endit:
        else
                ctx->invalid_bitmaps++;
 }
+
+static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thread_ctx)
+{
+       errcode_t       retval;
+       e2fsck_t        thread_context;
+
+       retval = ext2fs_get_mem(sizeof(struct e2fsck_struct), &thread_context);
+       if (retval) {
+               com_err(global_ctx->program_name, retval, "while allocating memory");
+               return retval;
+       }
+       memcpy(thread_context, global_ctx, sizeof(struct e2fsck_struct));
+       thread_context->fs->priv_data = thread_context;
+       thread_context->global_ctx = global_ctx;
+
+       *thread_ctx = thread_context;
+       return 0;
+}
+
+static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
+{
+       int     flags = global_ctx->flags;
+#ifdef HAVE_SETJMP_H
+       jmp_buf old_jmp;
+
+       memcpy(old_jmp, global_ctx->abort_loc, sizeof(jmp_buf));
+#endif
+       memcpy(global_ctx, thread_ctx, sizeof(struct e2fsck_struct));
+#ifdef HAVE_SETJMP_H
+       memcpy(global_ctx->abort_loc, old_jmp, sizeof(jmp_buf));
+#endif
+       /* Keep the global singal flags*/
+       global_ctx->flags |= (flags & E2F_FLAG_SIGNAL_MASK) |
+                            (global_ctx->flags & E2F_FLAG_SIGNAL_MASK);
+
+       global_ctx->fs->priv_data = global_ctx;
+       ext2fs_free_mem(&thread_ctx);
+       return 0;
+}
+
+void e2fsck_pass1_multithread(e2fsck_t ctx)
+{
+       errcode_t       retval;
+       e2fsck_t        thread_ctx;
+
+       retval = e2fsck_pass1_thread_prepare(ctx, &thread_ctx);
+       if (retval) {
+               com_err(ctx->program_name, 0,
+                       _("while preparing pass1 thread\n"));
+               ctx->flags |= E2F_FLAG_ABORT;
+               return;
+       }
+
+#ifdef HAVE_SETJMP_H
+       /*
+        * When fatal_error() happens, jump to here. The thread
+        * context's flags will be saved, but its abort_loc will
+        * be overwritten by original jump buffer for the later
+        * tests.
+        */
+       if (setjmp(thread_ctx->abort_loc)) {
+               thread_ctx->flags &= ~E2F_FLAG_SETJMP_OK;
+               e2fsck_pass1_thread_join(ctx, thread_ctx);
+               return;
+       }
+       thread_ctx->flags |= E2F_FLAG_SETJMP_OK;
+#endif
+
+       e2fsck_pass1_thread(thread_ctx);
+       retval = e2fsck_pass1_thread_join(ctx, thread_ctx);
+       if (retval) {
+               com_err(ctx->program_name, 0,
+                       _("while joining pass1 thread\n"));
+               ctx->flags |= E2F_FLAG_ABORT;
+               return;
+       }
+}
+
+void e2fsck_pass1(e2fsck_t ctx)
+{
+       e2fsck_pass1_multithread(ctx);
+}
+
 #undef FINISH_INODE_LOOP
 
 /*
@@ -2130,7 +2228,7 @@ static void process_inodes(e2fsck_t ctx, char *block_buf)
                ehandler_operation(buf);
                check_blocks(ctx, &pctx, block_buf,
                             &inodes_to_process[i].ea_ibody_quota);
-               if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+               if (e2fsck_should_abort(ctx))
                        break;
        }
        ctx->stashed_inode = old_stashed_inode;
@@ -3300,7 +3398,7 @@ static void check_blocks(e2fsck_t ctx, struct problem_context *pctx,
        inlinedata_fs = ext2fs_has_feature_inline_data(ctx->fs->super);
 
        if (check_ext_attr(ctx, pctx, block_buf, &ea_block_quota)) {
-               if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+               if (e2fsck_should_abort(ctx))
                        goto out;
                pb.num_blocks += EXT2FS_B2C(ctx->fs, ea_block_quota.blocks);
        }
@@ -3355,7 +3453,7 @@ static void check_blocks(e2fsck_t ctx, struct problem_context *pctx,
        }
        end_problem_latch(ctx, PR_LATCH_BLOCK);
        end_problem_latch(ctx, PR_LATCH_TOOBIG);
-       if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+       if (e2fsck_should_abort(ctx))
                goto out;
        if (pctx->errcode)
                fix_problem(ctx, PR_1_BLOCK_ITERATE, pctx);
@@ -3836,7 +3934,7 @@ static int process_bad_block(ext2_filsys fs,
                                *block_nr = 0;
                                return BLOCK_CHANGED;
                        }
-                       if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+                       if (e2fsck_should_abort(ctx))
                                return BLOCK_ABORT;
                } else
                        mark_block_used(ctx, blk);
@@ -3933,7 +4031,7 @@ static int process_bad_block(ext2_filsys fs,
                        *block_nr = 0;
                        return BLOCK_CHANGED;
                }
-               if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+               if (e2fsck_should_abort(ctx))
                        return BLOCK_ABORT;
                return 0;
        }
e2fsck/unix.c +1 -1
index 051b31a5..42f616e2 100644
@@ -1445,6 +1445,7 @@ int main (int argc, char *argv[])
        }
        reserve_stdio_fds();
 
+       ctx->global_ctx = NULL;
        set_up_logging(ctx);
        if (ctx->logf) {
                int i;
--

2.29.2.299.gdc1121823c-goog

#6 lore raw

Re: [RFC PATCH v3 02/61] e2fsck: copy context when using multi-thread fsck

In reply to Saranya Muruganandam
Quoted 6 lines · Saranya Muruganandam

From: Li Xi <[email protected]>

This patch only copy the context to a new one when -m is enabled. It doesn't actually start any thread. When pass1 test finishes, the new context is copied back to the original context.

Quoted 7 lines

Since the signal handler only changes the original context, so add global_ctx in "struct e2fsck_struct" and use that to check whether there is any signal of canceling.

This patch handles the long jump properly so that all the existing tests can be passed even the context has been copied. Otherwise, test f_expisize_ea_del would fail when aborting.

The patch description is a bit misleading. What this is really about is adding the infrastructure to start and join threads in pass #1. Since we presumably will add multiple threading support for other passes, the one-line summary and commit description should make this clear, so that in the future, when a future developer is trying to examine the 60+ commits in this patch series, it will be a bit easier for them to understand what is happening.

It might also be good to explain the patch series that this only starts a single thread, since this patch series is really only about adding the multi-threading machinery.

Some questions which immediately come up is whether it makes sense to have this machinery in e2fsck/pass1.c, or whether some of this is more general and should be in e2fsck/util.c --- although obviously some portion of the code when we are merging the results from the pass will be pass specific. Of course, none of this is in this commit, so it's hard for me to see whether or not it will make sense in the long run.

We can refactor the code later, or in a future patch series, but the point is that it's hard for me to tell whether the existing function breakdown makes the most amount of sense in the first reading of the patches. If you have an opinion of what's the better way to do things, having looked at the whole patch series, feel free to move some of the refactoring into the earlier patches; the patch series doesn't have to reflect the developmental history of the changes, and for the purposes of patch review, it's often simpler when you change earlier patches to simplify things --- although that can make it harder since then you'll have to rework later patches. (If it's too hard, it's fine to leave things as they are.)

- Ted

#7 lore raw

Re: [RFC PATCH v3 02/61] e2fsck: copy context when using multi-thread fsck

In reply to Saranya Muruganandam
Quoted 69 lines · Saranya Muruganandam

From: Li Xi <[email protected]>

This patch only copy the context to a new one when -m is enabled. It doesn't actually start any thread. When pass1 test finishes, the new context is copied back to the original context.

Since the signal handler only changes the original context, so add global_ctx in "struct e2fsck_struct" and use that to check whether there is any signal of canceling.

This patch handles the long jump properly so that all the existing tests can be passed even the context has been copied. Otherwise, test f_expisize_ea_del would fail when aborting.

---
 e2fsck/pass1.c | 114 +++++++++++++++++++++++++++++++++++++++++++++----
 e2fsck/unix.c  |   1 +
 2 files changed, 107 insertions(+), 8 deletions(-)
e2fsck/pass1.c +18 -3
index 8eecd958..64d237d3 100644
@@ -1144,7 +1144,22 @@ static int quota_inum_is_reserved(ext2_filsys fs, ext2_ino_t ino)
 	return 0;
 }
 
-void e2fsck_pass1(e2fsck_t ctx)
+static int e2fsck_should_abort(e2fsck_t ctx)
+{
+	e2fsck_t global_ctx;
+
+	if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+		return 1;
+
+	if (ctx->global_ctx) {
+		global_ctx = ctx->global_ctx;
+		if (global_ctx->flags & E2F_FLAG_SIGNAL_MASK)
+			return 1;
+	}
+	return 0;
+}
+
+void e2fsck_pass1_thread(e2fsck_t ctx)
 {
 	int	i;
 	__u64	max_sizes;
@@ -1360,7 +1375,7 @@ void e2fsck_pass1(e2fsck_t ctx)
 		if (ino > ino_threshold)
 			pass1_readahead(ctx, &ra_group, &ino_threshold);
 		ehandler_operation(old_op);
-		if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+		if (e2fsck_should_abort(ctx))
 			goto endit;
 		if (pctx.errcode == EXT2_ET_BAD_BLOCK_IN_INODE_TABLE) {
 			/*
@@ -1955,7 +1970,7 @@ void e2fsck_pass1(e2fsck_t ctx)
 		if (process_inode_count >= ctx->process_inode_size) {
 			process_inodes(ctx, block_buf);
 
-			if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+			if (e2fsck_should_abort(ctx))
 				goto endit;
 		}
 	}

e2fsck/pass1.c endit: · line 2092

Quoted 10 lines
e2fsck/pass1.c +7 -0
@@ -2068,6 +2083,89 @@ endit:
 	else
 		ctx->invalid_bitmaps++;
 }
+
+static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thread_ctx)
+{
+	errcode_t	retval;
+	e2fsck_t	thread_context;
+
+	retval = ext2fs_get_mem(sizeof(struct e2fsck_struct), &thread_context);

Hm, so I guess the strategy here is that parallel e2fsck makes per-thread copies of the ext2_filsys and e2fsck_t global contexts? And then after the threaded parts complete, each thread merges its per-thread contexts back into the global one, right?

This means that we have to be careful to track which fields in those cloned contexts have been updated by the thread so that we can copy them back and not lose any data.

I'm wondering if for future maintainability it would be better to track the per-thread data in a separate structure to make it very explicit which data (sub)structures are effectively per-thread and hence don't require locking?

(I ask that mostly because I'm having a hard time figuring out which fields are supposed to be shared and which ones aren't...)

--D

Quoted 139 lines
e2fsck/pass1.c +81 -5
+	if (retval) {
+		com_err(global_ctx->program_name, retval, "while allocating memory");
+		return retval;
+	}
+	memcpy(thread_context, global_ctx, sizeof(struct e2fsck_struct));
+	thread_context->fs->priv_data = thread_context;
+	thread_context->global_ctx = global_ctx;
+
+	*thread_ctx = thread_context;
+	return 0;
+}
+
+static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
+{
+	int	flags = global_ctx->flags;
+#ifdef HAVE_SETJMP_H
+	jmp_buf	old_jmp;
+
+	memcpy(old_jmp, global_ctx->abort_loc, sizeof(jmp_buf));
+#endif
+	memcpy(global_ctx, thread_ctx, sizeof(struct e2fsck_struct));
+#ifdef HAVE_SETJMP_H
+	memcpy(global_ctx->abort_loc, old_jmp, sizeof(jmp_buf));
+#endif
+	/* Keep the global singal flags*/
+	global_ctx->flags |= (flags & E2F_FLAG_SIGNAL_MASK) |
+			     (global_ctx->flags & E2F_FLAG_SIGNAL_MASK);
+
+	global_ctx->fs->priv_data = global_ctx;
+	ext2fs_free_mem(&thread_ctx);
+	return 0;
+}
+
+void e2fsck_pass1_multithread(e2fsck_t ctx)
+{
+	errcode_t	retval;
+	e2fsck_t	thread_ctx;
+
+	retval = e2fsck_pass1_thread_prepare(ctx, &thread_ctx);
+	if (retval) {
+		com_err(ctx->program_name, 0,
+			_("while preparing pass1 thread\n"));
+		ctx->flags |= E2F_FLAG_ABORT;
+		return;
+	}
+
+#ifdef HAVE_SETJMP_H
+	/*
+	 * When fatal_error() happens, jump to here. The thread
+	 * context's flags will be saved, but its abort_loc will
+	 * be overwritten by original jump buffer for the later
+	 * tests.
+	 */
+	if (setjmp(thread_ctx->abort_loc)) {
+		thread_ctx->flags &= ~E2F_FLAG_SETJMP_OK;
+		e2fsck_pass1_thread_join(ctx, thread_ctx);
+		return;
+	}
+	thread_ctx->flags |= E2F_FLAG_SETJMP_OK;
+#endif
+
+	e2fsck_pass1_thread(thread_ctx);
+	retval = e2fsck_pass1_thread_join(ctx, thread_ctx);
+	if (retval) {
+		com_err(ctx->program_name, 0,
+			_("while joining pass1 thread\n"));
+		ctx->flags |= E2F_FLAG_ABORT;
+		return;
+	}
+}
+
+void e2fsck_pass1(e2fsck_t ctx)
+{
+	e2fsck_pass1_multithread(ctx);
+}
+
 #undef FINISH_INODE_LOOP
 
 /*
@@ -2130,7 +2228,7 @@ static void process_inodes(e2fsck_t ctx, char *block_buf)
 		ehandler_operation(buf);
 		check_blocks(ctx, &pctx, block_buf,
 			     &inodes_to_process[i].ea_ibody_quota);
-		if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+		if (e2fsck_should_abort(ctx))
 			break;
 	}
 	ctx->stashed_inode = old_stashed_inode;
@@ -3300,7 +3398,7 @@ static void check_blocks(e2fsck_t ctx, struct problem_context *pctx,
 	inlinedata_fs = ext2fs_has_feature_inline_data(ctx->fs->super);
 
 	if (check_ext_attr(ctx, pctx, block_buf, &ea_block_quota)) {
-		if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+		if (e2fsck_should_abort(ctx))
 			goto out;
 		pb.num_blocks += EXT2FS_B2C(ctx->fs, ea_block_quota.blocks);
 	}
@@ -3355,7 +3453,7 @@ static void check_blocks(e2fsck_t ctx, struct problem_context *pctx,
 	}
 	end_problem_latch(ctx, PR_LATCH_BLOCK);
 	end_problem_latch(ctx, PR_LATCH_TOOBIG);
-	if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+	if (e2fsck_should_abort(ctx))
 		goto out;
 	if (pctx->errcode)
 		fix_problem(ctx, PR_1_BLOCK_ITERATE, pctx);
@@ -3836,7 +3934,7 @@ static int process_bad_block(ext2_filsys fs,
 				*block_nr = 0;
 				return BLOCK_CHANGED;
 			}
-			if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+			if (e2fsck_should_abort(ctx))
 				return BLOCK_ABORT;
 		} else
 			mark_block_used(ctx, blk);
@@ -3933,7 +4031,7 @@ static int process_bad_block(ext2_filsys fs,
 			*block_nr = 0;
 			return BLOCK_CHANGED;
 		}
-		if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+		if (e2fsck_should_abort(ctx))
 			return BLOCK_ABORT;
 		return 0;
 	}
e2fsck/unix.c +1 -0
index 051b31a5..42f616e2 100644
@@ -1445,6 +1445,7 @@ int main (int argc, char *argv[])
 	}
 	reserve_stdio_fds();
 
+	ctx->global_ctx = NULL;
 	set_up_logging(ctx);
 	if (ctx->logf) {
 		int i;
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#8 lore raw

Re: [RFC PATCH v3 02/61] e2fsck: copy context when using multi-thread fsck

Quoted 78 lines · Darrick J. Wong
Quoted 76 lines · Saranya Muruganandam

From: Li Xi <[email protected]>

This patch only copy the context to a new one when -m is enabled. It doesn't actually start any thread. When pass1 test finishes, the new context is copied back to the original context.

Since the signal handler only changes the original context, so add global_ctx in "struct e2fsck_struct" and use that to check whether there is any signal of canceling.

This patch handles the long jump properly so that all the existing tests can be passed even the context has been copied. Otherwise, test f_expisize_ea_del would fail when aborting.

---
 e2fsck/pass1.c | 114 +++++++++++++++++++++++++++++++++++++++++++++----
 e2fsck/unix.c  |   1 +
 2 files changed, 107 insertions(+), 8 deletions(-)
e2fsck/pass1.c +22 -3
index 8eecd958..64d237d3 100644
@@ -1144,7 +1144,22 @@ static int quota_inum_is_reserved(ext2_filsys fs, ext2_ino_t ino)
      return 0;
 }
 
-void e2fsck_pass1(e2fsck_t ctx)
+static int e2fsck_should_abort(e2fsck_t ctx)
+{
+     e2fsck_t global_ctx;
+
+     if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+             return 1;
+
+     if (ctx->global_ctx) {
+             global_ctx = ctx->global_ctx;
+             if (global_ctx->flags & E2F_FLAG_SIGNAL_MASK)
+                     return 1;
+     }
+     return 0;
+}
+
+void e2fsck_pass1_thread(e2fsck_t ctx)
 {
      int     i;
      __u64   max_sizes;
@@ -1360,7 +1375,7 @@ void e2fsck_pass1(e2fsck_t ctx)
              if (ino > ino_threshold)
                      pass1_readahead(ctx, &ra_group, &ino_threshold);
              ehandler_operation(old_op);
-             if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+             if (e2fsck_should_abort(ctx))
                      goto endit;
              if (pctx.errcode == EXT2_ET_BAD_BLOCK_IN_INODE_TABLE) {
                      /*
@@ -1955,7 +1970,7 @@ void e2fsck_pass1(e2fsck_t ctx)
              if (process_inode_count >= ctx->process_inode_size) {
                      process_inodes(ctx, block_buf);
 
-                     if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+                     if (e2fsck_should_abort(ctx))
                              goto endit;
              }
      }
@@ -2068,6 +2083,89 @@ endit:
      else
              ctx->invalid_bitmaps++;
 }
+
+static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thread_ctx)
+{
+     errcode_t       retval;
Quoted 8 lines

e2fsck/pass1.c endit: · line 2092

Quoted 3 lines
e2fsck/pass1.c +3 -0
+     e2fsck_t        thread_context;
+
+     retval = ext2fs_get_mem(sizeof(struct e2fsck_struct), &thread_context);

Hm, so I guess the strategy here is that parallel e2fsck makes per-thread copies of the ext2_filsys and e2fsck_t global contexts? And then after the threaded parts complete, each thread merges its per-thread contexts back into the global one, right?

Yes.

Quoted 9 lines

This means that we have to be careful to track which fields in those cloned contexts have been updated by the thread so that we can copy them back and not lose any data.

I'm wondering if for future maintainability it would be better to track the per-thread data in a separate structure to make it very explicit which data (sub)structures are effectively per-thread and hence don't require locking?

Maybe use a per-thread structure is better maintained, but i am not sure we could remove locking completely.

Locking is mostly used for fix, because fixing is serialized now and for some global structure which could be used seldomly but could simplify codes.

Quoted 8 lines

(I ask that mostly because I'm having a hard time figuring out which fields are supposed to be shared and which ones aren't...)

--D

Quoted 2 lines
e2fsck/pass1.c +2 -0
+     if (retval) {
+             com_err(global_ctx->program_name, retval, "while allocating memory");
#9 lore raw

Re: [RFC PATCH v3 02/61] e2fsck: copy context when using multi-thread fsck

Quoted 100 lines · Wang Shilong
Quoted 78 lines · Darrick J. Wong
Quoted 76 lines · Saranya Muruganandam

From: Li Xi <[email protected]>

This patch only copy the context to a new one when -m is enabled. It doesn't actually start any thread. When pass1 test finishes, the new context is copied back to the original context.

Since the signal handler only changes the original context, so add global_ctx in "struct e2fsck_struct" and use that to check whether there is any signal of canceling.

This patch handles the long jump properly so that all the existing tests can be passed even the context has been copied. Otherwise, test f_expisize_ea_del would fail when aborting.

---
 e2fsck/pass1.c | 114 +++++++++++++++++++++++++++++++++++++++++++++----
 e2fsck/unix.c  |   1 +
 2 files changed, 107 insertions(+), 8 deletions(-)
e2fsck/pass1.c +22 -3
index 8eecd958..64d237d3 100644
@@ -1144,7 +1144,22 @@ static int quota_inum_is_reserved(ext2_filsys fs, ext2_ino_t ino)
      return 0;
 }
 
-void e2fsck_pass1(e2fsck_t ctx)
+static int e2fsck_should_abort(e2fsck_t ctx)
+{
+     e2fsck_t global_ctx;
+
+     if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+             return 1;
+
+     if (ctx->global_ctx) {
+             global_ctx = ctx->global_ctx;
+             if (global_ctx->flags & E2F_FLAG_SIGNAL_MASK)
+                     return 1;
+     }
+     return 0;
+}
+
+void e2fsck_pass1_thread(e2fsck_t ctx)
 {
      int     i;
      __u64   max_sizes;
@@ -1360,7 +1375,7 @@ void e2fsck_pass1(e2fsck_t ctx)
              if (ino > ino_threshold)
                      pass1_readahead(ctx, &ra_group, &ino_threshold);
              ehandler_operation(old_op);
-             if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+             if (e2fsck_should_abort(ctx))
                      goto endit;
              if (pctx.errcode == EXT2_ET_BAD_BLOCK_IN_INODE_TABLE) {
                      /*
@@ -1955,7 +1970,7 @@ void e2fsck_pass1(e2fsck_t ctx)
              if (process_inode_count >= ctx->process_inode_size) {
                      process_inodes(ctx, block_buf);
 
-                     if (ctx->flags & E2F_FLAG_SIGNAL_MASK)
+                     if (e2fsck_should_abort(ctx))
                              goto endit;
              }
      }
@@ -2068,6 +2083,89 @@ endit:
      else
              ctx->invalid_bitmaps++;
 }
+
+static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thread_ctx)
+{
+     errcode_t       retval;
Quoted 8 lines

e2fsck/pass1.c endit: · line 2092

Quoted 3 lines
e2fsck/pass1.c +3 -0
+     e2fsck_t        thread_context;
+
+     retval = ext2fs_get_mem(sizeof(struct e2fsck_struct), &thread_context);

Hm, so I guess the strategy here is that parallel e2fsck makes per-thread copies of the ext2_filsys and e2fsck_t global contexts? And then after the threaded parts complete, each thread merges its per-thread contexts back into the global one, right?

Yes.

Quoted 9 lines

This means that we have to be careful to track which fields in those cloned contexts have been updated by the thread so that we can copy them back and not lose any data.

I'm wondering if for future maintainability it would be better to track the per-thread data in a separate structure to make it very explicit which data (sub)structures are effectively per-thread and hence don't require locking?

Quoted 6 lines

Maybe use a per-thread structure is better maintained, but i am not sure we could remove locking completely.

Locking is mostly used for fix, because fixing is serialized now and for some global structure which could be used seldomly but could simplify codes.

<nod> I was assuming that you'd still put a lock in the global structure and use it for data fields that aren't so frequently accessed.

--D

Quoted 8 lines
Quoted 8 lines

(I ask that mostly because I'm having a hard time figuring out which fields are supposed to be shared and which ones aren't...)

--D

Quoted 2 lines
e2fsck/pass1.c +2 -0
+     if (retval) {
+             com_err(global_ctx->program_name, retval, "while allocating memory");
#10 lore raw

RFC PATCH v3 3/61 e2fsck: copy fs when using multi-thread fsck

In reply to Saranya Muruganandam

From: Li Xi <[email protected]>

This patch only copy the fs to a new one when -m is enabled. It doesn't actually start any thread. When pass1 test finishes, the new fs is copied back to the original context.

This patch handles the fs fields in dblist, inode_map and block_map properly.

---
 e2fsck/pass1.c | 41 +++++++++++++++++++++++++++++++++++++----
 1 file changed, 37 insertions(+), 4 deletions(-)
e2fsck/pass1.c +37 -4
index 64d237d3..5b4947b0 100644
@@ -49,6 +49,8 @@
 
 #include "e2fsck.h"
 #include <ext2fs/ext2_ext_attr.h>
+/* todo remove this finally */
+#include <ext2fs/ext2fsP.h>
 #include <e2p/e2p.h>
 
 #include "problem.h"
@@ -2084,10 +2086,23 @@ endit:
 		ctx->invalid_bitmaps++;
 }
 
+static void e2fsck_pass1_copy_fs(ext2_filsys dest, ext2_filsys src)
+{
+	memcpy(dest, src, sizeof(struct struct_ext2_filsys));
+	if (dest->dblist)
+		dest->dblist->fs = dest;
+	if (dest->inode_map)
+		dest->inode_map->fs = dest;
+	if (dest->block_map)
+		dest->block_map->fs = dest;
+}
+
 static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thread_ctx)
 {
 	errcode_t	retval;
 	e2fsck_t	thread_context;
+	ext2_filsys	thread_fs;
+	ext2_filsys	global_fs = global_ctx->fs;
 
 	retval = ext2fs_get_mem(sizeof(struct e2fsck_struct), &thread_context);
 	if (retval) {
@@ -2095,18 +2110,32 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thre
 		return retval;
 	}
 	memcpy(thread_context, global_ctx, sizeof(struct e2fsck_struct));
-	thread_context->fs->priv_data = thread_context;
 	thread_context->global_ctx = global_ctx;
 
+	retval = ext2fs_get_mem(sizeof(struct struct_ext2_filsys), &thread_fs);
+	if (retval) {
+		com_err(global_ctx->program_name, retval, "while allocating memory");
+		goto out_context;
+	}
+
+	e2fsck_pass1_copy_fs(thread_fs, global_fs);
+	thread_fs->priv_data = thread_context;
+
+	thread_context->fs = thread_fs;
 	*thread_ctx = thread_context;
 	return 0;
+out_context:
+	ext2fs_free_mem(&thread_context);
+	return retval;
 }
 
 static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
 {
-	int	flags = global_ctx->flags;
+	int		flags = global_ctx->flags;
+	ext2_filsys	thread_fs = thread_ctx->fs;
+	ext2_filsys	global_fs = global_ctx->fs;
 #ifdef HAVE_SETJMP_H
-	jmp_buf	old_jmp;
+	jmp_buf		old_jmp;
 
 	memcpy(old_jmp, global_ctx->abort_loc, sizeof(jmp_buf));
 #endif
@@ -2118,7 +2147,11 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
 	global_ctx->flags |= (flags & E2F_FLAG_SIGNAL_MASK) |
 			     (global_ctx->flags & E2F_FLAG_SIGNAL_MASK);
 
-	global_ctx->fs->priv_data = global_ctx;
+	e2fsck_pass1_copy_fs(global_fs, thread_fs);
+	global_fs->priv_data = global_ctx;
+	global_ctx->fs = global_fs;
+
+	ext2fs_free_mem(&thread_ctx->fs);
 	ext2fs_free_mem(&thread_ctx);
 	return 0;
 }
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#11 lore raw

Re: [RFC PATCH v3 03/61] e2fsck: copy fs when using multi-thread fsck

Quoted 6 lines · Saranya Muruganandam

From: Li Xi <[email protected]>

This patch only copy the fs to a new one when -m is enabled. It doesn't actually start any thread. When pass1 test finishes, the new fs is copied back to the original context.

Quoted 6 lines

This patch handles the fs fields in dblist, inode_map and block_map properly.

I'm a bit surprised that we're not adding a ext2fs_clone_fs() function, but instead creating an e2fsck_pass1_copy_fs() function.

Again, what's going to happen when we need to copy the fs structure for other passes. Also, just simply copying the fs structure seems dangerous; how are we going to know which allocated substructures were created before the fs structure was clonsed (and hence can't be safely freed in a copy of the fs structure), and which substructures were allocated *after* the fs structure is cloned, in which case they need to be freed when the cloned fs structuers are released?

Perhaps the simplest thing to do is to assume that *everything* is cloned. If that results in too much memory consumed, maybe we can add a set of flags indicating which structures should *not* be freed for a particular fs structure, with the assumption that they will be freed when the top-level master fs structure is closed. There may be some potential traps with this, since in some cases a substructure can be released and re-allocated, in which case, if that substructure is shared by child fs structures, they could be left pointing at already freed copies of the data structures.

Basically, even if this works (and the fact that I saw a crash with a data structure being double-freed in the f_multithread_ok test strongly suggesting that it isn't), I'd be worried about making sure that the resulting architecture is one which is robust and maintainable, and not leaving traps for future developers when they don't realize what the assumptions are about shared substructures. Which strongly suggests that this should be a first-class aspect of libext2fs and it should be clearly documented how sharing does or doesn't work.

Cheers,

- Ted

#12 lore raw

RFC PATCH v3 4/61 e2fsck: clear icache when using multi-thread fsck

In reply to Saranya Muruganandam

From: Wang Shilong <[email protected]>

icache of fs will be rebuilt when needed, so after copying fs, icache can be inited to NULL.

---
 e2fsck/pass1.c | 37 ++++++++++++++++++++++++++++++++++---
 1 file changed, 34 insertions(+), 3 deletions(-)
e2fsck/pass1.c +34 -3
index 5b4947b0..ba513d91 100644
@@ -2086,8 +2086,10 @@ endit:
 		ctx->invalid_bitmaps++;
 }
 
-static void e2fsck_pass1_copy_fs(ext2_filsys dest, ext2_filsys src)
+static errcode_t e2fsck_pass1_copy_fs(ext2_filsys dest, ext2_filsys src)
 {
+	errcode_t	retval;
+
 	memcpy(dest, src, sizeof(struct struct_ext2_filsys));
 	if (dest->dblist)
 		dest->dblist->fs = dest;
@@ -2095,6 +2097,29 @@ static void e2fsck_pass1_copy_fs(ext2_filsys dest, ext2_filsys src)
 		dest->inode_map->fs = dest;
 	if (dest->block_map)
 		dest->block_map->fs = dest;
+
+	/* icache will be rebuilt if needed, so do not copy from @src */
+	src->icache = NULL;
+	return 0;
+}
+
+static void e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
+{
+	struct ext2_inode_cache *icache = dest->icache;
+
+	memcpy(dest, src, sizeof(struct struct_ext2_filsys));
+	if (dest->dblist)
+		dest->dblist->fs = dest;
+	if (dest->inode_map)
+		dest->inode_map->fs = dest;
+	if (dest->block_map)
+		dest->block_map->fs = dest;
+	dest->icache = icache;
+
+	if (src->icache) {
+		ext2fs_free_inode_cache(src->icache);
+		src->icache = NULL;
+	}
 }
 
 static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thread_ctx)
@@ -2118,12 +2143,18 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thre
 		goto out_context;
 	}
 
-	e2fsck_pass1_copy_fs(thread_fs, global_fs);
+	retval = e2fsck_pass1_copy_fs(thread_fs, global_fs);
+	if (retval) {
+		com_err(global_ctx->program_name, retval, "while copying fs");
+		goto out_fs;
+	}
 	thread_fs->priv_data = thread_context;
 
 	thread_context->fs = thread_fs;
 	*thread_ctx = thread_context;
 	return 0;
+out_fs:
+	ext2fs_free_mem(&thread_fs);
 out_context:
 	ext2fs_free_mem(&thread_context);
 	return retval;
@@ -2147,7 +2178,7 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
 	global_ctx->flags |= (flags & E2F_FLAG_SIGNAL_MASK) |
 			     (global_ctx->flags & E2F_FLAG_SIGNAL_MASK);
 
-	e2fsck_pass1_copy_fs(global_fs, thread_fs);
+	e2fsck_pass1_merge_fs(global_fs, thread_fs);
 	global_fs->priv_data = global_ctx;
 	global_ctx->fs = global_fs;
 
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#13 lore raw

Re: [RFC PATCH v3 04/61] e2fsck: clear icache when using multi-thread fsck

Quoted 8 lines · Saranya Muruganandam

From: Wang Shilong <[email protected]>

icache of fs will be rebuilt when needed, so after copying fs, icache can be inited to NULL.

So this is part of the whole "how is the fs structure is shared question". It's going to be better if this is all in a single patch, instead of being spread out into separate patches. Certainly from a review perspective, it's going to be much simpler....

Quoted 8 lines
e2fsck/pass1.c +8 -0
+static void e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
+{
+	struct ext2_inode_cache *icache = dest->icache;
+
+	memcpy(dest, src, sizeof(struct struct_ext2_filsys));
+	if (dest->dblist)
+		dest->dblist->fs = dest;
+	if (dest->inode_map)

e2fsck/pass1.c static void e2fsck_pass1_copy_fs(ext2_filsys dest, ext2_filsys src) · line 2123

Quoted 10 lines
e2fsck/pass1.c +9 -0
+		dest->inode_map->fs = dest;
+	if (dest->block_map)
+		dest->block_map->fs = dest;
+	dest->icache = icache;
+
+	if (src->icache) {
+		ext2fs_free_inode_cache(src->icache);
+		src->icache = NULL;
+	}
 }

I'm not sure how the merge_fs function is supposed to work. Right now it's not doing much in the way of merging; I'm guessing how it works is going to be done in a future patch, but this is going to make review a bit tricky.

.... peaking ahead in the patch series. Yeah, I think we need to think very carefully about how we do stuff here. It looks very ad hoc, with some aspects of the merge functionality in e2fsck/pass1.c, and some in libext2fs.

I wonder if we wouldn't be better off with the concept of a top-level fs structure, and child fs structures which point back to the top-level fs structure --- much like what is currently being done with the e2fsck_t structure. So if fs->parent_fs is non-NULL, we know that this is a child fs structure, and there are very clear rules about how things like io_managers are shared --- or not. Note that some io_managers, such as undo io manager, fundamentally *have* to be shared, which implies that we need to have some kind of flag indicating whether or not an io_manager is thread-aware or not. And if an io_manager is not thread-aware, then it wouldn't be safe to use it in multi-threading mode.

I'm going to stop doing a fine-grained review of this patch series, and it seems pretty clear to me that we need to have some fundamental architectural discussions about how to make multi-threading work. An ad-hoc approach is great for a prototype, as this surfaces all of these sorts of questions --- but I think we need to now stop, and think very carefully about how to make multi-threading support something which can be a long-term maintainable code base. And that means figuring out what the semantics should be about various shared data sructures, and then documenting how things are supposed to work very explicitly. (I note that many of these functions don't have much in the way of documentation, which is unfortunate from the prespective of some future developer who is wishing to further extend this work.)

- Ted

#14 lore raw

RFC PATCH v3 5/61 e2fsck: add assert when copying context

In reply to Saranya Muruganandam

From: Li Xi <[email protected]>

Adding the assert would simplify the copying of context.

---
 e2fsck/pass1.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
e2fsck/pass1.c +14 -0
index ba513d91..d16bedd3 100644
@@ -46,6 +46,7 @@
 #ifdef HAVE_ERRNO_H
 #include <errno.h>
 #endif
+#include <assert.h>
 
 #include "e2fsck.h"
 #include <ext2fs/ext2_ext_attr.h>
@@ -2129,6 +2130,19 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thre
 	ext2_filsys	thread_fs;
 	ext2_filsys	global_fs = global_ctx->fs;
 
+	assert(global_ctx->inode_used_map == NULL);
+	assert(global_ctx->inode_dir_map == NULL);
+	assert(global_ctx->inode_bb_map == NULL);
+	assert(global_ctx->inode_imagic_map == NULL);
+	assert(global_ctx->inode_reg_map == NULL);
+	assert(global_ctx->inodes_to_rebuild == NULL);
+
+	assert(global_ctx->block_found_map == NULL);
+	assert(global_ctx->block_dup_map == NULL);
+	assert(global_ctx->block_ea_map == NULL);
+	assert(global_ctx->block_metadata_map == NULL);
+	assert(global_ctx->fs->dblist == NULL);
+
 	retval = ext2fs_get_mem(sizeof(struct e2fsck_struct), &thread_context);
 	if (retval) {
 		com_err(global_ctx->program_name, retval, "while allocating memory");
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#15 lore raw

RFC PATCH v3 6/61 e2fsck: copy bitmaps when copying context

In reply to Saranya Muruganandam

From: Li Xi <[email protected]>

This patch copies bitmap when the copying context. In the multi-thread fsck, each thread use different bitmap that copied from the glboal bitmap. And Bitmaps from multiple threads will be merged into a global one after the pass1 finishes.

---
 e2fsck/pass1.c | 151 +++++++++++++++++++++++++++++++++++++++++++++----
 1 file changed, 139 insertions(+), 12 deletions(-)
e2fsck/pass1.c +139 -12
index d16bedd3..3a4286e1 100644
@@ -2087,6 +2087,22 @@ endit:
 		ctx->invalid_bitmaps++;
 }
 
+static errcode_t e2fsck_pass1_copy_bitmap(ext2_filsys fs, ext2fs_generic_bitmap *src,
+					  ext2fs_generic_bitmap *dest)
+{
+	errcode_t ret;
+
+	ret = ext2fs_copy_bitmap(*src, dest);
+	if (ret)
+		return ret;
+
+	(*dest)->fs = fs;
+	ext2fs_free_generic_bmap(*src);
+	*src = NULL;
+
+	return 0;
+}
+
 static errcode_t e2fsck_pass1_copy_fs(ext2_filsys dest, ext2_filsys src)
 {
 	errcode_t	retval;
@@ -2094,33 +2110,52 @@ static errcode_t e2fsck_pass1_copy_fs(ext2_filsys dest, ext2_filsys src)
 	memcpy(dest, src, sizeof(struct struct_ext2_filsys));
 	if (dest->dblist)
 		dest->dblist->fs = dest;
-	if (dest->inode_map)
-		dest->inode_map->fs = dest;
-	if (dest->block_map)
-		dest->block_map->fs = dest;
+	if (src->block_map) {
+		retval = e2fsck_pass1_copy_bitmap(dest, &src->block_map,
+						  &dest->block_map);
+		if (retval)
+			return retval;
+	}
+	if (src->inode_map) {
+		retval = e2fsck_pass1_copy_bitmap(dest, &src->inode_map,
+						  &dest->inode_map);
+		if (retval)
+			return retval;
+	}
 
 	/* icache will be rebuilt if needed, so do not copy from @src */
 	src->icache = NULL;
 	return 0;
 }
 
-static void e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
+static int e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
 {
 	struct ext2_inode_cache *icache = dest->icache;
+	errcode_t	retval = 0;
 
 	memcpy(dest, src, sizeof(struct struct_ext2_filsys));
 	if (dest->dblist)
 		dest->dblist->fs = dest;
-	if (dest->inode_map)
-		dest->inode_map->fs = dest;
-	if (dest->block_map)
-		dest->block_map->fs = dest;
+	if (src->inode_map) {
+		retval = e2fsck_pass1_copy_bitmap(dest, &src->inode_map,
+						  &dest->inode_map);
+		if (retval)
+			return retval;
+	}
+	if (src->block_map) {
+		retval = e2fsck_pass1_copy_bitmap(dest, &src->block_map,
+						  &dest->block_map);
+		if (retval)
+			return retval;
+	}
 	dest->icache = icache;
 
 	if (src->icache) {
 		ext2fs_free_inode_cache(src->icache);
 		src->icache = NULL;
 	}
+
+	return retval;
 }
 
 static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thread_ctx)
@@ -2174,8 +2209,9 @@ out_context:
 	return retval;
 }
 
-static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
+static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx)
 {
+	errcode_t	retval;
 	int		flags = global_ctx->flags;
 	ext2_filsys	thread_fs = thread_ctx->fs;
 	ext2_filsys	global_fs = global_ctx->fs;
@@ -2192,13 +2228,104 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
 	global_ctx->flags |= (flags & E2F_FLAG_SIGNAL_MASK) |
 			     (global_ctx->flags & E2F_FLAG_SIGNAL_MASK);
 
-	e2fsck_pass1_merge_fs(global_fs, thread_fs);
+	retval = e2fsck_pass1_merge_fs(global_fs, thread_fs);
+	if (retval) {
+		com_err(global_ctx->program_name, 0, _("while merging fs\n"));
+		return retval;
+	}
 	global_fs->priv_data = global_ctx;
 	global_ctx->fs = global_fs;
 
+	if (thread_ctx->inode_used_map) {
+		retval = e2fsck_pass1_copy_bitmap(global_fs,
+					&thread_ctx->inode_used_map,
+					&global_ctx->inode_used_map);
+		if (retval)
+			return retval;
+	}
+	if (thread_ctx->inode_bad_map) {
+		retval = e2fsck_pass1_copy_bitmap(global_fs,
+					&thread_ctx->inode_bad_map,
+					&global_ctx->inode_bad_map);
+		if (retval)
+			return retval;
+	}
+	if (thread_ctx->inode_dir_map) {
+		retval = e2fsck_pass1_copy_bitmap(global_fs,
+					&thread_ctx->inode_dir_map,
+					&global_ctx->inode_dir_map);
+		if (retval)
+			return retval;
+	}
+	if (thread_ctx->inode_bb_map) {
+		retval = e2fsck_pass1_copy_bitmap(global_fs,
+					&thread_ctx->inode_bb_map,
+					&global_ctx->inode_bb_map);
+		if (retval)
+			return retval;
+	}
+	if (thread_ctx->inode_imagic_map) {
+		retval = e2fsck_pass1_copy_bitmap(global_fs,
+					&thread_ctx->inode_imagic_map,
+					&global_ctx->inode_imagic_map);
+		if (retval)
+			return retval;
+	}
+	if (thread_ctx->inode_reg_map) {
+		retval = e2fsck_pass1_copy_bitmap(global_fs,
+					&thread_ctx->inode_reg_map,
+					&global_ctx->inode_reg_map);
+		if (retval)
+			return retval;
+	}
+	if (thread_ctx->inodes_to_rebuild) {
+		retval = e2fsck_pass1_copy_bitmap(global_fs,
+					&thread_ctx->inodes_to_rebuild,
+					&global_ctx->inodes_to_rebuild);
+		if (retval)
+			return retval;
+	}
+	if (thread_ctx->block_found_map) {
+		retval = e2fsck_pass1_copy_bitmap(global_fs,
+					&thread_ctx->block_found_map,
+					&global_ctx->block_found_map);
+		if (retval)
+			return retval;
+	}
+	if (thread_ctx->block_dup_map) {
+		retval = e2fsck_pass1_copy_bitmap(global_fs,
+					&thread_ctx->block_dup_map,
+					&global_ctx->block_dup_map);
+		if (retval)
+			return retval;
+	}
+	if (thread_ctx->block_ea_map) {
+		retval = e2fsck_pass1_copy_bitmap(global_fs,
+					&thread_ctx->block_ea_map,
+					&global_ctx->block_ea_map);
+		if (retval)
+			return retval;
+	}
+	if (thread_ctx->block_metadata_map) {
+		retval = e2fsck_pass1_copy_bitmap(global_fs,
+					&thread_ctx->block_metadata_map,
+					&global_ctx->block_metadata_map);
+		if (retval)
+			return retval;
+	}
+
+	return 0;
+}
+
+static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
+{
+	errcode_t	retval;
+
+	retval = e2fsck_pass1_thread_join_one(global_ctx, thread_ctx);
 	ext2fs_free_mem(&thread_ctx->fs);
 	ext2fs_free_mem(&thread_ctx);
-	return 0;
+
+	return retval;
 }
 
 void e2fsck_pass1_multithread(e2fsck_t ctx)
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#16 lore raw

RFC PATCH v3 7/61 e2fsck: copy badblocks when copying fs

In reply to Saranya Muruganandam

From: Wang Shilong <[email protected]>

This patch copies badblocks when the copying fs.

---
 e2fsck/pass1.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)
e2fsck/pass1.c +13 -0
index 3a4286e1..14508dd8 100644
@@ -2123,6 +2123,12 @@ static errcode_t e2fsck_pass1_copy_fs(ext2_filsys dest, ext2_filsys src)
 			return retval;
 	}
 
+	if (src->badblocks) {
+		retval = ext2fs_badblocks_copy(src->badblocks, &dest->badblocks);
+		if (retval)
+			return retval;
+	}
+
 	/* icache will be rebuilt if needed, so do not copy from @src */
 	src->icache = NULL;
 	return 0;
@@ -2155,6 +2161,13 @@ static int e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
 		src->icache = NULL;
 	}
 
+	if (src->badblocks) {
+		retval = ext2fs_badblocks_copy(src->badblocks, &dest->badblocks);
+
+		ext2fs_badblocks_list_free(src->badblocks);
+		src->badblocks = NULL;
+	}
+
 	return retval;
 }
 
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#17 lore raw

RFC PATCH v3 8/61 e2fsck: open io-channel when copying fs

In reply to Saranya Muruganandam

From: Li Xi <[email protected]>

This patch also add writethrough flag to the thread io-channel. When multiple threads write the same disk, we don't want the data being saved in memory cache. This will be useful in the future, but even without that flag, the tests can be passed too.

This patch also cleanup the io channel cache of the global context. Otherwise, after pass1 step, the next steps would use old data saved in the cache. And the cached data might have already been overwritten in pass1.

---
 e2fsck/e2fsck.h      |  3 +++
 e2fsck/pass1.c       | 46 ++++++++++++++++++++++++++++++++++++++----
 e2fsck/unix.c        |  1 +
 lib/ext2fs/ext2_io.h |  2 ++
 lib/ext2fs/ext2fs.h  |  3 +++
 lib/ext2fs/openfs.c  | 48 +++++++++++++++++++++++++++++++-------------
 lib/ext2fs/undo_io.c | 19 ++++++++++++++++++
 lib/ext2fs/unix_io.c | 24 +++++++++++++++++++---
 8 files changed, 125 insertions(+), 21 deletions(-)
e2fsck/e2fsck.h +3 -0
index e2784248..1416f15e 100644
@@ -228,6 +228,8 @@ typedef struct e2fsck_struct *e2fsck_t;
 #define MAX_EXTENT_DEPTH_COUNT 5
 
 struct e2fsck_struct {
+	/* Global context to get the cancel flag */
+	e2fsck_t		global_ctx;
 	ext2_filsys fs;
 	const char *program_name;
 	char *filesystem_name;
@@ -247,6 +249,7 @@ struct e2fsck_struct {
 	ext2_ino_t free_inodes;
 	int	mount_flags;
 	int	openfs_flags;
+	io_manager io_manager;
 	blkid_cache blkid;	/* blkid cache */
 
 #ifdef HAVE_SETJMP_H
e2fsck/pass1.c +42 -4

84 lines not on this page · raw message

e2fsck/unix.c +1 -0
index 42f616e2..f973af62 100644
@@ -1525,6 +1525,7 @@ restart:
 	}
 
 	ctx->openfs_flags = flags;
+	ctx->io_manager = io_ptr;
 	retval = try_open_fs(ctx, flags, io_ptr, &fs);
 
 	if (!ctx->superblock && !(ctx->options & E2F_OPT_PREEN) &&
lib/ext2fs/ext2_io.h +2 -0
index 5540900a..4ad2fec8 100644
@@ -81,6 +81,7 @@ struct struct_io_manager {
 	errcode_t (*write_blk)(io_channel channel, unsigned long block,
 			       int count, const void *data);
 	errcode_t (*flush)(io_channel channel);
+	errcode_t (*flush_cleanup)(io_channel channel);
 	errcode_t (*write_byte)(io_channel channel, unsigned long offset,
 				int count, const void *data);
 	errcode_t (*set_option)(io_channel channel, const char *option,
@@ -113,6 +114,7 @@ struct struct_io_manager {
 #define io_channel_read_blk(c,b,n,d)	((c)->manager->read_blk((c),b,n,d))
 #define io_channel_write_blk(c,b,n,d)	((c)->manager->write_blk((c),b,n,d))
 #define io_channel_flush(c) 		((c)->manager->flush((c)))
+#define io_channel_flush_cleanup(c) 	((c)->manager->flush_cleanup((c)))
 #define io_channel_bumpcount(c)		((c)->refcount++)
 
 /* io_manager.c */
lib/ext2fs/ext2fs.h +3 -0
index 69c8a3ff..5b76d02e 100644
@@ -1643,6 +1643,9 @@ extern errcode_t ext2fs_open2(const char *name, const char *io_options,
 			      int flags, int superblock,
 			      unsigned int block_size, io_manager manager,
 			      ext2_filsys *ret_fs);
+errcode_t ext2fs_open_channel(ext2_filsys fs, const char *io_options,
+			      io_manager manager, int flags,
+			      int blocksize);
 /*
  * The dgrp_t argument to these two functions is not actually a group number
  * but a block number offset within a group table!  Convert with the formula
lib/ext2fs/openfs.c +34 -14

68 lines not on this page · raw message

lib/ext2fs/undo_io.c +19 -0
index 19862414..1391a534 100644
@@ -1020,6 +1020,24 @@ static errcode_t undo_flush(io_channel channel)
 	return retval;
 }
 
+/*
+ * Flush data buffers to disk and cleanup the cache.
+ */
+static errcode_t undo_flush_cleanup(io_channel channel)
+{
+	errcode_t	retval = 0;
+	struct undo_private_data *data;
+
+	EXT2_CHECK_MAGIC(channel, EXT2_ET_MAGIC_IO_CHANNEL);
+	data = (struct undo_private_data *) channel->private_data;
+	EXT2_CHECK_MAGIC(data, EXT2_ET_MAGIC_UNIX_IO_CHANNEL);
+
+	if (data->real)
+		retval = io_channel_flush_cleanup(data->real);
+
+	return retval;
+}
+
 static errcode_t undo_set_option(io_channel channel, const char *option,
 				 const char *arg)
 {
@@ -1091,6 +1109,7 @@ static struct struct_io_manager struct_undo_manager = {
 	.read_blk	= undo_read_blk,
 	.write_blk	= undo_write_blk,
 	.flush		= undo_flush,
+	.flush_cleanup	= undo_flush_cleanup,
 	.write_byte	= undo_write_byte,
 	.set_option	= undo_set_option,
 	.get_stats	= undo_get_stats,
lib/ext2fs/unix_io.c +21 -3
index 628e60c3..c9defd4b 100644
@@ -1030,9 +1030,9 @@ static errcode_t unix_write_byte(io_channel channel, unsigned long offset,
 }
 
 /*
- * Flush data buffers to disk.
+ * Flush data buffers to disk and invalidate cache if needed
  */
-static errcode_t unix_flush(io_channel channel)
+static errcode_t _unix_flush(io_channel channel, int invalidate)
 {
 	struct unix_private_data *data;
 	errcode_t retval = 0;
@@ -1042,7 +1042,7 @@ static errcode_t unix_flush(io_channel channel)
 	EXT2_CHECK_MAGIC(data, EXT2_ET_MAGIC_UNIX_IO_CHANNEL);
 
 #ifndef NO_IO_CACHE
-	retval = flush_cached_blocks(channel, data, 0);
+	retval = flush_cached_blocks(channel, data, invalidate);
 #endif
 #ifdef HAVE_FSYNC
 	if (!retval && fsync(data->dev) != 0)
@@ -1051,6 +1051,22 @@ static errcode_t unix_flush(io_channel channel)
 	return retval;
 }
 
+/*
+ * Flush data buffers to disk.
+ */
+static errcode_t unix_flush(io_channel channel)
+{
+	return _unix_flush(channel, 0);
+}
+
+/*
+ * Flush data buffers to disk and invalidate cache.
+ */
+static errcode_t unix_flush_cleanup(io_channel channel)
+{
+	return _unix_flush(channel, 1);
+}
+
 static errcode_t unix_set_option(io_channel channel, const char *option,
 				 const char *arg)
 {
@@ -1225,6 +1241,7 @@ static struct struct_io_manager struct_unix_manager = {
 	.discard	= unix_discard,
 	.cache_readahead	= unix_cache_readahead,
 	.zeroout	= unix_zeroout,
+	.flush_cleanup	= unix_flush_cleanup,
 };
 
 io_manager unix_io_manager = &struct_unix_manager;
@@ -1246,6 +1263,7 @@ static struct struct_io_manager struct_unixfd_manager = {
 	.discard	= unix_discard,
 	.cache_readahead	= unix_cache_readahead,
 	.zeroout	= unix_zeroout,
+	.flush_cleanup	= unix_flush_cleanup,
 };
 
 io_manager unixfd_io_manager = &struct_unixfd_manager;
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#18 lore raw

Re: [RFC PATCH v3 08/61] e2fsck: open io-channel when copying fs

Quoted 11 lines · Saranya Muruganandam

From: Li Xi <[email protected]>

This patch also add writethrough flag to the thread io-channel. When multiple threads write the same disk, we don't want the data being saved in memory cache. This will be useful in the future, but even without that flag, the tests can be passed too.

This patch also cleanup the io channel cache of the global context. Otherwise, after pass1 step, the next steps would use old data saved in the cache. And the cached data might have already been overwritten in pass1.

See my previous comments about why io_managers will almost certainly need to be thread-aware. This commit modifies undo_io.c, but it can't possibly work as-is, since you can't have multiple copies of the undo manager from different threads trying to update a single undo file. So these changes, by themselves, can't possibly be sufficient.

Instead of doing things incrementally, my suggestion is that we figure out how to make io_managers thread-safe and we get it right *first*, instead of making incremental changes throughout the patch series. I'd also suggest that we figure out some kind of test framework so we can test io_managers in isolation, so we can do stress tests, as well as functional correctness tests as unit tests.

Once we have that, let's merge these changes in incrementally into e2fsprogs, so we can have clean, well-designed and well-tested low-level infrastructure, which will be easier for us to review.

lib/ext2fs/ext2_io.h struct struct_io_manager { · line 87

Quoted 12 lines
lib/ext2fs/ext2_io.h +1 -0
index 5540900a..4ad2fec8 100644
@@ -81,6 +81,7 @@ struct struct_io_manager {
 	errcode_t (*write_blk)(io_channel channel, unsigned long block,
 			       int count, const void *data);
 	errcode_t (*flush)(io_channel channel);
+	errcode_t (*flush_cleanup)(io_channel channel);
 	errcode_t (*write_byte)(io_channel channel, unsigned long offset,
 				int count, const void *data);
 	errcode_t (*set_option)(io_channel channel, const char *option,

Please don't add new functions into the middle of struct_io_manager. There is a long reserved[14] to add padding to the structure for a reason. Add a new function pointer just before the reserved[] array, and then decrement the padding count.

The reason for this is that it's technically allowed for applications to provide their own io_manager to the library, which may be stacked on top of an some other io_manager (as is the case for undo iomanager). Or it might because the userspace application is providing their own io manager to interface with some other OS --- maybe Windows, or Fuschia in the future, who knows?

So if we add a new function pointer to the middle of struct_io_manager, this breaks the ABI, and that's a Bad Thing, as it may cause surprises in the future for applications which are using shared libraries and we update with a newer version of the shared library without doing a major version bump of the shared library.

- Ted

#19 lore raw

Re: [RFC PATCH v3 08/61] e2fsck: open io-channel when copying fs

Quoted 8 lines · Theodore Y. Ts'o
Quoted 7 lines · Saranya Muruganandam

From: Li Xi <[email protected]>

This patch also add writethrough flag to the thread io-channel. When multiple threads write the same disk, we don't want the data being saved in memory cache. This will be useful in the future, but even without that flag, the tests can be passed too.

Quoted 4 lines
Quoted 4 lines

This patch also cleanup the io channel cache of the global context. Otherwise, after pass1 step, the next steps would use old data saved in the cache. And the cached data might have already been overwritten in pass1.

Thinking about this so more, what I'd suggest is the following.

1) We define a new flag, IO_FLAG_THREADED, which instructs the io_manager that it should provide thread-safety. For all io managers defined in libext2fs, if they do not support the flag, they will return a newly defined error code, EXT2_ET_IO_CHANNEL_NO_SUPPORT_FLAG. Ideally, all of them should support IO_FLAG_THREADED, even undo_io, even if that means just throwing a pthread rwlock around all of its read/write operations.

2) We also define a two options which can be set via io_channel_set_option(), so that the "writethrough" and "cache" options can be set to on or off via "writethrough=on", "cache=off", "cache=on", etc. This allows parallel reads to the unix io manager to avoid needing any locks --- since pthread doesn't support spinlocks, and mutexes are going to be rather expensive for highly contended operations such as parallel read operations when reading bitmaps.

(In fact, some of the bugs that I saw when playing with the parallel bitmap reading was probably cacused by the fact that there were multiple threads trying to play with unix_io's cache without any locking protection; and there's no real benefit to having any kind of caching when reading the block bitmaps, and if we spin off separate io_channels for each thread while reading the block bitmaps, the cache will be *purely* useless. The cache also doesn't do us much good when we are reading the bitmaps, or during pass 1, even if we aren't multi-threading, so having a way to turn off the cache is probably a mild performance improvement even in the non-threaded case.)

And as I think I mentioned earlier, let's do some of these preparatory changes to support parallel fsck *first*, since it'll be a lot easier to review a bunch of shorter, patch series which are less than, say, 3-6 patches at a time, and once those get in, we can do with the next step --- as opposed to a monster set of 61 commits.

- Ted

P.S. Finally, if we use a single shared io_manager which is threading aware, I don't think we'll need the flush_cleanup() operation proposed in this patch, since the cache will always be consistent.

#20 lore raw

RFC PATCH v3 9/61 e2fsck: create logs for mult-threads

In reply to Saranya Muruganandam

From: Li Xi <[email protected]>

When multi-threads are used, different logs should be created for different threads. Each thread has log files with suffix of ".$THREAD_INDEX".

And this patch adds f_multithread_logfile test case.

---
 e2fsck/e2fsck.h                      |  6 ++++--
 e2fsck/logfile.c                     | 10 ++++++++-
 e2fsck/pass1.c                       | 23 +++++++++++++++-----
 tests/f_multithread_logfile/expect.1 | 23 ++++++++++++++++++++
 tests/f_multithread_logfile/image.gz |  1 +
 tests/f_multithread_logfile/name     |  1 +
 tests/f_multithread_logfile/script   | 32 ++++++++++++++++++++++++++++
 7 files changed, 88 insertions(+), 8 deletions(-)
 create mode 100644 tests/f_multithread_logfile/expect.1
 create mode 120000 tests/f_multithread_logfile/image.gz
 create mode 100644 tests/f_multithread_logfile/name
 create mode 100644 tests/f_multithread_logfile/script
e2fsck/e2fsck.h +4 -2
index 1416f15e..5ad0fe93 100644
@@ -308,8 +308,10 @@ struct e2fsck_struct {
 	/*
 	 * For pass1_check_directory and pass1_get_blocks
 	 */
-	ext2_ino_t stashed_ino;
-	struct ext2_inode *stashed_inode;
+	ext2_ino_t		stashed_ino;
+	struct ext2_inode	*stashed_inode;
+	/* Thread index, if global_ctx is null, this field is unused */
+	int			thread_index;
 
 	/*
 	 * Location of the lost and found directory
e2fsck/logfile.c +9 -1
index 63e9a12f..c5505d27 100644
@@ -16,6 +16,7 @@
 #include <sys/types.h>
 #include <sys/stat.h>
 #include <fcntl.h>
+#include <assert.h>
 
 #include "e2fsck.h"
 #include <pwd.h>
@@ -291,6 +292,8 @@ static FILE *set_up_log_file(e2fsck_t ctx, const char *key, const char *fn)
 	struct string s, s1, s2;
 	char *s0 = 0, *log_dir = 0, *log_fn = 0;
 	int log_dir_wait = 0;
+	int string_size;
+	char string_index[10];
 
 	s.s = s1.s = s2.s = 0;
 
@@ -307,6 +310,12 @@ static FILE *set_up_log_file(e2fsck_t ctx, const char *key, const char *fn)
 		goto out;
 
 	expand_logfn(ctx, log_fn, &s);
+	if (ctx->global_ctx) {
+		sprintf(string_index, "%d", ctx->thread_index);
+		append_string(&s, ".", 1);
+		append_string(&s, string_index, 0);
+	}
+
 	if ((log_fn[0] == '/') || !log_dir || !log_dir[0])
 		s0 = s.s;
 
@@ -325,7 +334,6 @@ static FILE *set_up_log_file(e2fsck_t ctx, const char *key, const char *fn)
 		append_string(&s2, log_dir, 0);
 		append_string(&s2, "/", 1);
 		append_string(&s2, s.s, 0);
-		printf("%s\n", s2.s);
 	}
 
 	if (s0)
e2fsck/pass1.c +18 -5
index 10efa0ed..bae47a7f 100644
@@ -2250,6 +2250,9 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thre
 	}
 	thread_fs->priv_data = thread_context;
 
+	thread_context->thread_index = 0;
+	set_up_logging(thread_context);
+
 	thread_context->fs = thread_fs;
 	*thread_ctx = thread_context;
 	return 0;
@@ -2262,12 +2265,14 @@ out_context:
 
 static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx)
 {
-	errcode_t	retval;
-	int		flags = global_ctx->flags;
-	ext2_filsys	thread_fs = thread_ctx->fs;
-	ext2_filsys	global_fs = global_ctx->fs;
+	errcode_t	 retval;
+	int		 flags = global_ctx->flags;
+	ext2_filsys	 thread_fs = thread_ctx->fs;
+	ext2_filsys	 global_fs = global_ctx->fs;
+	FILE		*global_logf = global_ctx->logf;
+	FILE		*global_problem_logf = global_ctx->problem_logf;
 #ifdef HAVE_SETJMP_H
-	jmp_buf		old_jmp;
+	jmp_buf		 old_jmp;
 
 	memcpy(old_jmp, global_ctx->abort_loc, sizeof(jmp_buf));
 #endif
@@ -2286,6 +2291,8 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
 	}
 	global_fs->priv_data = global_ctx;
 	global_ctx->fs = global_fs;
+	global_ctx->logf = global_logf;
+	global_ctx->problem_logf = global_problem_logf;
 
 	if (thread_ctx->inode_used_map) {
 		retval = e2fsck_pass1_copy_bitmap(global_fs,
@@ -2374,6 +2381,12 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
 
 	retval = e2fsck_pass1_thread_join_one(global_ctx, thread_ctx);
 	ext2fs_free_mem(&thread_ctx->fs);
+	if (thread_ctx->logf)
+		fclose(thread_ctx->logf);
+	if (thread_ctx->problem_logf) {
+		fputs("</problem_log>\n", thread_ctx->problem_logf);
+		fclose(thread_ctx->problem_logf);
+	}
 	ext2fs_free_mem(&thread_ctx);
 
 	return retval;
tests/f_multithread_logfile/expect.1 new file +23 -0
new file mode 100644
index 00000000..e2b954d0
@@ -0,0 +1,23 @@
+ext2fs_open2: Bad magic number in super-block
+../e2fsck/e2fsck: Superblock invalid, trying backup blocks...
+Pass 1: Checking inodes, blocks, and sizes
+Pass 2: Checking directory structure
+Pass 3: Checking directory connectivity
+Pass 4: Checking reference counts
+Pass 5: Checking group summary information
+Free blocks count wrong for group #0 (7987, counted=7982).
+Fix? yes
+
+Free blocks count wrong (11602, counted=11597).
+Fix? yes
+
+Free inodes count wrong for group #0 (1493, counted=1488).
+Fix? yes
+
+Free inodes count wrong (2997, counted=2992).
+Fix? yes
+
+
+test_filesys: ***** FILE SYSTEM WAS MODIFIED *****
+test_filesys: 16/3008 files (0.0% non-contiguous), 403/12000 blocks
+Exit status is 1
tests/f_multithread_logfile/image.gz new file +1 -0
new file mode 120000
index 00000000..0fd40018
@@ -0,0 +1 @@
+../f_zero_super/image.gz
\ No newline at end of file
tests/f_multithread_logfile/name new file +1 -0
new file mode 100644
index 00000000..faaabc3b
@@ -0,0 +1 @@
+test "e2fsck -m" option works with "-E log_filename="
tests/f_multithread_logfile/script new file +32 -0
new file mode 100644
index 00000000..4f9ca6f8
@@ -0,0 +1,32 @@
+LOG_FNAME="f_multithread_logfile_xxx"
+FSCK_OPT="-fy -m -y -E log_filename=$LOG_FNAME"
+SKIP_VERIFY="true"
+ONE_PASS_ONLY="true"
+SKIP_CLEANUP="true"
+
+rm -f $LOG_FNAME.* $LOG_FNAME
+
+. $cmd_dir/run_e2fsck
+
+rm -f $test_name.ok $test_name.failed
+cmp -s $OUT1 $EXP1
+status1=$?
+
+if [ "$status1" -eq 0 ]; then
+	if [ ! -f $LOG_FNAME -o ! -f $LOG_FNAME.0 ]; then
+		echo "$LOG_FNAME or $LOG_FNAME.0 is not created" > $test_name.failed
+		echo "$test_name: $test_description: failed"
+	else
+		echo "$test_name: $test_description: ok"
+		touch $test_name.ok
+	fi
+else
+	diff $DIFF_OPTS $test_dir/expect.1 \
+		$test_name.1.log >> $test_name.failed
+        echo "$test_name: $test_description: failed"
+fi
+
+unset IMAGE FSCK_OPT SECOND_FSCK_OPT OUT1 OUT2 EXP1 EXP2
+unset SKIP_VERIFY SKIP_CLEANUP SKIP_GUNZIP ONE_PASS_ONLY PREP_CMD
+unset DESCRIPTION SKIP_UNLINK AFTER_CMD PASS_ZERO
+unset LOG_FINAME
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#21 lore raw

Re: [RFC PATCH v3 09/61] e2fsck: create logs for mult-threads

Quoted 7 lines · Saranya Muruganandam

From: Li Xi <[email protected]>

When multi-threads are used, different logs should be created for different threads. Each thread has log files with suffix of ".$THREAD_INDEX".

And this patch adds f_multithread_logfile test case.

I'm not convinced this is the best approach; why not add a (thread-aware) log abstraction? That way there can be a single logfile, which is going to be much more user/admin/SRE-friendly.

We also need to add some kind of mutex if we are multi-threading in e2fsck/message.c, since POSIX guarantees that the dio functions are thread-safe --- but only on a per function basis. So if we don't want the output to the terminal to potentially be scrambled, we need to take a mutex in print_e2fsck_message() if we are in multi-threading mode, so that only one thread is printing to the terminal at a time.

(And certainly if we are going to be asking a question of the user, we *definitely* need to be taking some kind of mutex first!)

- Ted

#22 lore raw

RFC PATCH v3 10/61 e2fsck: optionally configure one pfsck thread

In reply to Saranya Muruganandam

From: Li Xi <[email protected]>

This patch creates only one thread to do pass1 check. The same codes can be used to create multiple threads, but other functions need to be modified to get ready for that.

pfsck support will be enabled with if configured with --enable-pfsck option.

---
 configure         |  81 +++++++++++++++++++-----
 configure.ac      |  19 ++++++
 e2fsck/e2fsck.h   |  13 ++++
 e2fsck/logfile.c  |   2 +
 e2fsck/pass1.c    | 153 ++++++++++++++++++++++++++++++++++++++++------
 e2fsck/unix.c     |  10 +++
 lib/config.h.in   |   3 +
 tests/test_one.in |   8 +++
 8 files changed, 256 insertions(+), 33 deletions(-)
configure +66 -15
index d90188af..1bb7a325 100755
@@ -755,6 +755,7 @@ SET_MAKE
 VERSION
 PACKAGE
 GETTEXT_PACKAGE
+PTHREAD_LIB
 TDB_MAN_COMMENT
 TDB_CMT
 UUIDD_CMT
@@ -847,7 +848,6 @@ infodir
 docdir
 oldincludedir
 includedir
-runstatedir
 localstatedir
 sharedstatedir
 sysconfdir
@@ -912,6 +912,7 @@ enable_mmp
 enable_tdb
 enable_bmap_stats
 enable_bmap_stats_ops
+enable_pfsck
 enable_nls
 enable_threads
 with_gnu_ld
@@ -984,7 +985,6 @@ datadir='${datarootdir}'
 sysconfdir='${prefix}/etc'
 sharedstatedir='${prefix}/com'
 localstatedir='${prefix}/var'
-runstatedir='${localstatedir}/run'
 includedir='${prefix}/include'
 oldincludedir='/usr/include'
 docdir='${datarootdir}/doc/${PACKAGE}'
@@ -1237,15 +1237,6 @@ do
   | -silent | --silent | --silen | --sile | --sil)
     silent=yes ;;
 
-  -runstatedir | --runstatedir | --runstatedi | --runstated \
-  | --runstate | --runstat | --runsta | --runst | --runs \
-  | --run | --ru | --r)
-    ac_prev=runstatedir ;;
-  -runstatedir=* | --runstatedir=* | --runstatedi=* | --runstated=* \
-  | --runstate=* | --runstat=* | --runsta=* | --runst=* | --runs=* \
-  | --run=* | --ru=* | --r=*)
-    runstatedir=$ac_optarg ;;
-
   -sbindir | --sbindir | --sbindi | --sbind | --sbin | --sbi | --sb)
     ac_prev=sbindir ;;
   -sbindir=* | --sbindir=* | --sbindi=* | --sbind=* | --sbin=* \
@@ -1383,7 +1374,7 @@ fi
 for ac_var in	exec_prefix prefix bindir sbindir libexecdir datarootdir \
 		datadir sysconfdir sharedstatedir localstatedir includedir \
 		oldincludedir docdir infodir htmldir dvidir pdfdir psdir \
-		libdir localedir mandir runstatedir
+		libdir localedir mandir
 do
   eval ac_val=\$$ac_var
   # Remove trailing slashes.
@@ -1536,7 +1527,6 @@ Fine tuning of the installation directories:
   --sysconfdir=DIR        read-only single-machine data [PREFIX/etc]
   --sharedstatedir=DIR    modifiable architecture-independent data [PREFIX/com]
   --localstatedir=DIR     modifiable single-machine data [PREFIX/var]
-  --runstatedir=DIR       modifiable per-process data [LOCALSTATEDIR/run]
   --libdir=DIR            object code libraries [EPREFIX/lib]
   --includedir=DIR        C header files [PREFIX/include]
   --oldincludedir=DIR     C header files for non-gcc [/usr/include]
@@ -1599,6 +1589,7 @@ Optional Features:
   --disable-tdb           disable tdb support
   --disable-bmap-stats    disable collection of bitmap stats.
   --enable-bmap-stats-ops enable collection of additional bitmap stats
+  --enable-pfsck     enable parallel e2fsck
   --disable-nls           do not use Native Language Support
   --enable-threads={posix|solaris|pth|windows}
                           specify multithreading API
@@ -6121,6 +6112,68 @@ $as_echo "Disabling additional bitmap statistics by default" >&6; }
 
 fi
 
+PTHREAD_LIB=''
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for pthread_join in -lpthread" >&5
+$as_echo_n "checking for pthread_join in -lpthread... " >&6; }
+if ${ac_cv_lib_pthread_pthread_join+:} false; then :
+  $as_echo_n "(cached) " >&6
+else
+  ac_check_lib_save_LIBS=$LIBS
+LIBS="-lpthread  $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h.  */
+
+/* Override any GCC internal prototype to avoid an error.
+   Use char because int might match the return type of a GCC
+   builtin and then its argument prototype would still apply.  */
+#ifdef __cplusplus
+extern "C"
+#endif
+char pthread_join ();
+int
+main ()
+{
+return pthread_join ();
+  ;
+  return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+  ac_cv_lib_pthread_pthread_join=yes
+else
+  ac_cv_lib_pthread_pthread_join=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+    conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_pthread_pthread_join" >&5
+$as_echo "$ac_cv_lib_pthread_pthread_join" >&6; }
+if test "x$ac_cv_lib_pthread_pthread_join" = xyes; then :
+  PTHREAD_LIB=-pthread
+fi
+
+
+# Check whether --enable-pfsck was given.
+if test "${enable_pfsck+set}" = set; then :
+  enableval=$enable_pfsck; if test "$enableval" = "no" || test -z "PTHREAD_LIB"
+then
+	{ $as_echo "$as_me:${as_lineno-$LINENO}: result: Disabling parallel e2fsck" >&5
+$as_echo "Disabling parallel e2fsck" >&6; }
+else
+
+$as_echo "#define CONFIG_PFSCK 1" >>confdefs.h
+
+	{ $as_echo "$as_me:${as_lineno-$LINENO}: result: Enabling parallel e2fsck" >&5
+$as_echo "Enabling parallel e2fsck" >&6; }
+fi
+
+else
+  { $as_echo "$as_me:${as_lineno-$LINENO}: result: Disabling parallel e2fsck" >&5
+$as_echo "Disabling parallel e2fsck" >&6; }
+
+fi
+
 MAKEFILE_LIBRARY=$srcdir/lib/Makefile.library
 
 GETTEXT_PACKAGE=e2fsprogs
@@ -7338,8 +7391,6 @@ main ()
     if (*(data + i) != *(data3 + i))
       return 14;
   close (fd);
-  free (data);
-  free (data3);
   return 0;
 }
 _ACEOF
configure.ac +19 -0
index 7d921074..e73dbf50 100644
@@ -877,6 +877,25 @@ fi
 AC_MSG_RESULT([Disabling additional bitmap statistics by default])
 )
 dnl
+dnl handle --enable-pfsck
+dnl
+PTHREAD_LIB=''
+AC_CHECK_LIB(pthread,pthread_join,PTHREAD_LIB=-pthread)
+AC_SUBST(PTHREAD_LIB)
+AC_ARG_ENABLE([pfsck],
+[  --enable-pfsck     enable parallel e2fsck],
+if test "$enableval" = "no" || test -z "PTHREAD_LIB"
+then
+	AC_MSG_RESULT([Disabling parallel e2fsck])
+else
+	AC_DEFINE(CONFIG_PFSCK, 1,
+		[Define to 1 if parallel e2fsck is enabled])
+	AC_MSG_RESULT([Enabling parallel e2fsck])
+fi
+,
+AC_MSG_RESULT([Disabling parallel e2fsck])
+)
+dnl
 dnl
 dnl
 MAKEFILE_LIBRARY=$srcdir/lib/Makefile.library
e2fsck/e2fsck.h +13 -0
index 5ad0fe93..ccb66ae7 100644
@@ -426,6 +426,19 @@ struct e2fsck_struct {
 	char *undo_file;
 };
 
+#ifdef CONFIG_PFSCK
+struct e2fsck_thread_info {
+	/* ID returned by pthread_create() */
+	pthread_t		 eti_thread_id;
+	/* Application-defined thread index */
+	int			 eti_thread_index;
+	/* Thread has been started */
+	int			 eti_started;
+	/* Context used for this thread */
+	e2fsck_t		 eti_thread_ctx;
+};
+#endif
+
 /* Data structures to evaluate whether an extent tree needs rebuilding. */
 struct extent_tree_level {
 	unsigned int	num_extents;
e2fsck/logfile.c +2 -0
index c5505d27..8bda6b81 100644
@@ -310,11 +310,13 @@ static FILE *set_up_log_file(e2fsck_t ctx, const char *key, const char *fn)
 		goto out;
 
 	expand_logfn(ctx, log_fn, &s);
+#ifdef CONFIG_PFSCK
 	if (ctx->global_ctx) {
 		sprintf(string_index, "%d", ctx->thread_index);
 		append_string(&s, ".", 1);
 		append_string(&s, string_index, 0);
 	}
+#endif
 
 	if ((log_fn[0] == '/') || !log_dir || !log_dir[0])
 		s0 = s.s;
e2fsck/pass1.c +135 -18

197 lines not on this page · raw message

e2fsck/unix.c +10 -0
index f973af62..30c2bf31 100644
@@ -82,7 +82,9 @@ static void usage(e2fsck_t ctx)
 
 	fprintf(stderr, "%s", _("\nEmergency help:\n"
 		" -p                   Automatic repair (no questions)\n"
+#ifdef CONFIG_PFSCK
 		" -m                   multiple threads to speedup fsck\n"
+#endif
 		" -n                   Make no changes to the filesystem\n"
 		" -y                   Assume \"yes\" to all questions\n"
 		" -c                   Check for bad blocks and add them to the badblock list\n"
@@ -849,7 +851,11 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx)
 	phys_mem_kb = get_memory_size() / 1024;
 	ctx->readahead_kb = ~0ULL;
 
+#ifdef CONFIG_PFSCK
 	while ((c = getopt(argc, argv, "pamnyrcC:B:dE:fvtFVM:b:I:j:P:l:L:N:SsDkz:")) != EOF)
+#else
+	while ((c = getopt(argc, argv, "panyrcC:B:dE:fvtFVM:b:I:j:P:l:L:N:SsDkz:")) != EOF)
+#endif
 		switch (c) {
 		case 'C':
 			ctx->progress = e2fsck_update_progress;
@@ -890,9 +896,11 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx)
 			}
 			ctx->options |= E2F_OPT_PREEN;
 			break;
+#ifdef CONFIG_PFSCK
 		case 'm':
 			ctx->options |= E2F_OPT_MULTITHREAD;
 			break;
+#endif
 		case 'n':
 			if (ctx->options & (E2F_OPT_YES|E2F_OPT_PREEN))
 				goto conflict_opt;
@@ -1011,6 +1019,7 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx)
 			_("The -n and -l/-L options are incompatible."));
 		fatal_error(ctx, 0);
 	}
+#ifdef CONFIG_PFSCK
 	if (ctx->options & E2F_OPT_MULTITHREAD) {
 		if ((ctx->options & (E2F_OPT_YES|E2F_OPT_NO|E2F_OPT_PREEN)) == 0) {
 			com_err(ctx->program_name, 0, "%s",
@@ -1023,6 +1032,7 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx)
 			fatal_error(ctx, 0);
 		}
 	}
+#endif
 	if (ctx->options & E2F_OPT_NO)
 		ctx->options |= E2F_OPT_READONLY;
 
lib/config.h.in +3 -0
index b448482c..21efda62 100644
@@ -18,6 +18,9 @@
 /* Define to 1 to enable mmp support */
 #undef CONFIG_MMP
 
+/* Define to 1 if parallel e2fsck is enabled */
+#undef CONFIG_PFSCK
+
 /* Define to 1 to enable tdb support */
 #undef CONFIG_TDB
 
tests/test_one.in +8 -0
index 5d7607ad..c0a0b4fd 100644
@@ -27,6 +27,7 @@ esac
 
 test_dir=$1
 cmd_dir=$SRCDIR
+pfsck_enabled="no"
 
 if test "$TEST_CONFIG"x = x; then
 	TEST_CONFIG=$SRCDIR/test_config
@@ -52,6 +53,13 @@ else
 	test_description=
 fi
 
+$FSCK --help 2>&1 | grep -q -w -- -m && pfsck_enabled=yes
+if [ "$pfsck_enabled" != "yes" ] ; then
+	echo "$test_dir" | grep -q multithread &&
+	echo "$test_name: $test_description: skipped (pfsck disabled)" &&
+	exit 0
+fi
+
 if [ -n "$SKIP_SLOW_TESTS" -a -f $test_dir/is_slow_test ]; then
     echo "$test_name: $test_description: skipped (slow test)"
     exit 0
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#23 lore raw

Re: [RFC PATCH v3 10/61] e2fsck: optionally configure one pfsck thread

Quoted 8 lines · Saranya Muruganandam

From: Li Xi <[email protected]>

This patch creates only one thread to do pass1 check. The same codes can be used to create multiple threads, but other functions need to be modified to get ready for that.

pfsck support will be enabled with if configured with --enable-pfsck option.

This patch should probably be merged with patch #1 ("e2fsck: add -m option for multithread"). That will make it be *much* easier to review.

What I'd actually suggest is that we add some multi-threading support into libext2fs as separate commits, and put them *first* in the patch series. They should ideally have their own unit tests so they can be individually tested, but that way we can also review the library changes *first* to make sure they make sense, and that way the reviewer will understand what the library functions are before reviewing the e2fsck patches.

I'd then also follow that up with a commit which compiles e2fsck with the pthread library (and we need to make sure that is portable across different OS's, and that it works when linking both statically and dynamically), and then follow that up with e2fsck utility functions that make life easier for the multi-threading changes --- for example, a thread-safe logging abstraction.

Finally, I'd then add support to enable parallel fsck via a configure option, with the default controlled by whether or not pthread support is enabled. (My main concern will be that there may be some minimal C libraries which don't have threading support, when e2fsprogs is compiled in some embedded environment.)

I'm not convinced that it makes sense to have a command-line thread to enable multi-threading. What probably makes more sense is to have extended e2fsck option (using -E multithread, which takes an optional argument indicating how many threads should be used). In production, the default of whether or not multi-threading should be enabled would be via /etc/e2fsck.conf.

- Ted

#24 lore raw

RFC PATCH v3 11/61 e2fsck: add start/end group for thread

In reply to Saranya Muruganandam

From: Li Xi <[email protected]>

When multi-threads are used for check, each thread needs to jump to different group in pass1 check. This patch adds the group jumping support. But still, only one thread is used to check.

---
 e2fsck/e2fsck.h           | 23 ++++++++++++++++--
 e2fsck/logfile.c          |  3 ++-
 e2fsck/pass1.c            | 51 +++++++++++++++++++++++++++++++++------
 e2fsck/problem.c          |  5 ++++
 e2fsck/problem.h          |  3 +++
 lib/ext2fs/ext2_err.et.in |  3 +++
 6 files changed, 78 insertions(+), 10 deletions(-)
e2fsck/e2fsck.h +21 -2
index ccb66ae7..ba1af6bf 100644
@@ -227,6 +227,22 @@ typedef struct e2fsck_struct *e2fsck_t;
 
 #define MAX_EXTENT_DEPTH_COUNT 5
 
+#ifdef CONFIG_PFSCK
+/*
+ * Fields that used for multi-thread
+ */
+struct e2fsck_thread {
+	/* Thread index */
+	int		et_thread_index;
+	/* The start group number for this thread */
+	dgrp_t		et_group_start;
+	/* The end (not included) group number for this thread*/
+	dgrp_t		et_group_end;
+	/* The next group number to check */
+	dgrp_t		et_group_next;
+};
+#endif
+
 struct e2fsck_struct {
 	/* Global context to get the cancel flag */
 	e2fsck_t		global_ctx;
@@ -310,8 +326,11 @@ struct e2fsck_struct {
 	 */
 	ext2_ino_t		stashed_ino;
 	struct ext2_inode	*stashed_inode;
-	/* Thread index, if global_ctx is null, this field is unused */
-	int			thread_index;
+
+	/* if @global_ctx is null, this field is unused */
+#ifdef CONFIG_PFSCK
+	struct e2fsck_thread	 thread_info;
+#endif
 
 	/*
 	 * Location of the lost and found directory
e2fsck/logfile.c +2 -1
index 8bda6b81..d177e4b5 100644
@@ -312,7 +312,8 @@ static FILE *set_up_log_file(e2fsck_t ctx, const char *key, const char *fn)
 	expand_logfn(ctx, log_fn, &s);
 #ifdef CONFIG_PFSCK
 	if (ctx->global_ctx) {
-		sprintf(string_index, "%d", ctx->thread_index);
+		sprintf(string_index, "%d",
+			ctx->thread_info.et_thread_index);
 		append_string(&s, ".", 1);
 		append_string(&s, string_index, 0);
 	}
e2fsck/pass1.c +44 -7

96 lines not on this page · raw message

e2fsck/problem.c +5 -0
index e79c853b..22c2652c 100644
@@ -1269,6 +1269,11 @@ static struct e2fsck_problem problem_table[] = {
 	  N_("Encrypted @i %i has corrupt encryption @a.\n"),
 	  PROMPT_CLEAR_INODE, 0, 0, 0, 0 },
 
+	/* Failed to goto block group */
+	{ PR_1_SCAN_GOTO,
+	  N_("failed to goto block group"),
+          PROMPT_NONE, PR_FATAL, 0, 0, 0 },
+
 	/* Pass 1b errors */
 
 	/* Pass 1B: Rescan for duplicate/bad blocks */
e2fsck/problem.h +3 -0
index 4185e517..e28adebb 100644
@@ -707,6 +707,9 @@ struct problem_context {
 /* Encrypted inode has corrupt encryption extended attribute */
 #define PR_1_CORRUPT_ENCRYPTION_XATTR		0x01008B
 
+/* Failed to goto block group */
+#define PR_1_SCAN_GOTO				0x0100A0
+
 /*
  * Pass 1b errors
  */
lib/ext2fs/ext2_err.et.in +3 -0
index 0c76fee6..cdb37423 100644
@@ -548,4 +548,7 @@ ec	EXT2_ET_EA_INODE_CORRUPTED,
 ec	EXT2_ET_NO_GDESC,
 	"Group descriptors not loaded"
 
+ec	EXT2_ET_SCAN_FINISHED,
+	"Scanning finished"
+
 	end
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#25 lore raw

RFC PATCH v3 12/61 e2fsck: split groups to different threads

In reply to Saranya Muruganandam

From: Li Xi <[email protected]>

The start/end groups of a thread is calculated according to the thread number. But still, only one thread is used to check.

---
 e2fsck/pass1.c | 23 +++++++++++++++++------
 1 file changed, 17 insertions(+), 6 deletions(-)
e2fsck/pass1.c +17 -6
index bb32511f..fd354529 100644
@@ -2232,13 +2232,14 @@ static int e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
 }
 
 static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thread_ctx,
-					     int thread_index)
+					     int thread_index, int num_threads)
 {
 	errcode_t		retval;
 	e2fsck_t		thread_context;
 	ext2_filsys		thread_fs;
 	ext2_filsys		global_fs = global_ctx->fs;
 	struct e2fsck_thread	*tinfo;
+	dgrp_t			average_group;
 
 	assert(global_ctx->inode_used_map == NULL);
 	assert(global_ctx->inode_dir_map == NULL);
@@ -2278,11 +2279,20 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thre
 	thread_context->thread_info.et_thread_index = thread_index;
 	set_up_logging(thread_context);
 
-	assert(thread_index == 0);
+	/*
+	 * Distribute work to multiple threads:
+	 * Each thread work on fs->group_desc_count / nthread groups.
+	 */
 	tinfo = &thread_context->thread_info;
-	tinfo->et_group_start = 0;
-	tinfo->et_group_next = 0;
-	tinfo->et_group_end = thread_fs->group_desc_count;
+	average_group = thread_fs->group_desc_count / num_threads;
+	if (average_group == 0)
+		average_group = 1;
+	tinfo->et_group_start = average_group * thread_index;
+	if (thread_index == num_threads - 1)
+		tinfo->et_group_end = thread_fs->group_desc_count;
+	else
+		tinfo->et_group_end = average_group * (thread_index + 1);
+	tinfo->et_group_next = tinfo->et_group_start;
 
 	thread_context->fs = thread_fs;
 	*thread_ctx = thread_context;
@@ -2506,7 +2516,8 @@ static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo,
 	for (i = 0; i < num_threads; i++) {
 		tmp_pinfo = &infos[i];
 		tmp_pinfo->eti_thread_index = i;
-		retval = e2fsck_pass1_thread_prepare(global_ctx, &thread_ctx, i);
+		retval = e2fsck_pass1_thread_prepare(global_ctx, &thread_ctx,
+						     i, num_threads);
 		if (retval) {
 			com_err(global_ctx->program_name, retval,
 				_("while preparing pass1 thread\n"));
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#26 lore raw

RFC PATCH v3 13/61 e2fsck: print thread log properly

In reply to Saranya Muruganandam

From: Li Xi <[email protected]>

When multi-thread fsck is enabled, logs printed from multiple threads could overlap with each other. The overlap sometimes makes the logs unreadable because log_out() is used multiple times for a single line.

This patch adds leading [Thread XXX] to each logs if multi-thread is enabed by -m option.

This patch also adds message to show the group ranges and inode numbers for each thread, which is useful for debuging multi-thread check.

---
 e2fsck/e2fsck.h                      |  4 ++
 e2fsck/pass1.c                       | 22 +++++++++-
 e2fsck/problem.c                     |  6 +++
 e2fsck/util.c                        | 61 ++++++++++++++++++++++++++--
 tests/f_multithread/expect.1         |  4 +-
 tests/f_multithread_logfile/expect.1 |  4 +-
 tests/f_multithread_no/expect.1      |  4 +-
 7 files changed, 98 insertions(+), 7 deletions(-)
e2fsck/e2fsck.h +4 -0
index ba1af6bf..06893f67 100644
@@ -240,6 +240,10 @@ struct e2fsck_thread {
 	dgrp_t		et_group_end;
 	/* The next group number to check */
 	dgrp_t		et_group_next;
+	/* Scanned inode number */
+	ext2_ino_t	et_inode_number;
+	char		et_log_buf[2048];
+	char		et_log_length;
 };
 #endif
 
e2fsck/pass1.c +21 -1
index fd354529..528f0a6b 100644
@@ -1441,6 +1441,10 @@ void e2fsck_pass1_run(e2fsck_t ctx)
 		}
 		if (!ino)
 			break;
+#ifdef CONFIG_PFSCK
+		if (ctx->global_ctx)
+		        ctx->thread_info.et_inode_number++;
+#endif
 		pctx.ino = ino;
 		pctx.inode = inode;
 		ctx->stashed_ino = ino;
@@ -2293,7 +2297,12 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thre
 	else
 		tinfo->et_group_end = average_group * (thread_index + 1);
 	tinfo->et_group_next = tinfo->et_group_start;
-
+	tinfo->et_inode_number = 0;
+	tinfo->et_log_buf[0] = '\0';
+	tinfo->et_log_length = 0;
+	if (thread_context->options & E2F_OPT_MULTITHREAD)
+		log_out(thread_context, _("Scan group range [%d, %d)\n"),
+			tinfo->et_group_start, tinfo->et_group_end);
 	thread_context->fs = thread_fs;
 	*thread_ctx = thread_context;
 	return 0;
@@ -2334,6 +2343,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
 	global_ctx->fs = global_fs;
 	global_ctx->logf = global_logf;
 	global_ctx->problem_logf = global_problem_logf;
+	global_ctx->global_ctx = NULL;
 
 	if (thread_ctx->inode_used_map) {
 		retval = e2fsck_pass1_copy_bitmap(global_fs,
@@ -2483,6 +2493,12 @@ static void *e2fsck_pass1_thread(void *arg)
 	e2fsck_pass1_run(thread_ctx);
 
 out:
+	if (thread_ctx->options & E2F_OPT_MULTITHREAD)
+		log_out(thread_ctx,
+			_("Scanned group range [%lu, %lu), inodes %lu\n"),
+			thread_ctx->thread_info.et_group_start,
+			thread_ctx->thread_info.et_group_end,
+			thread_ctx->thread_info.et_inode_number);
 	return NULL;
 }
 
@@ -2618,6 +2634,10 @@ static errcode_t scan_callback(ext2_filsys fs,
 	if (ctx->global_ctx) {
 		tinfo = &ctx->thread_info;
 		tinfo->et_group_next++;
+		if (ctx->options & E2F_OPT_DEBUG &&
+		    ctx->options & E2F_OPT_MULTITHREAD)
+			log_out(ctx, _("group %d finished\n"),
+				tinfo->et_group_next);
 		if (tinfo->et_group_next >= tinfo->et_group_end)
 			return EXT2_ET_SCAN_FINISHED;
 	}
e2fsck/problem.c +6 -0
index 22c2652c..1ff6b028 100644
@@ -2481,6 +2481,12 @@ int fix_problem(e2fsck_t ctx, problem_t code, struct problem_context *pctx)
 	if (*message)
 		message = _(message);
 	if (!suppress) {
+#ifdef	CONFIG_PFSCK
+		if ((ctx->options & E2F_OPT_MULTITHREAD) && ctx->global_ctx)
+			printf("[Thread %d] ",
+			       ctx->thread_info.et_thread_index);
+#endif
+
 		if ((ctx->options & E2F_OPT_PREEN) &&
 		    !(ptr->flags & PR_PREEN_NOHDR)) {
 			printf("%s: ", ctx->device_name ?
e2fsck/util.c +58 -3

77 lines not on this page · raw message

tests/f_multithread/expect.1 +3 -1
index e2b954d0..8d2acd2b 100644
@@ -1,6 +1,8 @@
 ext2fs_open2: Bad magic number in super-block
 ../e2fsck/e2fsck: Superblock invalid, trying backup blocks...
-Pass 1: Checking inodes, blocks, and sizes
+[Thread 0] Scan group range [0, 2)
+[Thread 0] Pass 1: Checking inodes, blocks, and sizes
+[Thread 0] Scanned group range [0, 2), inodes 3008
 Pass 2: Checking directory structure
 Pass 3: Checking directory connectivity
 Pass 4: Checking reference counts
tests/f_multithread_logfile/expect.1 +3 -1
index e2b954d0..8d2acd2b 100644
@@ -1,6 +1,8 @@
 ext2fs_open2: Bad magic number in super-block
 ../e2fsck/e2fsck: Superblock invalid, trying backup blocks...
-Pass 1: Checking inodes, blocks, and sizes
+[Thread 0] Scan group range [0, 2)
+[Thread 0] Pass 1: Checking inodes, blocks, and sizes
+[Thread 0] Scanned group range [0, 2), inodes 3008
 Pass 2: Checking directory structure
 Pass 3: Checking directory connectivity
 Pass 4: Checking reference counts
tests/f_multithread_no/expect.1 +3 -1
index d14c4083..f85a3382 100644
@@ -1,6 +1,8 @@
 ext2fs_open2: Bad magic number in super-block
 ../e2fsck/e2fsck: Superblock invalid, trying backup blocks...
-Pass 1: Checking inodes, blocks, and sizes
+[Thread 0] Scan group range [0, 2)
+[Thread 0] Pass 1: Checking inodes, blocks, and sizes
+[Thread 0] Scanned group range [0, 2), inodes 3008
 Pass 2: Checking directory structure
 Pass 3: Checking directory connectivity
 Pass 4: Checking reference counts
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#27 lore raw

Re: [RFC PATCH v3 13/61] e2fsck: print thread log properly

Quoted 9 lines · Saranya Muruganandam

From: Li Xi <[email protected]>

When multi-thread fsck is enabled, logs printed from multiple threads could overlap with each other. The overlap sometimes makes the logs unreadable because log_out() is used multiple times for a single line.

This patch adds leading [Thread XXX] to each logs if multi-thread is enabed by -m option.

Given that the leading "[Thread XXX]" is output using a separate printf, this is still going to result in potential overlapped messages:

e2fsck/util.c out: · line 100

Quoted 9 lines
e2fsck/util.c +9 -0
+#ifdef CONFIG_PFSCK
+static void thread_log_out(struct e2fsck_thread *tinfo)
+{
+	printf("[Thread %d] %s", tinfo->et_thread_index,
+	       tinfo->et_log_buf);
+	tinfo->et_log_length = 0;
+	tinfo->et_log_buf[0] = '\0';
+}
+#endif

Instead of using a separate 2k buffer for each thread, why not just have a mutex so only one thread can be printing to the terminal and/or log files at one time?

- Ted

#28 lore raw

RFC PATCH v3 14/61 e2fsck: merge bitmaps after thread completes

In reply to Saranya Muruganandam

From: Wang Shilong <[email protected]>

A new method merge_bmap has been added to bitmap operations. But only red-black bitmap has that operation now.

---
 e2fsck/pass1.c            | 239 +++++++++++++++++++++++---------------
 lib/ext2fs/bitmaps.c      |  10 ++
 lib/ext2fs/blkmap64_rb.c  |  65 +++++++++++
 lib/ext2fs/bmap64.h       |   4 +
 lib/ext2fs/ext2fs.h       |   8 ++
 lib/ext2fs/gen_bitmap64.c |  29 +++++
 6 files changed, 264 insertions(+), 91 deletions(-)
e2fsck/pass1.c +148 -91
index 528f0a6b..9e4abad0 100644
@@ -2124,12 +2124,38 @@ static errcode_t e2fsck_pass1_copy_bitmap(ext2_filsys fs, ext2fs_generic_bitmap
 		return ret;
 
 	(*dest)->fs = fs;
-	ext2fs_free_generic_bmap(*src);
-	*src = NULL;
 
 	return 0;
 }
 
+static void e2fsck_pass1_free_bitmap(ext2fs_generic_bitmap *bitmap)
+{
+	if (*bitmap) {
+		ext2fs_free_generic_bmap(*bitmap);
+		*bitmap = NULL;
+	}
+
+}
+
+static errcode_t e2fsck_pass1_merge_bitmap(ext2_filsys fs, ext2fs_generic_bitmap *src,
+					  ext2fs_generic_bitmap *dest)
+{
+	errcode_t ret = 0;
+
+	if (*src) {
+		if (*dest == NULL) {
+			*dest = *src;
+			*src = NULL;
+		} else {
+			ret = ext2fs_merge_bitmap(*src, *dest, NULL, NULL);
+			if (ret)
+				return ret;
+		}
+		(*dest)->fs = fs;
+	}
+
+	return 0;
+}
 
 static errcode_t e2fsck_pass1_copy_fs(ext2_filsys dest, e2fsck_t src_context,
 				      ext2_filsys src)
@@ -2137,6 +2163,8 @@ static errcode_t e2fsck_pass1_copy_fs(ext2_filsys dest, e2fsck_t src_context,
 	errcode_t	retval;
 
 	memcpy(dest, src, sizeof(struct struct_ext2_filsys));
+	dest->inode_map = NULL;
+	dest->block_map = NULL;
 	if (dest->dblist)
 		dest->dblist->fs = dest;
 	if (src->block_map) {
@@ -2196,42 +2224,50 @@ static int e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
 	errcode_t retval = 0;
 	io_channel dest_io;
 	io_channel dest_image_io;
+	ext2fs_inode_bitmap inode_map;
+	ext2fs_block_bitmap block_map;
 
 	dest_io = dest->io;
 	dest_image_io = dest->image_io;
+	inode_map = dest->inode_map;
+	block_map = dest->block_map;
 
 	memcpy(dest, src, sizeof(struct struct_ext2_filsys));
 	dest->io = dest_io;
 	dest->image_io = dest_image_io;
 	dest->icache = icache;
+	dest->inode_map = inode_map;
+	dest->block_map = block_map;
 	if (dest->dblist)
 		dest->dblist->fs = dest;
-	if (src->inode_map) {
-		retval = e2fsck_pass1_copy_bitmap(dest, &src->inode_map,
-						  &dest->inode_map);
-		if (retval)
-			return retval;
-	}
-	if (src->block_map) {
-		retval = e2fsck_pass1_copy_bitmap(dest, &src->block_map,
-						  &dest->block_map);
-		if (retval)
-			return retval;
-	}
 
 	if (src->icache) {
 		ext2fs_free_inode_cache(src->icache);
 		src->icache = NULL;
 	}
 
+	retval = e2fsck_pass1_merge_bitmap(dest, &src->inode_map,
+					   &dest->inode_map);
+	if (retval)
+		goto out;
+
+	retval = e2fsck_pass1_merge_bitmap(dest, &src->block_map,
+					  &dest->block_map);
+	if (retval)
+		goto out;
+
 	if (src->badblocks) {
 		retval = ext2fs_badblocks_copy(src->badblocks, &dest->badblocks);
 
 		ext2fs_badblocks_list_free(src->badblocks);
 		src->badblocks = NULL;
 	}
-
+out:
 	io_channel_close(src->io);
+	if (src->inode_map)
+		ext2fs_free_generic_bmap(src->inode_map);
+	if (src->block_map)
+		ext2fs_free_generic_bmap(src->block_map);
 	return retval;
 }
 
@@ -2321,6 +2357,18 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
 	ext2_filsys	 global_fs = global_ctx->fs;
 	FILE		*global_logf = global_ctx->logf;
 	FILE		*global_problem_logf = global_ctx->problem_logf;
+	ext2fs_inode_bitmap inode_bad_map = global_ctx->inode_bad_map;
+	ext2fs_inode_bitmap inode_used_map = global_ctx->inode_used_map;
+	ext2fs_inode_bitmap inode_dir_map = global_ctx->inode_dir_map;
+	ext2fs_inode_bitmap inode_bb_map = global_ctx->inode_bb_map;
+	ext2fs_inode_bitmap inode_imagic_map = global_ctx->inode_imagic_map;
+	ext2fs_inode_bitmap inode_reg_map = global_ctx->inode_reg_map;
+	ext2fs_block_bitmap block_found_map = global_ctx->block_found_map;
+	ext2fs_block_bitmap block_dup_map = global_ctx->block_dup_map;
+	ext2fs_block_bitmap block_ea_map = global_ctx->block_ea_map;
+	ext2fs_block_bitmap block_metadata_map = global_ctx->block_metadata_map;
+	ext2fs_block_bitmap inodes_to_rebuild = global_ctx->inodes_to_rebuild;
+
 #ifdef HAVE_SETJMP_H
 	jmp_buf		 old_jmp;
 
@@ -2330,6 +2378,19 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
 #ifdef HAVE_SETJMP_H
 	memcpy(global_ctx->abort_loc, old_jmp, sizeof(jmp_buf));
 #endif
+
+	global_ctx->inode_used_map = inode_used_map;
+	global_ctx->inode_bad_map = inode_bad_map;
+	global_ctx->inode_dir_map = inode_dir_map;
+	global_ctx->inode_bb_map = inode_bb_map;
+	global_ctx->inode_imagic_map = inode_imagic_map;
+	global_ctx->inodes_to_rebuild = inodes_to_rebuild;
+	global_ctx->inode_reg_map = inode_reg_map;
+	global_ctx->block_found_map = block_found_map;
+	global_ctx->block_dup_map = block_dup_map;
+	global_ctx->block_ea_map = block_ea_map;
+	global_ctx->block_metadata_map = block_metadata_map;
+
 	/* Keep the global singal flags*/
 	global_ctx->flags |= (flags & E2F_FLAG_SIGNAL_MASK) |
 			     (global_ctx->flags & E2F_FLAG_SIGNAL_MASK);
@@ -2345,83 +2406,62 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
 	global_ctx->problem_logf = global_problem_logf;
 	global_ctx->global_ctx = NULL;
 
-	if (thread_ctx->inode_used_map) {
-		retval = e2fsck_pass1_copy_bitmap(global_fs,
-					&thread_ctx->inode_used_map,
-					&global_ctx->inode_used_map);
-		if (retval)
-			return retval;
-	}
-	if (thread_ctx->inode_bad_map) {
-		retval = e2fsck_pass1_copy_bitmap(global_fs,
-					&thread_ctx->inode_bad_map,
-					&global_ctx->inode_bad_map);
-		if (retval)
-			return retval;
-	}
-	if (thread_ctx->inode_dir_map) {
-		retval = e2fsck_pass1_copy_bitmap(global_fs,
+	retval = e2fsck_pass1_merge_bitmap(global_fs,
+				&thread_ctx->inode_used_map,
+				&global_ctx->inode_used_map);
+	if (retval)
+		return retval;
+
+	retval = e2fsck_pass1_merge_bitmap(global_fs,
+				&thread_ctx->inode_bad_map,
+				&global_ctx->inode_bad_map);
+	if (retval)
+		return retval;
+	retval = e2fsck_pass1_merge_bitmap(global_fs,
 					&thread_ctx->inode_dir_map,
 					&global_ctx->inode_dir_map);
-		if (retval)
-			return retval;
-	}
-	if (thread_ctx->inode_bb_map) {
-		retval = e2fsck_pass1_copy_bitmap(global_fs,
-					&thread_ctx->inode_bb_map,
-					&global_ctx->inode_bb_map);
-		if (retval)
-			return retval;
-	}
-	if (thread_ctx->inode_imagic_map) {
-		retval = e2fsck_pass1_copy_bitmap(global_fs,
-					&thread_ctx->inode_imagic_map,
-					&global_ctx->inode_imagic_map);
-		if (retval)
-			return retval;
-	}
-	if (thread_ctx->inode_reg_map) {
-		retval = e2fsck_pass1_copy_bitmap(global_fs,
-					&thread_ctx->inode_reg_map,
-					&global_ctx->inode_reg_map);
-		if (retval)
-			return retval;
-	}
-	if (thread_ctx->inodes_to_rebuild) {
-		retval = e2fsck_pass1_copy_bitmap(global_fs,
-					&thread_ctx->inodes_to_rebuild,
-					&global_ctx->inodes_to_rebuild);
-		if (retval)
-			return retval;
-	}
-	if (thread_ctx->block_found_map) {
-		retval = e2fsck_pass1_copy_bitmap(global_fs,
-					&thread_ctx->block_found_map,
-					&global_ctx->block_found_map);
-		if (retval)
-			return retval;
-	}
-	if (thread_ctx->block_dup_map) {
-		retval = e2fsck_pass1_copy_bitmap(global_fs,
-					&thread_ctx->block_dup_map,
-					&global_ctx->block_dup_map);
-		if (retval)
-			return retval;
-	}
-	if (thread_ctx->block_ea_map) {
-		retval = e2fsck_pass1_copy_bitmap(global_fs,
-					&thread_ctx->block_ea_map,
-					&global_ctx->block_ea_map);
-		if (retval)
-			return retval;
-	}
-	if (thread_ctx->block_metadata_map) {
-		retval = e2fsck_pass1_copy_bitmap(global_fs,
-					&thread_ctx->block_metadata_map,
-					&global_ctx->block_metadata_map);
-		if (retval)
-			return retval;
-	}
+	if (retval)
+		return retval;
+	retval = e2fsck_pass1_merge_bitmap(global_fs,
+				&thread_ctx->inode_bb_map,
+				&global_ctx->inode_bb_map);
+	if (retval)
+		return retval;
+	retval = e2fsck_pass1_merge_bitmap(global_fs,
+				&thread_ctx->inode_imagic_map,
+				&global_ctx->inode_imagic_map);
+	if (retval)
+		return retval;
+	retval = e2fsck_pass1_merge_bitmap(global_fs,
+				&thread_ctx->inode_reg_map,
+				&global_ctx->inode_reg_map);
+	if (retval)
+		return retval;
+	retval = e2fsck_pass1_merge_bitmap(global_fs,
+				&thread_ctx->inodes_to_rebuild,
+				&global_ctx->inodes_to_rebuild);
+	if (retval)
+		return retval;
+	retval = e2fsck_pass1_merge_bitmap(global_fs,
+				&thread_ctx->block_found_map,
+				&global_ctx->block_found_map);
+	if (retval)
+		return retval;
+	retval = e2fsck_pass1_merge_bitmap(global_fs,
+				&thread_ctx->block_dup_map,
+				&global_ctx->block_dup_map);
+	if (retval)
+		return retval;
+	retval = e2fsck_pass1_merge_bitmap(global_fs,
+				&thread_ctx->block_ea_map,
+				&global_ctx->block_ea_map);
+	if (retval)
+		return retval;
+	retval = e2fsck_pass1_merge_bitmap(global_fs,
+				&thread_ctx->block_metadata_map,
+				&global_ctx->block_metadata_map);
+	if (retval)
+		return retval;
 
 	return 0;
 }
@@ -2438,6 +2478,17 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
 		fputs("</problem_log>\n", thread_ctx->problem_logf);
 		fclose(thread_ctx->problem_logf);
 	}
+	e2fsck_pass1_free_bitmap(&thread_ctx->inode_used_map);
+	e2fsck_pass1_free_bitmap(&thread_ctx->inode_bad_map);
+	e2fsck_pass1_free_bitmap(&thread_ctx->inode_dir_map);
+	e2fsck_pass1_free_bitmap(&thread_ctx->inode_bb_map);
+	e2fsck_pass1_free_bitmap(&thread_ctx->inode_imagic_map);
+	e2fsck_pass1_free_bitmap(&thread_ctx->inode_reg_map);
+	e2fsck_pass1_free_bitmap(&thread_ctx->inodes_to_rebuild);
+	e2fsck_pass1_free_bitmap(&thread_ctx->block_found_map);
+	e2fsck_pass1_free_bitmap(&thread_ctx->block_dup_map);
+	e2fsck_pass1_free_bitmap(&thread_ctx->block_ea_map);
+	e2fsck_pass1_free_bitmap(&thread_ctx->block_metadata_map);
 	ext2fs_free_mem(&thread_ctx);
 
 	return retval;
@@ -2464,7 +2515,13 @@ static int e2fsck_pass1_threads_join(struct e2fsck_thread_info *infos,
 			if (ret == 0)
 				ret = rc;
 		}
-		e2fsck_pass1_thread_join(global_ctx, infos[i].eti_thread_ctx);
+		rc = e2fsck_pass1_thread_join(global_ctx, infos[i].eti_thread_ctx);
+		if (rc) {
+			com_err(global_ctx->program_name, rc,
+				_("while joining pass1 thread\n"));
+			if (ret == 0)
+				ret = rc;
+		}
 	}
 	free(infos);
 
lib/ext2fs/bitmaps.c +10 -0
index 834a3962..000df234 100644
@@ -45,6 +45,16 @@ errcode_t ext2fs_copy_bitmap(ext2fs_generic_bitmap src,
 {
 	return (ext2fs_copy_generic_bmap(src, dest));
 }
+
+errcode_t ext2fs_merge_bitmap(ext2fs_generic_bitmap src,
+			      ext2fs_generic_bitmap dest,
+			      ext2fs_generic_bitmap dup,
+			      ext2fs_generic_bitmap dup_allowed)
+{
+	return ext2fs_merge_generic_bmap(src, dest, dup,
+					 dup_allowed);
+}
+
 void ext2fs_set_bitmap_padding(ext2fs_generic_bitmap map)
 {
 	ext2fs_set_generic_bmap_padding(map);
lib/ext2fs/blkmap64_rb.c +65 -0

76 lines not on this page · raw message

lib/ext2fs/bmap64.h +4 -0
index de334548..555193ee 100644
@@ -72,6 +72,10 @@ struct ext2_bitmap_ops {
 	void	(*free_bmap)(ext2fs_generic_bitmap_64 bitmap);
 	errcode_t (*copy_bmap)(ext2fs_generic_bitmap_64 src,
 			     ext2fs_generic_bitmap_64 dest);
+	errcode_t (*merge_bmap)(ext2fs_generic_bitmap_64 src,
+				ext2fs_generic_bitmap_64 dest,
+				ext2fs_generic_bitmap_64 dup,
+				ext2fs_generic_bitmap_64 dup_allowed);
 	errcode_t (*resize_bmap)(ext2fs_generic_bitmap_64 bitmap,
 			       __u64 new_end,
 			       __u64 new_real_end);
lib/ext2fs/ext2fs.h +8 -0
index 5b76d02e..0aa1d94e 100644
@@ -837,6 +837,10 @@ extern void ext2fs_free_block_bitmap(ext2fs_block_bitmap bitmap);
 extern void ext2fs_free_inode_bitmap(ext2fs_inode_bitmap bitmap);
 extern errcode_t ext2fs_copy_bitmap(ext2fs_generic_bitmap src,
 				    ext2fs_generic_bitmap *dest);
+errcode_t ext2fs_merge_bitmap(ext2fs_generic_bitmap src,
+			      ext2fs_generic_bitmap dest,
+			      ext2fs_generic_bitmap dup,
+			      ext2fs_generic_bitmap dup_allowed);
 extern errcode_t ext2fs_write_inode_bitmap(ext2_filsys fs);
 extern errcode_t ext2fs_write_block_bitmap (ext2_filsys fs);
 extern errcode_t ext2fs_read_inode_bitmap (ext2_filsys fs);
@@ -1433,6 +1437,10 @@ void ext2fs_set_generic_bmap_padding(ext2fs_generic_bitmap bmap);
 errcode_t ext2fs_resize_generic_bmap(ext2fs_generic_bitmap bmap,
 				     __u64 new_end,
 				     __u64 new_real_end);
+errcode_t ext2fs_merge_generic_bmap(ext2fs_generic_bitmap gen_src,
+                                    ext2fs_generic_bitmap gen_dest,
+				    ext2fs_generic_bitmap gen_dup,
+				    ext2fs_generic_bitmap dup_allowed);
 errcode_t ext2fs_compare_generic_bmap(errcode_t neq,
 				      ext2fs_generic_bitmap bm1,
 				      ext2fs_generic_bitmap bm2);
lib/ext2fs/gen_bitmap64.c +29 -0
index b2370667..50617a34 100644
@@ -344,6 +344,35 @@ errcode_t ext2fs_copy_generic_bmap(ext2fs_generic_bitmap gen_src,
 	return 0;
 }
 
+errcode_t ext2fs_merge_generic_bmap(ext2fs_generic_bitmap gen_src,
+				    ext2fs_generic_bitmap gen_dest,
+				    ext2fs_generic_bitmap gen_dup,
+				    ext2fs_generic_bitmap gen_dup_allowed)
+{
+	ext2fs_generic_bitmap_64 src = (ext2fs_generic_bitmap_64)gen_src;
+	ext2fs_generic_bitmap_64 dest = (ext2fs_generic_bitmap_64)gen_dest;
+	ext2fs_generic_bitmap_64 dup = (ext2fs_generic_bitmap_64)gen_dup;
+	ext2fs_generic_bitmap_64 dup_allowed = (ext2fs_generic_bitmap_64)gen_dup_allowed;
+
+	if (!src || !dest)
+		return EINVAL;
+
+	if (!EXT2FS_IS_64_BITMAP(src) || !EXT2FS_IS_64_BITMAP(dest) ||
+	    (dup && !EXT2FS_IS_64_BITMAP(dup)) ||
+		(dup_allowed && !EXT2FS_IS_64_BITMAP(dup_allowed)))
+		return EINVAL;
+
+	if (src->bitmap_ops != dest->bitmap_ops ||
+	    (dup && src->bitmap_ops != dup->bitmap_ops) ||
+	    (dup_allowed && src->bitmap_ops != dup_allowed->bitmap_ops))
+		return EINVAL;
+
+	if (src->bitmap_ops->merge_bmap == NULL)
+		return EOPNOTSUPP;
+
+	return src->bitmap_ops->merge_bmap(src, dest, dup, dup_allowed);
+}
+
 errcode_t ext2fs_resize_generic_bmap(ext2fs_generic_bitmap gen_bmap,
 				     __u64 new_end,
 				     __u64 new_real_end)
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#29 lore raw

Re: [RFC PATCH v3 14/61] e2fsck: merge bitmaps after thread completes

Quoted 8 lines · Saranya Muruganandam

From: Wang Shilong <[email protected]>

A new method merge_bmap has been added to bitmap operations. But only red-black bitmap has that operation now.

There are hacks to deal with the fact that only the read-block bitmap backend supports merge_bmap in this patch series. Why not implement that for *all* of the back ends, along with some unit tests; it's going to be less work than working around failures due to partial implementation of merge_bmap --- and the workarounds weren't complete in any case; you can force the use of a particular back-end via /etc/e2fsck.conf --- see the function e2fsck_allocate_{block,inode}_bmap() in e2fsck/util.c.

This was mainly way of forcing additional testing of the backend implementations, and also so we could test/tune whether certain backends were preferable for certain file systems for different ways in which block/inode bitmaps are used by e2fsck --- but it's possible for an sysadmin / SRE to explicitly set them on a production e2fsck, and it would be preferable if things didn't blow up because we took a shortcut and didn't implement merge_bmap for all of the bitmap
backends. (It's *really* not that hard, anyway....)

e2fsck/pass1.c static errcode_t e2fsck_pass1_copy_bitmap(ext2_filsys fs, ext2fs_generic_bitmap · line 2138

Quoted 8 lines
e2fsck/pass1.c +8 -0
+static void e2fsck_pass1_free_bitmap(ext2fs_generic_bitmap *bitmap)
+{
+	if (*bitmap) {
+		ext2fs_free_generic_bmap(*bitmap);
+		*bitmap = NULL;
+	}
+
+}

I'm not sure why this is needed? ext2fs_free_Generic_bmap() will already return if NULL is passed into it.

Also, by the end of the patch series, there are no callers of this function....

And again, please separate out the libext2fs changes from the e2fsck changes, do the libext2fs changes first, and there should really be some unit tests of merge_bmap so we can validate that it works correctly before we drop it into use with e2fsck.

There should also be better documentation of what dup and dup_allowed in merge_bmap().

- Ted

#30 lore raw

RFC PATCH v3 15/61 e2fsck: do not change global variables

In reply to Saranya Muruganandam

From: Li Xi <[email protected]>

Global variables used in pass1 check are changed to local variables in this patch. This will avoid conflict between threads.

---
 e2fsck/pass1.c | 75 +++++++++++++++++++++++++++++++-------------------
 1 file changed, 47 insertions(+), 28 deletions(-)
e2fsck/pass1.c +47 -28
index 9e4abad0..d4a2e707 100644
@@ -86,7 +86,6 @@ static void alloc_bb_map(e2fsck_t ctx);
 static void alloc_imagic_map(e2fsck_t ctx);
 static void mark_inode_bad(e2fsck_t ctx, ino_t ino);
 static void handle_fs_bad_blocks(e2fsck_t ctx);
-static void process_inodes(e2fsck_t ctx, char *block_buf);
 static EXT2_QSORT_TYPE process_inode_cmp(const void *a, const void *b);
 static errcode_t scan_callback(ext2_filsys fs, ext2_inode_scan scan,
 				  dgrp_t group, void * priv_data);
@@ -121,15 +120,15 @@ struct process_inode_block {
 };
 
 struct scan_callback_struct {
-	e2fsck_t	ctx;
-	char		*block_buf;
+	e2fsck_t			 ctx;
+	char				*block_buf;
+	struct process_inode_block	*inodes_to_process;
+	int				*process_inode_count;
 };
 
-/*
- * For the inodes to process list.
- */
-static struct process_inode_block *inodes_to_process;
-static int process_inode_count;
+static void process_inodes(e2fsck_t ctx, char *block_buf,
+			   struct process_inode_block *inodes_to_process,
+			   int *process_inode_count);
 
 static __u64 ext2_max_sizes[EXT2_MAX_BLOCK_LOG_SIZE -
 			    EXT2_MIN_BLOCK_LOG_SIZE + 1];
@@ -1168,7 +1167,6 @@ static int e2fsck_should_abort(e2fsck_t ctx)
 void e2fsck_pass1_run(e2fsck_t ctx)
 {
 	int	i;
-	__u64	max_sizes;
 	ext2_filsys fs = ctx->fs;
 	ext2_ino_t	ino = 0;
 	struct ext2_inode *inode = NULL;
@@ -1191,6 +1189,8 @@ void e2fsck_pass1_run(e2fsck_t ctx)
 	ext2_ino_t	ino_threshold = 0;
 	dgrp_t		ra_group = 0;
 	struct ea_quota	ea_ibody_quota;
+	struct process_inode_block *inodes_to_process;
+	int		process_inode_count;
 
 	init_resource_track(&rtrack, ctx->fs->io);
 	clear_problem_context(&pctx);
@@ -1215,17 +1215,6 @@ void e2fsck_pass1_run(e2fsck_t ctx)
 	mtrace_print("Pass 1");
 #endif
 
-#define EXT2_BPP(bits) (1ULL << ((bits) - 2))
-
-	for (i = EXT2_MIN_BLOCK_LOG_SIZE; i <= EXT2_MAX_BLOCK_LOG_SIZE; i++) {
-		max_sizes = EXT2_NDIR_BLOCKS + EXT2_BPP(i);
-		max_sizes = max_sizes + EXT2_BPP(i) * EXT2_BPP(i);
-		max_sizes = max_sizes + EXT2_BPP(i) * EXT2_BPP(i) * EXT2_BPP(i);
-		max_sizes = (max_sizes * (1UL << i));
-		ext2_max_sizes[i - EXT2_MIN_BLOCK_LOG_SIZE] = max_sizes;
-	}
-#undef EXT2_BPP
-
 	imagic_fs = ext2fs_has_feature_imagic_inodes(sb);
 	extent_fs = ext2fs_has_feature_extents(sb);
 	inlinedata_fs = ext2fs_has_feature_inline_data(sb);
@@ -1349,6 +1338,8 @@ void e2fsck_pass1_run(e2fsck_t ctx)
 	ctx->stashed_inode = inode;
 	scan_struct.ctx = ctx;
 	scan_struct.block_buf = block_buf;
+	scan_struct.inodes_to_process = inodes_to_process;
+	scan_struct.process_inode_count = &process_inode_count;
 	ext2fs_set_inode_callback(scan, scan_callback, &scan_struct);
 	if (ctx->progress && ((ctx->progress)(ctx, 1, 0,
 					      ctx->fs->group_desc_count)))
@@ -1997,13 +1988,15 @@ void e2fsck_pass1_run(e2fsck_t ctx)
 			goto endit;
 
 		if (process_inode_count >= ctx->process_inode_size) {
-			process_inodes(ctx, block_buf);
+			process_inodes(ctx, block_buf, inodes_to_process,
+				       &process_inode_count);
 
 			if (e2fsck_should_abort(ctx))
 				goto endit;
 		}
 	}
-	process_inodes(ctx, block_buf);
+	process_inodes(ctx, block_buf, inodes_to_process,
+		       &process_inode_count);
 	ext2fs_close_inode_scan(scan);
 	scan = NULL;
 
@@ -2113,6 +2106,27 @@ endit:
 		ctx->invalid_bitmaps++;
 }
 
+static void init_ext2_max_sizes()
+{
+	int	i;
+	__u64	max_sizes;
+
+	/*
+	 * Init ext2_max_sizes which will be immutable and shared between
+	 * threads
+	 */
+#define EXT2_BPP(bits) (1ULL << ((bits) - 2))
+
+	for (i = EXT2_MIN_BLOCK_LOG_SIZE; i <= EXT2_MAX_BLOCK_LOG_SIZE; i++) {
+		max_sizes = EXT2_NDIR_BLOCKS + EXT2_BPP(i);
+		max_sizes = max_sizes + EXT2_BPP(i) * EXT2_BPP(i);
+		max_sizes = max_sizes + EXT2_BPP(i) * EXT2_BPP(i) * EXT2_BPP(i);
+		max_sizes = (max_sizes * (1UL << i));
+		ext2_max_sizes[i - EXT2_MIN_BLOCK_LOG_SIZE] = max_sizes;
+	}
+#undef EXT2_BPP
+}
+
 #ifdef CONFIG_PFSCK
 static errcode_t e2fsck_pass1_copy_bitmap(ext2_filsys fs, ext2fs_generic_bitmap *src,
 					  ext2fs_generic_bitmap *dest)
@@ -2656,6 +2670,7 @@ out_abort:
 void e2fsck_pass1(e2fsck_t ctx)
 {
 
+	init_ext2_max_sizes();
 #ifdef CONFIG_PFSCK
 	e2fsck_pass1_multithread(ctx);
 #else
@@ -2680,7 +2695,9 @@ static errcode_t scan_callback(ext2_filsys fs,
 	scan_struct = (struct scan_callback_struct *) priv_data;
 	ctx = scan_struct->ctx;
 
-	process_inodes((e2fsck_t) fs->priv_data, scan_struct->block_buf);
+	process_inodes((e2fsck_t) fs->priv_data, scan_struct->block_buf,
+		       scan_struct->inodes_to_process,
+		       scan_struct->process_inode_count);
 
 	if (ctx->progress)
 		if ((ctx->progress)(ctx, 1, group+1,
@@ -2706,7 +2723,9 @@ static errcode_t scan_callback(ext2_filsys fs,
 /*
  * Process the inodes in the "inodes to process" list.
  */
-static void process_inodes(e2fsck_t ctx, char *block_buf)
+static void process_inodes(e2fsck_t ctx, char *block_buf,
+			   struct process_inode_block *inodes_to_process,
+			   int *process_inode_count)
 {
 	int			i;
 	struct ext2_inode	*old_stashed_inode;
@@ -2718,15 +2737,15 @@ static void process_inodes(e2fsck_t ctx, char *block_buf)
 #if 0
 	printf("begin process_inodes: ");
 #endif
-	if (process_inode_count == 0)
+	if (*process_inode_count == 0)
 		return;
 	old_operation = ehandler_operation(0);
 	old_stashed_inode = ctx->stashed_inode;
 	old_stashed_ino = ctx->stashed_ino;
-	qsort(inodes_to_process, process_inode_count,
+	qsort(inodes_to_process, *process_inode_count,
 		      sizeof(struct process_inode_block), process_inode_cmp);
 	clear_problem_context(&pctx);
-	for (i=0; i < process_inode_count; i++) {
+	for (i=0; i < *process_inode_count; i++) {
 		pctx.inode = ctx->stashed_inode =
 			(struct ext2_inode *) &inodes_to_process[i].inode;
 		pctx.ino = ctx->stashed_ino = inodes_to_process[i].ino;
@@ -2744,7 +2763,7 @@ static void process_inodes(e2fsck_t ctx, char *block_buf)
 	}
 	ctx->stashed_inode = old_stashed_inode;
 	ctx->stashed_ino = old_stashed_ino;
-	process_inode_count = 0;
+	*process_inode_count = 0;
 #if 0
 	printf("end process inodes\n");
 #endif
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#31 lore raw

RFC PATCH v3 16/61 e2fsck: optimize the inserting of dir_info_db

In reply to Saranya Muruganandam

From: Li Xi <[email protected]>

Binary search is now used when inserting an dir info to the array. Memmove is now used when moving array. Both of them improves the performance of inserting.

This patch is also a prepartion for the merging of two dir db arrays.

---
 e2fsck/dirinfo.c | 172 ++++++++++++++++++++++++++++++-----------------
 1 file changed, 112 insertions(+), 60 deletions(-)
e2fsck/dirinfo.c +112 -60
index 49d624c5..28baaca2 100644
@@ -7,6 +7,7 @@
 
 #undef DIRINFO_DEBUG
 
+#include <assert.h>
 #include "config.h"
 #include "e2fsck.h"
 #include <sys/stat.h>
@@ -122,6 +123,104 @@ static void setup_db(e2fsck_t ctx)
 				       "directory map");
 }
 
+/*
+ * Return the min index that has ino larger or equal to @ino
+ * If not found, return -ENOENT
+ */
+static int
+e2fsck_dir_info_min_larger_equal(struct dir_info_db *dir_info,
+				 ext2_ino_t ino, ext2_ino_t *index)
+{
+	ext2_ino_t low = 0;
+	ext2_ino_t mid, high;
+	ext2_ino_t tmp_ino;
+	int found = 0;
+
+	if (dir_info->count == 0)
+		return -ENOENT;
+
+	high = dir_info->count - 1;
+	while (low <= high) {
+		/* sum may overflow, but result will fit into mid again */
+		mid = (unsigned long long)(low + high) / 2;
+		tmp_ino = dir_info->array[mid].ino;
+		if (ino == tmp_ino) {
+			*index = mid;
+			found = 1;
+			return 0;
+		} else if (ino < tmp_ino) {
+			/*
+			 * The mid ino is larger than @ino, remember the index
+			 * here so we won't miss this ino
+			 */
+			*index = mid;
+			found = 1;
+			if (mid == 0)
+				break;
+			high = mid - 1;
+		} else {
+			low = mid + 1;
+		}
+	}
+
+	if (found)
+		return 0;
+
+	return -ENOENT;
+}
+
+/*
+ *
+ * Insert an inode into the sorted array. The array should have at least one
+ * free slot.
+ *
+ * Normally, add_dir_info is called with each inode in
+ * sequential order; but once in a while (like when pass 3
+ * needs to recreate the root directory or lost+found
+ * directory) it is called out of order.  In those cases, we
+ * need to move the dir_info entries down to make room, since
+ * the dir_info array needs to be sorted by inode number for
+ * get_dir_info()'s sake.
+ */
+static void e2fsck_insert_dir_info(struct dir_info_db *dir_info, ext2_ino_t ino, ext2_ino_t parent)
+{
+	ext2_ino_t		index;
+	struct dir_info		*dir;
+	size_t			dir_size = sizeof(*dir);
+	struct dir_info		*array = dir_info->array;
+	ext2_ino_t		array_count = dir_info->count;
+	int			err;
+
+	/*
+	 * Removing this check won't break anything. But since seqential ino
+	 * inserting happens a lot, this check avoids binary search.
+	 */
+	if (array_count == 0 || array[array_count - 1].ino < ino) {
+		dir = &array[array_count];
+		dir_info->count++;
+		goto out;
+	}
+
+	err = e2fsck_dir_info_min_larger_equal(dir_info, ino, &index);
+	if (err >= 0 && array[index].ino == ino) {
+		dir = &array[index];
+		goto out;
+	}
+	if (err < 0) {
+		dir = &array[array_count];
+		dir_info->count++;
+		goto out;
+	}
+
+	dir = &array[index];
+	memmove((char *)dir + dir_size, dir, dir_size * (array_count - index));
+	dir_info->count++;
+out:
+	dir->ino = ino;
+	dir->dotdot = parent;
+	dir->parent = parent;
+}
+
 /*
  * This subroutine is called during pass1 to create a directory info
  * entry.  During pass1, the passed-in parent is 0; it will get filled
@@ -171,30 +270,7 @@ void e2fsck_add_dir_info(e2fsck_t ctx, ext2_ino_t ino, ext2_ino_t parent)
 	}
 #endif
 
-	/*
-	 * Normally, add_dir_info is called with each inode in
-	 * sequential order; but once in a while (like when pass 3
-	 * needs to recreate the root directory or lost+found
-	 * directory) it is called out of order.  In those cases, we
-	 * need to move the dir_info entries down to make room, since
-	 * the dir_info array needs to be sorted by inode number for
-	 * get_dir_info()'s sake.
-	 */
-	if (ctx->dir_info->count &&
-	    ctx->dir_info->array[ctx->dir_info->count-1].ino >= ino) {
-		for (i = ctx->dir_info->count-1; i > 0; i--)
-			if (ctx->dir_info->array[i-1].ino < ino)
-				break;
-		dir = &ctx->dir_info->array[i];
-		if (dir->ino != ino)
-			for (j = ctx->dir_info->count++; j > i; j--)
-				ctx->dir_info->array[j] = ctx->dir_info->array[j-1];
-	} else
-		dir = &ctx->dir_info->array[ctx->dir_info->count++];
-
-	dir->ino = ino;
-	dir->dotdot = parent;
-	dir->parent = parent;
+	e2fsck_insert_dir_info(ctx->dir_info, ino, parent);
 }
 
 /*
@@ -204,7 +280,8 @@ void e2fsck_add_dir_info(e2fsck_t ctx, ext2_ino_t ino, ext2_ino_t parent)
 static struct dir_info *e2fsck_get_dir_info(e2fsck_t ctx, ext2_ino_t ino)
 {
 	struct dir_info_db	*db = ctx->dir_info;
-	ext2_ino_t low, high, mid;
+	ext2_ino_t		index;
+	int			err;
 
 	if (!db)
 		return 0;
@@ -245,44 +322,19 @@ static struct dir_info *e2fsck_get_dir_info(e2fsck_t ctx, ext2_ino_t ino)
 	if (db->last_lookup && db->last_lookup->ino == ino)
 		return db->last_lookup;
 
-	low = 0;
-	high = ctx->dir_info->count - 1;
-	if (ino == ctx->dir_info->array[low].ino) {
+	err = e2fsck_dir_info_min_larger_equal(ctx->dir_info, ino, &index);
+	if (err < 0)
+		return NULL;
+	assert(ino <= ctx->dir_info->array[index].ino);
+	if (ino == ctx->dir_info->array[index].ino) {
 #ifdef DIRINFO_DEBUG
-		printf("(%u,%u,%u)\n", ino,
-		       ctx->dir_info->array[low].dotdot,
-		       ctx->dir_info->array[low].parent);
+		printf("(%d,%d,%d)\n", ino,
+		       ctx->dir_info->array[index].dotdot,
+		       ctx->dir_info->array[index].parent);
 #endif
-		return &ctx->dir_info->array[low];
+		return &ctx->dir_info->array[index];
 	}
-	if (ino == ctx->dir_info->array[high].ino) {
-#ifdef DIRINFO_DEBUG
-		printf("(%u,%u,%u)\n", ino,
-		       ctx->dir_info->array[high].dotdot,
-		       ctx->dir_info->array[high].parent);
-#endif
-		return &ctx->dir_info->array[high];
-	}
-
-	while (low < high) {
-		/* sum may overflow, but result will fit into mid again */
-		mid = (unsigned long long)(low + high) / 2;
-		if (mid == low || mid == high)
-			break;
-		if (ino == ctx->dir_info->array[mid].ino) {
-#ifdef DIRINFO_DEBUG
-			printf("(%u,%u,%u)\n", ino,
-			       ctx->dir_info->array[mid].dotdot,
-			       ctx->dir_info->array[mid].parent);
-#endif
-			return &ctx->dir_info->array[mid];
-		}
-		if (ino < ctx->dir_info->array[mid].ino)
-			high = mid;
-		else
-			low = mid;
-	}
-	return 0;
+	return NULL;
 }
 
 static void e2fsck_put_dir_info(e2fsck_t ctx EXT2FS_NO_TDB_UNUSED,
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#32 lore raw

RFC PATCH v3 17/61 e2fsck: merge dir_info after thread finishes

In reply to Saranya Muruganandam

From: Li Xi <[email protected]>

dir_info need be merged after thread finish.

---
 e2fsck/dirinfo.c | 66 ++++++++++++++++++++++++++++++++++++++++++++++++
 e2fsck/e2fsck.h  |  2 ++
 e2fsck/pass1.c   | 58 +++++++++++++++++++++++++++++++++++++++---
 3 files changed, 123 insertions(+), 3 deletions(-)
e2fsck/dirinfo.c +66 -0
index 28baaca2..b4adaa6e 100644
@@ -169,6 +169,72 @@ e2fsck_dir_info_min_larger_equal(struct dir_info_db *dir_info,
 	return -ENOENT;
 }
 
+/*
+ * Merge two sorted dir info to @dest
+ */
+void e2fsck_merge_dir_info(e2fsck_t ctx, struct dir_info_db *src,
+			   struct dir_info_db *dest)
+{
+	size_t		 size_dir_info = sizeof(struct dir_info);
+	ext2_ino_t	 size = dest->size;
+	struct dir_info	 *src_array = src->array;
+	struct dir_info	 *dest_array = dest->array;
+	ext2_ino_t	 src_count = src->count;
+	ext2_ino_t	 dest_count = dest->count;
+	ext2_ino_t	 total_count = src_count + dest_count;
+	struct dir_info	*tmp_array;
+	struct dir_info	*array_ptr;
+	ext2_ino_t	 src_index = 0;
+	ext2_ino_t	 dest_index = 0;
+
+	if (src->count == 0)
+		return;
+
+	if (size < total_count)
+		size = total_count;
+
+	if (size < src->size)
+		size = src->size;
+
+	tmp_array = e2fsck_allocate_memory(ctx, size * size_dir_info,
+					    "directory map");
+	array_ptr = tmp_array;
+	/*
+	 * This can be improved by binary search and memcpy, but codes
+	 * would be more complex. And if the groups distributed to each
+	 * thread are strided, this implementation won't be too bad
+	 * comparing to the optimiztion.
+	 */
+	while (src_index < src_count || dest_index < dest_count) {
+		if (src_index >= src_count) {
+			memcpy(array_ptr, &dest_array[dest_index],
+			       (dest_count - dest_index) * size_dir_info);
+			break;
+		}
+		if (dest_index >= dest_count) {
+			memcpy(array_ptr, &src_array[src_index],
+			       (src_count - src_index) * size_dir_info);
+			break;
+		}
+		if (src_array[src_index].ino < dest_array[dest_index].ino) {
+			*array_ptr = src_array[src_index];
+			src_index++;
+		} else {
+			assert(src_array[src_index].ino >
+			       dest_array[dest_index].ino);
+			*array_ptr = dest_array[dest_index];
+			dest_index++;
+		}
+		array_ptr++;
+	}
+
+	if (dest->array)
+		ext2fs_free_mem(&dest->array);
+	dest->array = tmp_array;
+	dest->size = size;
+	dest->count = total_count;
+}
+
 /*
  *
  * Insert an inode into the sorted array. The array should have at least one
e2fsck/e2fsck.h +2 -0
index 06893f67..6783ed05 100644
@@ -507,6 +507,8 @@ extern void read_bad_blocks_file(e2fsck_t ctx, const char *bad_blocks_file,
 
 /* dirinfo.c */
 extern void e2fsck_add_dir_info(e2fsck_t ctx, ext2_ino_t ino, ext2_ino_t parent);
+void e2fsck_merge_dir_info(e2fsck_t ctx, struct dir_info_db *src,
+                           struct dir_info_db *dest);
 extern void e2fsck_free_dir_info(e2fsck_t ctx);
 extern int e2fsck_get_num_dirinfo(e2fsck_t ctx);
 extern struct dir_info_iter *e2fsck_dir_info_iter_begin(e2fsck_t ctx);
e2fsck/pass1.c +55 -3

94 lines not on this page · raw message

Signature · 1 line
2.29.2.299.gdc1121823c-goog
#33 lore raw

RFC PATCH v3 18/61 e2fsck: rbtree bitmap for dir

In reply to Saranya Muruganandam

From: Wang Shilong <[email protected]>

Only rbtree support merge operation now, use it for bitmaps.

---
 e2fsck/pass1.c | 1 +
 1 file changed, 1 insertion(+)
e2fsck/pass1.c +1 -0
index ef6b2d13..2147f64b 100644
@@ -1235,6 +1235,7 @@ void e2fsck_pass1_run(e2fsck_t ctx)
 	}
 	pctx.errcode = e2fsck_allocate_inode_bitmap(fs,
 			_("directory inode map"),
+			ctx->global_ctx ? EXT2FS_BMAP64_RBTREE :
 			EXT2FS_BMAP64_AUTODIR,
 			"inode_dir_map", &ctx->inode_dir_map);
 	if (pctx.errcode) {
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#34 lore raw

RFC PATCH v3 19/61 e2fsck: merge badblocks after thread finishes

In reply to Saranya Muruganandam

From: Wang Shilong <[email protected]>

Badblocks should be merged properly after threads finish.

---
 e2fsck/pass1.c         | 19 +++++++---
 lib/ext2fs/badblocks.c | 85 ++++++++++++++++++++++++++++++++++++++----
 lib/ext2fs/ext2fs.h    |  2 +
 lib/ext2fs/ext2fsP.h   |  1 -
 4 files changed, 94 insertions(+), 13 deletions(-)
e2fsck/pass1.c +14 -5
index 2147f64b..da42323d 100644
@@ -2180,6 +2180,7 @@ static errcode_t e2fsck_pass1_copy_fs(ext2_filsys dest, e2fsck_t src_context,
 	memcpy(dest, src, sizeof(struct struct_ext2_filsys));
 	dest->inode_map = NULL;
 	dest->block_map = NULL;
+	dest->badblocks = NULL;
 	if (dest->dblist)
 		dest->dblist->fs = dest;
 	if (src->block_map) {
@@ -2196,7 +2197,8 @@ static errcode_t e2fsck_pass1_copy_fs(ext2_filsys dest, e2fsck_t src_context,
 	}
 
 	if (src->badblocks) {
-		retval = ext2fs_badblocks_copy(src->badblocks, &dest->badblocks);
+		retval = ext2fs_badblocks_copy(src->badblocks,
+					       &dest->badblocks);
 		if (retval)
 			return retval;
 	}
@@ -2241,11 +2243,13 @@ static int e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
 	io_channel dest_image_io;
 	ext2fs_inode_bitmap inode_map;
 	ext2fs_block_bitmap block_map;
+	ext2_badblocks_list badblocks;
 
 	dest_io = dest->io;
 	dest_image_io = dest->image_io;
 	inode_map = dest->inode_map;
 	block_map = dest->block_map;
+	badblocks = dest->badblocks;
 
 	memcpy(dest, src, sizeof(struct struct_ext2_filsys));
 	dest->io = dest_io;
@@ -2253,6 +2257,7 @@ static int e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
 	dest->icache = icache;
 	dest->inode_map = inode_map;
 	dest->block_map = block_map;
+	dest->badblocks = badblocks;
 	if (dest->dblist)
 		dest->dblist->fs = dest;
 
@@ -2272,10 +2277,12 @@ static int e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
 		goto out;
 
 	if (src->badblocks) {
-		retval = ext2fs_badblocks_copy(src->badblocks, &dest->badblocks);
-
-		ext2fs_badblocks_list_free(src->badblocks);
-		src->badblocks = NULL;
+		if (dest->badblocks == NULL)
+			retval = ext2fs_badblocks_copy(src->badblocks,
+						       &dest->badblocks);
+		else
+			retval = ext2fs_badblocks_merge(src->badblocks,
+							dest->badblocks);
 	}
 out:
 	io_channel_close(src->io);
@@ -2283,6 +2290,8 @@ out:
 		ext2fs_free_generic_bmap(src->inode_map);
 	if (src->block_map)
 		ext2fs_free_generic_bmap(src->block_map);
+	if (src->badblocks)
+		ext2fs_badblocks_list_free(src->badblocks);
 	return retval;
 }
 
lib/ext2fs/badblocks.c +78 -7

109 lines not on this page · raw message

lib/ext2fs/ext2fs.h +2 -0
index 0aa1d94e..ac548311 100644
@@ -814,6 +814,8 @@ extern int ext2fs_badblocks_list_iterate(ext2_badblocks_iterate iter,
 extern void ext2fs_badblocks_list_iterate_end(ext2_badblocks_iterate iter);
 extern errcode_t ext2fs_badblocks_copy(ext2_badblocks_list src,
 				       ext2_badblocks_list *dest);
+extern errcode_t ext2fs_badblocks_merge(ext2_badblocks_list src,
+					ext2_badblocks_list dest);
 extern int ext2fs_badblocks_equal(ext2_badblocks_list bb1,
 				  ext2_badblocks_list bb2);
 extern int ext2fs_u32_list_count(ext2_u32_list bb);
lib/ext2fs/ext2fsP.h +0 -1
index ad8b7d52..02df759a 100644
@@ -34,7 +34,6 @@ struct ext2_struct_u32_list {
 	int	num;
 	int	size;
 	__u32	*list;
-	int	badblocks_flags;
 };
 
 struct ext2_struct_u32_iterate {
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#35 lore raw

RFC PATCH v3 20/61 e2fsck: merge icounts after thread finishes

In reply to Saranya Muruganandam

From: Li Xi <[email protected]>

Merge inode_count and inode_link_info properly after threads finish.

---
 e2fsck/pass1.c      |  47 ++++++++++++++++++++
 lib/ext2fs/ext2fs.h |   1 +
 lib/ext2fs/icount.c | 103 ++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 151 insertions(+)
e2fsck/pass1.c +47 -0
index da42323d..cdecd7c2 100644
@@ -2388,6 +2388,41 @@ static void e2fsck_pass1_merge_dir_info(e2fsck_t global_ctx, e2fsck_t thread_ctx
 			      global_ctx->dir_info);
 }
 
+static inline errcode_t
+e2fsck_pass1_merge_icount(ext2_icount_t *dest_icount,
+			  ext2_icount_t *src_icount)
+{
+	if (*src_icount) {
+		if (*dest_icount == NULL) {
+			*dest_icount = *src_icount;
+			*src_icount = NULL;
+		} else {
+			errcode_t ret;
+
+			ret = ext2fs_icount_merge(*src_icount,
+						  *dest_icount);
+			if (ret)
+				return ret;
+		}
+	}
+
+	return 0;
+}
+
+static errcode_t e2fsck_pass1_merge_icounts(e2fsck_t global_ctx, e2fsck_t thread_ctx)
+{
+	errcode_t ret;
+
+	ret = e2fsck_pass1_merge_icount(&global_ctx->inode_count,
+					&thread_ctx->inode_count);
+	if (ret)
+		return ret;
+	ret = e2fsck_pass1_merge_icount(&global_ctx->inode_link_info,
+					&thread_ctx->inode_link_info);
+
+	return ret;
+}
+
 static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx)
 {
 	errcode_t	 retval;
@@ -2408,6 +2443,8 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
 	ext2fs_block_bitmap block_ea_map = global_ctx->block_ea_map;
 	ext2fs_block_bitmap block_metadata_map = global_ctx->block_metadata_map;
 	ext2fs_block_bitmap inodes_to_rebuild = global_ctx->inodes_to_rebuild;
+	ext2_icount_t inode_count = global_ctx->inode_count;
+	ext2_icount_t inode_link_info = global_ctx->inode_link_info;
 
 #ifdef HAVE_SETJMP_H
 	jmp_buf		 old_jmp;
@@ -2432,6 +2469,8 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
 	global_ctx->block_metadata_map = block_metadata_map;
 	global_ctx->dir_info = dir_info;
 	e2fsck_pass1_merge_dir_info(global_ctx, thread_ctx);
+	global_ctx->inode_count = inode_count;
+	global_ctx->inode_link_info = inode_link_info;
 
 	/* Keep the global singal flags*/
 	global_ctx->flags |= (flags & E2F_FLAG_SIGNAL_MASK) |
@@ -2447,6 +2486,12 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
 	global_ctx->logf = global_logf;
 	global_ctx->problem_logf = global_problem_logf;
 	global_ctx->global_ctx = NULL;
+	retval = e2fsck_pass1_merge_icounts(global_ctx, thread_ctx);
+	if (retval) {
+		com_err(global_ctx->program_name, 0,
+			_("while merging icounts\n"));
+		return retval;
+	}
 
 	retval = e2fsck_pass1_merge_bitmap(global_fs,
 				&thread_ctx->inode_used_map,
@@ -2532,6 +2577,8 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
 	e2fsck_pass1_free_bitmap(&thread_ctx->block_ea_map);
 	e2fsck_pass1_free_bitmap(&thread_ctx->block_metadata_map);
 	e2fsck_free_dir_info(thread_ctx);
+	ext2fs_free_icount(thread_ctx->inode_count);
+	ext2fs_free_icount(thread_ctx->inode_link_info);
 	ext2fs_free_mem(&thread_ctx);
 
 	return retval;
lib/ext2fs/ext2fs.h +1 -0
index ac548311..28662d44 100644
@@ -1513,6 +1513,7 @@ extern errcode_t ext2fs_icount_decrement(ext2_icount_t icount, ext2_ino_t ino,
 					 __u16 *ret);
 extern errcode_t ext2fs_icount_store(ext2_icount_t icount, ext2_ino_t ino,
 				     __u16 count);
+extern errcode_t ext2fs_icount_merge(ext2_icount_t src, ext2_icount_t dest);
 extern ext2_ino_t ext2fs_get_icount_size(ext2_icount_t icount);
 errcode_t ext2fs_icount_validate(ext2_icount_t icount, FILE *);
 
lib/ext2fs/icount.c +103 -0

115 lines not on this page · raw message

Signature · 1 line
2.29.2.299.gdc1121823c-goog
#36 lore raw

RFC PATCH v3 21/61 e2fsck: merge dblist after thread finishes

In reply to Saranya Muruganandam

From: Li Xi <[email protected]>

Merge dblist properly.

---
 e2fsck/pass1.c      | 19 +++++++++++++++++++
 lib/ext2fs/dblist.c | 36 ++++++++++++++++++++++++++++++++++++
 lib/ext2fs/ext2fs.h |  1 +
 3 files changed, 56 insertions(+)
e2fsck/pass1.c +19 -0
index cdecd7c2..75298d9d 100644
@@ -2244,12 +2244,14 @@ static int e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
 	ext2fs_inode_bitmap inode_map;
 	ext2fs_block_bitmap block_map;
 	ext2_badblocks_list badblocks;
+	ext2_dblist dblist;
 
 	dest_io = dest->io;
 	dest_image_io = dest->image_io;
 	inode_map = dest->inode_map;
 	block_map = dest->block_map;
 	badblocks = dest->badblocks;
+	dblist = dest->dblist;
 
 	memcpy(dest, src, sizeof(struct struct_ext2_filsys));
 	dest->io = dest_io;
@@ -2258,6 +2260,7 @@ static int e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
 	dest->inode_map = inode_map;
 	dest->block_map = block_map;
 	dest->badblocks = badblocks;
+	dest->dblist = dblist;
 	if (dest->dblist)
 		dest->dblist->fs = dest;
 
@@ -2276,6 +2279,19 @@ static int e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
 	if (retval)
 		goto out;
 
+	if (src->dblist) {
+		if (dest->dblist) {
+			retval = ext2fs_merge_dblist(src->dblist,
+						     dest->dblist);
+			if (retval)
+				goto out;
+		} else {
+			dest->dblist = src->dblist;
+			dest->dblist->fs = dest;
+			src->dblist = NULL;
+		}
+	}
+
 	if (src->badblocks) {
 		if (dest->badblocks == NULL)
 			retval = ext2fs_badblocks_copy(src->badblocks,
@@ -2292,6 +2308,9 @@ out:
 		ext2fs_free_generic_bmap(src->block_map);
 	if (src->badblocks)
 		ext2fs_badblocks_list_free(src->badblocks);
+	if (src->dblist)
+		ext2fs_free_dblist(src->dblist);
+
 	return retval;
 }
 
lib/ext2fs/dblist.c +36 -0
index bbdb221d..1fdd8f43 100644
@@ -119,6 +119,42 @@ errcode_t ext2fs_copy_dblist(ext2_dblist src, ext2_dblist *dest)
 	return 0;
 }
 
+/*
+ * Merge a directory block list @src to @dest
+ */
+errcode_t ext2fs_merge_dblist(ext2_dblist src, ext2_dblist dest)
+{
+	unsigned long long src_count = src->count;
+	unsigned long long dest_count = dest->count;
+	unsigned long long size = src_count + dest_count;
+	size_t size_entry = sizeof(struct ext2_db_entry2);
+	struct ext2_db_entry2 *array, *array2;
+	errcode_t retval;
+
+	if (src_count == 0)
+		return 0;
+
+	if (src->sorted || (dest->sorted && dest_count != 0))
+		return EINVAL;
+
+	retval = ext2fs_get_array(size, size_entry, &array);
+	if (retval)
+		return retval;
+
+	array2 = array;
+	memcpy(array, src->list, src_count * size_entry);
+	array += src_count;
+	memcpy(array, dest->list, dest_count * size_entry);
+	ext2fs_free_mem(&dest->list);
+
+	dest->list = array2;
+	dest->count = src_count + dest_count;
+	dest->size = size;
+	dest->sorted = 0;
+
+	return 0;
+}
+
 /*
  * Close a directory block list
  *
lib/ext2fs/ext2fs.h +1 -0
index 28662d44..0fa0e22f 100644
@@ -1122,6 +1122,7 @@ extern errcode_t ext2fs_add_dir_block(ext2_dblist dblist, ext2_ino_t ino,
 				      blk_t blk, int blockcnt);
 extern errcode_t ext2fs_add_dir_block2(ext2_dblist dblist, ext2_ino_t ino,
 				       blk64_t blk, e2_blkcnt_t blockcnt);
+extern errcode_t ext2fs_merge_dblist(ext2_dblist src, ext2_dblist dest);
 extern void ext2fs_dblist_sort(ext2_dblist dblist,
 			       EXT2_QSORT_TYPE (*sortfunc)(const void *,
 							   const void *));
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#37 lore raw

RFC PATCH v3 22/61 e2fsck: add debug codes for multiple threads

In reply to Saranya Muruganandam

From: Li Xi <[email protected]>

These debug codes are added to run the multiple pass1 check thread one by one in order. If all the codes are correct, fsck of multiple threads should have exactly the same outcome with single thread.

---
 e2fsck/e2fsck.h | 16 ++++++++++++++++
 e2fsck/pass1.c  | 29 +++++++++++++++++++++++++++++
 2 files changed, 45 insertions(+)
e2fsck/e2fsck.h +16 -0
index 6783ed05..972c8410 100644
@@ -450,6 +450,18 @@ struct e2fsck_struct {
 };
 
 #ifdef CONFIG_PFSCK
+#ifdef DEBUG_THREADS
+/*
+ * Enabling DEBUG_THREADS would cause the parallel
+ * fsck threads run sequentially.
+ */
+struct e2fsck_thread_debug {
+	pthread_mutex_t	etd_mutex;
+	pthread_cond_t	etd_cond;
+	int		etd_finished_threads;
+};
+#endif
+
 struct e2fsck_thread_info {
 	/* ID returned by pthread_create() */
 	pthread_t		 eti_thread_id;
@@ -459,7 +471,11 @@ struct e2fsck_thread_info {
 	int			 eti_started;
 	/* Context used for this thread */
 	e2fsck_t		 eti_thread_ctx;
+#ifdef DEBUG_THREADS
+	struct e2fsck_thread_debug	*eti_debug;
+#endif
 };
+
 #endif
 
 /* Data structures to evaluate whether an extent tree needs rebuilding. */
e2fsck/pass1.c +29 -0
index 75298d9d..f36b3e70 100644
@@ -2641,6 +2641,18 @@ static void *e2fsck_pass1_thread(void *arg)
 {
 	struct e2fsck_thread_info	*info = arg;
 	e2fsck_t			 thread_ctx = info->eti_thread_ctx;
+#ifdef DEBUG_THREADS
+	struct e2fsck_thread_debug	*thread_debug = info->eti_debug;
+#endif
+
+#ifdef DEBUG_THREADS
+	pthread_mutex_lock(&thread_debug->etd_mutex);
+	while (info->eti_thread_index > thread_debug->etd_finished_threads) {
+		pthread_cond_wait(&thread_debug->etd_cond,
+				  &thread_debug->etd_mutex);
+	}
+	pthread_mutex_unlock(&thread_debug->etd_mutex);
+#endif
 
 #ifdef HAVE_SETJMP_H
 	/*
@@ -2665,6 +2677,14 @@ out:
 			thread_ctx->thread_info.et_group_start,
 			thread_ctx->thread_info.et_group_end,
 			thread_ctx->thread_info.et_inode_number);
+
+#ifdef DEBUG_THREADS
+	pthread_mutex_lock(&thread_debug->etd_mutex);
+	thread_debug->etd_finished_threads++;
+	pthread_cond_broadcast(&thread_debug->etd_cond);
+	pthread_mutex_unlock(&thread_debug->etd_mutex);
+#endif
+
 	return NULL;
 }
 
@@ -2678,6 +2698,12 @@ static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo,
 	struct e2fsck_thread_info	*tmp_pinfo;
 	int				 i;
 	e2fsck_t			 thread_ctx;
+#ifdef DEBUG_THREADS
+	struct e2fsck_thread_debug	 thread_debug =
+		{PTHREAD_MUTEX_INITIALIZER, PTHREAD_COND_INITIALIZER, 0};
+
+	thread_debug.etd_finished_threads = 0;
+#endif
 
 	retval = pthread_attr_init(&attr);
 	if (retval) {
@@ -2698,6 +2724,9 @@ static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo,
 	for (i = 0; i < num_threads; i++) {
 		tmp_pinfo = &infos[i];
 		tmp_pinfo->eti_thread_index = i;
+#ifdef DEBUG_THREADS
+		tmp_pinfo->eti_debug = &thread_debug;
+#endif
 		retval = e2fsck_pass1_thread_prepare(global_ctx, &thread_ctx,
 						     i, num_threads);
 		if (retval) {
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#38 lore raw

RFC PATCH v3 23/61 e2fsck: merge counts after threads finish

In reply to Saranya Muruganandam

From: Wang Shilong <[email protected]>

Merge counts properly.

---
 e2fsck/pass1.c | 34 ++++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)
e2fsck/pass1.c +34 -0
index f36b3e70..af0ff724 100644
@@ -2464,6 +2464,23 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
 	ext2fs_block_bitmap inodes_to_rebuild = global_ctx->inodes_to_rebuild;
 	ext2_icount_t inode_count = global_ctx->inode_count;
 	ext2_icount_t inode_link_info = global_ctx->inode_link_info;
+	__u32 fs_directory_count = global_ctx->fs_directory_count;
+	__u32 fs_regular_count = global_ctx->fs_regular_count;
+	__u32 fs_blockdev_count = global_ctx->fs_blockdev_count;
+	__u32 fs_chardev_count = global_ctx->fs_chardev_count;
+	__u32 fs_links_count = global_ctx->fs_links_count;
+	__u32 fs_symlinks_count = global_ctx->fs_symlinks_count;
+	__u32 fs_fast_symlinks_count = global_ctx->fs_fast_symlinks_count;
+	__u32 fs_fifo_count = global_ctx->fs_fifo_count;
+	__u32 fs_total_count = global_ctx->fs_total_count;
+	__u32 fs_badblocks_count = global_ctx->fs_badblocks_count;
+	__u32 fs_sockets_count = global_ctx->fs_sockets_count;
+	__u32 fs_ind_count = global_ctx->fs_ind_count;
+	__u32 fs_dind_count = global_ctx->fs_dind_count;
+	__u32 fs_tind_count = global_ctx->fs_tind_count;
+	__u32 fs_fragmented = global_ctx->fs_fragmented;
+	__u32 fs_fragmented_dir = global_ctx->fs_fragmented_dir;
+	__u32 large_files = global_ctx->large_files;
 
 #ifdef HAVE_SETJMP_H
 	jmp_buf		 old_jmp;
@@ -2490,6 +2507,23 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
 	e2fsck_pass1_merge_dir_info(global_ctx, thread_ctx);
 	global_ctx->inode_count = inode_count;
 	global_ctx->inode_link_info = inode_link_info;
+	global_ctx->fs_directory_count += fs_directory_count;
+	global_ctx->fs_regular_count += fs_regular_count;
+	global_ctx->fs_blockdev_count += fs_blockdev_count;
+	global_ctx->fs_chardev_count += fs_chardev_count;
+	global_ctx->fs_links_count += fs_links_count;
+	global_ctx->fs_symlinks_count += fs_symlinks_count;
+	global_ctx->fs_fast_symlinks_count += fs_fast_symlinks_count;
+	global_ctx->fs_fifo_count += fs_fifo_count;
+	global_ctx->fs_total_count += fs_total_count;
+	global_ctx->fs_badblocks_count += fs_badblocks_count;
+	global_ctx->fs_sockets_count += fs_sockets_count;
+	global_ctx->fs_ind_count += fs_ind_count;
+	global_ctx->fs_dind_count += fs_dind_count;
+	global_ctx->fs_tind_count += fs_tind_count;
+	global_ctx->fs_fragmented += fs_fragmented;
+	global_ctx->fs_fragmented_dir += fs_fragmented_dir;
+	global_ctx->large_files += large_files;
 
 	/* Keep the global singal flags*/
 	global_ctx->flags |= (flags & E2F_FLAG_SIGNAL_MASK) |
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#39 lore raw

RFC PATCH v3 24/61 e2fsck: merge fs flags when threads finish

In reply to Saranya Muruganandam

From: Li Xi <[email protected]>

merge fs flags properly.

---
 e2fsck/pass1.c | 5 +++++
 1 file changed, 5 insertions(+)
e2fsck/pass1.c +5 -0
index af0ff724..783e14f0 100644
@@ -2245,6 +2245,7 @@ static int e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
 	ext2fs_block_bitmap block_map;
 	ext2_badblocks_list badblocks;
 	ext2_dblist dblist;
+	int flags;
 
 	dest_io = dest->io;
 	dest_image_io = dest->image_io;
@@ -2252,6 +2253,7 @@ static int e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
 	block_map = dest->block_map;
 	badblocks = dest->badblocks;
 	dblist = dest->dblist;
+	flags = dest->flags;
 
 	memcpy(dest, src, sizeof(struct struct_ext2_filsys));
 	dest->io = dest_io;
@@ -2263,6 +2265,9 @@ static int e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
 	dest->dblist = dblist;
 	if (dest->dblist)
 		dest->dblist->fs = dest;
+	dest->flags = src->flags | flags;
+	if (!(src->flags & EXT2_FLAG_VALID) || !(flags & EXT2_FLAG_VALID))
+		ext2fs_unmark_valid(dest);
 
 	if (src->icache) {
 		ext2fs_free_inode_cache(src->icache);
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#40 lore raw

RFC PATCH v3 25/61 e2fsck: merge dx_dir_info after threads finish

In reply to Saranya Muruganandam

From: Wang Shilong <[email protected]>

Merge properly.

---
 e2fsck/dx_dirinfo.c | 64 +++++++++++++++++++++++++++++++++++++++++++++
 e2fsck/e2fsck.h     |  1 +
 e2fsck/pass1.c      | 23 ++++++++++++++++
 3 files changed, 88 insertions(+)
e2fsck/dx_dirinfo.c +64 -0
index caca3e30..91954572 100644
@@ -5,6 +5,7 @@
  * under the terms of the GNU Public License.
  */
 
+#include <assert.h>
 #include "config.h"
 #include "e2fsck.h"
 
@@ -79,6 +80,69 @@ void e2fsck_add_dx_dir(e2fsck_t ctx, ext2_ino_t ino, struct ext2_inode *inode,
 				       "dx_block info array");
 }
 
+/*
+ * Merge two sorted dir info to @dest
+ */
+void e2fsck_merge_dx_dir(e2fsck_t global_ctx, e2fsck_t thread_ctx)
+{
+	struct dx_dir_info *src_array = thread_ctx->dx_dir_info;
+	struct dx_dir_info *dest_array = global_ctx->dx_dir_info;
+	size_t size_dx_info = sizeof(struct dx_dir_info);
+	ext2_ino_t size = global_ctx->dx_dir_info_size;
+	ext2_ino_t src_count = thread_ctx->dx_dir_info_count;
+	ext2_ino_t dest_count = global_ctx->dx_dir_info_count;
+	ext2_ino_t total_count = src_count + dest_count;
+	struct dx_dir_info *array;
+	struct dx_dir_info *array_ptr;
+	ext2_ino_t src_index = 0, dest_index = 0;
+
+	if (thread_ctx->dx_dir_info_count == 0)
+		return;
+
+	if (size < total_count)
+		size = total_count;
+
+	array = e2fsck_allocate_memory(global_ctx, size * size_dx_info,
+				       "directory map");
+	array_ptr = array;
+	/*
+	 * This can be improved by binary search and memcpy, but codes
+	 * would be more complex. And if the groups distributed to each
+	 * thread are strided, this implementation won't be too bad
+	 * comparing to the optimiztion.
+	 */
+	while (src_index < src_count || dest_index < dest_count) {
+		if (src_index >= src_count) {
+			memcpy(array_ptr, &dest_array[dest_index],
+			       (dest_count - dest_index) * size_dx_info);
+			break;
+		}
+		if (dest_index >= dest_count) {
+			memcpy(array_ptr, &src_array[src_index],
+			       (src_count - src_index) * size_dx_info);
+			break;
+		}
+		if (src_array[src_index].ino < dest_array[dest_index].ino) {
+			*array_ptr = src_array[src_index];
+			src_index++;
+		} else {
+			assert(src_array[src_index].ino >
+			       dest_array[dest_index].ino);
+			*array_ptr = dest_array[dest_index];
+			dest_index++;
+		}
+		array_ptr++;
+	}
+
+	if (global_ctx->dx_dir_info)
+		ext2fs_free_mem(&global_ctx->dx_dir_info);
+	if (thread_ctx->dx_dir_info)
+		ext2fs_free_mem(&thread_ctx->dx_dir_info);
+	global_ctx->dx_dir_info = array;
+	global_ctx->dx_dir_info_size = size;
+	global_ctx->dx_dir_info_count = total_count;
+}
+
 /*
  * get_dx_dir_info() --- given an inode number, try to find the directory
  * information entry for it.
e2fsck/e2fsck.h +1 -0
index 972c8410..c3b0af34 100644
@@ -539,6 +539,7 @@ extern int e2fsck_dir_info_get_parent(e2fsck_t ctx, ext2_ino_t ino,
 				      ext2_ino_t *parent);
 extern int e2fsck_dir_info_get_dotdot(e2fsck_t ctx, ext2_ino_t ino,
 				      ext2_ino_t *dotdot);
+extern void e2fsck_merge_dx_dir(e2fsck_t global_ctx, e2fsck_t thread_ctx);
 
 /* dx_dirinfo.c */
 extern void e2fsck_add_dx_dir(e2fsck_t ctx, ext2_ino_t ino,
e2fsck/pass1.c +23 -0
index 783e14f0..7095e8b4 100644
@@ -2412,6 +2412,22 @@ static void e2fsck_pass1_merge_dir_info(e2fsck_t global_ctx, e2fsck_t thread_ctx
 			      global_ctx->dir_info);
 }
 
+static void e2fsck_pass1_merge_dx_dir(e2fsck_t global_ctx, e2fsck_t thread_ctx)
+{
+	if (thread_ctx->dx_dir_info == NULL)
+		return;
+
+	if (global_ctx->dx_dir_info == NULL) {
+		global_ctx->dx_dir_info = thread_ctx->dx_dir_info;
+		global_ctx->dx_dir_info_size = thread_ctx->dx_dir_info_size;
+		global_ctx->dx_dir_info_count = thread_ctx->dx_dir_info_count;
+		thread_ctx->dx_dir_info = NULL;
+		return;
+	}
+
+	e2fsck_merge_dx_dir(global_ctx, thread_ctx);
+}
+
 static inline errcode_t
 e2fsck_pass1_merge_icount(ext2_icount_t *dest_icount,
 			  ext2_icount_t *src_icount)
@@ -2457,6 +2473,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
 	FILE		*global_problem_logf = global_ctx->problem_logf;
 	ext2fs_inode_bitmap inode_bad_map = global_ctx->inode_bad_map;
 	struct dir_info_db *dir_info = global_ctx->dir_info;
+	struct dx_dir_info *dx_dir_info = global_ctx->dx_dir_info;
 	ext2fs_inode_bitmap inode_used_map = global_ctx->inode_used_map;
 	ext2fs_inode_bitmap inode_dir_map = global_ctx->inode_dir_map;
 	ext2fs_inode_bitmap inode_bb_map = global_ctx->inode_bb_map;
@@ -2486,6 +2503,8 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
 	__u32 fs_fragmented = global_ctx->fs_fragmented;
 	__u32 fs_fragmented_dir = global_ctx->fs_fragmented_dir;
 	__u32 large_files = global_ctx->large_files;
+	ext2_ino_t dx_dir_info_size = global_ctx->dx_dir_info_size;
+	ext2_ino_t dx_dir_info_count = global_ctx->dx_dir_info_count;
 
 #ifdef HAVE_SETJMP_H
 	jmp_buf		 old_jmp;
@@ -2510,6 +2529,10 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
 	global_ctx->block_metadata_map = block_metadata_map;
 	global_ctx->dir_info = dir_info;
 	e2fsck_pass1_merge_dir_info(global_ctx, thread_ctx);
+	global_ctx->dx_dir_info = dx_dir_info;
+	global_ctx->dx_dir_info_count = dx_dir_info_count;
+	global_ctx->dx_dir_info_size = dx_dir_info_size;
+	e2fsck_pass1_merge_dx_dir(global_ctx, thread_ctx);
 	global_ctx->inode_count = inode_count;
 	global_ctx->inode_link_info = inode_link_info;
 	global_ctx->fs_directory_count += fs_directory_count;
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#41 lore raw

RFC PATCH v3 26/61 e2fsck: merge dirs_to_hash when threads finish

In reply to Saranya Muruganandam

From: Wang Shilong <[email protected]>

@dirs_to_hash list need be merged after threads finish, test covered by t_dangerous.

---
 e2fsck/pass1.c | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)
e2fsck/pass1.c +30 -0
index 7095e8b4..5378d7da 100644
@@ -2463,6 +2463,25 @@ static errcode_t e2fsck_pass1_merge_icounts(e2fsck_t global_ctx, e2fsck_t thread
 	return ret;
 }
 
+static errcode_t e2fsck_pass1_merge_dirs_to_hash(e2fsck_t global_ctx,
+						 e2fsck_t thread_ctx)
+{
+	errcode_t retval = 0;
+
+	if (!thread_ctx->dirs_to_hash)
+		return 0;
+
+	if (!global_ctx->dirs_to_hash)
+		retval = ext2fs_badblocks_copy(thread_ctx->dirs_to_hash,
+					       &global_ctx->dirs_to_hash);
+	else
+		retval = ext2fs_badblocks_merge(thread_ctx->dirs_to_hash,
+						global_ctx->dirs_to_hash);
+
+	return retval;
+}
+
+
 static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx)
 {
 	errcode_t	 retval;
@@ -2505,6 +2524,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
 	__u32 large_files = global_ctx->large_files;
 	ext2_ino_t dx_dir_info_size = global_ctx->dx_dir_info_size;
 	ext2_ino_t dx_dir_info_count = global_ctx->dx_dir_info_count;
+	ext2_u32_list dirs_to_hash = global_ctx->dirs_to_hash;
 
 #ifdef HAVE_SETJMP_H
 	jmp_buf		 old_jmp;
@@ -2574,6 +2594,14 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
 		return retval;
 	}
 
+	global_ctx->dirs_to_hash = dirs_to_hash;
+	retval = e2fsck_pass1_merge_dirs_to_hash(global_ctx, thread_ctx);
+	if (retval) {
+		com_err(global_ctx->program_name, 0,
+			_("while merging dirs to hash\n"));
+		return retval;
+	}
+
 	retval = e2fsck_pass1_merge_bitmap(global_fs,
 				&thread_ctx->inode_used_map,
 				&global_ctx->inode_used_map);
@@ -2660,6 +2688,8 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
 	e2fsck_free_dir_info(thread_ctx);
 	ext2fs_free_icount(thread_ctx->inode_count);
 	ext2fs_free_icount(thread_ctx->inode_link_info);
+	if (thread_ctx->dirs_to_hash)
+		ext2fs_badblocks_list_free(thread_ctx->dirs_to_hash);
 	ext2fs_free_mem(&thread_ctx);
 
 	return retval;
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#42 lore raw

RFC PATCH v3 27/61 e2fsck: merge context flags properly

In reply to Saranya Muruganandam

From: Wang Shilong <[email protected]>

e2fsck might restart after pass1, so we should keep flags if possible, this patch try to fix f_illitable_flexbg failure

---
 e2fsck/pass1.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)
e2fsck/pass1.c +1 -3
index 5378d7da..35ab9cae 100644
@@ -2573,9 +2573,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
 	global_ctx->fs_fragmented_dir += fs_fragmented_dir;
 	global_ctx->large_files += large_files;
 
-	/* Keep the global singal flags*/
-	global_ctx->flags |= (flags & E2F_FLAG_SIGNAL_MASK) |
-			     (global_ctx->flags & E2F_FLAG_SIGNAL_MASK);
+	global_ctx->flags |= flags;
 
 	retval = e2fsck_pass1_merge_fs(global_fs, thread_fs);
 	if (retval) {
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#43 lore raw

RFC PATCH v3 28/61 e2fsck: merge quota context after threads finish

In reply to Saranya Muruganandam

From: Wang Shilong <[email protected]>

Every threads calculate its own quota accounting,
merge them after threads finish.

---
 e2fsck/pass1.c        | 15 ++++++++++++++-
 lib/support/mkquota.c | 39 +++++++++++++++++++++++++++++++++++++++
 lib/support/quotaio.h |  3 +++
 3 files changed, 56 insertions(+), 1 deletion(-)
e2fsck/pass1.c +14 -1
index 35ab9cae..f2476261 100644
@@ -2388,6 +2388,12 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thre
 		log_out(thread_context, _("Scan group range [%d, %d)\n"),
 			tinfo->et_group_start, tinfo->et_group_end);
 	thread_context->fs = thread_fs;
+	retval = quota_init_context(&thread_context->qctx, thread_fs, 0);
+	if (retval) {
+		com_err(global_ctx->program_name, retval,
+			"while init quota context");
+		goto out_fs;
+	}
 	*thread_ctx = thread_context;
 	return 0;
 out_fs:
@@ -2481,7 +2487,6 @@ static errcode_t e2fsck_pass1_merge_dirs_to_hash(e2fsck_t global_ctx,
 	return retval;
 }
 
-
 static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx)
 {
 	errcode_t	 retval;
@@ -2525,6 +2530,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
 	ext2_ino_t dx_dir_info_size = global_ctx->dx_dir_info_size;
 	ext2_ino_t dx_dir_info_count = global_ctx->dx_dir_info_count;
 	ext2_u32_list dirs_to_hash = global_ctx->dirs_to_hash;
+	quota_ctx_t qctx = global_ctx->qctx;
 
 #ifdef HAVE_SETJMP_H
 	jmp_buf		 old_jmp;
@@ -2600,6 +2606,12 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
 		return retval;
 	}
 
+	global_ctx->qctx = qctx;
+	retval = quota_merge_and_update_usage(global_ctx->qctx,
+					      thread_ctx->qctx);
+	if (retval)
+		return retval;
+
 	retval = e2fsck_pass1_merge_bitmap(global_fs,
 				&thread_ctx->inode_used_map,
 				&global_ctx->inode_used_map);
@@ -2688,6 +2700,7 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
 	ext2fs_free_icount(thread_ctx->inode_link_info);
 	if (thread_ctx->dirs_to_hash)
 		ext2fs_badblocks_list_free(thread_ctx->dirs_to_hash);
+	quota_release_context(&thread_ctx->qctx);
 	ext2fs_free_mem(&thread_ctx);
 
 	return retval;
lib/support/mkquota.c +39 -0
index 6f7ae6d6..c16a3d0a 100644
@@ -639,6 +639,45 @@ out:
 	return err;
 }
 
+static errcode_t merge_usage(dict_t *dest, dict_t *src)
+{
+	dnode_t *n;
+	struct dquot *src_dq, *dest_dq;
+
+	for (n = dict_first(src); n; n = dict_next(src, n)) {
+		src_dq = dnode_get(n);
+		if (!src_dq)
+			continue;
+		dest_dq = get_dq(dest, src_dq->dq_id);
+		if (dest_dq == NULL)
+			return -ENOMEM;
+		dest_dq->dq_dqb.dqb_curspace += src_dq->dq_dqb.dqb_curspace;
+		dest_dq->dq_dqb.dqb_curinodes += src_dq->dq_dqb.dqb_curinodes;
+	}
+
+	return 0;
+}
+
+
+errcode_t quota_merge_and_update_usage(quota_ctx_t dest_qctx,
+					quota_ctx_t src_qctx)
+{
+	dict_t *dict;
+	enum quota_type	qtype;
+	errcode_t retval = 0;
+
+	for (qtype = 0; qtype < MAXQUOTAS; qtype++) {
+		dict = src_qctx->quota_dict[qtype];
+		if (!dict)
+			continue;
+		retval = merge_usage(dest_qctx->quota_dict[qtype], dict);
+		if (retval)
+			break;
+	}
+
+	return retval;
+}
+
 /*
  * Compares the measured quota in qctx->quota_dict with that in the quota inode
  * on disk and updates the limits in qctx->quota_dict. 'usage_inconsistent' is
lib/support/quotaio.h +3 -0
index 60689700..bca295a1 100644
@@ -40,6 +40,7 @@
 #include "ext2fs/ext2_fs.h"
 #include "ext2fs/ext2fs.h"
 #include "dqblk_v2.h"
+#include "support/dict.h"
 
 typedef int64_t qsize_t;	/* Type in which we store size limitations */
 
@@ -233,6 +234,8 @@ int quota_file_exists(ext2_filsys fs, enum quota_type qtype);
 void quota_set_sb_inum(ext2_filsys fs, ext2_ino_t ino, enum quota_type qtype);
 errcode_t quota_compare_and_update(quota_ctx_t qctx, enum quota_type qtype,
 				   int *usage_inconsistent);
+errcode_t quota_merge_and_update_usage(quota_ctx_t dest_qctx,
+					quota_ctx_t src_qctx);
 int parse_quota_opts(const char *opts, int (*func)(char *));
 
 /* parse_qtype.c */
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#44 lore raw

RFC PATCH v3 29/61 e2fsck: serialize fix operations

In reply to Saranya Muruganandam

From: Wang Shilong <[email protected]>

Allow different threads to fix at the same time could be dangerous and error-prone now, and most of time parallel scanning and checking is important.

So this patch adds a mutex to serialize
fix operations during pass1.

And the good benefit of this, we don't need block allocations and free, superblock updates protection any more, since only fix operations during pass1 could touch them.

---
 e2fsck/e2fsck.h |  6 +++++
 e2fsck/pass1.c  | 63 +++++++++++++++++++++++++++++++++++++++++++------
 e2fsck/util.c   | 38 +++++++++++++++++++++++++++++
 3 files changed, 100 insertions(+), 7 deletions(-)
e2fsck/e2fsck.h +6 -0
index c3b0af34..777d8b96 100644
@@ -447,6 +447,10 @@ struct e2fsck_struct {
 
 	/* Undo file */
 	char *undo_file;
+#ifdef CONFIG_PFSCK
+	/* serialize fix operation for multiple threads */
+	pthread_mutex_t		 fs_fix_mutex;
+#endif
 };
 
 #ifdef CONFIG_PFSCK
@@ -744,6 +748,8 @@ extern errcode_t e2fsck_allocate_subcluster_bitmap(ext2_filsys fs,
 						   const char *profile_name,
 						   ext2fs_block_bitmap *ret);
 unsigned long long get_memory_size(void);
+extern void e2fsck_pass1_fix_lock(e2fsck_t ctx);
+extern void e2fsck_pass1_fix_unlock(e2fsck_t ctx);
 
 /* unix.c */
 extern void e2fsck_clear_progbar(e2fsck_t ctx);
e2fsck/pass1.c +56 -7

215 lines not on this page · raw message

e2fsck/util.c +38 -0
index a388bd70..8eec477c 100644
@@ -42,6 +42,10 @@
 #include <sys/sysctl.h>
 #endif
 
+#ifdef CONFIG_PFSCK
+#include <pthread.h>
+#endif
+
 #include "e2fsck.h"
 
 extern e2fsck_t e2fsck_global_ctx;   /* Try your very best not to use this! */
@@ -565,13 +569,45 @@ void e2fsck_read_inode_full(e2fsck_t ctx, unsigned long ino,
 	}
 }
 
+#ifdef CONFIG_PFSCK
+void e2fsck_pass1_fix_lock(e2fsck_t ctx)
+{
+	e2fsck_t global_ctx = ctx->global_ctx;
+	if (!global_ctx)
+		global_ctx = ctx;
+
+	pthread_mutex_lock(&global_ctx->fs_fix_mutex);
+}
+
+void e2fsck_pass1_fix_unlock(e2fsck_t ctx)
+{
+	e2fsck_t global_ctx = ctx->global_ctx;
+	if (!global_ctx)
+		global_ctx = ctx;
+
+	pthread_mutex_unlock(&global_ctx->fs_fix_mutex);
+}
+#else
+void e2fsck_pass1_fix_lock(e2fsck_t ctx)
+{
+
+}
+
+void e2fsck_pass1_fix_unlock(e2fsck_t ctx)
+{
+
+}
+#endif
+
 void e2fsck_write_inode_full(e2fsck_t ctx, unsigned long ino,
 			     struct ext2_inode * inode, int bufsize,
 			     const char *proc)
 {
 	errcode_t retval;
 
+	e2fsck_pass1_fix_lock(ctx);
 	retval = ext2fs_write_inode_full(ctx->fs, ino, inode, bufsize);
+	e2fsck_pass1_fix_unlock(ctx);
 	if (retval) {
 		com_err("ext2fs_write_inode", retval,
 			_("while writing inode %lu in %s"), ino, proc);
@@ -584,7 +620,9 @@ void e2fsck_write_inode(e2fsck_t ctx, unsigned long ino,
 {
 	errcode_t retval;
 
+	e2fsck_pass1_fix_lock(ctx);
 	retval = ext2fs_write_inode(ctx->fs, ino, inode);
+	e2fsck_pass1_fix_unlock(ctx);
 	if (retval) {
 		com_err("ext2fs_write_inode", retval,
 			_("while writing inode %lu in %s"), ino, proc);
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#45 lore raw

RFC PATCH v3 30/61 e2fsck: move some fixes out of parallel pthreads

In reply to Saranya Muruganandam

From: Wang Shilong <[email protected]>

We could only use @found_map_block to find free blocks after we have collectd all used blocks, so something like handle_fs_bad_blocks(), ext2fs_create_resize_inode(), e2fsck_pass1_dupblocks() really should be handled after all threads has been finished.

---
 e2fsck/e2fsck.h                      |   4 +
 e2fsck/pass1.c                       | 306 ++++++++++++++++-----------
 e2fsck/util.c                        |  36 +++-
 tests/f_multithread/expect.1         |   2 +-
 tests/f_multithread_logfile/expect.1 |   2 +-
 tests/f_multithread_no/expect.1      |   2 +-
 6 files changed, 216 insertions(+), 136 deletions(-)
e2fsck/e2fsck.h +4 -0
index 777d8b96..fecc8bbf 100644
@@ -450,6 +450,8 @@ struct e2fsck_struct {
 #ifdef CONFIG_PFSCK
 	/* serialize fix operation for multiple threads */
 	pthread_mutex_t		 fs_fix_mutex;
+	/* protect block_found_map, block_dup_map */
+	pthread_mutex_t		 fs_block_map_mutex;
 #endif
 };
 
@@ -750,6 +752,8 @@ extern errcode_t e2fsck_allocate_subcluster_bitmap(ext2_filsys fs,
 unsigned long long get_memory_size(void);
 extern void e2fsck_pass1_fix_lock(e2fsck_t ctx);
 extern void e2fsck_pass1_fix_unlock(e2fsck_t ctx);
+extern void e2fsck_pass1_block_map_lock(e2fsck_t ctx);
+extern void e2fsck_pass1_block_map_unlock(e2fsck_t ctx);
 
 /* unix.c */
 extern void e2fsck_clear_progbar(e2fsck_t ctx);
e2fsck/pass1.c +181 -125

543 lines not on this page · raw message

e2fsck/util.c +28 -8
index 8eec477c..3dcfa86a 100644
@@ -570,23 +570,34 @@ void e2fsck_read_inode_full(e2fsck_t ctx, unsigned long ino,
 }
 
 #ifdef CONFIG_PFSCK
+#define e2fsck_get_lock_context(ctx)		\
+	e2fsck_t global_ctx = ctx->global_ctx;	\
+	if (!global_ctx)			\
+		global_ctx = ctx;		\
+
 void e2fsck_pass1_fix_lock(e2fsck_t ctx)
 {
-	e2fsck_t global_ctx = ctx->global_ctx;
-	if (!global_ctx)
-		global_ctx = ctx;
-
+	e2fsck_get_lock_context(ctx);
 	pthread_mutex_lock(&global_ctx->fs_fix_mutex);
 }
 
 void e2fsck_pass1_fix_unlock(e2fsck_t ctx)
 {
-	e2fsck_t global_ctx = ctx->global_ctx;
-	if (!global_ctx)
-		global_ctx = ctx;
-
+	e2fsck_get_lock_context(ctx);
 	pthread_mutex_unlock(&global_ctx->fs_fix_mutex);
 }
+
+void e2fsck_pass1_block_map_lock(e2fsck_t ctx)
+{
+	e2fsck_get_lock_context(ctx);
+	pthread_mutex_lock(&global_ctx->fs_block_map_mutex);
+}
+
+void e2fsck_pass1_block_map_unlock(e2fsck_t ctx)
+{
+	e2fsck_get_lock_context(ctx);
+	pthread_mutex_unlock(&global_ctx->fs_block_map_mutex);
+}
 #else
 void e2fsck_pass1_fix_lock(e2fsck_t ctx)
 {
@@ -596,6 +607,15 @@ void e2fsck_pass1_fix_lock(e2fsck_t ctx)
 void e2fsck_pass1_fix_unlock(e2fsck_t ctx)
 {
 
+}
+
+void e2fsck_pass1_block_map_lock(e2fsck_t ctx)
+{
+
+}
+
+void e2fsck_pass1_block_map_unlock(e2fsck_t ctx)
+{
 }
 #endif
 
tests/f_multithread/expect.1 +1 -1
index 8d2acd2b..4db68d9e 100644
@@ -1,7 +1,7 @@
 ext2fs_open2: Bad magic number in super-block
 ../e2fsck/e2fsck: Superblock invalid, trying backup blocks...
+Pass 1: Checking inodes, blocks, and sizes
 [Thread 0] Scan group range [0, 2)
-[Thread 0] Pass 1: Checking inodes, blocks, and sizes
 [Thread 0] Scanned group range [0, 2), inodes 3008
 Pass 2: Checking directory structure
 Pass 3: Checking directory connectivity
tests/f_multithread_logfile/expect.1 +1 -1
index 8d2acd2b..4db68d9e 100644
@@ -1,7 +1,7 @@
 ext2fs_open2: Bad magic number in super-block
 ../e2fsck/e2fsck: Superblock invalid, trying backup blocks...
+Pass 1: Checking inodes, blocks, and sizes
 [Thread 0] Scan group range [0, 2)
-[Thread 0] Pass 1: Checking inodes, blocks, and sizes
 [Thread 0] Scanned group range [0, 2), inodes 3008
 Pass 2: Checking directory structure
 Pass 3: Checking directory connectivity
tests/f_multithread_no/expect.1 +1 -1
index f85a3382..eda2fcac 100644
@@ -1,7 +1,7 @@
 ext2fs_open2: Bad magic number in super-block
 ../e2fsck/e2fsck: Superblock invalid, trying backup blocks...
+Pass 1: Checking inodes, blocks, and sizes
 [Thread 0] Scan group range [0, 2)
-[Thread 0] Pass 1: Checking inodes, blocks, and sizes
 [Thread 0] Scanned group range [0, 2), inodes 3008
 Pass 2: Checking directory structure
 Pass 3: Checking directory connectivity
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#46 lore raw

RFC PATCH v3 31/61 e2fsck: split and merge invalid bitmaps

In reply to Saranya Muruganandam

From: Wang Shilong <[email protected]>

Invalid bitmaps are splitted per thread, and we
should merge them after thread finish.

---
 e2fsck/pass1.c | 71 ++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 71 insertions(+)
e2fsck/pass1.c +71 -0
index 49bdba21..29954e88 100644
@@ -2379,6 +2379,62 @@ out:
 	return retval;
 }
 
+static void e2fsck_pass1_copy_invalid_bitmaps(e2fsck_t global_ctx,
+					      e2fsck_t thread_ctx)
+{
+	dgrp_t i, j;
+	dgrp_t grp_start = thread_ctx->thread_info.et_group_start;
+	dgrp_t grp_end = thread_ctx->thread_info.et_group_end;
+	dgrp_t total = grp_end - grp_start;
+
+	thread_ctx->invalid_inode_bitmap_flag =
+			e2fsck_allocate_memory(global_ctx, sizeof(int) * total,
+						"invalid_inode_bitmap");
+	thread_ctx->invalid_block_bitmap_flag =
+			e2fsck_allocate_memory(global_ctx, sizeof(int) * total,
+					       "invalid_block_bitmap");
+	thread_ctx->invalid_inode_table_flag =
+			e2fsck_allocate_memory(global_ctx, sizeof(int) * total,
+					       "invalid_inode_table");
+
+	memcpy(thread_ctx->invalid_block_bitmap_flag,
+	       &global_ctx->invalid_block_bitmap_flag[grp_start],
+	       total * sizeof(int));
+	memcpy(thread_ctx->invalid_inode_bitmap_flag,
+	       &global_ctx->invalid_inode_bitmap_flag[grp_start],
+	       total * sizeof(int));
+	memcpy(thread_ctx->invalid_inode_table_flag,
+	       &global_ctx->invalid_inode_table_flag[grp_start],
+	       total * sizeof(int));
+
+	thread_ctx->invalid_bitmaps = 0;
+	for (i = grp_start, j = 0; i < grp_end; i++, j++) {
+		if (thread_ctx->invalid_block_bitmap_flag[j])
+			thread_ctx->invalid_bitmaps++;
+		if (thread_ctx->invalid_inode_bitmap_flag[j])
+			thread_ctx->invalid_bitmaps++;
+		if (thread_ctx->invalid_inode_table_flag[j])
+			thread_ctx->invalid_bitmaps++;
+	}
+}
+
+static void e2fsck_pass1_merge_invalid_bitmaps(e2fsck_t global_ctx,
+					       e2fsck_t thread_ctx)
+{
+	dgrp_t i, j;
+	dgrp_t grp_start = thread_ctx->thread_info.et_group_start;
+	dgrp_t grp_end = thread_ctx->thread_info.et_group_end;
+	dgrp_t total = grp_end - grp_start;
+
+	memcpy(&global_ctx->invalid_block_bitmap_flag[grp_start],
+	       thread_ctx->invalid_block_bitmap_flag, total * sizeof(int));
+	memcpy(&global_ctx->invalid_inode_bitmap_flag[grp_start],
+	       thread_ctx->invalid_inode_bitmap_flag, total * sizeof(int));
+	memcpy(&global_ctx->invalid_inode_table_flag[grp_start],
+	       thread_ctx->invalid_inode_table_flag, total * sizeof(int));
+	global_ctx->invalid_bitmaps += thread_ctx->invalid_bitmaps;
+}
+
 static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thread_ctx,
 					     int thread_index, int num_threads)
 {
@@ -2455,6 +2511,7 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thre
 		goto out_fs;
 	}
 	*thread_ctx = thread_context;
+	e2fsck_pass1_copy_invalid_bitmaps(global_ctx, thread_context);
 	return 0;
 out_fs:
 	ext2fs_free_mem(&thread_fs);
@@ -2589,6 +2646,10 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
 	ext2_ino_t dx_dir_info_count = global_ctx->dx_dir_info_count;
 	ext2_u32_list dirs_to_hash = global_ctx->dirs_to_hash;
 	quota_ctx_t qctx = global_ctx->qctx;
+	int *invalid_block_bitmap_flag = global_ctx->invalid_block_bitmap_flag;
+	int *invalid_inode_bitmap_flag = global_ctx->invalid_inode_bitmap_flag;
+	int *invalid_inode_table_flag  = global_ctx->invalid_inode_table_flag;
+	int invalid_bitmaps = global_ctx->invalid_bitmaps;
 
 #ifdef HAVE_SETJMP_H
 	jmp_buf		 old_jmp;
@@ -2667,6 +2728,11 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
 					      thread_ctx->qctx);
 	if (retval)
 		return retval;
+	global_ctx->invalid_block_bitmap_flag = invalid_block_bitmap_flag;
+	global_ctx->invalid_inode_bitmap_flag = invalid_inode_bitmap_flag;
+	global_ctx->invalid_inode_table_flag = invalid_inode_table_flag;
+	global_ctx->invalid_bitmaps = invalid_bitmaps;
+	e2fsck_pass1_merge_invalid_bitmaps(global_ctx, thread_ctx);
 
 	retval = e2fsck_pass1_merge_bitmap(global_fs,
 				&thread_ctx->inode_used_map,
@@ -2739,6 +2805,9 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
 	if (thread_ctx->dirs_to_hash)
 		ext2fs_badblocks_list_free(thread_ctx->dirs_to_hash);
 	quota_release_context(&thread_ctx->qctx);
+	ext2fs_free_mem(&thread_ctx->invalid_block_bitmap_flag);
+	ext2fs_free_mem(&thread_ctx->invalid_inode_bitmap_flag);
+	ext2fs_free_mem(&thread_ctx->invalid_inode_table_flag);
 	ext2fs_free_mem(&thread_ctx);
 
 	return retval;
@@ -2752,6 +2821,8 @@ static int e2fsck_pass1_threads_join(struct e2fsck_thread_info *infos,
 	int				 i;
 	struct e2fsck_thread_info	*pinfo;
 
+	/* merge invalid bitmaps will recalculate it */
+	global_ctx->invalid_bitmaps = 0;
 	for (i = 0; i < num_threads; i++) {
 		pinfo = &infos[i];
 
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#47 lore raw

Re: [RFC PATCH v3 31/61] e2fsck: split and merge invalid bitmaps

Quoted 4 lines · Saranya Muruganandam

From: Wang Shilong <[email protected]>

Invalid bitmaps are splitted per thread, and we
should merge them after thread finish.

For a large filesystem, would it make more sense to merge results periodically to reduce the peak memory consumption? That might not be all that high of a peak though since the end merges could be deleting records from the per-thread data structure after each succesful insertion merge.

--D

Quoted 126 lines
---
 e2fsck/pass1.c | 71 ++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 71 insertions(+)
e2fsck/pass1.c +71 -0
index 49bdba21..29954e88 100644
@@ -2379,6 +2379,62 @@ out:
 	return retval;
 }
 
+static void e2fsck_pass1_copy_invalid_bitmaps(e2fsck_t global_ctx,
+					      e2fsck_t thread_ctx)
+{
+	dgrp_t i, j;
+	dgrp_t grp_start = thread_ctx->thread_info.et_group_start;
+	dgrp_t grp_end = thread_ctx->thread_info.et_group_end;
+	dgrp_t total = grp_end - grp_start;
+
+	thread_ctx->invalid_inode_bitmap_flag =
+			e2fsck_allocate_memory(global_ctx, sizeof(int) * total,
+						"invalid_inode_bitmap");
+	thread_ctx->invalid_block_bitmap_flag =
+			e2fsck_allocate_memory(global_ctx, sizeof(int) * total,
+					       "invalid_block_bitmap");
+	thread_ctx->invalid_inode_table_flag =
+			e2fsck_allocate_memory(global_ctx, sizeof(int) * total,
+					       "invalid_inode_table");
+
+	memcpy(thread_ctx->invalid_block_bitmap_flag,
+	       &global_ctx->invalid_block_bitmap_flag[grp_start],
+	       total * sizeof(int));
+	memcpy(thread_ctx->invalid_inode_bitmap_flag,
+	       &global_ctx->invalid_inode_bitmap_flag[grp_start],
+	       total * sizeof(int));
+	memcpy(thread_ctx->invalid_inode_table_flag,
+	       &global_ctx->invalid_inode_table_flag[grp_start],
+	       total * sizeof(int));
+
+	thread_ctx->invalid_bitmaps = 0;
+	for (i = grp_start, j = 0; i < grp_end; i++, j++) {
+		if (thread_ctx->invalid_block_bitmap_flag[j])
+			thread_ctx->invalid_bitmaps++;
+		if (thread_ctx->invalid_inode_bitmap_flag[j])
+			thread_ctx->invalid_bitmaps++;
+		if (thread_ctx->invalid_inode_table_flag[j])
+			thread_ctx->invalid_bitmaps++;
+	}
+}
+
+static void e2fsck_pass1_merge_invalid_bitmaps(e2fsck_t global_ctx,
+					       e2fsck_t thread_ctx)
+{
+	dgrp_t i, j;
+	dgrp_t grp_start = thread_ctx->thread_info.et_group_start;
+	dgrp_t grp_end = thread_ctx->thread_info.et_group_end;
+	dgrp_t total = grp_end - grp_start;
+
+	memcpy(&global_ctx->invalid_block_bitmap_flag[grp_start],
+	       thread_ctx->invalid_block_bitmap_flag, total * sizeof(int));
+	memcpy(&global_ctx->invalid_inode_bitmap_flag[grp_start],
+	       thread_ctx->invalid_inode_bitmap_flag, total * sizeof(int));
+	memcpy(&global_ctx->invalid_inode_table_flag[grp_start],
+	       thread_ctx->invalid_inode_table_flag, total * sizeof(int));
+	global_ctx->invalid_bitmaps += thread_ctx->invalid_bitmaps;
+}
+
 static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thread_ctx,
 					     int thread_index, int num_threads)
 {
@@ -2455,6 +2511,7 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thre
 		goto out_fs;
 	}
 	*thread_ctx = thread_context;
+	e2fsck_pass1_copy_invalid_bitmaps(global_ctx, thread_context);
 	return 0;
 out_fs:
 	ext2fs_free_mem(&thread_fs);
@@ -2589,6 +2646,10 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
 	ext2_ino_t dx_dir_info_count = global_ctx->dx_dir_info_count;
 	ext2_u32_list dirs_to_hash = global_ctx->dirs_to_hash;
 	quota_ctx_t qctx = global_ctx->qctx;
+	int *invalid_block_bitmap_flag = global_ctx->invalid_block_bitmap_flag;
+	int *invalid_inode_bitmap_flag = global_ctx->invalid_inode_bitmap_flag;
+	int *invalid_inode_table_flag  = global_ctx->invalid_inode_table_flag;
+	int invalid_bitmaps = global_ctx->invalid_bitmaps;
 
 #ifdef HAVE_SETJMP_H
 	jmp_buf		 old_jmp;
@@ -2667,6 +2728,11 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
 					      thread_ctx->qctx);
 	if (retval)
 		return retval;
+	global_ctx->invalid_block_bitmap_flag = invalid_block_bitmap_flag;
+	global_ctx->invalid_inode_bitmap_flag = invalid_inode_bitmap_flag;
+	global_ctx->invalid_inode_table_flag = invalid_inode_table_flag;
+	global_ctx->invalid_bitmaps = invalid_bitmaps;
+	e2fsck_pass1_merge_invalid_bitmaps(global_ctx, thread_ctx);
 
 	retval = e2fsck_pass1_merge_bitmap(global_fs,
 				&thread_ctx->inode_used_map,
@@ -2739,6 +2805,9 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
 	if (thread_ctx->dirs_to_hash)
 		ext2fs_badblocks_list_free(thread_ctx->dirs_to_hash);
 	quota_release_context(&thread_ctx->qctx);
+	ext2fs_free_mem(&thread_ctx->invalid_block_bitmap_flag);
+	ext2fs_free_mem(&thread_ctx->invalid_inode_bitmap_flag);
+	ext2fs_free_mem(&thread_ctx->invalid_inode_table_flag);
 	ext2fs_free_mem(&thread_ctx);
 
 	return retval;
@@ -2752,6 +2821,8 @@ static int e2fsck_pass1_threads_join(struct e2fsck_thread_info *infos,
 	int				 i;
 	struct e2fsck_thread_info	*pinfo;
 
+	/* merge invalid bitmaps will recalculate it */
+	global_ctx->invalid_bitmaps = 0;
 	for (i = 0; i < num_threads; i++) {
 		pinfo = &infos[i];
 
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#48 lore raw

Re: [RFC PATCH v3 31/61] e2fsck: split and merge invalid bitmaps

Quoted 4 lines · Darrick J. Wong
Quoted 2 lines · Saranya Muruganandam

From: Wang Shilong <[email protected]>

Quoted 8 lines
Quoted 2 lines

Invalid bitmaps are splitted per thread, and we
should merge them after thread finish.

For a large filesystem, would it make more sense to merge results periodically to reduce the peak memory consumption? That might not be all that high of a peak though since the end merges could be deleting records from the per-thread data structure after each succesful insertion merge.

That could be possible optimization, comparing to original memory usage, pfsck did not increase memory usage too much, it just try to add some extra structure overhead X number of threads.

And from our benchmarking on PiB filesystem, we saw similar memory peak numbers for pfsck, we might post them in the changelog next time.

Quoted 129 lines

--D

Quoted 126 lines
---
 e2fsck/pass1.c | 71 ++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 71 insertions(+)
e2fsck/pass1.c +71 -1
index 49bdba21..29954e88 100644
@@ -2379,6 +2379,62 @@ out:
      return retval;
 }
 
+static void e2fsck_pass1_copy_invalid_bitmaps(e2fsck_t global_ctx,
+                                           e2fsck_t thread_ctx)
+{
+     dgrp_t i, j;
+     dgrp_t grp_start = thread_ctx->thread_info.et_group_start;
+     dgrp_t grp_end = thread_ctx->thread_info.et_group_end;
+     dgrp_t total = grp_end - grp_start;
+
+     thread_ctx->invalid_inode_bitmap_flag =
+                     e2fsck_allocate_memory(global_ctx, sizeof(int) * total,
+                                             "invalid_inode_bitmap");
+     thread_ctx->invalid_block_bitmap_flag =
+                     e2fsck_allocate_memory(global_ctx, sizeof(int) * total,
+                                            "invalid_block_bitmap");
+     thread_ctx->invalid_inode_table_flag =
+                     e2fsck_allocate_memory(global_ctx, sizeof(int) * total,
+                                            "invalid_inode_table");
+
+     memcpy(thread_ctx->invalid_block_bitmap_flag,
+            &global_ctx->invalid_block_bitmap_flag[grp_start],
+            total * sizeof(int));
+     memcpy(thread_ctx->invalid_inode_bitmap_flag,
+            &global_ctx->invalid_inode_bitmap_flag[grp_start],
+            total * sizeof(int));
+     memcpy(thread_ctx->invalid_inode_table_flag,
+            &global_ctx->invalid_inode_table_flag[grp_start],
+            total * sizeof(int));
+
+     thread_ctx->invalid_bitmaps = 0;
+     for (i = grp_start, j = 0; i < grp_end; i++, j++) {
+             if (thread_ctx->invalid_block_bitmap_flag[j])
+                     thread_ctx->invalid_bitmaps++;
+             if (thread_ctx->invalid_inode_bitmap_flag[j])
+                     thread_ctx->invalid_bitmaps++;
+             if (thread_ctx->invalid_inode_table_flag[j])
+                     thread_ctx->invalid_bitmaps++;
+     }
+}
+
+static void e2fsck_pass1_merge_invalid_bitmaps(e2fsck_t global_ctx,
+                                            e2fsck_t thread_ctx)
+{
+     dgrp_t i, j;
+     dgrp_t grp_start = thread_ctx->thread_info.et_group_start;
+     dgrp_t grp_end = thread_ctx->thread_info.et_group_end;
+     dgrp_t total = grp_end - grp_start;
+
+     memcpy(&global_ctx->invalid_block_bitmap_flag[grp_start],
+            thread_ctx->invalid_block_bitmap_flag, total * sizeof(int));
+     memcpy(&global_ctx->invalid_inode_bitmap_flag[grp_start],
+            thread_ctx->invalid_inode_bitmap_flag, total * sizeof(int));
+     memcpy(&global_ctx->invalid_inode_table_flag[grp_start],
+            thread_ctx->invalid_inode_table_flag, total * sizeof(int));
+     global_ctx->invalid_bitmaps += thread_ctx->invalid_bitmaps;
+}
+
 static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thread_ctx,
                                           int thread_index, int num_threads)
 {
@@ -2455,6 +2511,7 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thre
              goto out_fs;
      }
      *thread_ctx = thread_context;
+     e2fsck_pass1_copy_invalid_bitmaps(global_ctx, thread_context);
      return 0;
 out_fs:
      ext2fs_free_mem(&thread_fs);
@@ -2589,6 +2646,10 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
      ext2_ino_t dx_dir_info_count = global_ctx->dx_dir_info_count;
      ext2_u32_list dirs_to_hash = global_ctx->dirs_to_hash;
      quota_ctx_t qctx = global_ctx->qctx;
+     int *invalid_block_bitmap_flag = global_ctx->invalid_block_bitmap_flag;
+     int *invalid_inode_bitmap_flag = global_ctx->invalid_inode_bitmap_flag;
+     int *invalid_inode_table_flag  = global_ctx->invalid_inode_table_flag;
+     int invalid_bitmaps = global_ctx->invalid_bitmaps;
 
 #ifdef HAVE_SETJMP_H
      jmp_buf          old_jmp;
@@ -2667,6 +2728,11 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
                                            thread_ctx->qctx);
      if (retval)
              return retval;
+     global_ctx->invalid_block_bitmap_flag = invalid_block_bitmap_flag;
+     global_ctx->invalid_inode_bitmap_flag = invalid_inode_bitmap_flag;
+     global_ctx->invalid_inode_table_flag = invalid_inode_table_flag;
+     global_ctx->invalid_bitmaps = invalid_bitmaps;
+     e2fsck_pass1_merge_invalid_bitmaps(global_ctx, thread_ctx);
 
      retval = e2fsck_pass1_merge_bitmap(global_fs,
                              &thread_ctx->inode_used_map,
@@ -2739,6 +2805,9 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
      if (thread_ctx->dirs_to_hash)
              ext2fs_badblocks_list_free(thread_ctx->dirs_to_hash);
      quota_release_context(&thread_ctx->qctx);
+     ext2fs_free_mem(&thread_ctx->invalid_block_bitmap_flag);
+     ext2fs_free_mem(&thread_ctx->invalid_inode_bitmap_flag);
+     ext2fs_free_mem(&thread_ctx->invalid_inode_table_flag);
      ext2fs_free_mem(&thread_ctx);
 
      return retval;
@@ -2752,6 +2821,8 @@ static int e2fsck_pass1_threads_join(struct e2fsck_thread_info *infos,
      int                              i;
      struct e2fsck_thread_info       *pinfo;
 
+     /* merge invalid bitmaps will recalculate it */
+     global_ctx->invalid_bitmaps = 0;
      for (i = 0; i < num_threads; i++) {
              pinfo = &infos[i];
 
--

2.29.2.299.gdc1121823c-goog

#49 lore raw

RFC PATCH v3 32/61 e2fsck: merge EA blocks properly

In reply to Saranya Muruganandam

From: Wang Shilong <[email protected]>

EA blocks might be shared, merge them carefully.

---
 e2fsck/e2fsck.h |   1 +
 e2fsck/pass1.c  | 245 +++++++++++++++++++++++++++++++++++++++++-------
 2 files changed, 213 insertions(+), 33 deletions(-)
e2fsck/e2fsck.h +1 -0
index fecc8bbf..192a534c 100644
@@ -299,6 +299,7 @@ struct e2fsck_struct {
 
 	ext2_refcount_t	refcount;
 	ext2_refcount_t refcount_extra;
+	ext2_refcount_t refcount_orig;
 
 	/*
 	 * Quota blocks and inodes to be charged for each ea block.
e2fsck/pass1.c +212 -33

342 lines not on this page · raw message

Signature · 1 line
2.29.2.299.gdc1121823c-goog
#50 lore raw

RFC PATCH v3 33/61 e2fsck: kickoff mutex lock for block found map

In reply to Saranya Muruganandam

From: Wang Shilong <[email protected]>

Now @block_found_map is no longer shared by multiple threads, and @block_dup_map need be checked again after threads finish.

---
 e2fsck/e2fsck.h                   |   9 +-
 e2fsck/pass1.c                    | 172 ++++++++++++++++++------------
 e2fsck/util.c                     |  34 ++++--
 lib/ext2fs/bitops.h               |   2 +
 lib/ext2fs/gen_bitmap64.c         |  33 ++++++
 tests/f_itable_collision/expect.1 |   3 -
 6 files changed, 172 insertions(+), 81 deletions(-)
e2fsck/e2fsck.h +6 -3
index 192a534c..d4b472f5 100644
@@ -202,6 +202,7 @@ struct resource_track {
 #define E2F_FLAG_TIME_INSANE	0x2000 /* Time is insane */
 #define E2F_FLAG_PROBLEMS_FIXED	0x4000 /* At least one problem was fixed */
 #define E2F_FLAG_ALLOC_OK	0x8000 /* Can we allocate blocks? */
+#define E2F_FLAG_DUP_BLOCK	0x20000 /* dup block found during pass1 */
 
 #define E2F_RESET_FLAGS (E2F_FLAG_TIME_INSANE | E2F_FLAG_PROBLEMS_FIXED)
 
@@ -452,7 +453,7 @@ struct e2fsck_struct {
 	/* serialize fix operation for multiple threads */
 	pthread_mutex_t		 fs_fix_mutex;
 	/* protect block_found_map, block_dup_map */
-	pthread_mutex_t		 fs_block_map_mutex;
+	pthread_rwlock_t	 fs_block_map_rwlock;
 #endif
 };
 
@@ -753,8 +754,10 @@ extern errcode_t e2fsck_allocate_subcluster_bitmap(ext2_filsys fs,
 unsigned long long get_memory_size(void);
 extern void e2fsck_pass1_fix_lock(e2fsck_t ctx);
 extern void e2fsck_pass1_fix_unlock(e2fsck_t ctx);
-extern void e2fsck_pass1_block_map_lock(e2fsck_t ctx);
-extern void e2fsck_pass1_block_map_unlock(e2fsck_t ctx);
+extern void e2fsck_pass1_block_map_w_lock(e2fsck_t ctx);
+extern void e2fsck_pass1_block_map_w_unlock(e2fsck_t ctx);
+extern void e2fsck_pass1_block_map_r_lock(e2fsck_t ctx);
+extern void e2fsck_pass1_block_map_r_unlock(e2fsck_t ctx);
 
 /* unix.c */
 extern void e2fsck_clear_progbar(e2fsck_t ctx);
e2fsck/pass1.c +103 -69

352 lines not on this page · raw message

e2fsck/util.c +28 -6
index 3dcfa86a..3a84abb6 100644
@@ -587,17 +587,29 @@ void e2fsck_pass1_fix_unlock(e2fsck_t ctx)
 	pthread_mutex_unlock(&global_ctx->fs_fix_mutex);
 }
 
-void e2fsck_pass1_block_map_lock(e2fsck_t ctx)
+void e2fsck_pass1_block_map_w_lock(e2fsck_t ctx)
 {
 	e2fsck_get_lock_context(ctx);
-	pthread_mutex_lock(&global_ctx->fs_block_map_mutex);
+	pthread_rwlock_wrlock(&global_ctx->fs_block_map_rwlock);
 }
 
-void e2fsck_pass1_block_map_unlock(e2fsck_t ctx)
+void e2fsck_pass1_block_map_w_unlock(e2fsck_t ctx)
 {
 	e2fsck_get_lock_context(ctx);
-	pthread_mutex_unlock(&global_ctx->fs_block_map_mutex);
+	pthread_rwlock_unlock(&global_ctx->fs_block_map_rwlock);
 }
+
+void e2fsck_pass1_block_map_r_lock(e2fsck_t ctx)
+{
+	e2fsck_get_lock_context(ctx);
+	pthread_rwlock_rdlock(&global_ctx->fs_block_map_rwlock);
+}
+
+void e2fsck_pass1_block_map_r_unlock(e2fsck_t ctx)
+{
+	e2fsck_get_lock_context(ctx);
+	pthread_rwlock_unlock(&global_ctx->fs_block_map_rwlock);
+ }
 #else
 void e2fsck_pass1_fix_lock(e2fsck_t ctx)
 {
@@ -608,14 +620,24 @@ void e2fsck_pass1_fix_unlock(e2fsck_t ctx)
 {
 
 }
+void e2fsck_pass1_block_map_w_lock(e2fsck_t ctx)
+{
 
-void e2fsck_pass1_block_map_lock(e2fsck_t ctx)
+}
+
+void e2fsck_pass1_block_map_w_unlock(e2fsck_t ctx)
 {
 
 }
 
-void e2fsck_pass1_block_map_unlock(e2fsck_t ctx)
+void e2fsck_pass1_block_map_r_lock(e2fsck_t ctx)
 {
+
+}
+
+void e2fsck_pass1_block_map_r_unlock(e2fsck_t ctx)
+{
+
 }
 #endif
 
lib/ext2fs/bitops.h +2 -0
index 505b3c9c..1facc8dd 100644
@@ -120,6 +120,8 @@ extern int ext2fs_test_block_bitmap_range(ext2fs_block_bitmap bitmap,
 extern void ext2fs_set_bitmap_padding(ext2fs_generic_bitmap map);
 extern __u32 ext2fs_get_generic_bitmap_start(ext2fs_generic_bitmap bitmap);
 extern __u32 ext2fs_get_generic_bitmap_end(ext2fs_generic_bitmap bitmap);
+extern int ext2fs_test_block_bitmap_range2_valid(ext2fs_block_bitmap bitmap,
+						blk64_t block, unsigned int num);
 
 /* 64-bit versions */
 
lib/ext2fs/gen_bitmap64.c +33 -0
index 50617a34..bdfed633 100644
@@ -725,6 +725,39 @@ int ext2fs_test_block_bitmap_range2(ext2fs_block_bitmap gen_bmap,
 	return bmap->bitmap_ops->test_clear_bmap_extent(bmap, block, num);
 }
 
+int ext2fs_test_block_bitmap_range2_valid(ext2fs_block_bitmap bitmap,
+					  blk64_t block, unsigned int num)
+{
+	ext2fs_generic_bitmap_64 bmap = (ext2fs_generic_bitmap_64)bitmap;
+	__u64	end = block + num;
+
+	if (!bmap)
+		return 0;
+
+	if (EXT2FS_IS_32_BITMAP(bmap)) {
+		if ((block & ~0xffffffffULL) ||
+		    ((block+num-1) & ~0xffffffffULL)) {
+			return 0;
+		}
+	}
+
+	if (!EXT2FS_IS_64_BITMAP(bmap))
+		return 0;
+
+	/* convert to clusters if necessary */
+	block >>= bmap->cluster_bits;
+	end += (1 << bmap->cluster_bits) - 1;
+	end >>= bmap->cluster_bits;
+	num = end - block;
+
+	if ((block < bmap->start) || (block > bmap->end) ||
+	    (block+num-1 > bmap->end))
+		return 0;
+
+	return 1;
+}
+
+
 void ext2fs_mark_block_bitmap_range2(ext2fs_block_bitmap gen_bmap,
 				     blk64_t block, unsigned int num)
 {
tests/f_itable_collision/expect.1 +0 -3
index 01c85d4d..7e98baa8 100644
@@ -1,6 +1,5 @@
 Pass 1: Checking inodes, blocks, and sizes
 Inode 12 block 37 conflicts with critical metadata, skipping block checks.
-Illegal block number passed to ext2fs_test_block_bitmap #268435455 for in-use block map
 Illegal block number passed to ext2fs_mark_block_bitmap #268435455 for in-use block map
 Inode 12, i_blocks is 48, should be 56.  Fix? yes
 
@@ -27,9 +26,7 @@ Clear inode? yes
 Restarting e2fsck from the beginning...
 Pass 1: Checking inodes, blocks, and sizes
 Inode 12 block 37 conflicts with critical metadata, skipping block checks.
-Illegal block number passed to ext2fs_test_block_bitmap #4294967294 for in-use block map
 Illegal block number passed to ext2fs_mark_block_bitmap #4294967294 for in-use block map
-Illegal block number passed to ext2fs_test_block_bitmap #268435455 for in-use block map
 Illegal block number passed to ext2fs_mark_block_bitmap #268435455 for in-use block map
 
 Running additional passes to resolve blocks claimed by more than one inode...
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#51 lore raw

RFC PATCH v3 34/61 e2fsck: allow admin specify number of threads

In reply to Saranya Muruganandam

From: Wang Shilong <[email protected]>

-m option is introduced to specify number of threads for pfsck.

---
 e2fsck/e2fsck.h                       |   2 +
 e2fsck/pass1.c                        | 165 ++++++++++++++++----------
 e2fsck/unix.c                         |  17 ++-
 tests/f_multithread/script            |   2 +-
 tests/f_multithread_completion/script |   2 +-
 tests/f_multithread_logfile/script    |   2 +-
 tests/f_multithread_no/script         |   2 +-
 tests/f_multithread_preen/script      |   2 +-
 tests/f_multithread_yes/script        |   2 +-
 9 files changed, 128 insertions(+), 68 deletions(-)
e2fsck/e2fsck.h +2 -0
index d4b472f5..f46a95ef 100644
@@ -450,6 +450,7 @@ struct e2fsck_struct {
 	/* Undo file */
 	char *undo_file;
 #ifdef CONFIG_PFSCK
+	__u32			 fs_num_threads;
 	/* serialize fix operation for multiple threads */
 	pthread_mutex_t		 fs_fix_mutex;
 	/* protect block_found_map, block_dup_map */
@@ -684,6 +685,7 @@ int check_backup_super_block(e2fsck_t ctx);
 void check_resize_inode(e2fsck_t ctx);
 
 /* util.c */
+#define E2FSCK_MAX_THREADS	(65536)
 extern void *e2fsck_allocate_memory(e2fsck_t ctx, unsigned long size,
 				    const char *description);
 extern int ask(e2fsck_t ctx, const char * string, int def);
e2fsck/pass1.c +104 -61

206 lines not on this page · raw message

e2fsck/unix.c +16 -1
index 30c2bf31..cd31bcd5 100644
@@ -819,6 +819,10 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx)
 	int 		res;		/* result of sscanf */
 #ifdef CONFIG_JBD_DEBUG
 	char 		*jbd_debug;
+#endif
+#ifdef CONFIG_PFSCK
+	char		*pm;
+	unsigned long	thread_num;
 #endif
 	unsigned long long phys_mem_kb;
 
@@ -852,7 +856,7 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx)
 	ctx->readahead_kb = ~0ULL;
 
 #ifdef CONFIG_PFSCK
-	while ((c = getopt(argc, argv, "pamnyrcC:B:dE:fvtFVM:b:I:j:P:l:L:N:SsDkz:")) != EOF)
+	while ((c = getopt(argc, argv, "pam:nyrcC:B:dE:fvtFVM:b:I:j:P:l:L:N:SsDkz:")) != EOF)
 #else
 	while ((c = getopt(argc, argv, "panyrcC:B:dE:fvtFVM:b:I:j:P:l:L:N:SsDkz:")) != EOF)
 #endif
@@ -898,7 +902,18 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx)
 			break;
 #ifdef CONFIG_PFSCK
 		case 'm':
+			thread_num = strtoul(optarg, &pm, 0);
+			if (*pm)
+				fatal_error(ctx,
+					_("Invalid multiple thread num.\n"));
+			if (thread_num > E2FSCK_MAX_THREADS) {
+				fprintf(stderr,
+					_("threads %lu too large (max %lu)\n"),
+					thread_num, E2FSCK_MAX_THREADS);
+				fatal_error(ctx, 0);
+			}
 			ctx->options |= E2F_OPT_MULTITHREAD;
+			ctx->fs_num_threads = thread_num;
 			break;
 #endif
 		case 'n':
tests/f_multithread/script +1 -1
index 0fe96cd0..83cd0f03 100644
@@ -1,4 +1,4 @@
-FSCK_OPT="-fy -m"
+FSCK_OPT="-fy -m1"
 SECOND_FSCK_OPT=-yf
 
 . $cmd_dir/run_e2fsck
tests/f_multithread_completion/script +1 -1
index bf23cd61..0ec13816 100644
@@ -1,4 +1,4 @@
-FSCK_OPT="-fy -m -C 1"
+FSCK_OPT="-fy -m1 -C 1"
 SECOND_FSCK_OPT=-yf
 
 . $cmd_dir/run_e2fsck
tests/f_multithread_logfile/script +1 -1
index 4f9ca6f8..dbb65319 100644
@@ -1,5 +1,5 @@
 LOG_FNAME="f_multithread_logfile_xxx"
-FSCK_OPT="-fy -m -y -E log_filename=$LOG_FNAME"
+FSCK_OPT="-fy -m1 -y -E log_filename=$LOG_FNAME"
 SKIP_VERIFY="true"
 ONE_PASS_ONLY="true"
 SKIP_CLEANUP="true"
tests/f_multithread_no/script +1 -1
index b93deb3a..db791e11 100644
@@ -1,4 +1,4 @@
-FSCK_OPT="-fn -m"
+FSCK_OPT="-fn -m1"
 SECOND_FSCK_OPT=-yf
 
 . $cmd_dir/run_e2fsck
tests/f_multithread_preen/script +1 -1
index ecb79cd6..8965f4a7 100644
@@ -1,4 +1,4 @@
-FSCK_OPT="-fp -m"
+FSCK_OPT="-fp -m1"
 SECOND_FSCK_OPT=-yf
 
 . $cmd_dir/run_e2fsck
tests/f_multithread_yes/script +1 -1
index 38891f6a..8b4aa9b8 100644
@@ -1,4 +1,4 @@
-FSCK_OPT="-f -m"
+FSCK_OPT="-f -m1"
 SECOND_FSCK_OPT=-yf
 
 . $cmd_dir/run_e2fsck
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#52 lore raw

RFC PATCH v3 35/61 e2fsck: adjust number of threads

In reply to Saranya Muruganandam

From: Wang Shilong <[email protected]>

number of threads should not exceed flex bg numbers, and output messages if we adjust threads number.

---
 e2fsck/pass1.c      | 32 +++++++++++++++-----------------
 lib/ext2fs/ext2fs.h | 32 +++++++++++++++++++++++++++++++-
 2 files changed, 46 insertions(+), 18 deletions(-)
e2fsck/pass1.c +15 -17
index 30365d23..e2387fe3 100644
@@ -1282,6 +1282,7 @@ static void e2fsck_pass1_set_thread_num(e2fsck_t ctx)
 	}
 out:
 	ctx->fs_num_threads = num_threads;
+	ctx->fs->fs_num_threads = num_threads;
 }
 #endif
 
@@ -2551,14 +2552,14 @@ static void e2fsck_pass1_merge_invalid_bitmaps(e2fsck_t global_ctx,
 }
 
 static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thread_ctx,
-					     int thread_index, int num_threads)
+					     int thread_index, int num_threads,
+					     dgrp_t average_group)
 {
 	errcode_t		retval;
 	e2fsck_t		thread_context;
 	ext2_filsys		thread_fs;
 	ext2_filsys		global_fs = global_ctx->fs;
 	struct e2fsck_thread	*tinfo;
-	dgrp_t			average_group;
 
 	assert(global_ctx->inode_used_map == NULL);
 	assert(global_ctx->inode_dir_map == NULL);
@@ -2605,16 +2606,9 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thre
 	thread_context->thread_info.et_thread_index = thread_index;
 	set_up_logging(thread_context);
 
-	/*
-	 * Distribute work to multiple threads:
-	 * Each thread work on fs->group_desc_count / nthread groups.
-	 */
 	tinfo = &thread_context->thread_info;
-	average_group = thread_fs->group_desc_count / num_threads;
-	if (average_group == 0)
-		average_group = 1;
 	tinfo->et_group_start = average_group * thread_index;
-	if (thread_index == num_threads - 1)
+	if (thread_index == global_fs->fs_num_threads - 1)
 		tinfo->et_group_end = thread_fs->group_desc_count;
 	else
 		tinfo->et_group_end = average_group * (thread_index + 1);
@@ -3130,12 +3124,13 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
 }
 
 static int e2fsck_pass1_threads_join(struct e2fsck_thread_info *infos,
-				      int num_threads, e2fsck_t global_ctx)
+				     e2fsck_t global_ctx)
 {
 	errcode_t			 rc;
 	errcode_t			 ret = 0;
 	int				 i;
 	struct e2fsck_thread_info	*pinfo;
+	int				 num_threads = global_ctx->fs_num_threads;
 
 	/* merge invalid bitmaps will recalculate it */
 	global_ctx->invalid_bitmaps = 0;
@@ -3217,7 +3212,7 @@ out:
 }
 
 static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo,
-				      int num_threads, e2fsck_t global_ctx)
+				      e2fsck_t global_ctx)
 {
 	struct e2fsck_thread_info	*infos;
 	pthread_attr_t			 attr;
@@ -3226,6 +3221,8 @@ static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo,
 	struct e2fsck_thread_info	*tmp_pinfo;
 	int				 i;
 	e2fsck_t			 thread_ctx;
+	dgrp_t				 average_group;
+	int				 num_threads = global_ctx->fs_num_threads;
 #ifdef DEBUG_THREADS
 	struct e2fsck_thread_debug	 thread_debug =
 		{PTHREAD_MUTEX_INITIALIZER, PTHREAD_COND_INITIALIZER, 0};
@@ -3249,6 +3246,7 @@ static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo,
 		return retval;
 	}
 
+	average_group = ext2fs_get_avg_group(global_ctx->fs);
 	for (i = 0; i < num_threads; i++) {
 		tmp_pinfo = &infos[i];
 		tmp_pinfo->eti_thread_index = i;
@@ -3256,7 +3254,8 @@ static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo,
 		tmp_pinfo->eti_debug = &thread_debug;
 #endif
 		retval = e2fsck_pass1_thread_prepare(global_ctx, &thread_ctx,
-						     i, num_threads);
+						     i, num_threads,
+						     average_group);
 		if (retval) {
 			com_err(global_ctx->program_name, retval,
 				_("while preparing pass1 thread\n"));
@@ -3286,7 +3285,7 @@ static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo,
 	}
 
 	if (retval) {
-		e2fsck_pass1_threads_join(infos, num_threads, global_ctx);
+		e2fsck_pass1_threads_join(infos, global_ctx);
 		return retval;
 	}
 	*pinfo = infos;
@@ -3296,17 +3295,16 @@ static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo,
 static void e2fsck_pass1_multithread(e2fsck_t global_ctx)
 {
 	struct e2fsck_thread_info *infos = NULL;
-	int num_threads = global_ctx->fs_num_threads;
 	errcode_t retval;
 
-	retval = e2fsck_pass1_threads_start(&infos, num_threads, global_ctx);
+	retval = e2fsck_pass1_threads_start(&infos, global_ctx);
 	if (retval) {
 		com_err(global_ctx->program_name, retval,
 			_("while starting pass1 threads\n"));
 		goto out_abort;
 	}
 
-	retval = e2fsck_pass1_threads_join(infos, num_threads, global_ctx);
+	retval = e2fsck_pass1_threads_join(infos, global_ctx);
 	if (retval) {
 		com_err(global_ctx->program_name, retval,
 			_("while joining pass1 threads\n"));
lib/ext2fs/ext2fs.h +31 -1
index 0fa0e22f..83f2af07 100644
@@ -255,10 +255,11 @@ struct struct_ext2_filsys {
 	int				cluster_ratio_bits;
 	__u16				default_bitmap_type;
 	__u16				pad;
+	__u32				fs_num_threads;
 	/*
 	 * Reserved for future expansion
 	 */
-	__u32				reserved[5];
+	__u32				reserved[4];
 
 	/*
 	 * Reserved for the use of the calling application.
@@ -2121,6 +2122,35 @@ ext2fs_const_inode(const struct ext2_inode_large * large_inode)
 	return (const struct ext2_inode *) large_inode;
 }
 
+static dgrp_t ext2fs_get_avg_group(ext2_filsys fs)
+{
+#ifdef CONFIG_PFSCK
+	dgrp_t average_group;
+	unsigned flexbg_size;
+
+	if (fs->fs_num_threads <= 1)
+		return fs->group_desc_count;
+
+	average_group = fs->group_desc_count / fs->fs_num_threads;
+	if (average_group <= 1)
+		return 1;
+
+	if (ext2fs_has_feature_flex_bg(fs->super)) {
+		int times = 1;
+
+		flexbg_size = 1 << fs->super->s_log_groups_per_flex;
+		if (average_group % flexbg_size) {
+			times = average_group / flexbg_size;
+			average_group = times * flexbg_size;
+		}
+	}
+
+	return average_group;
+#else
+	return fs->group_desc_count;
+#endif
+}
+
 #undef _INLINE_
 #endif
 
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#53 lore raw

RFC PATCH v3 36/61 e2fsck: fix readahead for pfsck of pass1

In reply to Saranya Muruganandam

From: Wang Shilong <[email protected]>

Several improvments for this patch:

1) move readahead_kb detection to preparing phase.
2) inode readahead should be aware of thread block group boundary.
3) make readahead_kb aware of multiple threads.

---
 e2fsck/pass1.c | 28 +++++++++++++++++++---------
 1 file changed, 19 insertions(+), 9 deletions(-)
e2fsck/pass1.c +19 -9
index e2387fe3..ad3bd8be 100644
@@ -1093,16 +1093,19 @@ out:
 static void pass1_readahead(e2fsck_t ctx, dgrp_t *group, ext2_ino_t *next_ino)
 {
 	ext2_ino_t inodes_in_group = 0, inodes_per_block, inodes_per_buffer;
-	dgrp_t start = *group, grp;
+	dgrp_t start = *group, grp, grp_end = ctx->fs->group_desc_count;
 	blk64_t blocks_to_read = 0;
 	errcode_t err = EXT2_ET_INVALID_ARGUMENT;
 
+#ifdef CONFIG_PFSCK
+	grp_end = ctx->thread_info.et_group_end;
+#endif
 	if (ctx->readahead_kb == 0)
 		goto out;
 
 	/* Keep iterating groups until we have enough to readahead */
 	inodes_per_block = EXT2_INODES_PER_BLOCK(ctx->fs->super);
-	for (grp = start; grp < ctx->fs->group_desc_count; grp++) {
+	for (grp = start; grp < grp_end; grp++) {
 		if (ext2fs_bg_flags_test(ctx->fs, grp, EXT2_BG_INODE_UNINIT))
 			continue;
 		inodes_in_group = ctx->fs->super->s_inodes_per_group -
@@ -1295,12 +1298,25 @@ static errcode_t e2fsck_pass1_prepare(e2fsck_t ctx)
 {
 	struct problem_context pctx;
 	ext2_filsys fs = ctx->fs;
+	unsigned long long readahead_kb;
 
 	init_ext2_max_sizes();
-#ifdef	CONFIG_PFSCK
+#ifdef CONFIG_PFSCK
 	e2fsck_pass1_set_thread_num(ctx);
 #endif
+	/* If we can do readahead, figure out how many groups to pull in. */
+	if (!e2fsck_can_readahead(ctx->fs))
+		ctx->readahead_kb = 0;
+	else if (ctx->readahead_kb == ~0ULL)
+		ctx->readahead_kb = e2fsck_guess_readahead(ctx->fs);
 
+#ifdef CONFIG_PFSCK
+	/* don't use more than 1/10 of memory for threads checking */
+	readahead_kb = get_memory_size() / (10 * ctx->fs_num_threads);
+	/* maybe better disable RA if this is too small? */
+	if (ctx->readahead_kb > readahead_kb)
+		ctx->readahead_kb = readahead_kb;
+#endif
 	clear_problem_context(&pctx);
 	if (!(ctx->options & E2F_OPT_PREEN))
 		fix_problem(ctx, PR_1_PASS_HEADER, &pctx);
@@ -1477,13 +1493,7 @@ void e2fsck_pass1_run(e2fsck_t ctx)
 	init_resource_track(&rtrack, ctx->fs->io);
 	clear_problem_context(&pctx);
 
-	/* If we can do readahead, figure out how many groups to pull in. */
-	if (!e2fsck_can_readahead(ctx->fs))
-		ctx->readahead_kb = 0;
-	else if (ctx->readahead_kb == ~0ULL)
-		ctx->readahead_kb = e2fsck_guess_readahead(ctx->fs);
 	pass1_readahead(ctx, &ra_group, &ino_threshold);
-
 	if (ext2fs_has_feature_dir_index(fs->super) &&
 	    !(ctx->options & E2F_OPT_NO)) {
 		if (ext2fs_u32_list_create(&ctx->dirs_to_hash, 50))
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#54 lore raw

RFC PATCH v3 37/61 e2fsck: merge options after threads finish

In reply to Saranya Muruganandam

From: Wang Shilong <[email protected]>

It will be possible that threads might append E2F_OPT_YES, so we need merge options to global, test f_yesall cover this.

---
 e2fsck/pass1.c | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)
e2fsck/pass1.c +4 -5
index ad3bd8be..1a68a2fb 100644
@@ -2935,6 +2935,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
 	ext2_refcount_t ea_inode_refs = global_ctx->ea_inode_refs;
 	ext2fs_block_bitmap  block_found_map = global_ctx->block_found_map;
 	ext2fs_block_bitmap  block_dup_map = global_ctx->block_dup_map;
+	int options = global_ctx->options;
 
 #ifdef HAVE_SETJMP_H
 	jmp_buf		 old_jmp;
@@ -2987,7 +2988,8 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
 	global_ctx->fs_fragmented += fs_fragmented;
 	global_ctx->fs_fragmented_dir += fs_fragmented_dir;
 	global_ctx->large_files += large_files;
-
+	/* threads might enable E2F_OPT_YES */
+	global_ctx->options |= options;
 	global_ctx->flags |= flags;
 
 	retval = e2fsck_pass1_merge_fs(global_fs, thread_fs);
@@ -3022,10 +3024,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
 					      thread_ctx->qctx);
 	if (retval)
 		return retval;
-	global_ctx->invalid_block_bitmap_flag = invalid_block_bitmap_flag;
-	global_ctx->invalid_inode_bitmap_flag = invalid_inode_bitmap_flag;
-	global_ctx->invalid_inode_table_flag = invalid_inode_table_flag;
-	global_ctx->invalid_bitmaps = invalid_bitmaps;
+
 	e2fsck_pass1_merge_invalid_bitmaps(global_ctx, thread_ctx);
 
 	retval = e2fsck_pass1_merge_bitmap(global_fs,
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#55 lore raw

Re: [RFC PATCH v3 37/61] e2fsck: merge options after threads finish

Quoted 4 lines · Saranya Muruganandam

From: Wang Shilong <[email protected]>

It will be possible that threads might append E2F_OPT_YES, so we need merge options to global, test f_yesall cover this.

I don't get it -- presumably fix_problem already has to take some sort of a lock to complain the console and (possibly) wait for input, right? Why not just set E2F_OPT_YES right then and there and have it take effect across all threads immediately? I guess you'd have to add another lock to protect the options bitflags.

--D

Quoted 43 lines
---
 e2fsck/pass1.c | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)
e2fsck/pass1.c +4 -5
index ad3bd8be..1a68a2fb 100644
@@ -2935,6 +2935,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
 	ext2_refcount_t ea_inode_refs = global_ctx->ea_inode_refs;
 	ext2fs_block_bitmap  block_found_map = global_ctx->block_found_map;
 	ext2fs_block_bitmap  block_dup_map = global_ctx->block_dup_map;
+	int options = global_ctx->options;
 
 #ifdef HAVE_SETJMP_H
 	jmp_buf		 old_jmp;
@@ -2987,7 +2988,8 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
 	global_ctx->fs_fragmented += fs_fragmented;
 	global_ctx->fs_fragmented_dir += fs_fragmented_dir;
 	global_ctx->large_files += large_files;
-
+	/* threads might enable E2F_OPT_YES */
+	global_ctx->options |= options;
 	global_ctx->flags |= flags;
 
 	retval = e2fsck_pass1_merge_fs(global_fs, thread_fs);
@@ -3022,10 +3024,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
 					      thread_ctx->qctx);
 	if (retval)
 		return retval;
-	global_ctx->invalid_block_bitmap_flag = invalid_block_bitmap_flag;
-	global_ctx->invalid_inode_bitmap_flag = invalid_inode_bitmap_flag;
-	global_ctx->invalid_inode_table_flag = invalid_inode_table_flag;
-	global_ctx->invalid_bitmaps = invalid_bitmaps;
+
 	e2fsck_pass1_merge_invalid_bitmaps(global_ctx, thread_ctx);
 
 	retval = e2fsck_pass1_merge_bitmap(global_fs,
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#56 lore raw

RFC PATCH v3 38/61 e2fsck: reset lost_and_found after threads finish

In reply to Saranya Muruganandam

From: Wang Shilong <[email protected]>

This should not be kept, the reaons is similar to what e2fsck_pass1 has done before.

---
 e2fsck/pass1.c | 5 +++++
 1 file changed, 5 insertions(+)
e2fsck/pass1.c +5 -0
index 1a68a2fb..09bfef44 100644
@@ -2991,6 +2991,11 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
 	/* threads might enable E2F_OPT_YES */
 	global_ctx->options |= options;
 	global_ctx->flags |= flags;
+	/*
+	 * The l+f inode may have been cleared, so zap it now and
+	 * later passes will recalculate it if necessary
+	 */
+	global_ctx->lost_and_found = 0;
 
 	retval = e2fsck_pass1_merge_fs(global_fs, thread_fs);
 	if (retval) {
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#57 lore raw

RFC PATCH v3 39/61 e2fsck: merge extent depth count after threads finish

In reply to Saranya Muruganandam

From: Wang Shilong <[email protected]>

tests covered by f_extent_htree.

---
 e2fsck/pass1.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)
e2fsck/pass1.c +10 -1
index 09bfef44..0a872028 100644
@@ -2935,8 +2935,11 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
 	ext2_refcount_t ea_inode_refs = global_ctx->ea_inode_refs;
 	ext2fs_block_bitmap  block_found_map = global_ctx->block_found_map;
 	ext2fs_block_bitmap  block_dup_map = global_ctx->block_dup_map;
-	int options = global_ctx->options;
+	int options = global_ctx->options, i;
+	__u32 extent_depth_count[MAX_EXTENT_DEPTH_COUNT];
 
+	memcpy(extent_depth_count, global_ctx->extent_depth_count,
+	       sizeof(extent_depth_count));
 #ifdef HAVE_SETJMP_H
 	jmp_buf		 old_jmp;
 
@@ -2996,6 +2999,12 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
 	 * later passes will recalculate it if necessary
 	 */
 	global_ctx->lost_and_found = 0;
+	memcpy(global_ctx->extent_depth_count, extent_depth_count,
+	       sizeof(extent_depth_count));
+	/* merge extent depth count */
+	for (i = 0; i < MAX_EXTENT_DEPTH_COUNT; i++)
+		global_ctx->extent_depth_count[i] +=
+			thread_ctx->extent_depth_count[i];
 
 	retval = e2fsck_pass1_merge_fs(global_fs, thread_fs);
 	if (retval) {
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#58 lore raw

RFC PATCH v3 40/61 e2fsck: simplify e2fsck context merging codes

In reply to Saranya Muruganandam

From: Wang Shilong <[email protected]>

We tried to copy thread context to global context directly and then copy back some saved variables before merging.

Since we have finished almost all necessary variables in the e2fsck context, we could simplify codes, and this could help us understand what is missing rather than hide problems.

---
 e2fsck/pass1.c | 151 ++++++++++---------------------------------------
 1 file changed, 31 insertions(+), 120 deletions(-)
e2fsck/pass1.c +31 -120
index 0a872028..03d7f455 100644
@@ -2432,6 +2432,7 @@ static int e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
 	ext2_badblocks_list badblocks;
 	ext2_dblist dblist;
 	int flags;
+	e2fsck_t dest_ctx = dest->priv_data;
 
 	dest_io = dest->io;
 	dest_image_io = dest->image_io;
@@ -2449,6 +2450,7 @@ static int e2fsck_pass1_merge_fs(ext2_filsys dest, ext2_filsys src)
 	dest->block_map = block_map;
 	dest->badblocks = badblocks;
 	dest->dblist = dblist;
+	dest->priv_data = dest_ctx;
 	if (dest->dblist)
 		dest->dblist->fs = dest;
 	dest->flags = src->flags | flags;
@@ -2882,140 +2884,51 @@ static errcode_t e2fsck_pass1_merge_ea_refcount(e2fsck_t global_ctx,
 	return retval;
 }
 
-static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx)
+static errcode_t e2fsck_pass1_merge_context(e2fsck_t global_ctx,
+					    e2fsck_t thread_ctx)
 {
-	errcode_t	 retval;
-	int		 flags = global_ctx->flags;
-	ext2_filsys	 thread_fs = thread_ctx->fs;
-	ext2_filsys	 global_fs = global_ctx->fs;
-	FILE		*global_logf = global_ctx->logf;
-	FILE		*global_problem_logf = global_ctx->problem_logf;
-	ext2fs_inode_bitmap inode_bad_map = global_ctx->inode_bad_map;
-	struct dir_info_db *dir_info = global_ctx->dir_info;
-	struct dx_dir_info *dx_dir_info = global_ctx->dx_dir_info;
-	ext2fs_inode_bitmap inode_used_map = global_ctx->inode_used_map;
-	ext2fs_inode_bitmap inode_dir_map = global_ctx->inode_dir_map;
-	ext2fs_inode_bitmap inode_bb_map = global_ctx->inode_bb_map;
-	ext2fs_inode_bitmap inode_imagic_map = global_ctx->inode_imagic_map;
-	ext2fs_inode_bitmap inode_reg_map = global_ctx->inode_reg_map;
-	ext2fs_block_bitmap inodes_to_rebuild = global_ctx->inodes_to_rebuild;
-	ext2_icount_t inode_count = global_ctx->inode_count;
-	ext2_icount_t inode_link_info = global_ctx->inode_link_info;
-	__u32 fs_directory_count = global_ctx->fs_directory_count;
-	__u32 fs_regular_count = global_ctx->fs_regular_count;
-	__u32 fs_blockdev_count = global_ctx->fs_blockdev_count;
-	__u32 fs_chardev_count = global_ctx->fs_chardev_count;
-	__u32 fs_links_count = global_ctx->fs_links_count;
-	__u32 fs_symlinks_count = global_ctx->fs_symlinks_count;
-	__u32 fs_fast_symlinks_count = global_ctx->fs_fast_symlinks_count;
-	__u32 fs_fifo_count = global_ctx->fs_fifo_count;
-	__u32 fs_total_count = global_ctx->fs_total_count;
-	__u32 fs_badblocks_count = global_ctx->fs_badblocks_count;
-	__u32 fs_sockets_count = global_ctx->fs_sockets_count;
-	__u32 fs_ind_count = global_ctx->fs_ind_count;
-	__u32 fs_dind_count = global_ctx->fs_dind_count;
-	__u32 fs_tind_count = global_ctx->fs_tind_count;
-	__u32 fs_fragmented = global_ctx->fs_fragmented;
-	__u32 fs_fragmented_dir = global_ctx->fs_fragmented_dir;
-	__u32 large_files = global_ctx->large_files;
-	ext2_ino_t dx_dir_info_size = global_ctx->dx_dir_info_size;
-	ext2_ino_t dx_dir_info_count = global_ctx->dx_dir_info_count;
-	ext2_u32_list dirs_to_hash = global_ctx->dirs_to_hash;
-	quota_ctx_t qctx = global_ctx->qctx;
-	int *invalid_block_bitmap_flag = global_ctx->invalid_block_bitmap_flag;
-	int *invalid_inode_bitmap_flag = global_ctx->invalid_inode_bitmap_flag;
-	int *invalid_inode_table_flag  = global_ctx->invalid_inode_table_flag;
-	int invalid_bitmaps = global_ctx->invalid_bitmaps;
-	ext2_refcount_t refcount = global_ctx->refcount;
-	ext2_refcount_t refcount_extra = global_ctx->refcount_extra;
-	ext2_refcount_t refcount_orig = global_ctx->refcount_orig;
-	ext2_refcount_t ea_block_quota_blocks = global_ctx->ea_block_quota_blocks;
-	ext2_refcount_t ea_block_quota_inodes = global_ctx->ea_block_quota_inodes;
-	ext2fs_block_bitmap block_ea_map = global_ctx->block_ea_map;
-	ext2_refcount_t ea_inode_refs = global_ctx->ea_inode_refs;
-	ext2fs_block_bitmap  block_found_map = global_ctx->block_found_map;
-	ext2fs_block_bitmap  block_dup_map = global_ctx->block_dup_map;
-	int options = global_ctx->options, i;
-	__u32 extent_depth_count[MAX_EXTENT_DEPTH_COUNT];
-
-	memcpy(extent_depth_count, global_ctx->extent_depth_count,
-	       sizeof(extent_depth_count));
-#ifdef HAVE_SETJMP_H
-	jmp_buf		 old_jmp;
-
-	memcpy(old_jmp, global_ctx->abort_loc, sizeof(jmp_buf));
-#endif
-	memcpy(global_ctx, thread_ctx, sizeof(struct e2fsck_struct));
-#ifdef HAVE_SETJMP_H
-	memcpy(global_ctx->abort_loc, old_jmp, sizeof(jmp_buf));
-#endif
+	ext2_filsys global_fs = global_ctx->fs;
+	errcode_t retval;
+	int i;
 
-	global_ctx->inode_used_map = inode_used_map;
-	global_ctx->inode_bad_map = inode_bad_map;
-	global_ctx->inode_dir_map = inode_dir_map;
-	global_ctx->inode_bb_map = inode_bb_map;
-	global_ctx->inode_imagic_map = inode_imagic_map;
-	global_ctx->inodes_to_rebuild = inodes_to_rebuild;
-	global_ctx->inode_reg_map = inode_reg_map;
-	global_ctx->block_dup_map = block_dup_map;
-	global_ctx->block_found_map = block_found_map;
-	global_ctx->dir_info = dir_info;
-	e2fsck_pass1_merge_dir_info(global_ctx, thread_ctx);
-	global_ctx->dx_dir_info = dx_dir_info;
-	global_ctx->dx_dir_info_count = dx_dir_info_count;
-	global_ctx->dx_dir_info_size = dx_dir_info_size;
-	e2fsck_pass1_merge_dx_dir(global_ctx, thread_ctx);
-	global_ctx->inode_count = inode_count;
-	global_ctx->inode_link_info = inode_link_info;
-	global_ctx->refcount = refcount;
-	global_ctx->refcount_extra = refcount_extra;
-	global_ctx->refcount_orig = refcount_orig;
-	global_ctx->ea_block_quota_blocks = ea_block_quota_blocks;
-	global_ctx->ea_block_quota_inodes = ea_block_quota_inodes;
-	global_ctx->block_ea_map = block_ea_map;
-	global_ctx->ea_inode_refs = ea_inode_refs;
-	global_ctx->fs_directory_count += fs_directory_count;
-	global_ctx->fs_regular_count += fs_regular_count;
-	global_ctx->fs_blockdev_count += fs_blockdev_count;
-	global_ctx->fs_chardev_count += fs_chardev_count;
-	global_ctx->fs_links_count += fs_links_count;
-	global_ctx->fs_symlinks_count += fs_symlinks_count;
-	global_ctx->fs_fast_symlinks_count += fs_fast_symlinks_count;
-	global_ctx->fs_fifo_count += fs_fifo_count;
-	global_ctx->fs_total_count += fs_total_count;
-	global_ctx->fs_badblocks_count += fs_badblocks_count;
-	global_ctx->fs_sockets_count += fs_sockets_count;
-	global_ctx->fs_ind_count += fs_ind_count;
-	global_ctx->fs_dind_count += fs_dind_count;
-	global_ctx->fs_tind_count += fs_tind_count;
-	global_ctx->fs_fragmented += fs_fragmented;
-	global_ctx->fs_fragmented_dir += fs_fragmented_dir;
-	global_ctx->large_files += large_files;
+	global_ctx->fs_directory_count += thread_ctx->fs_directory_count;
+	global_ctx->fs_regular_count += thread_ctx->fs_regular_count;
+	global_ctx->fs_blockdev_count += thread_ctx->fs_blockdev_count;
+	global_ctx->fs_chardev_count += thread_ctx->fs_chardev_count;
+	global_ctx->fs_links_count += thread_ctx->fs_links_count;
+	global_ctx->fs_symlinks_count += thread_ctx->fs_symlinks_count;
+	global_ctx->fs_fast_symlinks_count += thread_ctx->fs_fast_symlinks_count;
+	global_ctx->fs_fifo_count += thread_ctx->fs_fifo_count;
+	global_ctx->fs_total_count += thread_ctx->fs_total_count;
+	global_ctx->fs_badblocks_count += thread_ctx->fs_badblocks_count;
+	global_ctx->fs_sockets_count += thread_ctx->fs_sockets_count;
+	global_ctx->fs_ind_count += thread_ctx->fs_ind_count;
+	global_ctx->fs_dind_count += thread_ctx->fs_dind_count;
+	global_ctx->fs_tind_count += thread_ctx->fs_tind_count;
+	global_ctx->fs_fragmented += thread_ctx->fs_fragmented;
+	global_ctx->fs_fragmented_dir += thread_ctx->fs_fragmented_dir;
+	global_ctx->large_files += thread_ctx->large_files;
 	/* threads might enable E2F_OPT_YES */
-	global_ctx->options |= options;
-	global_ctx->flags |= flags;
+	global_ctx->options |= thread_ctx->options;
+	global_ctx->flags |= thread_ctx->flags;
 	/*
 	 * The l+f inode may have been cleared, so zap it now and
 	 * later passes will recalculate it if necessary
 	 */
 	global_ctx->lost_and_found = 0;
-	memcpy(global_ctx->extent_depth_count, extent_depth_count,
-	       sizeof(extent_depth_count));
 	/* merge extent depth count */
 	for (i = 0; i < MAX_EXTENT_DEPTH_COUNT; i++)
 		global_ctx->extent_depth_count[i] +=
 			thread_ctx->extent_depth_count[i];
 
-	retval = e2fsck_pass1_merge_fs(global_fs, thread_fs);
+	e2fsck_pass1_merge_dir_info(global_ctx, thread_ctx);
+	e2fsck_pass1_merge_dx_dir(global_ctx, thread_ctx);
+
+	retval = e2fsck_pass1_merge_fs(global_ctx->fs, thread_ctx->fs);
 	if (retval) {
 		com_err(global_ctx->program_name, 0, _("while merging fs\n"));
 		return retval;
 	}
-	global_fs->priv_data = global_ctx;
-	global_ctx->fs = global_fs;
-	global_ctx->logf = global_logf;
-	global_ctx->problem_logf = global_problem_logf;
-	global_ctx->global_ctx = NULL;
 	retval = e2fsck_pass1_merge_icounts(global_ctx, thread_ctx);
 	if (retval) {
 		com_err(global_ctx->program_name, 0,
@@ -3023,7 +2936,6 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
 		return retval;
 	}
 
-	global_ctx->dirs_to_hash = dirs_to_hash;
 	retval = e2fsck_pass1_merge_dirs_to_hash(global_ctx, thread_ctx);
 	if (retval) {
 		com_err(global_ctx->program_name, 0,
@@ -3033,7 +2945,6 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx
 
 	e2fsck_pass1_merge_ea_inode_refs(global_ctx, thread_ctx);
 	e2fsck_pass1_merge_ea_refcount(global_ctx, thread_ctx);
-	global_ctx->qctx = qctx;
 	retval = quota_merge_and_update_usage(global_ctx->qctx,
 					      thread_ctx->qctx);
 	if (retval)
@@ -3107,7 +3018,7 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
 {
 	errcode_t	retval;
 
-	retval = e2fsck_pass1_thread_join_one(global_ctx, thread_ctx);
+	retval = e2fsck_pass1_merge_context(global_ctx, thread_ctx);
 	ext2fs_free_mem(&thread_ctx->fs);
 	if (thread_ctx->logf)
 		fclose(thread_ctx->logf);
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#59 lore raw

RFC PATCH v3 41/61 e2fsck: set E2F_FLAG_ALLOC_OK after threads

In reply to Saranya Muruganandam

From: Wang Shilong <[email protected]>

Only flag ALLOC OK after all threads finished without problem.

---
 e2fsck/pass1.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)
e2fsck/pass1.c +8 -2
index 03d7f455..62345bb6 100644
@@ -1379,9 +1379,12 @@ static void e2fsck_pass1_post(e2fsck_t ctx)
 {
 	struct problem_context pctx;
 	ext2_filsys fs = ctx->fs;
+	char *block_buf;
 
-	char *block_buf =
-		(char *)e2fsck_allocate_memory(ctx, ctx->fs->blocksize * 3,
+	if (e2fsck_should_abort(ctx))
+		return;
+
+	block_buf = (char *)e2fsck_allocate_memory(ctx, ctx->fs->blocksize * 3,
 					      "block interate buffer");
 	reserve_block_for_root_repair(ctx);
 	reserve_block_for_lnf_repair(ctx);
@@ -1459,6 +1462,8 @@ static void e2fsck_pass1_post(e2fsck_t ctx)
 		ext2fs_free_mem(&block_buf);
 		ctx->flags &= ~E2F_FLAG_DUP_BLOCK;
 	}
+
+	ctx->flags |= E2F_FLAG_ALLOC_OK;
 }
 
 
@@ -2290,6 +2295,7 @@ void e2fsck_pass1_run(e2fsck_t ctx)
 	}
 
 	ctx->flags |= E2F_FLAG_ALLOC_OK;
+	ext2fs_free_mem(&inodes_to_process);
 endit:
 	e2fsck_use_inode_shortcuts(ctx, 0);
 	ext2fs_free_mem(&inodes_to_process);
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#60 lore raw

RFC PATCH v3 42/61 e2fsck: wait fix thread finish before checking

In reply to Saranya Muruganandam

From: Wang Shilong <[email protected]>

Before proceeding next inodes, waitting existed
fixing finished.

---
 e2fsck/e2fsck.c |  3 +++
 e2fsck/e2fsck.h |  5 +++-
 e2fsck/pass1.c  | 65 +++++++++++++++++++++++++++++++++++++++++++------
 e2fsck/util.c   | 56 +++++++++++++++++++++++++++++++++++++++---
 4 files changed, 117 insertions(+), 12 deletions(-)
e2fsck/e2fsck.c +3 -0
index d8be566f..1fd57504 100644
@@ -178,6 +178,9 @@ errcode_t e2fsck_reset_context(e2fsck_t ctx)
 	ctx->fs_fragmented = 0;
 	ctx->fs_fragmented_dir = 0;
 	ctx->large_files = 0;
+#ifdef CONFIG_PFSCK
+	ctx->fs_need_locking = 0;
+#endif
 
 	for (i=0; i < MAX_EXTENT_DEPTH_COUNT; i++)
 		ctx->extent_depth_count[i] = 0;
e2fsck/e2fsck.h +4 -1
index f46a95ef..a66772c1 100644
@@ -451,8 +451,9 @@ struct e2fsck_struct {
 	char *undo_file;
 #ifdef CONFIG_PFSCK
 	__u32			 fs_num_threads;
+	int			 fs_need_locking;
 	/* serialize fix operation for multiple threads */
-	pthread_mutex_t		 fs_fix_mutex;
+	pthread_rwlock_t	 fs_fix_rwlock;
 	/* protect block_found_map, block_dup_map */
 	pthread_rwlock_t	 fs_block_map_rwlock;
 #endif
@@ -514,6 +515,8 @@ extern int e2fsck_strnlen(const char * s, int count);
 
 extern void e2fsck_pass1(e2fsck_t ctx);
 extern void e2fsck_pass1_dupblocks(e2fsck_t ctx, char *block_buf);
+extern void e2fsck_pass1_check_lock(e2fsck_t ctx);
+extern void e2fsck_pass1_check_unlock(e2fsck_t ctx);
 extern void e2fsck_pass2(e2fsck_t ctx);
 extern void e2fsck_pass3(e2fsck_t ctx);
 extern void e2fsck_pass4(e2fsck_t ctx);
e2fsck/pass1.c +57 -8

235 lines not on this page · raw message

e2fsck/util.c +53 -3

104 lines not on this page · raw message

Signature · 1 line
2.29.2.299.gdc1121823c-goog
#61 lore raw

RFC PATCH v3 43/61 e2fsck: cleanup e2fsck_pass1_thread_join()

In reply to Saranya Muruganandam

From: Wang Shilong <[email protected]>

Use e2fsck_reset_context() to free memory to simpify codes.

---
 e2fsck/pass1.c | 34 ++++++++--------------------------
 1 file changed, 8 insertions(+), 26 deletions(-)
e2fsck/pass1.c +8 -26
index 5e62e357..60f70111 100644
@@ -3079,32 +3079,14 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
 		fputs("</problem_log>\n", thread_ctx->problem_logf);
 		fclose(thread_ctx->problem_logf);
 	}
-	e2fsck_pass1_free_bitmap(&thread_ctx->inode_used_map);
-	e2fsck_pass1_free_bitmap(&thread_ctx->inode_bad_map);
-	e2fsck_pass1_free_bitmap(&thread_ctx->inode_dir_map);
-	e2fsck_pass1_free_bitmap(&thread_ctx->inode_bb_map);
-	e2fsck_pass1_free_bitmap(&thread_ctx->inode_imagic_map);
-	e2fsck_pass1_free_bitmap(&thread_ctx->inode_reg_map);
-	e2fsck_pass1_free_bitmap(&thread_ctx->inodes_to_rebuild);
-	e2fsck_pass1_free_bitmap(&thread_ctx->block_found_map);
-	e2fsck_pass1_free_bitmap(&thread_ctx->block_ea_map);
-	if (thread_ctx->refcount)
-		ea_refcount_free(thread_ctx->refcount);
-	if (thread_ctx->refcount_extra)
-		ea_refcount_free(thread_ctx->refcount_extra);
-	if (thread_ctx->ea_inode_refs)
-		ea_refcount_free(thread_ctx->ea_inode_refs);
-	if (thread_ctx->refcount_orig)
-		ea_refcount_free(thread_ctx->refcount_orig);
-	e2fsck_free_dir_info(thread_ctx);
-	ext2fs_free_icount(thread_ctx->inode_count);
-	ext2fs_free_icount(thread_ctx->inode_link_info);
-	if (thread_ctx->dirs_to_hash)
-		ext2fs_badblocks_list_free(thread_ctx->dirs_to_hash);
-	quota_release_context(&thread_ctx->qctx);
-	ext2fs_free_mem(&thread_ctx->invalid_block_bitmap_flag);
-	ext2fs_free_mem(&thread_ctx->invalid_inode_bitmap_flag);
-	ext2fs_free_mem(&thread_ctx->invalid_inode_table_flag);
+
+	/*
+	 * @block_metadata_map and @block_dup_map are
+	 * shared, so we don't free them.
+	 */
+	thread_ctx->block_metadata_map = NULL;
+	thread_ctx->block_dup_map = NULL;
+	e2fsck_reset_context(thread_ctx);
 	ext2fs_free_mem(&thread_ctx);
 
 	return retval;
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#62 lore raw

RFC PATCH v3 44/61 e2fsck: avoid too much memory allocation for pfsck

In reply to Saranya Muruganandam

From: Wang Shilong <[email protected]>

e2fsck init memory according to filesystem inodes/dir numbers recorded in the superblock, this should be aware of filesystem number of threads, otherwise, oom happen.

---
 lib/ext2fs/dblist.c | 2 ++
 lib/ext2fs/icount.c | 4 ++++
 2 files changed, 6 insertions(+)
lib/ext2fs/dblist.c +2 -0
index 1fdd8f43..c4e712fd 100644
@@ -58,6 +58,8 @@ static errcode_t make_dblist(ext2_filsys fs, ext2_ino_t size,
 		if (retval)
 			goto cleanup;
 		dblist->size = (num_dirs * 2) + 12;
+		if (fs->fs_num_threads)
+			dblist->size /= fs->fs_num_threads;
 	}
 	len = (size_t) sizeof(struct ext2_db_entry2) * dblist->size;
 	dblist->count = count;
lib/ext2fs/icount.c +4 -0
index 766eccca..48665c7e 100644
@@ -237,6 +237,8 @@ errcode_t ext2fs_create_icount_tdb(ext2_filsys fs EXT2FS_NO_TDB_UNUSED,
 	 * value.
 	 */
 	num_inodes = fs->super->s_inodes_count - fs->super->s_free_inodes_count;
+	if (fs->fs_num_threads)
+		num_inodes /= fs->fs_num_threads;
 
 	icount->tdb = tdb_open(fn, num_inodes, TDB_NOLOCK | TDB_NOSYNC,
 			       O_RDWR | O_CREAT | O_TRUNC, 0600);
@@ -288,6 +290,8 @@ errcode_t ext2fs_create_icount2(ext2_filsys fs, int flags, unsigned int size,
 		if (retval)
 			goto errout;
 		icount->size += fs->super->s_inodes_count / 50;
+		if (fs->fs_num_threads)
+			icount->size /= fs->fs_num_threads;
 	}
 
 	bytes = (size_t) (icount->size * sizeof(struct ext2_icount_el));
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#63 lore raw

RFC PATCH v3 45/61 e2fsck: make default smallest RA size to 1M

In reply to Saranya Muruganandam

From: Wang Shilong <[email protected]>

If we have a smaller inodes per group, default ra size could be very small(etc 128KiB), this hurts performances.

Tune above 128K to 1M, i see pass1 time drop down from 677.12 seconds to 246 secons with 32 threads.

---
 e2fsck/readahead.c | 4 ++++
 1 file changed, 4 insertions(+)
e2fsck/readahead.c +4 -0
index 38d4ec42..40b73664 100644
@@ -234,6 +234,8 @@ int e2fsck_can_readahead(ext2_filsys fs)
 	return err != EXT2_ET_OP_NOT_SUPPORTED;
 }
 
+#define MIN_DEFAULT_RA	(1024 * 1024)
+
 unsigned long long e2fsck_guess_readahead(ext2_filsys fs)
 {
 	unsigned long long guess;
@@ -245,6 +247,8 @@ unsigned long long e2fsck_guess_readahead(ext2_filsys fs)
 	 * in e2fsck runtime.
 	 */
 	guess = 2ULL * fs->blocksize * fs->inode_blocks_per_group;
+	if (guess < MIN_DEFAULT_RA)
+		guess = MIN_DEFAULT_RA;
 
 	/* Disable RA if it'd use more 1/50th of RAM. */
 	if (get_memory_size() > (guess * 50))
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#64 lore raw

RFC PATCH v3 46/61 ext2fs: parallel bitmap loading

In reply to Saranya Muruganandam

From: Wang Shilong <[email protected]>

In our benchmarking for PiB size filesystem, pass5 takes 10446s to finish and 99.5% of time takes on reading bitmaps.

It makes sense to reading bitmaps using multiple threads, a quickly benchmark show 10446s to 626s with 64 threads.

---
 lib/ext2fs/rw_bitmaps.c | 289 ++++++++++++++++++++++++++++++++++------
 1 file changed, 250 insertions(+), 39 deletions(-)
lib/ext2fs/rw_bitmaps.c +250 -39
index d80c9eb8..960a6913 100644
@@ -23,6 +23,7 @@
 #ifdef HAVE_SYS_TYPES_H
 #include <sys/types.h>
 #endif
+#include <pthread.h>
 
 #include "ext2_fs.h"
 #include "ext2fs.h"
@@ -205,22 +206,12 @@ static int bitmap_tail_verify(unsigned char *bitmap, int first, int last)
 	return 1;
 }
 
-static errcode_t read_bitmaps(ext2_filsys fs, int do_inode, int do_block)
+static errcode_t read_bitmaps_range_prepare(ext2_filsys fs, int do_inode, int do_block)
 {
-	dgrp_t i;
-	char *block_bitmap = 0, *inode_bitmap = 0;
-	char *buf;
 	errcode_t retval;
 	int block_nbytes = EXT2_CLUSTERS_PER_GROUP(fs->super) / 8;
 	int inode_nbytes = EXT2_INODES_PER_GROUP(fs->super) / 8;
-	int tail_flags = 0;
-	int csum_flag;
-	unsigned int	cnt;
-	blk64_t	blk;
-	blk64_t	blk_itr = EXT2FS_B2C(fs, fs->super->s_first_data_block);
-	blk64_t   blk_cnt;
-	ext2_ino_t ino_itr = 1;
-	ext2_ino_t ino_cnt;
+	char *buf;
 
 	EXT2_CHECK_MAGIC(fs, EXT2_ET_MAGIC_EXT2FS_FILSYS);
 
@@ -230,11 +221,10 @@ static errcode_t read_bitmaps(ext2_filsys fs, int do_inode, int do_block)
 
 	fs->write_bitmaps = ext2fs_write_bitmaps;
 
-	csum_flag = ext2fs_has_group_desc_csum(fs);
-
 	retval = ext2fs_get_mem(strlen(fs->device_name) + 80, &buf);
 	if (retval)
 		return retval;
+
 	if (do_block) {
 		if (fs->block_map)
 			ext2fs_free_block_bitmap(fs->block_map);
@@ -243,11 +233,8 @@ static errcode_t read_bitmaps(ext2_filsys fs, int do_inode, int do_block)
 		retval = ext2fs_allocate_block_bitmap(fs, buf, &fs->block_map);
 		if (retval)
 			goto cleanup;
-		retval = io_channel_alloc_buf(fs->io, 0, &block_bitmap);
-		if (retval)
-			goto cleanup;
-	} else
-		block_nbytes = 0;
+	}
+
 	if (do_inode) {
 		if (fs->inode_map)
 			ext2fs_free_inode_bitmap(fs->inode_map);
@@ -256,13 +243,69 @@ static errcode_t read_bitmaps(ext2_filsys fs, int do_inode, int do_block)
 		retval = ext2fs_allocate_inode_bitmap(fs, buf, &fs->inode_map);
 		if (retval)
 			goto cleanup;
-		retval = io_channel_alloc_buf(fs->io, 0, &inode_bitmap);
+	}
+	ext2fs_free_mem(&buf);
+
+	return retval;
+
+cleanup:
+	if (do_block) {
+		ext2fs_free_block_bitmap(fs->block_map);
+		fs->block_map = 0;
+	}
+	if (do_inode) {
+		ext2fs_free_inode_bitmap(fs->inode_map);
+		fs->inode_map = 0;
+	}
+	if (buf)
+		ext2fs_free_mem(&buf);
+	return retval;
+}
+
+static errcode_t read_bitmaps_range_start(ext2_filsys fs, int do_inode, int do_block,
+					  dgrp_t start, dgrp_t end, pthread_mutex_t *mutex,
+					  io_channel io)
+{
+	dgrp_t i;
+	char *block_bitmap = 0, *inode_bitmap = 0;
+	char *buf;
+	errcode_t retval;
+	int block_nbytes = EXT2_CLUSTERS_PER_GROUP(fs->super) / 8;
+	int inode_nbytes = EXT2_INODES_PER_GROUP(fs->super) / 8;
+	int tail_flags = 0;
+	int csum_flag;
+	unsigned int	cnt;
+	blk64_t	blk;
+	blk64_t	blk_itr = EXT2FS_B2C(fs, fs->super->s_first_data_block);
+	blk64_t   blk_cnt;
+	ext2_ino_t ino_itr = 1;
+	ext2_ino_t ino_cnt;
+	io_channel this_io;
+
+	if (!io)
+		this_io = fs->io;
+	else
+		this_io = io;
+
+	csum_flag = ext2fs_has_group_desc_csum(fs);
+
+	if (do_block) {
+		retval = io_channel_alloc_buf(this_io, 0, &block_bitmap);
+		if (retval)
+			goto cleanup;
+	} else {
+		block_nbytes = 0;
+	}
+
+	if (do_inode) {
+		retval = io_channel_alloc_buf(this_io, 0, &inode_bitmap);
 		if (retval)
 			goto cleanup;
-	} else
+	} else {
 		inode_nbytes = 0;
-	ext2fs_free_mem(&buf);
+	}
 
+	/* io should be null */
 	if (fs->flags & EXT2_FLAG_IMAGE_FILE) {
 		blk = (ext2fs_le32_to_cpu(fs->image_header->offset_inodemap) / fs->blocksize);
 		ino_cnt = fs->super->s_inodes_count;
@@ -303,7 +346,9 @@ static errcode_t read_bitmaps(ext2_filsys fs, int do_inode, int do_block)
 		goto success_cleanup;
 	}
 
-	for (i = 0; i < fs->group_desc_count; i++) {
+	blk_itr += ((blk64_t)start * (block_nbytes << 3));
+	ino_itr += ((blk64_t)start * (inode_nbytes << 3));
+	for (i = start; i <= end; i++) {
 		if (block_bitmap) {
 			blk = ext2fs_block_bitmap_loc(fs, i);
 			if ((csum_flag &&
@@ -312,7 +357,7 @@ static errcode_t read_bitmaps(ext2_filsys fs, int do_inode, int do_block)
 			    (blk >= ext2fs_blocks_count(fs->super)))
 				blk = 0;
 			if (blk) {
-				retval = io_channel_read_blk64(fs->io, blk,
+				retval = io_channel_read_blk64(this_io, blk,
 							       1, block_bitmap);
 				if (retval) {
 					retval = EXT2_ET_BLOCK_BITMAP_READ;
@@ -333,8 +378,12 @@ static errcode_t read_bitmaps(ext2_filsys fs, int do_inode, int do_block)
 			} else
 				memset(block_bitmap, 0, block_nbytes);
 			cnt = block_nbytes << 3;
+			if (mutex)
+				pthread_mutex_lock(mutex);
 			retval = ext2fs_set_block_bitmap_range2(fs->block_map,
 					       blk_itr, cnt, block_bitmap);
+			if (mutex)
+				pthread_mutex_unlock(mutex);
 			if (retval)
 				goto cleanup;
 			blk_itr += block_nbytes << 3;
@@ -347,7 +396,7 @@ static errcode_t read_bitmaps(ext2_filsys fs, int do_inode, int do_block)
 			    (blk >= ext2fs_blocks_count(fs->super)))
 				blk = 0;
 			if (blk) {
-				retval = io_channel_read_blk64(fs->io, blk,
+				retval = io_channel_read_blk64(this_io, blk,
 							       1, inode_bitmap);
 				if (retval) {
 					retval = EXT2_ET_INODE_BITMAP_READ;
@@ -369,29 +418,28 @@ static errcode_t read_bitmaps(ext2_filsys fs, int do_inode, int do_block)
 			} else
 				memset(inode_bitmap, 0, inode_nbytes);
 			cnt = inode_nbytes << 3;
+			if (mutex)
+				pthread_mutex_lock(mutex);
 			retval = ext2fs_set_inode_bitmap_range2(fs->inode_map,
 					       ino_itr, cnt, inode_bitmap);
+			if (mutex)
+				pthread_mutex_unlock(mutex);
 			if (retval)
 				goto cleanup;
 			ino_itr += inode_nbytes << 3;
 		}
 	}
 
-	/* Mark group blocks for any BLOCK_UNINIT groups */
-	if (do_block) {
-		retval = mark_uninit_bg_group_blocks(fs);
-		if (retval)
-			goto cleanup;
-	}
-
 success_cleanup:
-	if (inode_bitmap) {
-		ext2fs_free_mem(&inode_bitmap);
-		fs->flags &= ~EXT2_FLAG_IBITMAP_TAIL_PROBLEM;
-	}
-	if (block_bitmap) {
-		ext2fs_free_mem(&block_bitmap);
-		fs->flags &= ~EXT2_FLAG_BBITMAP_TAIL_PROBLEM;
+	if (start == 0 && end == fs->group_desc_count - 1) {
+		if (inode_bitmap) {
+			ext2fs_free_mem(&inode_bitmap);
+			fs->flags &= ~EXT2_FLAG_IBITMAP_TAIL_PROBLEM;
+		}
+		if (block_bitmap) {
+			ext2fs_free_mem(&block_bitmap);
+			fs->flags &= ~EXT2_FLAG_BBITMAP_TAIL_PROBLEM;
+		}
 	}
 	fs->flags |= tail_flags;
 	return 0;
@@ -412,6 +460,169 @@ cleanup:
 	if (buf)
 		ext2fs_free_mem(&buf);
 	return retval;
+
+}
+
+static errcode_t read_bitmaps_range_end(ext2_filsys fs, int do_inode, int do_block)
+{
+	errcode_t retval = 0;
+
+	/* Mark group blocks for any BLOCK_UNINIT groups */
+	if (do_block) {
+		retval = mark_uninit_bg_group_blocks(fs);
+		if (retval)
+			goto cleanup;
+	}
+
+	return retval;
+cleanup:
+	if (do_block) {
+		ext2fs_free_block_bitmap(fs->block_map);
+		fs->block_map = 0;
+	}
+	if (do_inode) {
+		ext2fs_free_inode_bitmap(fs->inode_map);
+		fs->inode_map = 0;
+	}
+	return retval;
+}
+
+static errcode_t read_bitmaps_range(ext2_filsys fs, int do_inode, int do_block,
+				    dgrp_t start, dgrp_t end)
+{
+	errcode_t retval;
+
+	retval = read_bitmaps_range_prepare(fs, do_inode, do_block);
+	if (retval)
+		return retval;
+
+	retval = read_bitmaps_range_start(fs, do_inode, do_block, start, end, NULL, NULL);
+	if (retval)
+		return retval;
+
+	return read_bitmaps_range_end(fs, do_inode, do_block);
+}
+
+#ifdef CONFIG_PFSCK
+struct read_bitmaps_thread_info {
+	ext2_filsys	rbt_fs;
+	int 		rbt_do_inode;
+	int		rbt_do_block;
+	dgrp_t		rbt_grp_start;
+	dgrp_t		rbt_grp_end;
+	errcode_t	rbt_retval;
+	pthread_mutex_t *rbt_mutex;
+	io_channel      rbt_io;
+};
+
+static void* read_bitmaps_thread(void *data)
+{
+	struct read_bitmaps_thread_info *rbt = data;
+
+	rbt->rbt_retval = read_bitmaps_range_start(rbt->rbt_fs,
+				rbt->rbt_do_inode, rbt->rbt_do_block,
+				rbt->rbt_grp_start, rbt->rbt_grp_end,
+				rbt->rbt_mutex, rbt->rbt_io);
+	return NULL;
+}
+#endif
+
+static errcode_t read_bitmaps(ext2_filsys fs, int do_inode, int do_block)
+{
+#ifdef CONFIG_PFSCK
+	pthread_attr_t	attr;
+	int num_threads = fs->fs_num_threads;
+	pthread_t *thread_ids = NULL;
+	struct read_bitmaps_thread_info *thread_infos = NULL;
+	pthread_mutex_t rbt_mutex = PTHREAD_MUTEX_INITIALIZER;
+	errcode_t retval;
+	errcode_t rc;
+	dgrp_t average_group;
+	int i;
+	io_manager manager = unix_io_manager;
+#else
+	int num_threads = 1;
+#endif
+
+	if (num_threads <= 1 || (fs->flags & EXT2_FLAG_IMAGE_FILE))
+		return read_bitmaps_range(fs, do_inode, do_block, 0, fs->group_desc_count - 1);
+
+#ifdef CONFIG_PFSCK
+	retval = pthread_attr_init(&attr);
+	if (retval)
+		return retval;
+
+	thread_ids = calloc(sizeof(pthread_t), num_threads);
+	if (!thread_ids)
+		return -ENOMEM;
+
+	thread_infos = calloc(sizeof(struct read_bitmaps_thread_info),
+				num_threads);
+	if (!thread_infos)
+		goto out;
+
+	average_group = ext2fs_get_avg_group(fs);
+	retval = read_bitmaps_range_prepare(fs, do_inode, do_block);
+	if (retval)
+		goto out;
+
+	fprintf(stdout, "Multiple threads triggered to read bitmaps\n");
+	for (i = 0; i < num_threads; i++) {
+		thread_infos[i].rbt_fs = fs;
+		thread_infos[i].rbt_do_inode = do_inode;
+		thread_infos[i].rbt_do_block = do_block;
+		thread_infos[i].rbt_mutex = &rbt_mutex;
+		if (i == 0)
+			thread_infos[i].rbt_grp_start = 0;
+		else
+			thread_infos[i].rbt_grp_start = average_group * i + 1;
+
+		if (i == num_threads - 1)
+			thread_infos[i].rbt_grp_end = fs->group_desc_count - 1;
+		else
+			thread_infos[i].rbt_grp_end = average_group * (i + 1);
+		retval = manager->open(fs->device_name, IO_FLAG_RW,
+					&thread_infos[i].rbt_io);
+		if (retval)
+			break;
+		io_channel_set_blksize(thread_infos[i].rbt_io, fs->io->block_size);
+		retval = pthread_create(&thread_ids[i], &attr,
+					&read_bitmaps_thread, &thread_infos[i]);
+		if (retval) {
+			io_channel_close(thread_infos[i].rbt_io);
+			break;
+		}
+	}
+	for (i = 0; i < num_threads; i++) {
+		if (!thread_ids[i])
+			break;
+		rc = pthread_join(thread_ids[i], NULL);
+		if (rc && !retval)
+			retval = rc;
+		rc = thread_infos[i].rbt_retval;
+		if (rc && !retval)
+			retval = rc;
+		io_channel_close(thread_infos[i].rbt_io);
+	}
+out:
+	rc = pthread_attr_destroy(&attr);
+	if (rc && !retval)
+		retval = rc;
+	free(thread_infos);
+	free(thread_ids);
+
+	if (!retval)
+		retval = read_bitmaps_range_end(fs, do_inode, do_block);
+
+	if (!retval) {
+		if (do_inode)
+			fs->flags &= ~EXT2_FLAG_IBITMAP_TAIL_PROBLEM;
+		if (do_block)
+			fs->flags &= ~EXT2_FLAG_BBITMAP_TAIL_PROBLEM;
+	}
+
+	return retval;
+#endif
 }
 
 errcode_t ext2fs_read_inode_bitmap(ext2_filsys fs)
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#65 lore raw

Re: [RFC PATCH v3 46/61] ext2fs: parallel bitmap loading

Quoted 10 lines · Saranya Muruganandam

From: Wang Shilong <[email protected]>

In our benchmarking for PiB size filesystem, pass5 takes 10446s to finish and 99.5% of time takes on reading bitmaps.

It makes sense to reading bitmaps using multiple threads, a quickly benchmark show 10446s to 626s with 64 threads.

Note: This patch will *explode* with much hilarity if num_threads is greater than the number of block groups. That's because the ext2fs_get_avg_group() will return 1 if fs_num_threads is greater than group_desc_count.

This will result in the group start and end limits to go beyond the array boundaries.... and then *boom*. So there will probably need to be some kind of safety checks if the caller has set fs_num_threads to a value which is much larger than is appropriate for a given file system.

Speaking of which, relying on fs_num_threads being set by e2fsck means that we won't get the benefits of the parallel block bitmap reads for debugfs and dumpe2fs. So we should think about how the other tools should trigger read bitmaps. And this might be something that we want to do independent of whether we are doing parallel fsck.

Suggested approach:

1) Create create ext2fs_is_device_rotational() which returns whether or not a particular device is a HDD, or a non-rotational device (e.g., SSD, GCE PD, AWS EBS, etc.), using rotational as a proxy for "reading using multiple threads is a good thing".

2) Create an ext2fs_get_num_procs() which calls sysconf(_SC_NPROCESSOR_ONLN) if sysconf and _SC_NPROESSOR_ONLN is available. If not, there may be other OS-specific ways of determining the number of CPU's available.

3) If HAVE_PTHREADS and the number of block groups is greater than the number of CPU's * 2, a function that (for now) we drop in libsupport will set fs->fs_num_threads to the number of processors as the default. There may not be a reason to change the default for debugfs and dumpe2fs, but for e2fsck, this would be used for the default, but it could be over-ridden via "-E multithread=<number of threads>".

- Ted

#66 lore raw

RFC PATCH v3 47/61 e2fsck: update mmp block in one thread

In reply to Saranya Muruganandam

From: Wang Shilong <[email protected]>

For multiple threads, different threads will try to update mmp block at the same time, only allow one thread to update it.

---
 e2fsck/e2fsck.h |  1 +
 e2fsck/pass1.c  | 36 ++++++++++++++++++++++++++++++++++--
 2 files changed, 35 insertions(+), 2 deletions(-)
e2fsck/e2fsck.h +1 -0
index a66772c1..1469c3e1 100644
@@ -451,6 +451,7 @@ struct e2fsck_struct {
 	char *undo_file;
 #ifdef CONFIG_PFSCK
 	__u32			 fs_num_threads;
+	__u32			 mmp_update_thread;
 	int			 fs_need_locking;
 	/* serialize fix operation for multiple threads */
 	pthread_rwlock_t	 fs_fix_rwlock;
e2fsck/pass1.c +34 -2
index 60f70111..e98cda9f 100644
@@ -1497,7 +1497,7 @@ void e2fsck_pass1_run(e2fsck_t ctx)
 	dgrp_t		ra_group = 0;
 	struct ea_quota	ea_ibody_quota;
 	struct process_inode_block *inodes_to_process;
-	int		process_inode_count;
+	int		process_inode_count, check_mmp;
 
 	init_resource_track(&rtrack, ctx->fs->io);
 	clear_problem_context(&pctx);
@@ -1646,8 +1646,33 @@ void e2fsck_pass1_run(e2fsck_t ctx)
 #endif
 
 	while (1) {
+		check_mmp = 0;
 		e2fsck_pass1_check_lock(ctx);
-		if (ino % (fs->super->s_inodes_per_group * 4) == 1) {
+#ifdef	CONFIG_PFSCK
+		if (!ctx->mmp_update_thread) {
+			e2fsck_pass1_block_map_w_lock(ctx);
+			if (!ctx->mmp_update_thread) {
+				if (ctx->global_ctx)
+					ctx->mmp_update_thread =
+						ctx->thread_info.et_thread_index + 1;
+				else
+					ctx->mmp_update_thread = 1;
+				check_mmp = 1;
+			}
+			e2fsck_pass1_block_map_w_unlock(ctx);
+		}
+
+		/* only one active thread could update mmp block. */
+		e2fsck_pass1_block_map_r_lock(ctx);
+		if (!ctx->global_ctx || ctx->mmp_update_thread ==
+			(ctx->thread_info.et_thread_index + 1))
+			check_mmp = 1;
+		e2fsck_pass1_block_map_r_unlock(ctx);
+#else
+		check_mmp = 1;
+#endif
+
+		if (check_mmp && (ino % (fs->super->s_inodes_per_group * 4) == 1)) {
 			if (e2fsck_mmp_update(fs))
 				fatal_error(ctx, 0);
 		}
@@ -2365,6 +2390,13 @@ endit:
 		print_resource_track(ctx, _("Pass 1"), &rtrack, ctx->fs->io);
 	else
 		ctx->invalid_bitmaps++;
+#ifdef	CONFIG_PFSCK
+	/* reset update_thread after this thread exit */
+	e2fsck_pass1_block_map_w_lock(ctx);
+	if (ctx->mmp_update_thread)
+		ctx->mmp_update_thread = 0;
+	e2fsck_pass1_block_map_w_unlock(ctx);
+#endif
 }
 
 #ifdef CONFIG_PFSCK
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#67 lore raw

RFC PATCH v3 48/61 e2fsck: reset @inodes_to_rebuild if restart

In reply to Saranya Muruganandam

From: Wang Shilong <[email protected]>

Verify multiple thread on a corrupted images hit following bug:

pass1.c:2902: e2fsck_pass1_thread_prepare:
Assertion `global_ctx->inodes_to_rebuild == NULL' failed. Signal (6) SIGABRT si_code=SI_TKILL ./e2fsck/e2fsck[0x43829e] /lib64/libpthread.so.0(+0x14b20)[0x7f3b45135b20] /lib64/libc.so.6(gsignal+0x145)[0x7f3b44f2c625] /lib64/libc.so.6(abort+0x12b)[0x7f3b44f158d9] /lib64/libc.so.6(+0x257a9)[0x7f3b44f157a9] /lib64/libc.so.6(+0x34a66)[0x7f3b44f24a66] ./e2fsck/e2fsck(e2fsck_pass1+0x1662)[0x423572] ./e2fsck/e2fsck(e2fsck_run+0x5a)[0x41611a] ./e2fsck/e2fsck(main+0x1608)[0x4121b8] /lib64/libc.so.6(__libc_start_main+0xf3)[0x7f3b44f171a3] ./e2fsck/e2fsck(_start+0x2e)[0x413dde]

@inodes_to_rebuild could be not NULL after we restart pass1

---
 e2fsck/e2fsck.c | 4 ++++
 1 file changed, 4 insertions(+)
e2fsck/e2fsck.c +4 -0
index 1fd57504..a03550c0 100644
@@ -75,6 +75,10 @@ errcode_t e2fsck_reset_context(e2fsck_t ctx)
 		ext2fs_free_block_bitmap(ctx->block_found_map);
 		ctx->block_found_map = 0;
 	}
+	if (ctx->inodes_to_rebuild) {
+		ext2fs_free_inode_bitmap(ctx->inodes_to_rebuild);
+		ctx->inodes_to_rebuild = 0;
+	}
 	if (ctx->inode_link_info) {
 		ext2fs_free_icount(ctx->inode_link_info);
 		ctx->inode_link_info = 0;
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#68 lore raw

RFC PATCH v3 49/61 e2fsck: fix build for make rpm

In reply to Saranya Muruganandam

From: Wang Shilong <[email protected]>

link e2fsck with -lpthread properly to make rpm pass.

---
 MCONFIG.in         | 1 +
 e2fsck/Makefile.in | 9 +++++----
 2 files changed, 6 insertions(+), 4 deletions(-)
MCONFIG.in +1 -0
index 0598f21b..a85e24f1 100644
@@ -146,6 +146,7 @@ DEPLIBUUID = @DEPLIBUUID@
 DEPLIBSUPPORT = $(LIB)/libsupport@STATIC_LIB_EXT@
 DEPLIBBLKID = @DEPLIBBLKID@ @PRIVATE_LIBS_CMT@ $(DEPLIBUUID)
 TESTENV = LD_LIBRARY_PATH="$(LIB):$${LD_LIBRARY_PATH}" DYLD_LIBRARY_PATH="$(LIB):$${DYLD_LIBRARY_PATH}"
+SEM_INIT_LIB = @SEM_INIT_LIB@
 
 STATIC_LIBSS = $(LIB)/libss@STATIC_LIB_EXT@ @DLOPEN_LIB@
 STATIC_LIBCOM_ERR = $(LIB)/libcom_err@STATIC_LIB_EXT@ @SEM_INIT_LIB@
e2fsck/Makefile.in +5 -4
index 6e25d27d..0ec11952 100644
@@ -16,20 +16,21 @@ MANPAGES=	e2fsck.8
 FMANPAGES=	e2fsck.conf.5
 
 LIBS= $(LIBSUPPORT) $(LIBEXT2FS) $(LIBCOM_ERR) $(LIBBLKID) $(LIBUUID) \
-	$(LIBINTL) $(LIBE2P) $(LIBMAGIC) $(SYSLIBS)
+	$(LIBINTL) $(LIBE2P) $(LIBMAGIC) $(SYSLIBS) $(SEM_INIT_LIB)
 DEPLIBS= $(DEPLIBSUPPORT) $(LIBEXT2FS) $(DEPLIBCOM_ERR) $(DEPLIBBLKID) \
 	 $(DEPLIBUUID) $(DEPLIBE2P)
 
 STATIC_LIBS= $(STATIC_LIBSUPPORT) $(STATIC_LIBEXT2FS) $(STATIC_LIBCOM_ERR) \
 	     $(STATIC_LIBBLKID) $(STATIC_LIBUUID) $(LIBINTL) $(STATIC_LIBE2P) \
-	     $(LIBMAGIC) $(SYSLIBS)
+	     $(LIBMAGIC) $(SYSLIBS) $(SEM_INIT_LIB)
 STATIC_DEPLIBS= $(DEPSTATIC_LIBSUPPORT) $(STATIC_LIBEXT2FS) \
 		$(DEPSTATIC_LIBCOM_ERR) $(DEPSTATIC_LIBBLKID) \
 		$(DEPSTATIC_LIBUUID) $(DEPSTATIC_LIBE2P)
 
 PROFILED_LIBS= $(PROFILED_LIBSUPPORT) $(PROFILED_LIBEXT2FS) \
 	       $(PROFILED_LIBCOM_ERR) $(PROFILED_LIBBLKID) $(PROFILED_LIBUUID) \
-	       $(PROFILED_LIBE2P) $(LIBINTL) $(LIBMAGIC) $(SYSLIBS)
+	       $(PROFILED_LIBE2P) $(LIBINTL) $(LIBMAGIC) $(SYSLIBS) \
+	       $(SEM_INIT_LIB)
 PROFILED_DEPLIBS= $(DEPPROFILED_LIBSUPPORT) $(PROFILED_LIBEXT2FS) \
 		  $(DEPPROFILED_LIBCOM_ERR) $(DEPPROFILED_LIBBLKID) \
 		  $(DEPPROFILED_LIBUUID) $(DEPPROFILED_LIBE2P)
@@ -115,7 +116,7 @@ all-static:: e2fsck.static
 
 e2fsck: $(OBJS)  $(DEPLIBS)
 	$(E) "	LD $@"
-	$(Q) $(LD) $(ALL_LDFLAGS) $(RDYNAMIC) -o e2fsck $(OBJS) $(LIBS) 
+	$(Q) $(LD) $(ALL_LDFLAGS) $(RDYNAMIC) -o e2fsck $(OBJS) $(LIBS)
 
 e2fsck.static: $(OBJS) $(STATIC_DEPLIBS)
 	$(E) "	LD $@"
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#69 lore raw

RFC PATCH v3 50/61 e2fsck: move ext2fs_get_avg_group to rw_bitmaps.c

In reply to Saranya Muruganandam

From: Wang Shilong <[email protected]>

A bounch of ext2fs_get_avg_group() unused warning messages are annoying, move it to rw_bitmaps.c to make gcc happy.

Change-Id Ia7d372b7b5aadcbf5d94916f6f67363a2a9f0bfa

Reviewed-on: https://review.whamcloud.com/40060

---
 lib/ext2fs/ext2fs.h     | 29 +----------------------------
 lib/ext2fs/rw_bitmaps.c | 25 +++++++++++++++++++++++++
 2 files changed, 26 insertions(+), 28 deletions(-)
lib/ext2fs/ext2fs.h +1 -28
index 83f2af07..616c9412 100644
@@ -2122,34 +2122,7 @@ ext2fs_const_inode(const struct ext2_inode_large * large_inode)
 	return (const struct ext2_inode *) large_inode;
 }
 
-static dgrp_t ext2fs_get_avg_group(ext2_filsys fs)
-{
-#ifdef CONFIG_PFSCK
-	dgrp_t average_group;
-	unsigned flexbg_size;
-
-	if (fs->fs_num_threads <= 1)
-		return fs->group_desc_count;
-
-	average_group = fs->group_desc_count / fs->fs_num_threads;
-	if (average_group <= 1)
-		return 1;
-
-	if (ext2fs_has_feature_flex_bg(fs->super)) {
-		int times = 1;
-
-		flexbg_size = 1 << fs->super->s_log_groups_per_flex;
-		if (average_group % flexbg_size) {
-			times = average_group / flexbg_size;
-			average_group = times * flexbg_size;
-		}
-	}
-
-	return average_group;
-#else
-	return fs->group_desc_count;
-#endif
-}
+dgrp_t ext2fs_get_avg_group(ext2_filsys fs);
 
 #undef _INLINE_
 #endif
lib/ext2fs/rw_bitmaps.c +25 -0
index 960a6913..95de9b1c 100644
@@ -515,6 +515,31 @@ struct read_bitmaps_thread_info {
 	io_channel      rbt_io;
 };
 
+dgrp_t ext2fs_get_avg_group(ext2_filsys fs)
+{
+	dgrp_t average_group;
+	unsigned flexbg_size;
+
+	if (fs->fs_num_threads <= 1)
+		return fs->group_desc_count;
+
+	average_group = fs->group_desc_count / fs->fs_num_threads;
+	if (average_group <= 1)
+		return 1;
+
+	if (ext2fs_has_feature_flex_bg(fs->super)) {
+		int times = 1;
+
+		flexbg_size = 1 << fs->super->s_log_groups_per_flex;
+		if (average_group % flexbg_size) {
+			times = average_group / flexbg_size;
+			average_group = times * flexbg_size;
+		}
+	}
+
+	return average_group;
+}
+
 static void* read_bitmaps_thread(void *data)
 {
 	struct read_bitmaps_thread_info *rbt = data;
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#70 lore raw

Re: [RFC PATCH v3 50/61] e2fsck: move ext2fs_get_avg_group to rw_bitmaps.c

Quoted 4 lines · Saranya Muruganandam

From: Wang Shilong <[email protected]>

A bounch of ext2fs_get_avg_group() unused warning messages are annoying, move it to rw_bitmaps.c to make gcc happy.

This should be merged with the commit that initially defined ext2fs_get_avg_group() in the first place.

More generally, *all* of the "fix *" commits should be merged with the original commits. This will make it **so** much easier to review the commits.

- Ted

#71 lore raw

RFC PATCH v3 51/61 configure: enable pfsck by default

In reply to Saranya Muruganandam

From: Wang Shilong <[email protected]>

Since most of work has been done, compile e2fsprogs with pfsck enabled by default.

So it could testing widely now.

---
 configure    | 13 +++++++++++--
 configure.ac | 13 ++++++++++---
 2 files changed, 21 insertions(+), 5 deletions(-)
configure +11 -2
index 1bb7a325..511de4a1 100755
@@ -1589,7 +1589,7 @@ Optional Features:
   --disable-tdb           disable tdb support
   --disable-bmap-stats    disable collection of bitmap stats.
   --enable-bmap-stats-ops enable collection of additional bitmap stats
-  --enable-pfsck     enable parallel e2fsck
+  --disable-pfsck     disable parallel e2fsck
   --disable-nls           do not use Native Language Support
   --enable-threads={posix|solaris|pth|windows}
                           specify multithreading API
@@ -6169,8 +6169,17 @@ $as_echo "Enabling parallel e2fsck" >&6; }
 fi
 
 else
-  { $as_echo "$as_me:${as_lineno-$LINENO}: result: Disabling parallel e2fsck" >&5
+  if test -z "PTHREAD_LIB"
+then
+	{ $as_echo "$as_me:${as_lineno-$LINENO}: result: Disabling parallel e2fsck" >&5
 $as_echo "Disabling parallel e2fsck" >&6; }
+else
+
+$as_echo "#define CONFIG_PFSCK 1" >>confdefs.h
+
+	{ $as_echo "$as_me:${as_lineno-$LINENO}: result: Enabling parallel e2fsck by default" >&5
+$as_echo "Enabling parallel e2fsck by default" >&6; }
+fi
 
 fi
 
configure.ac +10 -3
index e73dbf50..2dacd6c8 100644
@@ -877,13 +877,13 @@ fi
 AC_MSG_RESULT([Disabling additional bitmap statistics by default])
 )
 dnl
-dnl handle --enable-pfsck
+dnl handle --disable-pfsck
 dnl
 PTHREAD_LIB=''
 AC_CHECK_LIB(pthread,pthread_join,PTHREAD_LIB=-pthread)
 AC_SUBST(PTHREAD_LIB)
 AC_ARG_ENABLE([pfsck],
-[  --enable-pfsck     enable parallel e2fsck],
+[  --disable-pfsck     disable parallel e2fsck],
 if test "$enableval" = "no" || test -z "PTHREAD_LIB"
 then
 	AC_MSG_RESULT([Disabling parallel e2fsck])
@@ -893,7 +893,14 @@ else
 	AC_MSG_RESULT([Enabling parallel e2fsck])
 fi
 ,
-AC_MSG_RESULT([Disabling parallel e2fsck])
+if test -z "PTHREAD_LIB"
+then
+	AC_MSG_RESULT([Disabling parallel e2fsck])
+else
+	AC_DEFINE(CONFIG_PFSCK, 1,
+		[Define to 1 if parallel e2fsck is enabled])
+	AC_MSG_RESULT([Enabling parallel e2fsck by default])
+fi
 )
 dnl
 dnl
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#72 lore raw

RFC PATCH v3 53/61 e2fsck: fix f_multithread_ok test

In reply to Saranya Muruganandam

From: Andreas Dilger <[email protected]>

Don't use $OUT for both the input amd output of a pipeline, as the output file is truncated befor the input is read.

Fix the handling in the failure case to generate the *.failed file, and print the actual $test_name instead of "test_name".

---
 tests/f_multithread_ok/script | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
tests/f_multithread_ok/script +4 -4
index 4010881b..c62c93ce 100644
@@ -5,15 +5,15 @@ SKIP_CLEANUP="true"
 
 . $cmd_dir/run_e2fsck
 
-cat $OUT1 | grep -v Thread > $OUT1
-rm -f $test_name.ok $test_name.failed
-cmp -s $OUT1 $EXP1
+grep -v Thread $OUT1 > $OUT1.tmp
+cmp -s $OUT1.tmp $EXP1
 status1=$?
 if [ "$status1" -eq 0 ]; then
 	echo "$test_name: $test_description: ok"
 	touch $test_name.ok
 else
-	echo "test_name: $test_description: failed"
+	echo "$test_name: $test_description: failed"
+	cmp $OUT1.tmp $EXP1 > $test_name.failed
 fi
 
 unset IMAGE FSCK_OPT SECOND_FSCK_OPT OUT1 OUT2 EXP1 EXP2 
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#73 lore raw

RFC PATCH v3 54/61 e2fsck: fix race in ext2fs_read_bitmaps()

In reply to Saranya Muruganandam

From: Wang Shilong <[email protected]>

During corruption testing hiting following segfault:

Multiple threads triggered to read bitmaps Signal (11) SIGSEGV si_code=SEGV_MAPERR fault addr=0x200 ./e2fsck[0x4382ae] /lib64/libpthread.so.0(+0x14b20)[0x7f5854d2fb20] ./e2fsck(ext2fs_rb_insert_color+0xc)[0x46ac0c] ./e2fsck[0x467bb4]
./e2fsck[0x467e6d]
./e2fsck[0x45ba95]
./e2fsck[0x45c124] /lib64/libpthread.so.0(+0x94e2)[0x7f5854d244e2] /lib64/libc.so.6(clone+0x43)[0x7f5854beb6c3]

Problem is @block_map might be set NULL if one of thread exit, move such kind of cleanup operation to main thread after all threads exit.

Another potential problem is e2fsck_read_bitmap() could be called during pass1, this need be serialized, serialize it in the pass1.

---
 e2fsck/pass1.c          |  2 +-
 lib/ext2fs/rw_bitmaps.c | 25 ++++++++-----------------
 2 files changed, 9 insertions(+), 18 deletions(-)
e2fsck/pass1.c +1 -1
index e98cda9f..3899d710 100644
@@ -4183,9 +4183,9 @@ report_problem:
 					}
 					continue;
 				}
+				e2fsck_pass1_fix_lock(ctx);
 				e2fsck_read_bitmaps(ctx);
 				pb->inode_modified = 1;
-				e2fsck_pass1_fix_lock(ctx);
 				pctx->errcode =
 					ext2fs_extent_delete(ehandle, 0);
 				e2fsck_pass1_fix_unlock(ctx);
lib/ext2fs/rw_bitmaps.c +8 -17
index 95de9b1c..eb791202 100644
@@ -445,14 +445,6 @@ success_cleanup:
 	return 0;
 
 cleanup:
-	if (do_block) {
-		ext2fs_free_block_bitmap(fs->block_map);
-		fs->block_map = 0;
-	}
-	if (do_inode) {
-		ext2fs_free_inode_bitmap(fs->inode_map);
-		fs->inode_map = 0;
-	}
 	if (inode_bitmap)
 		ext2fs_free_mem(&inode_bitmap);
 	if (block_bitmap)
@@ -463,9 +455,12 @@ cleanup:
 
 }
 
-static errcode_t read_bitmaps_range_end(ext2_filsys fs, int do_inode, int do_block)
+static errcode_t read_bitmaps_range_end(ext2_filsys fs, int do_inode, int do_block,
+					errcode_t retval)
 {
-	errcode_t retval = 0;
+
+	if (retval)
+		goto cleanup;
 
 	/* Mark group blocks for any BLOCK_UNINIT groups */
 	if (do_block) {
@@ -474,7 +469,7 @@ static errcode_t read_bitmaps_range_end(ext2_filsys fs, int do_inode, int do_blo
 			goto cleanup;
 	}
 
-	return retval;
+	return 0;
 cleanup:
 	if (do_block) {
 		ext2fs_free_block_bitmap(fs->block_map);
@@ -497,10 +492,8 @@ static errcode_t read_bitmaps_range(ext2_filsys fs, int do_inode, int do_block,
 		return retval;
 
 	retval = read_bitmaps_range_start(fs, do_inode, do_block, start, end, NULL, NULL);
-	if (retval)
-		return retval;
 
-	return read_bitmaps_range_end(fs, do_inode, do_block);
+	return read_bitmaps_range_end(fs, do_inode, do_block, retval);
 }
 
 #ifdef CONFIG_PFSCK
@@ -636,9 +629,7 @@ out:
 	free(thread_infos);
 	free(thread_ids);
 
-	if (!retval)
-		retval = read_bitmaps_range_end(fs, do_inode, do_block);
-
+	retval = read_bitmaps_range_end(fs, do_inode, do_block, retval);
 	if (!retval) {
 		if (do_inode)
 			fs->flags &= ~EXT2_FLAG_IBITMAP_TAIL_PROBLEM;
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#74 lore raw

RFC PATCH v3 55/61 e2fsck: fix readahead for pass1 without pfsck

In reply to Saranya Muruganandam

From: Wang Shilong <[email protected]>

If admin try fsck without -m option, codes try old behavior, thread information won't be inited either. @et_group_end is 0 thus readahead for pass1 will be totally disabled.

With the patch applied, we could get same performance number without pfsck as before.

---
 e2fsck/pass1.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
e2fsck/pass1.c +2 -1
index 3899d710..70826866 100644
@@ -1100,7 +1100,8 @@ static void pass1_readahead(e2fsck_t ctx, dgrp_t *group, ext2_ino_t *next_ino)
 	errcode_t err = EXT2_ET_INVALID_ARGUMENT;
 
 #ifdef CONFIG_PFSCK
-	grp_end = ctx->thread_info.et_group_end;
+	if (ctx->fs->fs_num_threads > 1)
+		grp_end = ctx->thread_info.et_group_end;
 #endif
 	if (ctx->readahead_kb == 0)
 		goto out;
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#75 lore raw

RFC PATCH v3 56/61 e2fsck: fix memory leaks with pfsck enabled

In reply to Saranya Muruganandam

From: Wang Shilong <[email protected]>

valgrind detected few memory leaks:

1) quota context is not released after merging.
2) three block bufs are not freed in read_bitmaps_range_start()
3) @refcount_orig should be released

---
 e2fsck/e2fsck.c         |  4 ++++
 e2fsck/pass1.c          |  1 +
 lib/ext2fs/rw_bitmaps.c | 14 ++++----------
 3 files changed, 9 insertions(+), 10 deletions(-)
e2fsck/e2fsck.c +4 -0
index a03550c0..e406f6dd 100644
@@ -102,6 +102,10 @@ errcode_t e2fsck_reset_context(e2fsck_t ctx)
 		ea_refcount_free(ctx->refcount_extra);
 		ctx->refcount_extra = 0;
 	}
+	if (ctx->refcount_orig) {
+		ea_refcount_free(ctx->refcount_orig);
+		ctx->refcount_orig = 0;
+	}
 	if (ctx->ea_block_quota_blocks) {
 		ea_refcount_free(ctx->ea_block_quota_blocks);
 		ctx->ea_block_quota_blocks = 0;
e2fsck/pass1.c +1 -0
index 70826866..7768119b 100644
@@ -3113,6 +3113,7 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
 		fclose(thread_ctx->problem_logf);
 	}
 
+	quota_release_context(&thread_ctx->qctx);
 	/*
 	 * @block_metadata_map and @block_dup_map are
 	 * shared, so we don't free them.
lib/ext2fs/rw_bitmaps.c +4 -10
index eb791202..5fde2632 100644
@@ -269,7 +269,7 @@ static errcode_t read_bitmaps_range_start(ext2_filsys fs, int do_inode, int do_b
 	dgrp_t i;
 	char *block_bitmap = 0, *inode_bitmap = 0;
 	char *buf;
-	errcode_t retval;
+	errcode_t retval = 0;
 	int block_nbytes = EXT2_CLUSTERS_PER_GROUP(fs->super) / 8;
 	int inode_nbytes = EXT2_INODES_PER_GROUP(fs->super) / 8;
 	int tail_flags = 0;
@@ -432,18 +432,12 @@ static errcode_t read_bitmaps_range_start(ext2_filsys fs, int do_inode, int do_b
 
 success_cleanup:
 	if (start == 0 && end == fs->group_desc_count - 1) {
-		if (inode_bitmap) {
-			ext2fs_free_mem(&inode_bitmap);
+		if (inode_bitmap)
 			fs->flags &= ~EXT2_FLAG_IBITMAP_TAIL_PROBLEM;
-		}
-		if (block_bitmap) {
-			ext2fs_free_mem(&block_bitmap);
+		if (block_bitmap)
 			fs->flags &= ~EXT2_FLAG_BBITMAP_TAIL_PROBLEM;
-		}
 	}
 	fs->flags |= tail_flags;
-	return 0;
-
 cleanup:
 	if (inode_bitmap)
 		ext2fs_free_mem(&inode_bitmap);
@@ -451,8 +445,8 @@ cleanup:
 		ext2fs_free_mem(&block_bitmap);
 	if (buf)
 		ext2fs_free_mem(&buf);
-	return retval;
 
+	return retval;
 }
 
 static errcode_t read_bitmaps_range_end(ext2_filsys fs, int do_inode, int do_block,
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#76 lore raw

RFC PATCH v3 57/61 ext2fs: fix to set tail flags with pfsck enabled

In reply to Saranya Muruganandam

From: Wang Shilong <[email protected]>

If any of block/inode bitmap block checksum error happen, tail flag should be set properly.

However, we firstly set tail flags in each thread, after threads finish we clear those tail problem wrongly.

This will make fsck miss bitmap checksum erors later, patch try to fix the problem by move all this kind of logic in read_bitmaps_range_end()

---
 lib/ext2fs/rw_bitmaps.c | 46 +++++++++++++++++------------------------
 1 file changed, 19 insertions(+), 27 deletions(-)
lib/ext2fs/rw_bitmaps.c +19 -27
index 5fde2632..b66ba773 100644
@@ -264,7 +264,7 @@ cleanup:
 
 static errcode_t read_bitmaps_range_start(ext2_filsys fs, int do_inode, int do_block,
 					  dgrp_t start, dgrp_t end, pthread_mutex_t *mutex,
-					  io_channel io)
+					  io_channel io, int *tail_flags)
 {
 	dgrp_t i;
 	char *block_bitmap = 0, *inode_bitmap = 0;
@@ -272,7 +272,6 @@ static errcode_t read_bitmaps_range_start(ext2_filsys fs, int do_inode, int do_b
 	errcode_t retval = 0;
 	int block_nbytes = EXT2_CLUSTERS_PER_GROUP(fs->super) / 8;
 	int inode_nbytes = EXT2_INODES_PER_GROUP(fs->super) / 8;
-	int tail_flags = 0;
 	int csum_flag;
 	unsigned int	cnt;
 	blk64_t	blk;
@@ -343,7 +342,7 @@ static errcode_t read_bitmaps_range_start(ext2_filsys fs, int do_inode, int do_b
 			blk_itr += cnt;
 			blk_cnt -= cnt;
 		}
-		goto success_cleanup;
+		goto cleanup;
 	}
 
 	blk_itr += ((blk64_t)start * (block_nbytes << 3));
@@ -374,7 +373,7 @@ static errcode_t read_bitmaps_range_start(ext2_filsys fs, int do_inode, int do_b
 				}
 				if (!bitmap_tail_verify((unsigned char *) block_bitmap,
 							block_nbytes, fs->blocksize - 1))
-					tail_flags |= EXT2_FLAG_BBITMAP_TAIL_PROBLEM;
+					*tail_flags |= EXT2_FLAG_BBITMAP_TAIL_PROBLEM;
 			} else
 				memset(block_bitmap, 0, block_nbytes);
 			cnt = block_nbytes << 3;
@@ -414,7 +413,7 @@ static errcode_t read_bitmaps_range_start(ext2_filsys fs, int do_inode, int do_b
 				}
 				if (!bitmap_tail_verify((unsigned char *) inode_bitmap,
 							inode_nbytes, fs->blocksize - 1))
-					tail_flags |= EXT2_FLAG_IBITMAP_TAIL_PROBLEM;
+					*tail_flags |= EXT2_FLAG_IBITMAP_TAIL_PROBLEM;
 			} else
 				memset(inode_bitmap, 0, inode_nbytes);
 			cnt = inode_nbytes << 3;
@@ -430,14 +429,6 @@ static errcode_t read_bitmaps_range_start(ext2_filsys fs, int do_inode, int do_b
 		}
 	}
 
-success_cleanup:
-	if (start == 0 && end == fs->group_desc_count - 1) {
-		if (inode_bitmap)
-			fs->flags &= ~EXT2_FLAG_IBITMAP_TAIL_PROBLEM;
-		if (block_bitmap)
-			fs->flags &= ~EXT2_FLAG_BBITMAP_TAIL_PROBLEM;
-	}
-	fs->flags |= tail_flags;
 cleanup:
 	if (inode_bitmap)
 		ext2fs_free_mem(&inode_bitmap);
@@ -450,7 +441,7 @@ cleanup:
 }
 
 static errcode_t read_bitmaps_range_end(ext2_filsys fs, int do_inode, int do_block,
-					errcode_t retval)
+					errcode_t retval, int tail_flags)
 {
 
 	if (retval)
@@ -461,7 +452,11 @@ static errcode_t read_bitmaps_range_end(ext2_filsys fs, int do_inode, int do_blo
 		retval = mark_uninit_bg_group_blocks(fs);
 		if (retval)
 			goto cleanup;
+		fs->flags &= ~EXT2_FLAG_BBITMAP_TAIL_PROBLEM;
 	}
+	if (do_inode)
+		fs->flags &= ~EXT2_FLAG_IBITMAP_TAIL_PROBLEM;
+	fs->flags |= tail_flags;
 
 	return 0;
 cleanup:
@@ -480,14 +475,16 @@ static errcode_t read_bitmaps_range(ext2_filsys fs, int do_inode, int do_block,
 				    dgrp_t start, dgrp_t end)
 {
 	errcode_t retval;
+	int tail_flags = 0;
 
 	retval = read_bitmaps_range_prepare(fs, do_inode, do_block);
 	if (retval)
 		return retval;
 
-	retval = read_bitmaps_range_start(fs, do_inode, do_block, start, end, NULL, NULL);
+	retval = read_bitmaps_range_start(fs, do_inode, do_block, start, end, NULL,
+					  NULL, &tail_flags);
 
-	return read_bitmaps_range_end(fs, do_inode, do_block, retval);
+	return read_bitmaps_range_end(fs, do_inode, do_block, retval, tail_flags);
 }
 
 #ifdef CONFIG_PFSCK
@@ -499,6 +496,7 @@ struct read_bitmaps_thread_info {
 	dgrp_t		rbt_grp_end;
 	errcode_t	rbt_retval;
 	pthread_mutex_t *rbt_mutex;
+	int		rbt_tail_flags;
 	io_channel      rbt_io;
 };
 
@@ -534,7 +532,7 @@ static void* read_bitmaps_thread(void *data)
 	rbt->rbt_retval = read_bitmaps_range_start(rbt->rbt_fs,
 				rbt->rbt_do_inode, rbt->rbt_do_block,
 				rbt->rbt_grp_start, rbt->rbt_grp_end,
-				rbt->rbt_mutex, rbt->rbt_io);
+				rbt->rbt_mutex, rbt->rbt_io, &rbt->rbt_tail_flags);
 	return NULL;
 }
 #endif
@@ -550,7 +548,7 @@ static errcode_t read_bitmaps(ext2_filsys fs, int do_inode, int do_block)
 	errcode_t retval;
 	errcode_t rc;
 	dgrp_t average_group;
-	int i;
+	int i, tail_flags = 0;
 	io_manager manager = unix_io_manager;
 #else
 	int num_threads = 1;
@@ -584,6 +582,7 @@ static errcode_t read_bitmaps(ext2_filsys fs, int do_inode, int do_block)
 		thread_infos[i].rbt_do_inode = do_inode;
 		thread_infos[i].rbt_do_block = do_block;
 		thread_infos[i].rbt_mutex = &rbt_mutex;
+		thread_infos[i].rbt_tail_flags = 0;
 		if (i == 0)
 			thread_infos[i].rbt_grp_start = 0;
 		else
@@ -614,6 +613,7 @@ static errcode_t read_bitmaps(ext2_filsys fs, int do_inode, int do_block)
 		rc = thread_infos[i].rbt_retval;
 		if (rc && !retval)
 			retval = rc;
+		tail_flags |= thread_infos[i].rbt_tail_flags;
 		io_channel_close(thread_infos[i].rbt_io);
 	}
 out:
@@ -623,15 +623,7 @@ out:
 	free(thread_infos);
 	free(thread_ids);
 
-	retval = read_bitmaps_range_end(fs, do_inode, do_block, retval);
-	if (!retval) {
-		if (do_inode)
-			fs->flags &= ~EXT2_FLAG_IBITMAP_TAIL_PROBLEM;
-		if (do_block)
-			fs->flags &= ~EXT2_FLAG_BBITMAP_TAIL_PROBLEM;
-	}
-
-	return retval;
+	return read_bitmaps_range_end(fs, do_inode, do_block, retval, tail_flags);
 #endif
 }
 
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#77 lore raw

RFC PATCH v3 58/61 e2fsck: misc cleanups for pfsck

In reply to Saranya Muruganandam

From: Andreas Dilger <[email protected]>

Add -m option description to e2fsck.8 man page.

Rename e2fsck_struct fs_num_threads to pfs_num_threads to avoid confusion with the ext2_filsys fs_num_threads field, and move thread_info to be together with the other CONFIG_PFSCK fields.

Move ext2_filsys fs_num_threads to fit into the __u16 "pad" field to avoid consuming one of the few remaining __u32 reserved fields.

Fix a few print format warnings.

---
 e2fsck/e2fsck.8.in  |  8 +++++++-
 e2fsck/e2fsck.h     | 13 +++++--------
 e2fsck/pass1.c      | 29 +++++++++++++++--------------
 e2fsck/unix.c       |  4 ++--
 lib/ext2fs/ext2fs.h |  5 ++---
 5 files changed, 31 insertions(+), 28 deletions(-)
e2fsck/e2fsck.8.in +7 -1
index 4e3890b2..404d07fe 100644
@@ -8,7 +8,7 @@ e2fsck \- check a Linux ext2/ext3/ext4 file system
 .SH SYNOPSIS
 .B e2fsck
 [
-.B \-pacnyrdfkvtDFV
+.B \-pacnyrdfkmvtDFV
 ]
 [
 .B \-b
@@ -329,6 +329,12 @@ Set the bad blocks list to be the list of blocks specified by
 option, except the bad blocks list is cleared before the blocks listed
 in the file are added to the bad blocks list.)
 .TP
+.B \-m " threads"
+Run e2fsck with up to the specified number of
+.IR threads .
+The actual number of threads may be lower, if the filesystem does not
+have enough block groups to effectively parallelize the workload.
+.TP
 .B \-n
 Open the filesystem read-only, and assume an answer of `no' to all
 questions.  Allows
e2fsck/e2fsck.h +5 -8
index 1469c3e1..362e128c 100644
@@ -243,8 +243,8 @@ struct e2fsck_thread {
 	dgrp_t		et_group_next;
 	/* Scanned inode number */
 	ext2_ino_t	et_inode_number;
-	char		et_log_buf[2048];
 	char		et_log_length;
+	char		et_log_buf[2048];
 };
 #endif
 
@@ -333,11 +333,6 @@ struct e2fsck_struct {
 	ext2_ino_t		stashed_ino;
 	struct ext2_inode	*stashed_inode;
 
-	/* if @global_ctx is null, this field is unused */
-#ifdef CONFIG_PFSCK
-	struct e2fsck_thread	 thread_info;
-#endif
-
 	/*
 	 * Location of the lost and found directory
 	 */
@@ -450,7 +445,9 @@ struct e2fsck_struct {
 	/* Undo file */
 	char *undo_file;
 #ifdef CONFIG_PFSCK
-	__u32			 fs_num_threads;
+	/* if @global_ctx is null, this field is unused */
+	struct e2fsck_thread	 thread_info;
+	__u32			 pfs_num_threads;
 	__u32			 mmp_update_thread;
 	int			 fs_need_locking;
 	/* serialize fix operation for multiple threads */
@@ -689,7 +686,7 @@ int check_backup_super_block(e2fsck_t ctx);
 void check_resize_inode(e2fsck_t ctx);
 
 /* util.c */
-#define E2FSCK_MAX_THREADS	(65536)
+#define E2FSCK_MAX_THREADS	(65535)
 extern void *e2fsck_allocate_memory(e2fsck_t ctx, unsigned long size,
 				    const char *description);
 extern int ask(e2fsck_t ctx, const char * string, int def);
e2fsck/pass1.c +15 -14
index 7768119b..8d4e2675 100644
@@ -1260,7 +1260,7 @@ static void e2fsck_pass1_set_thread_num(e2fsck_t ctx)
 {
 	unsigned flexbg_size = 1;
 	ext2_filsys fs = ctx->fs;
-	int num_threads = ctx->fs_num_threads;
+	int num_threads = ctx->pfs_num_threads;
 	int max_threads;
 
 	if (num_threads < 1) {
@@ -1280,6 +1280,8 @@ static void e2fsck_pass1_set_thread_num(e2fsck_t ctx)
 	max_threads = fs->group_desc_count / flexbg_size;
 	if (max_threads == 0)
 		max_threads = 1;
+	if (max_threads > E2FSCK_MAX_THREADS)
+		max_threads = E2FSCK_MAX_THREADS;
 
 	if (num_threads > max_threads) {
 		fprintf(stderr, "Use max possible thread num: %d instead\n",
@@ -1287,7 +1289,7 @@ static void e2fsck_pass1_set_thread_num(e2fsck_t ctx)
 		num_threads = max_threads;
 	}
 out:
-	ctx->fs_num_threads = num_threads;
+	ctx->pfs_num_threads = num_threads;
 	ctx->fs->fs_num_threads = num_threads;
 }
 #endif
@@ -1315,7 +1317,7 @@ static errcode_t e2fsck_pass1_prepare(e2fsck_t ctx)
 
 #ifdef CONFIG_PFSCK
 	/* don't use more than 1/10 of memory for threads checking */
-	readahead_kb = get_memory_size() / (10 * ctx->fs_num_threads);
+	readahead_kb = get_memory_size() / (10 * ctx->pfs_num_threads);
 	/* maybe better disable RA if this is too small? */
 	if (ctx->readahead_kb > readahead_kb)
 		ctx->readahead_kb = readahead_kb;
@@ -1373,7 +1375,7 @@ static errcode_t e2fsck_pass1_prepare(e2fsck_t ctx)
 #ifdef	CONFIG_PFSCK
 	pthread_rwlock_init(&ctx->fs_fix_rwlock, NULL);
 	pthread_rwlock_init(&ctx->fs_block_map_rwlock, NULL);
-	if (ctx->fs_num_threads > 1)
+	if (ctx->pfs_num_threads > 1)
 		ctx->fs_need_locking = 1;
 #endif
 
@@ -1633,7 +1635,7 @@ void e2fsck_pass1_run(e2fsck_t ctx)
 	if (ctx->global_ctx) {
 		if (ctx->options & E2F_OPT_DEBUG &&
 		    ctx->options & E2F_OPT_MULTITHREAD)
-			fprintf(stderr, "thread %d jumping to group %d\n",
+			fprintf(stderr, "thread %d jumping to group %u\n",
 					ctx->thread_info.et_thread_index,
 					ctx->thread_info.et_group_start);
 		pctx.errcode = ext2fs_inode_scan_goto_blockgroup(scan,
@@ -3129,11 +3131,11 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx)
 static int e2fsck_pass1_threads_join(struct e2fsck_thread_info *infos,
 				     e2fsck_t global_ctx)
 {
-	errcode_t			 rc;
-	errcode_t			 ret = 0;
-	int				 i;
-	struct e2fsck_thread_info	*pinfo;
-	int				 num_threads = global_ctx->fs_num_threads;
+	errcode_t rc;
+	errcode_t ret = 0;
+	struct e2fsck_thread_info *pinfo;
+	int num_threads = global_ctx->pfs_num_threads;
+	int i;
 
 	/* merge invalid bitmaps will recalculate it */
 	global_ctx->invalid_bitmaps = 0;
@@ -3199,7 +3201,7 @@ static void *e2fsck_pass1_thread(void *arg)
 out:
 	if (thread_ctx->options & E2F_OPT_MULTITHREAD)
 		log_out(thread_ctx,
-			_("Scanned group range [%lu, %lu), inodes %lu\n"),
+			_("Scanned group range [%u, %u), inodes %u\n"),
 			thread_ctx->thread_info.et_group_start,
 			thread_ctx->thread_info.et_group_end,
 			thread_ctx->thread_info.et_inode_number);
@@ -3225,7 +3227,7 @@ static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo,
 	int				 i;
 	e2fsck_t			 thread_ctx;
 	dgrp_t				 average_group;
-	int				 num_threads = global_ctx->fs_num_threads;
+	int num_threads = global_ctx->pfs_num_threads;
 #ifdef DEBUG_THREADS
 	struct e2fsck_thread_debug	 thread_debug =
 		{PTHREAD_MUTEX_INITIALIZER, PTHREAD_COND_INITIALIZER, 0};
@@ -3329,8 +3331,7 @@ void e2fsck_pass1(e2fsck_t ctx)
 	if (retval)
 		return;
 #ifdef CONFIG_PFSCK
-	if (ctx->fs_num_threads > 1 ||
-	    ctx->options & E2F_OPT_MULTITHREAD) {
+	if (ctx->pfs_num_threads > 1 || ctx->options & E2F_OPT_MULTITHREAD) {
 		need_single = 0;
 		e2fsck_pass1_multithread(ctx);
 	}
e2fsck/unix.c +2 -2
index cd31bcd5..bebc19ed 100644
@@ -908,12 +908,12 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx)
 					_("Invalid multiple thread num.\n"));
 			if (thread_num > E2FSCK_MAX_THREADS) {
 				fprintf(stderr,
-					_("threads %lu too large (max %lu)\n"),
+					_("threads %lu too large (max %u)\n"),
 					thread_num, E2FSCK_MAX_THREADS);
 				fatal_error(ctx, 0);
 			}
 			ctx->options |= E2F_OPT_MULTITHREAD;
-			ctx->fs_num_threads = thread_num;
+			ctx->pfs_num_threads = thread_num;
 			break;
 #endif
 		case 'n':
lib/ext2fs/ext2fs.h +2 -3
index 616c9412..f74303f4 100644
@@ -254,12 +254,11 @@ struct struct_ext2_filsys {
 	time_t				now;
 	int				cluster_ratio_bits;
 	__u16				default_bitmap_type;
-	__u16				pad;
-	__u32				fs_num_threads;
+	__u16				fs_num_threads;
 	/*
 	 * Reserved for future expansion
 	 */
-	__u32				reserved[4];
+	__u32				reserved[5];
 
 	/*
 	 * Reserved for the use of the calling application.
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#78 lore raw

RFC PATCH v3 59/61 e2fsck: update mmp block race

In reply to Saranya Muruganandam

From: Wang Shilong <[email protected]>

Update mmp block is only expected in one thread, @mmp_update_thread is used to get/set active thread number, however this should be set globally shared with different threads rather than be private per thread.

---
 e2fsck/pass1.c | 20 +++++++++-----------
 1 file changed, 9 insertions(+), 11 deletions(-)
e2fsck/pass1.c +9 -11
index 8d4e2675..a51fe20f 100644
@@ -1501,6 +1501,7 @@ void e2fsck_pass1_run(e2fsck_t ctx)
 	struct ea_quota	ea_ibody_quota;
 	struct process_inode_block *inodes_to_process;
 	int		process_inode_count, check_mmp;
+	e2fsck_t	global_ctx = ctx->global_ctx ? ctx->global_ctx : ctx;
 
 	init_resource_track(&rtrack, ctx->fs->io);
 	clear_problem_context(&pctx);
@@ -1652,14 +1653,11 @@ void e2fsck_pass1_run(e2fsck_t ctx)
 		check_mmp = 0;
 		e2fsck_pass1_check_lock(ctx);
 #ifdef	CONFIG_PFSCK
-		if (!ctx->mmp_update_thread) {
+		if (!global_ctx->mmp_update_thread) {
 			e2fsck_pass1_block_map_w_lock(ctx);
-			if (!ctx->mmp_update_thread) {
-				if (ctx->global_ctx)
-					ctx->mmp_update_thread =
-						ctx->thread_info.et_thread_index + 1;
-				else
-					ctx->mmp_update_thread = 1;
+			if (!global_ctx->mmp_update_thread) {
+				global_ctx->mmp_update_thread =
+					ctx->thread_info.et_thread_index + 1;
 				check_mmp = 1;
 			}
 			e2fsck_pass1_block_map_w_unlock(ctx);
@@ -1667,8 +1665,8 @@ void e2fsck_pass1_run(e2fsck_t ctx)
 
 		/* only one active thread could update mmp block. */
 		e2fsck_pass1_block_map_r_lock(ctx);
-		if (!ctx->global_ctx || ctx->mmp_update_thread ==
-			(ctx->thread_info.et_thread_index + 1))
+		if (global_ctx->mmp_update_thread ==
+		    ctx->thread_info.et_thread_index + 1)
 			check_mmp = 1;
 		e2fsck_pass1_block_map_r_unlock(ctx);
 #else
@@ -2396,8 +2394,8 @@ endit:
 #ifdef	CONFIG_PFSCK
 	/* reset update_thread after this thread exit */
 	e2fsck_pass1_block_map_w_lock(ctx);
-	if (ctx->mmp_update_thread)
-		ctx->mmp_update_thread = 0;
+	if (check_mmp)
+		global_ctx->mmp_update_thread = 0;
 	e2fsck_pass1_block_map_w_unlock(ctx);
 #endif
 }
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#79 lore raw

RFC PATCH v3 60/61 e2fsck: propagate number of threads

In reply to Saranya Muruganandam

Sometimes, such as in orphan_inode case, e2fsck_pass1 is called after reading the block bitmaps. This results in reading the block bitmap sequentially and multithreading only gets kicked in later. Fix the thread count earlier while setting up the file system.

---
 e2fsck/unix.c | 3 +++
 1 file changed, 3 insertions(+)
e2fsck/unix.c +3 -0
index bebc19ed..a2c6a178 100644
@@ -1729,6 +1729,9 @@ failure:
 
 	ctx->fs = fs;
 	fs->now = ctx->now;
+#ifdef CONFIG_PFSCK
+	fs->fs_num_threads = ctx->pfs_num_threads;
+#endif
 	sb = fs->super;
 
 	if (sb->s_rev_level > E2FSCK_CURRENT_REV) {
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#80 lore raw

Re: [RFC PATCH v3 60/61] e2fsck: propagate number of threads

Quoted 5 lines · Saranya Muruganandam

Sometimes, such as in orphan_inode case, e2fsck_pass1 is called after reading the block bitmaps. This results in reading the block bitmap sequentially and multithreading only gets kicked in later. Fix the thread count earlier while setting up the file system.

I wonder if read_bitmaps() should be using a different way of determining how many threads compared to how many threads are used by e2fsck.

Also, looking more closely, it's not at all clear we should fs_num_threads should be in the fs structure at all. It's actually used in three places. (a) to influence the default size of a dblist, if the size is not provided, (b) to influence the default size of an icount structure, if the size is not provided, (c) to figure out how many threads are needed in rw_bitmaps.c

(a) and (b) can be dealt with in e2fsck by having e2fsck pass in an explicit size. (In fact, for the dblist, we can do a better job by summing the number of directories in the block group range for each thread, instead of taking the total number of threads in the group, and dividing by "the averge number of block groups").

As far as (c) is concerned, perhaps it would be better to make it be explicit, via a new interface:

errcode_t ext2fs_rw_bitmaps(ext2_filsys fs, int flags, int num_threads);

Where flags is an OR of the following flags:

#define EXT2FS_BITMAPS_WRITE 	      0x0001
#define EXT2FS_BITMAPS_BLOCK 	      0x0002
#define EXT2FS_BITMAPS_INODE 	      0x0004

... and where num_threads is a hint that can be ignored if the pthreads interface is not available (via a HAVE_PTHREADS autoconf test).

If we do that, then there's no need to have fs->fs_num_threads at all, and that can purely be something which is local matter by the application (e.g., e2fsck, and maybe later other programs like debugfs's ncheck/icheck commands).

See? It's really, really important that we have well-defined interfaces in libext2fs, because that's how we keep tech debt in e2fsprogs down to a manageable level.

So it might be a good idea to separate out the rw_bitmaps patch() from the rest of the series, and create a much smaller patch series that has the autoconf test to see if pthreads are available, and if so, how it should be linked in (this would also subsume the "fix build for rpm" patch, except we would make sure that it works for all OS's, instead of serendiptously when Linux shared libraries were in use, and then adding a hack when it was noticed that the build configuration that rpm happened to use wasn't working).

By breaking down the patch series, we can make sure each change is done cleanly, and portably, and then we can use the individual pieces to to parallelize e2fsck. This will probably be better than trying to deal with this whole patch series as a monolithic thing.

What do folks think?

- Ted

#81 lore raw

RFC PATCH v3 61/61 e2fsck: Annotating fields in e2fsck_struct

In reply to Saranya Muruganandam

Adding information on fields in e2fsck_struct
on how they are used when running parallel fsck.

---
 e2fsck/e2fsck.h | 20 ++++++++++++++------
 1 file changed, 14 insertions(+), 6 deletions(-)
e2fsck/e2fsck.h +14 -6
index 362e128c..f15c383d 100644
@@ -251,7 +251,7 @@ struct e2fsck_thread {
 struct e2fsck_struct {
 	/* Global context to get the cancel flag */
 	e2fsck_t		global_ctx;
-	ext2_filsys fs;
+	ext2_filsys fs; /* [fs_fix_rwlock] */
 	const char *program_name;
 	char *filesystem_name;
 	char *device_name;
@@ -260,7 +260,9 @@ struct e2fsck_struct {
 	char	*log_fn;
 	FILE	*problem_logf;
 	char	*problem_log_fn;
-	int	flags;		/* E2fsck internal flags */
+	/* E2fsck internal flags.
+	 * shared by different threads for pass1 [fs_fix_rwlock] */
+	int	flags;
 	int	options;
 	unsigned blocksize;	/* blocksize */
 	blk64_t	use_superblock;	/* sb requested by user */
@@ -281,6 +283,7 @@ struct e2fsck_struct {
 	int (*progress)(e2fsck_t ctx, int pass, unsigned long cur,
 			unsigned long max);
 
+	/* The following inode bitmaps are separately used in thread_ctx Pass1*/
 	ext2fs_inode_bitmap inode_used_map; /* Inodes which are in use */
 	ext2fs_inode_bitmap inode_bad_map; /* Inodes which are bad somehow */
 	ext2fs_inode_bitmap inode_dir_map; /* Inodes which are directories */
@@ -288,12 +291,14 @@ struct e2fsck_struct {
 	ext2fs_inode_bitmap inode_imagic_map; /* AFS inodes */
 	ext2fs_inode_bitmap inode_reg_map; /* Inodes which are regular files*/
 
+	/* Following 3 protected by [fs_block_map_rwlock] */
 	ext2fs_block_bitmap block_found_map; /* Blocks which are in use */
 	ext2fs_block_bitmap block_dup_map; /* Blks referenced more than once */
 	ext2fs_block_bitmap block_ea_map; /* Blocks which are used by EA's */
 
 	/*
-	 * Inode count arrays
+	 * Inode count arrays.
+	 * Separately used in thread_ctx, pass1
 	 */
 	ext2_icount_t	inode_count;
 	ext2_icount_t inode_link_info;
@@ -315,7 +320,8 @@ struct e2fsck_struct {
 
 	/*
 	 * Array of flags indicating whether an inode bitmap, block
-	 * bitmap, or inode table is invalid
+	 * bitmap, or inode table is invalid.
+	 * Separately used in thread_ctx, pass1
 	 */
 	int *invalid_inode_bitmap_flag;
 	int *invalid_block_bitmap_flag;
@@ -328,7 +334,8 @@ struct e2fsck_struct {
 	char *block_buf;
 
 	/*
-	 * For pass1_check_directory and pass1_get_blocks
+	 * For pass1_check_directory and pass1_get_blocks.
+	 * Separately used in thread_ctx in pass1
 	 */
 	ext2_ino_t		stashed_ino;
 	struct ext2_inode	*stashed_inode;
@@ -387,6 +394,7 @@ struct e2fsck_struct {
 
 	/*
 	 * How we display the progress update (for unix)
+	 * shared by different threads for pass1 [fs_fix_rwlock]
 	 */
 	int progress_fd;
 	int progress_pos;
@@ -395,7 +403,7 @@ struct e2fsck_struct {
 	int interactive;	/* Are we connected directly to a tty? */
 	char start_meta[2], stop_meta[2];
 
-	/* File counts */
+	/* File counts. Separately used in thread_ctx, pass1 */
 	__u32 fs_directory_count;
 	__u32 fs_regular_count;
 	__u32 fs_blockdev_count;
Signature · 1 line
2.29.2.299.gdc1121823c-goog
#82 lore raw
In reply to Saranya Muruganandam
Quoted 28 lines · Saranya Muruganandam

Currently it has been popular that single disk could be more than TiB, etc 16Tib with only one single disk, with this trend, one single filesystem could be larger and larger and easily reach PiB with LUN system.

The journal filesystem like ext4 need be offline to do regular check and repair from time to time, however the problem is e2fsck still do this using single thread, this could be challenging at scale for two reasons:

1) even with readahead, IO speed still limits several tens MiB per second.
2) could not utilize CPU cores.

It could be challenging to try multh-threads for all phase of e2fsck, but as first step, we might try this for most time-consuming pass1, according to our benchmarking it cost of 80% time for whole e2fck phase.

Pass1 is trying to scanning all valid inode of filesystem and check it one by one, and the patchset idea is trying to split these to different threads and trying to do this at the same time, we try to merge these inodes and corresponding inode's extent information after threads finish.

To simplify complexity and make it less error-prone, the fix is still serialized, since most of time there will be only minor errors for filesystem, what's important for us is parallel reading and checking.

Here is a benchmarking on our Lustre filesystem with 1.2 PiB OSD ext4 based filesystem:

Quoted 7 lines

DDN SFA18KE StorageServer
DCR(DeClustering RAID) with 162 x HGST 10TB NL-SAS Tested Server
A Virtual Machine running on SFA18KE
8 x CPU cores (Xeon(R) Gold 6140)
150GB memory
CentoOS7.7 (Lustre patched kernel)

This introductory patch presumably came from the original patch series; hence "our Lustre file system". Just to make it clearer, it's probably better to make it clear who did which benchmarks. And Saranya, you might want to include your benchmark results since it will be easier for people to replicate.

Quoted 8 lines

I've tested the whole patch series using 'make test' of e2fsck itself, and i manually set default threads to 4 which still pass almost of test suite, failure cases are below:

f_h_badroot f_multithread f_multithread_logfile f_multithread_no f_multithread_ok

h_h_badroot failed because out of order checking output, and others are because of extra multiple threads log output.

And this "I" is Saranya, yes?

Quoted 15 lines

Andreas Dilger (2):
e2fsck: fix f_multithread_ok test
e2fsck: misc cleanups for pfsck

Li Xi (18):
e2fsck: add -m option for multithread
e2fsck: copy context when using multi-thread fsck
e2fsck: copy fs when using multi-thread fsck
e2fsck: add assert when copying context
e2fsck: copy bitmaps when copying context
e2fsck: open io-channel when copying fs
e2fsck: create logs for mult-threads
e2fsck: optionally configure one pfsck thread
e2fsck: add start/end group for thread
e2fsck: split groups to different threads

Quoted 8 lines

e2fsck: print thread log properly
e2fsck: do not change global variables
e2fsck: optimize the inserting of dir_info_db
e2fsck: merge dir_info after thread finishes
e2fsck: merge icounts after thread finishes
e2fsck: merge dblist after thread finishes
e2fsck: add debug codes for multiple threads
e2fsck: merge fs flags when threads finish

The fact that all of these patches are prefixed with e2fsck: hides the fact that some of these changes include changes to libext2fs. It's probably better to separate out the changes to libext2fs so we can pay special attention to issues of presering the ABI.

I'll talk more about this in the individual patches.

- Ted

#83 lore raw
In reply to Saranya Muruganandam
Quoted 8 lines · Saranya Muruganandam

I've tested the whole patch series using 'make test' of e2fsck itself, and i manually set default threads to 4 which still pass almost of test suite, failure cases are below:

f_h_badroot f_multithread f_multithread_logfile f_multithread_no f_multithread_ok

h_h_badroot failed because out of order checking output, and others are because of extra multiple threads log output.

I just tried the full series, and I'm only seeing one test failure. Unfortunately, it's f_multithread, and it's double free crash:

...
Pass 5: Checking group summary information
Multiple threads triggered to read bitmaps
double free or corruption (!prev)
Signal (6) SIGABRT si_code=SI_TKILL ../e2fsck/e2fsck(+0x45fab)[0x556589911fab] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140)[0x7fe52ec34140] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x141)[0x7fe52ea96c41] /lib/x86_64-linux-gnu/libc.so.6(abort+0x123)[0x7fe52ea80537] /lib/x86_64-linux-gnu/libc.so.6(+0x7e6c8)[0x7fe52ead96c8] /lib/x86_64-linux-gnu/libc.so.6(+0x859ba)[0x7fe52eae09ba] /lib/x86_64-linux-gnu/libc.so.6(+0x86ffc)[0x7fe52eae1ffc] ../lib/libext2fs.so.2(ext2fs_free_mem+0x23)[0x7fe52ed1091a] ../lib/libext2fs.so.2(+0x47329)[0x7fe52ed1f329] ../lib/libext2fs.so.2(+0x475b4)[0x7fe52ed1f5b4] /lib/x86_64-linux-gnu/libpthread.so.0(+0x8ea7)[0x7fe52ec28ea7] /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7fe52eb58d4f] Exit status is 8

Here's the contents of f_multithread_ok.1.log after running "./test_script --valgrind f_multithread_ok" in the tests directory:

Pass 1: Checking inodes, blocks, and sizes
[Thread 0] Scan group range [0, 1)
[Thread 1] Scan group range [1, 2)
[Thread 2] Scan group range [2, 3)
[Thread 3] Scan group range [3, 4)
[Thread 2] Scanned group range [2, 3), inodes 8192
[Thread 1] Scanned group range [1, 2), inodes 8192
[Thread 3] Scanned group range [3, 4), inodes 8192
[Thread 0] Scanned group range [0, 1), inodes 8192
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
Multiple threads triggered to read bitmaps
==182288== Thread 2:
==182288== Conditional jump or move depends on uninitialised value(s)

==182288==    at 0x488E31B: read_bitmaps_range_start (rw_bitmaps.c:437)
==182288==    by 0x488E5B3: read_bitmaps_thread (rw_bitmaps.c:532)
==182288==    by 0x4965EA6: start_thread (pthread_create.c:477)
==182288==    by 0x4A7CD4E: clone (clone.S:95)
==182288== 
==182288== Conditional jump or move depends on uninitialised value(s)
==182288==    at 0x4839961: free (vg_replace_malloc.c:538)
==182288==    by 0x487F919: ext2fs_free_mem (ext2fs.h:1891)
==182288==    by 0x488E328: read_bitmaps_range_start (rw_bitmaps.c:438)
==182288==    by 0x488E5B3: read_bitmaps_thread (rw_bitmaps.c:532)
==182288==    by 0x4965EA6: start_thread (pthread_create.c:477)
==182288==    by 0x4A7CD4E: clone (clone.S:95)
==182288== 
==182288== Invalid free() / delete / delete[] / realloc()
==182288==    at 0x48399AB: free (vg_replace_malloc.c:538)
==182288==    by 0x487F919: ext2fs_free_mem (ext2fs.h:1891)
==182288==    by 0x488E328: read_bitmaps_range_start (rw_bitmaps.c:438)
==182288==    by 0x488E5B3: read_bitmaps_thread (rw_bitmaps.c:532)
==182288==    by 0x4965EA6: start_thread (pthread_create.c:477)
==182288==    by 0x4A7CD4E: clone (clone.S:95)
==182288==  Address 0x4b46100 is 0 bytes inside a block of size 3,144 free'd
==182288==    at 0x48399AB: free (vg_replace_malloc.c:538)
==182288==    by 0x487F919: ext2fs_free_mem (ext2fs.h:1891)
==182288==    by 0x488E328: read_bitmaps_range_start (rw_bitmaps.c:438)
==182288==    by 0x488E5B3: read_bitmaps_thread (rw_bitmaps.c:532)
==182288==    by 0x4965EA6: start_thread (pthread_create.c:477)
==182288==    by 0x4A7CD4E: clone (clone.S:95)
==182288==  Block was alloc'd at
==182288==    at 0x483877F: malloc (vg_replace_malloc.c:307)
==182288==    by 0x487F7C9: ext2fs_get_mem (ext2fs.h:1847)
==182288==    by 0x11EE47: e2fsck_allocate_context (e2fsck.c:27)
==182288==    by 0x11B228: PRS (unix.c:829)
==182288==    by 0x11CD55: main (unix.c:1465)

...

- Ted