Skip to contentSkip to conversation

e2fsprogs: Support for orphan file feature

PATCH Applied Jan Kara 2 participants 15 messages Jun 16, 2021 – Aug 4, 2021

v3 of 6 Applied Latest is v6 →

lore
Share & tools
lore

Jan Kara

author
#0 lore raw

v3 cover e2fsprogs: Support for orphan file feature

Hello,

reviving this series after 6 years. This is the third version of support for
orphan file feature in e2fsprogs. The first patch removes feature bit for a feature which never happened and probably isn't living anywhere (if it still lives I can allocate a different bit...). Other two patches are fixes I've spotted while debugging this series. I'm not aware of any outstanding issue with the series, xfstests pass fine with it.

Changes since v2:
* rebased onto current master branch
* fixed various bugs I've spotted during testing
* added support for debugfs, dumpe2fs, e2image
* changed code to use dynamically allocated inode number instead of fixed one

Honza

author
#1 lore raw

PATCH v3 1/9 ext2fs: Drop HAS_SNAPSHOT feature

It has never been implemented and is dead for quite some time and unused AFAICT.

---
 lib/ext2fs/ext2_fs.h | 2 --
 1 file changed, 2 deletions(-)
lib/ext2fs/ext2_fs.h +0 -2
index e92a045205a9..6f1d5db4b482 100644
@@ -825,7 +825,6 @@ struct ext2_super_block {
 #define EXT4_FEATURE_RO_COMPAT_GDT_CSUM		0x0010
 #define EXT4_FEATURE_RO_COMPAT_DIR_NLINK	0x0020
 #define EXT4_FEATURE_RO_COMPAT_EXTRA_ISIZE	0x0040
-#define EXT4_FEATURE_RO_COMPAT_HAS_SNAPSHOT	0x0080
 #define EXT4_FEATURE_RO_COMPAT_QUOTA		0x0100
 #define EXT4_FEATURE_RO_COMPAT_BIGALLOC		0x0200
 /*
@@ -926,7 +925,6 @@ EXT4_FEATURE_RO_COMPAT_FUNCS(huge_file,		4, HUGE_FILE)
 EXT4_FEATURE_RO_COMPAT_FUNCS(gdt_csum,		4, GDT_CSUM)
 EXT4_FEATURE_RO_COMPAT_FUNCS(dir_nlink,		4, DIR_NLINK)
 EXT4_FEATURE_RO_COMPAT_FUNCS(extra_isize,	4, EXTRA_ISIZE)
-EXT4_FEATURE_RO_COMPAT_FUNCS(has_snapshot,	4, HAS_SNAPSHOT)
 EXT4_FEATURE_RO_COMPAT_FUNCS(quota,		4, QUOTA)
 EXT4_FEATURE_RO_COMPAT_FUNCS(bigalloc,		4, BIGALLOC)
 EXT4_FEATURE_RO_COMPAT_FUNCS(metadata_csum,	4, METADATA_CSUM)
Signature · 1 line
2.26.2
author
#2 lore raw

PATCH v3 2/9 quota: Do not account space used by project quota file to quota

In reply to Jan Kara

Project quota files have high inode numbers but are not accounted in quota usage. Do not track them when computing quota usage.

---
 lib/support/mkquota.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
lib/support/mkquota.c +5 -3
index fbc3833aee98..21a5c34d6921 100644
@@ -500,9 +500,11 @@ errcode_t quota_compute_usage(quota_ctx_t qctx)
 		}
 		if (ino == 0)
 			break;
-		if (inode->i_links_count &&
-		    (ino == EXT2_ROOT_INO ||
-		     ino >= EXT2_FIRST_INODE(fs->super))) {
+		if (!inode->i_links_count)
+			continue;
+		if (ino == EXT2_ROOT_INO ||
+		    (ino >= EXT2_FIRST_INODE(fs->super) &&
+		     ino != quota_type2inum(PRJQUOTA, fs->super))) {
 			space = ext2fs_get_stat_i_blocks(fs,
 						EXT2_INODE(inode)) << 9;
 			quota_data_add(qctx, inode, ino, space);
Signature · 1 line
2.26.2
#3 lore raw

Re: [PATCH 2/9] quota: Do not account space used by project quota file to quota

Quoted 4 lines · Jan Kara

Project quota files have high inode numbers but are not accounted in quota usage. Do not track them when computing quota usage.

Thanks, applied to the maint branch (since this is a bug fix unrelated to the orhpan inode feature).

It looks like this was an issue that wasn't picked up by our regression tests. Do you have an sample image that you used while you were developing this patch, that perhaps we should be adding to our regression test suite?

- Ted

author
#4 lore raw

Re: [PATCH 2/9] quota: Do not account space used by project quota file to quota

Quoted 6 lines · Theodore Ts'o
Quoted 4 lines · Jan Kara

Project quota files have high inode numbers but are not accounted in quota usage. Do not track them when computing quota usage.

Quoted 7 lines

Thanks, applied to the maint branch (since this is a bug fix unrelated to the orhpan inode feature).

It looks like this was an issue that wasn't picked up by our regression tests. Do you have an sample image that you used while you were developing this patch, that perhaps we should be adding to our regression test suite?

So I've uncovered this just by code inspection when adding orphan file feature. Checking this problem is not as simple as using a fs image (the problem manifests only if we add new quota type using tune2fs). I've created a test for it. I'll post it separately.

Honza

Signature · 2 lines
Jan Kara <[email protected]>
SUSE Labs, CR
author
#5 lore raw

PATCH v3 3/9 e2image: Dump quota files

In reply to Jan Kara

Dump quota files to resulting filesystem image. They are fs metadata.

---
 misc/e2image.c | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)
misc/e2image.c +6 -3
index 90a34bebc36d..ac00827e4628 100644
@@ -52,6 +52,7 @@ extern int optind;
 
 #include "support/nls-enable.h"
 #include "support/plausible.h"
+#include "support/quotaio.h"
 #include "../version.h"
 
 #define QCOW_OFLAG_COPIED     (1ULL << 63)
@@ -1364,9 +1365,11 @@ static void write_raw_image_file(ext2_filsys fs, int fd, int type, int flags,
 		pb.ino = ino;
 		pb.is_dir = LINUX_S_ISDIR(inode.i_mode);
 		if (LINUX_S_ISDIR(inode.i_mode) ||
-		    (LINUX_S_ISLNK(inode.i_mode) &&
-		     ext2fs_inode_has_valid_blocks2(fs, &inode)) ||
-		    ino == fs->super->s_journal_inum) {
+		    LINUX_S_ISLNK(inode.i_mode) ||
+		    ino == fs->super->s_journal_inum ||
+		    ino == quota_type2inum(USRQUOTA, fs->super) ||
+		    ino == quota_type2inum(GRPQUOTA, fs->super) ||
+		    ino == quota_type2inum(PRJQUOTA, fs->super)) {
 			retval = ext2fs_block_iterate3(fs, ino,
 					BLOCK_FLAG_READ_ONLY, block_buf,
 					process_dir_block, &pb);
Signature · 1 line
2.26.2
#6 lore raw

Re: [PATCH 3/9] e2image: Dump quota files

Quoted 3 lines · Jan Kara

Dump quota files to resulting filesystem image. They are fs metadata.

Thanks, applied to the maint branch.

- Ted

author
#7 lore raw

PATCH v3 4/9 libext2fs: Support for orphan file feature

In reply to Jan Kara

Add support for creating and deleting orphan file and a couple of utility functions that will be used in other tools.

---
 lib/e2p/feature.c           |   4 +
 lib/ext2fs/Makefile.in      |   2 +
 lib/ext2fs/ext2_fs.h        |  16 ++-
 lib/ext2fs/ext2fs.h         |  32 ++++-
 lib/ext2fs/orphan.c         | 252 ++++++++++++++++++++++++++++++++++++
 lib/ext2fs/swapfs.c         |   3 +-
 lib/ext2fs/tst_super_size.c |   3 +-
 lib/support/mkquota.c       |   3 +-
 8 files changed, 307 insertions(+), 8 deletions(-)
 create mode 100644 lib/ext2fs/orphan.c
lib/e2p/feature.c +4 -0
index 2291060214ff..29b7b1512400 100644
@@ -49,6 +49,8 @@ static struct feature feature_list[] = {
 			"fast_commit" },
 	{	E2P_FEATURE_COMPAT, EXT4_FEATURE_COMPAT_STABLE_INODES,
 			"stable_inodes" },
+	{	E2P_FEATURE_COMPAT, EXT4_FEATURE_COMPAT_ORPHAN_FILE,
+			"orphan_file" },
 
 	{	E2P_FEATURE_RO_INCOMPAT, EXT2_FEATURE_RO_COMPAT_SPARSE_SUPER,
 			"sparse_super" },
@@ -80,6 +82,8 @@ static struct feature feature_list[] = {
 			"shared_blocks"},
 	{	E2P_FEATURE_RO_INCOMPAT, EXT4_FEATURE_RO_COMPAT_VERITY,
 			"verity"},
+	{	E2P_FEATURE_RO_INCOMPAT, EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT,
+			"orphan_present" },
 
 	{	E2P_FEATURE_INCOMPAT, EXT2_FEATURE_INCOMPAT_COMPRESSION,
 			"compression" },
lib/ext2fs/Makefile.in +2 -0
index 5d9af86e520b..ffbfd7a7fc33 100644
@@ -115,6 +115,7 @@ OBJS= $(DEBUGFS_LIB_OBJS) $(RESIZE_LIB_OBJS) $(E2IMAGE_LIB_OBJS) \
 	newdir.o \
 	nls_utf8.o \
 	openfs.o \
+	orphan.o \
 	progress.o \
 	punch.o \
 	qcow2.o \
@@ -198,6 +199,7 @@ SRCS= ext2_err.c \
 	$(srcdir)/newdir.c \
 	$(srcdir)/nls_utf8.c \
 	$(srcdir)/openfs.c \
+	$(srcdir)/orphan.c \
 	$(srcdir)/progress.c \
 	$(srcdir)/punch.c \
 	$(srcdir)/qcow2.c \
lib/ext2fs/ext2_fs.h +14 -2
index 6f1d5db4b482..00809e7b92be 100644
@@ -761,7 +761,8 @@ struct ext2_super_block {
 	__u8    s_last_error_errcode;
 /*27c*/ __le16	s_encoding;		/* Filename charset encoding */
 	__le16	s_encoding_flags;	/* Filename charset encoding flags */
-	__le32	s_reserved[95];		/* Padding to the end of the block */
+	__le32  s_orphan_file_inum;	/* Inode for tracking orphan inodes */
+	__le32	s_reserved[94];		/* Padding to the end of the block */
 /*3fc*/	__u32	s_checksum;		/* crc32c(superblock) */
 };
 
@@ -816,7 +817,7 @@ struct ext2_super_block {
 #define EXT4_FEATURE_COMPAT_SPARSE_SUPER2	0x0200
 #define EXT4_FEATURE_COMPAT_FAST_COMMIT		0x0400
 #define EXT4_FEATURE_COMPAT_STABLE_INODES	0x0800
-
+#define EXT4_FEATURE_COMPAT_ORPHAN_FILE		0x1000
 
 #define EXT2_FEATURE_RO_COMPAT_SPARSE_SUPER	0x0001
 #define EXT2_FEATURE_RO_COMPAT_LARGE_FILE	0x0002
@@ -825,6 +826,7 @@ struct ext2_super_block {
 #define EXT4_FEATURE_RO_COMPAT_GDT_CSUM		0x0010
 #define EXT4_FEATURE_RO_COMPAT_DIR_NLINK	0x0020
 #define EXT4_FEATURE_RO_COMPAT_EXTRA_ISIZE	0x0040
+#define EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT	0x0080
 #define EXT4_FEATURE_RO_COMPAT_QUOTA		0x0100
 #define EXT4_FEATURE_RO_COMPAT_BIGALLOC		0x0200
 /*
@@ -918,6 +920,7 @@ EXT4_FEATURE_COMPAT_FUNCS(exclude_bitmap,	2, EXCLUDE_BITMAP)
 EXT4_FEATURE_COMPAT_FUNCS(sparse_super2,	4, SPARSE_SUPER2)
 EXT4_FEATURE_COMPAT_FUNCS(fast_commit,		4, FAST_COMMIT)
 EXT4_FEATURE_COMPAT_FUNCS(stable_inodes,	4, STABLE_INODES)
+EXT4_FEATURE_COMPAT_FUNCS(orphan_file,		4, ORPHAN_FILE)
 
 EXT4_FEATURE_RO_COMPAT_FUNCS(sparse_super,	2, SPARSE_SUPER)
 EXT4_FEATURE_RO_COMPAT_FUNCS(large_file,	2, LARGE_FILE)
@@ -933,6 +936,7 @@ EXT4_FEATURE_RO_COMPAT_FUNCS(readonly,		4, READONLY)
 EXT4_FEATURE_RO_COMPAT_FUNCS(project,		4, PROJECT)
 EXT4_FEATURE_RO_COMPAT_FUNCS(shared_blocks,	4, SHARED_BLOCKS)
 EXT4_FEATURE_RO_COMPAT_FUNCS(verity,		4, VERITY)
+EXT4_FEATURE_RO_COMPAT_FUNCS(orphan_present,	4, ORPHAN_PRESENT)
 
 EXT4_FEATURE_INCOMPAT_FUNCS(compression,	2, COMPRESSION)
 EXT4_FEATURE_INCOMPAT_FUNCS(filetype,		2, FILETYPE)
@@ -1100,6 +1104,14 @@ static inline unsigned int ext2fs_dir_rec_len(__u8 name_len,
 	return rec_len;
 }
 
+#define EXT4_ORPHAN_BLOCK_MAGIC 0x0b10ca04
+
+/* Structure at the tail of orphan block */
+struct ext4_orphan_block_tail {
+	__u32 ob_magic;
+	__u32 ob_checksum;
+};
+
 /*
  * Constants for ext4's extended time encoding
  */
lib/ext2fs/ext2fs.h +29 -3
index df150f0003f2..886e12793620 100644
@@ -620,7 +620,8 @@ typedef struct ext2_icount *ext2_icount_t;
 					 EXT2_FEATURE_COMPAT_EXT_ATTR|\
 					 EXT4_FEATURE_COMPAT_SPARSE_SUPER2|\
 					 EXT4_FEATURE_COMPAT_FAST_COMMIT|\
-					 EXT4_FEATURE_COMPAT_STABLE_INODES)
+					 EXT4_FEATURE_COMPAT_STABLE_INODES|\
+					 EXT4_FEATURE_COMPAT_ORPHAN_FILE)
 
 #ifdef CONFIG_MMP
 #define EXT4_LIB_INCOMPAT_MMP		EXT4_FEATURE_INCOMPAT_MMP
@@ -655,7 +656,8 @@ typedef struct ext2_icount *ext2_icount_t;
 					 EXT4_FEATURE_RO_COMPAT_READONLY |\
 					 EXT4_FEATURE_RO_COMPAT_PROJECT |\
 					 EXT4_FEATURE_RO_COMPAT_SHARED_BLOCKS |\
-					 EXT4_FEATURE_RO_COMPAT_VERITY)
+					 EXT4_FEATURE_RO_COMPAT_VERITY |\
+					 EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT)
 
 /*
  * These features are only allowed if EXT2_FLAG_SOFTSUPP_FEATURES is passed
@@ -1687,6 +1689,15 @@ errcode_t ext2fs_get_data_io(ext2_filsys fs, io_channel *old_io);
 errcode_t ext2fs_set_data_io(ext2_filsys fs, io_channel new_io);
 errcode_t ext2fs_rewrite_to_io(ext2_filsys fs, io_channel new_io);
 
+/* orphan.c */
+extern errcode_t ext2fs_create_orphan_file(ext2_filsys fs, blk_t num_blocks);
+extern errcode_t ext2fs_truncate_orphan_file(ext2_filsys fs);
+extern e2_blkcnt_t ext2fs_default_orphan_file_blocks(ext2_filsys fs);
+extern errcode_t ext2fs_orphan_file_block_csum_set(ext2_filsys fs,
+						   ext2_ino_t ino, char *buf);
+extern int ext2fs_orphan_file_block_csum_verify(ext2_filsys fs, ext2_ino_t ino,
+						char *buf);
+
 /* get_pathname.c */
 extern errcode_t ext2fs_get_pathname(ext2_filsys fs, ext2_ino_t dir, ext2_ino_t ino,
 			       char **name);
@@ -1840,7 +1851,9 @@ extern int ext2fs_dirent_file_type(const struct ext2_dir_entry *entry);
 extern void ext2fs_dirent_set_file_type(struct ext2_dir_entry *entry, int type);
 extern struct ext2_inode *ext2fs_inode(struct ext2_inode_large * large_inode);
 extern const struct ext2_inode *ext2fs_const_inode(const struct ext2_inode_large * large_inode);
-
+extern int ext2fs_inodes_per_orphan_block(ext2_filsys fs);
+extern struct ext4_orphan_block_tail *ext2fs_orphan_block_tail(ext2_filsys fs,
+							       char *buf);
 #endif
 
 /*
@@ -2150,6 +2163,19 @@ ext2fs_const_inode(const struct ext2_inode_large * large_inode)
 	return (const struct ext2_inode *) large_inode;
 }
 
+_INLINE_ int ext2fs_inodes_per_orphan_block(ext2_filsys fs)
+{
+	return (fs->blocksize - sizeof(struct ext4_orphan_block_tail)) /
+		sizeof(__u32);
+}
+
+_INLINE_ struct ext4_orphan_block_tail *
+ext2fs_orphan_block_tail(ext2_filsys fs, char *buf)
+{
+	return (struct ext4_orphan_block_tail *)(buf + fs->blocksize -
+		sizeof(struct ext4_orphan_block_tail));
+}
+
 #undef _INLINE_
 #endif
 
lib/ext2fs/orphan.c new file +252 -0

252 lines not on this page · raw message

lib/ext2fs/swapfs.c +2 -1
index 1006b2d2bd52..b844e7665999 100644
@@ -131,8 +131,9 @@ void ext2fs_swap_super(struct ext2_super_block * sb)
 	/* s_*_time_hi are __u8 and does not need swabbing */
 	sb->s_encoding = ext2fs_swab16(sb->s_encoding);
 	sb->s_encoding_flags = ext2fs_swab16(sb->s_encoding_flags);
+	sb->s_orphan_file_inum = ext2fs_swab32(sb->s_orphan_file_inum);
 	/* catch when new fields are used from s_reserved */
-	EXT2FS_BUILD_BUG_ON(sizeof(sb->s_reserved) != 95 * sizeof(__le32));
+	EXT2FS_BUILD_BUG_ON(sizeof(sb->s_reserved) != 94 * sizeof(__le32));
 	sb->s_checksum = ext2fs_swab32(sb->s_checksum);
 }
 
lib/ext2fs/tst_super_size.c +2 -1
index 80a5269bceb7..ad452dee8eb3 100644
@@ -152,7 +152,8 @@ int main(int argc, char **argv)
 	check_field(s_last_error_errcode, 1);
 	check_field(s_encoding, 2);
 	check_field(s_encoding_flags, 2);
-	check_field(s_reserved, 95 * 4);
+	check_field(s_orphan_file_inum, 4);
+	check_field(s_reserved, 94 * 4);
 	check_field(s_checksum, 4);
 	do_field("Superblock end", 0, 0, cur_offset, 1024);
 #endif
lib/support/mkquota.c +2 -1
index 21a5c34d6921..b790b93f52c2 100644
@@ -504,7 +504,8 @@ errcode_t quota_compute_usage(quota_ctx_t qctx)
 			continue;
 		if (ino == EXT2_ROOT_INO ||
 		    (ino >= EXT2_FIRST_INODE(fs->super) &&
-		     ino != quota_type2inum(PRJQUOTA, fs->super))) {
+		     ino != quota_type2inum(PRJQUOTA, fs->super) &&
+		     ino != fs->super->s_orphan_file_inum)) {
 			space = ext2fs_get_stat_i_blocks(fs,
 						EXT2_INODE(inode)) << 9;
 			quota_data_add(qctx, inode, ino, space);
Signature · 1 line
2.26.2
#8 lore raw

Re: [PATCH 4/9] libext2fs: Support for orphan file feature

lib/ext2fs/ext2_fs.h struct ext2_super_block { · line 831

Quoted 7 lines · Jan Kara
lib/ext2fs/ext2_fs.h +1 -0
@@ -825,6 +826,7 @@ struct ext2_super_block {
 #define EXT4_FEATURE_RO_COMPAT_GDT_CSUM		0x0010
 #define EXT4_FEATURE_RO_COMPAT_DIR_NLINK	0x0020
 #define EXT4_FEATURE_RO_COMPAT_EXTRA_ISIZE	0x0040
+#define EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT	0x0080
 #define EXT4_FEATURE_RO_COMPAT_QUOTA		0x0100
 #define EXT4_FEATURE_RO_COMPAT_BIGALLOC		0x0200

(This isn't a full review of the patch, but just a quick feedback of what I've noticed so far.)

Since Andreas has requested that we not get rid of the RO_COMPAT_SNAPSHOT, I'm using 0x10000 for EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT in my testing.

I also noted a number of new GCC warnings when running "make gcc-wall" on lib/ext2fs after applying this commit.

- Ted

/usr/projects/e2fsprogs/e2fsprogs/lib/ext2fs/orphan.c: In function ‘ext2fs_do_orphan_file_block_csum’:
/usr/projects/e2fsprogs/e2fsprogs/lib/ext2fs/orphan.c:58:30: warning: pointer targets in passing argument 2 of ‘ext2fs_crc32c_le’ differ in signedness [-Wpointer-sign]

   58 |  crc = ext2fs_crc32c_le(crc, buf, inodes_per_ob * sizeof(__u32));
      |                              ^~~
      |                              |
      |                              char *
In file included from /usr/projects/e2fsprogs/e2fsprogs/lib/ext2fs/ext2fsP.h:16,
                 from /usr/projects/e2fsprogs/e2fsprogs/lib/ext2fs/orphan.c:16:
/usr/projects/e2fsprogs/e2fsprogs/lib/ext2fs/ext2fs.h:1075:63: note: expected ‘const unsigned char *’ but argument is of type ‘char *’
 1075 | extern __u32 ext2fs_crc32c_le(__u32 crc, unsigned char const *p, size_t len);
      |                                          ~~~~~~~~~~~~~~~~~~~~~^
/usr/projects/e2fsprogs/e2fsprogs/lib/ext2fs/orphan.c: In function ‘ext2fs_do_orphan_file_block_csum’:
/usr/projects/e2fsprogs/e2fsprogs/lib/ext2fs/orphan.c:58:30: warning: pointer targets in passing argument 2 of ‘ext2fs_crc32c_le’ differ in signedness [-Wpointer-sign]
   58 |  crc = ext2fs_crc32c_le(crc, buf, inodes_per_ob * sizeof(__u32));
      |                              ^~~
      |                              |
      |                              char *
In file included from /usr/projects/e2fsprogs/e2fsprogs/lib/ext2fs/ext2fsP.h:16,
                 from /usr/projects/e2fsprogs/e2fsprogs/lib/ext2fs/orphan.c:16:
/usr/projects/e2fsprogs/e2fsprogs/lib/ext2fs/ext2fs.h:1075:63: note: expected ‘const unsigned char *’ but argument is of type ‘char *’
 1075 | extern __u32 ext2fs_crc32c_le(__u32 crc, unsigned char const *p, size_t len);
      |                                          ~~~~~~~~~~~~~~~~~~~~~^

make[1]: Leaving directory '/build/e2fsprogs/lib/ext2fs'

#9 lore raw

Re: [PATCH 4/9] libext2fs: Support for orphan file feature

Quoted 12 lines · Theodore Ts'o

lib/ext2fs/ext2_fs.h struct ext2_super_block { · line 831

Quoted 7 lines · Jan Kara
lib/ext2fs/ext2_fs.h +1 -0
@@ -825,6 +826,7 @@ struct ext2_super_block {
 #define EXT4_FEATURE_RO_COMPAT_GDT_CSUM		0x0010
 #define EXT4_FEATURE_RO_COMPAT_DIR_NLINK	0x0020
 #define EXT4_FEATURE_RO_COMPAT_EXTRA_ISIZE	0x0040
+#define EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT	0x0080
 #define EXT4_FEATURE_RO_COMPAT_QUOTA		0x0100
 #define EXT4_FEATURE_RO_COMPAT_BIGALLOC		0x0200

(This isn't a full review of the patch, but just a quick feedback of what I've noticed so far.)

Quoted 6 lines

Since Andreas has requested that we not get rid of the RO_COMPAT_SNAPSHOT, I'm using 0x10000 for EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT in my testing.

I also noted a number of new GCC warnings when running "make gcc-wall" on lib/ext2fs after applying this commit.

Sorry, I just realized I had used b4 to grab the v3 instead of the v4 of the patch series. Fortunately it looks like the 2nd and 3rd patches hadn't changed between the two patch series, so I'll regroup and try again this time with the v4 patches.

- Ted

author
#10 lore raw

PATCH v3 5/9 mke2fs: Add support for orphan_file feature

In reply to Jan Kara
---
 misc/mke2fs.8.in |  5 +++++
 misc/mke2fs.c    | 36 +++++++++++++++++++++++++++++++++++-
 2 files changed, 40 insertions(+), 1 deletion(-)
misc/mke2fs.8.in +5 -0
index 84248ffc9e1d..3747c93a02bf 100644
@@ -403,6 +403,11 @@ filesystem to change based on the user running \fBmke2fs\fR.
 Set a flag in the filesystem superblock indicating that it may be
 mounted using experimental kernel code, such as the ext4dev filesystem.
 .TP
+.BI orphan_file_size= size
+Set size of the file for tracking unlinked but still open inodes and inodes
+with truncate in progress. Larger file allows for better scalability, reserving
+a few blocks per cpu is ideal.
+.TP
 .B discard
 Attempt to discard blocks at mkfs time (discarding blocks initially is useful
 on solid state devices and sparse / thin-provisioned storage). When the device
misc/mke2fs.c +35 -1
index afbcf486bad2..8a5cf9b920e6 100644
@@ -94,6 +94,7 @@ static gid_t	root_gid;
 int	journal_size;
 int	journal_flags;
 int	journal_fc_size;
+static e2_blkcnt_t	orphan_file_blocks;
 static int	lazy_itable_init;
 static int	packed_meta_blocks;
 int		no_copy_xattrs;
@@ -1087,6 +1088,21 @@ static void parse_extended_opts(struct ext2_super_block *param,
 				continue;
 			}
 			encoding_flags = arg;
+		} else if (!strcmp(token, "orphan_file_size")) {
+			if (!arg) {
+				r_usage++;
+				badopt = token;
+				continue;
+			}
+			orphan_file_blocks = parse_num_blocks2(arg,
+						fs_param.s_log_block_size);
+			if (orphan_file_blocks == 0) {
+				fprintf(stderr,
+					_("Invalid size of orphan file %s\n"),
+					arg);
+				r_usage++;
+				continue;
+			}
 		} else {
 			r_usage++;
 			badopt = token;
@@ -1154,7 +1170,8 @@ static __u32 ok_features[3] = {
 		EXT2_FEATURE_COMPAT_EXT_ATTR |
 		EXT4_FEATURE_COMPAT_SPARSE_SUPER2 |
 		EXT4_FEATURE_COMPAT_FAST_COMMIT |
-		EXT4_FEATURE_COMPAT_STABLE_INODES,
+		EXT4_FEATURE_COMPAT_STABLE_INODES |
+		EXT4_FEATURE_COMPAT_ORPHAN_FILE,
 	/* Incompat */
 	EXT2_FEATURE_INCOMPAT_FILETYPE|
 		EXT3_FEATURE_INCOMPAT_EXTENTS|
@@ -3438,6 +3455,23 @@ no_journal:
 		fix_cluster_bg_counts(fs);
 	if (ext2fs_has_feature_quota(&fs_param))
 		create_quota_inodes(fs);
+	if (ext2fs_has_feature_orphan_file(&fs_param)) {
+		if (!ext2fs_has_feature_journal(&fs_param)) {
+			com_err(program_name, 0, _("cannot set orphan_file "
+				"flag without a journal."));
+			exit(1);
+		}
+		if (!orphan_file_blocks) {
+			orphan_file_blocks =
+				ext2fs_default_orphan_file_blocks(fs);
+		}
+		retval = ext2fs_create_orphan_file(fs, orphan_file_blocks);
+		if (retval) {
+			com_err(program_name, retval,
+				_("while creating orphan file"));
+			exit(1);
+		}
+	}
 
 	retval = mk_hugefiles(fs, device_name);
 	if (retval)
Signature · 1 line
2.26.2
author
#11 lore raw

PATCH v3 6/9 e2fsck: Add support for handling orphan file

In reply to Jan Kara
---
 e2fsck/e2fsck.h  |   1 +
 e2fsck/pass1.c   |  27 ++++
 e2fsck/pass4.c   |   2 +-
 e2fsck/problem.c |  80 +++++++++++
 e2fsck/problem.h |  50 +++++++
 e2fsck/super.c   | 347 +++++++++++++++++++++++++++++++++++++++++------
 e2fsck/unix.c    |  73 +++++++++-
 7 files changed, 536 insertions(+), 44 deletions(-)
e2fsck/e2fsck.h +1 -0
index 15d043ee4692..979cdfcb2740 100644
@@ -649,6 +649,7 @@ void sigcatcher_setup(void);
 void check_super_block(e2fsck_t ctx);
 int check_backup_super_block(e2fsck_t ctx);
 void check_resize_inode(e2fsck_t ctx);
+int check_init_orphan_file(e2fsck_t ctx);
 
 /* util.c */
 extern void *e2fsck_allocate_memory(e2fsck_t ctx, unsigned long size,
e2fsck/pass1.c +27 -0
index 9d4308956773..b8d8dd109096 100644
@@ -1778,6 +1778,32 @@ void e2fsck_pass1(e2fsck_t ctx)
 							inode_size, "pass1");
 				failed_csum = 0;
 			}
+		} else if (ino == fs->super->s_orphan_file_inum) {
+			ext2fs_mark_inode_bitmap2(ctx->inode_used_map, ino);
+			if (ext2fs_has_feature_orphan_file(fs->super)) {
+				if (!LINUX_S_ISREG(inode->i_mode) &&
+				    fix_problem(ctx, PR_1_ORPHAN_FILE_BAD_MODE,
+						&pctx)) {
+					inode->i_mode = LINUX_S_IFREG;
+					e2fsck_write_inode(ctx, ino, inode,
+							   "pass1");
+					failed_csum = 0;
+				}
+				check_blocks(ctx, &pctx, block_buf, NULL);
+				FINISH_INODE_LOOP(ctx, ino, &pctx, failed_csum);
+				continue;
+			}
+			if ((inode->i_links_count ||
+			     inode->i_blocks || inode->i_block[0]) &&
+			    fix_problem(ctx, PR_1_ORPHAN_FILE_NOT_CLEAR,
+					&pctx)) {
+				memset(inode, 0, inode_size);
+				ext2fs_icount_store(ctx->inode_link_info, ino,
+						    0);
+				e2fsck_write_inode_full(ctx, ino, inode,
+							inode_size, "pass1");
+				failed_csum = 0;
+			}
 		} else if (ino < EXT2_FIRST_INODE(fs->super)) {
 			problem_t problem = 0;
 
@@ -3484,6 +3510,7 @@ static void check_blocks(e2fsck_t ctx, struct problem_context *pctx,
 	}
 
 	if (ino != quota_type2inum(PRJQUOTA, fs->super) &&
+	    ino != fs->super->s_orphan_file_inum &&
 	    (ino == EXT2_ROOT_INO || ino >= EXT2_FIRST_INODE(ctx->fs->super)) &&
 	    !(inode->i_flags & EXT4_EA_INODE_FL)) {
 		quota_data_add(ctx->qctx, (struct ext2_inode_large *) inode,
e2fsck/pass4.c +1 -1
index 8c2d2f1fca12..f41eb849e567 100644
@@ -192,7 +192,7 @@ void e2fsck_pass4(e2fsck_t ctx)
 					goto errout;
 		}
 		if (i == quota_type2inum(PRJQUOTA, ctx->fs->super) ||
-		    i == EXT2_BAD_INO ||
+		    i == fs->super->s_orphan_file_inum || i == EXT2_BAD_INO ||
 		    (i > EXT2_ROOT_INO && i < EXT2_FIRST_INODE(fs->super)))
 			continue;
 		if (!(ext2fs_test_inode_bitmap2(ctx->inode_used_map, i)) ||
e2fsck/problem.c +80 -0

98 lines not on this page · raw message

e2fsck/problem.h +50 -0

68 lines not on this page · raw message

e2fsck/super.c +306 -41

390 lines not on this page · raw message

e2fsck/unix.c +71 -2

92 lines not on this page · raw message

Signature · 1 line
2.26.2
author
#12 lore raw

PATCH v3 7/9 tune2fs: Add support for orphan_file feature

In reply to Jan Kara
---
 misc/tune2fs.8.in |  5 +++
 misc/tune2fs.c    | 89 +++++++++++++++++++++++++++++++++++++++++++++--
 2 files changed, 92 insertions(+), 2 deletions(-)
misc/tune2fs.8.in +5 -0
index b963f30edef3..849f94b68d6e 100644
@@ -257,6 +257,11 @@ program.
 This superblock setting is only honored in 2.6.35+ kernels;
 and not at all by the ext2 and ext3 file system drivers.
 .TP
+.BI orphan_file_size= size
+Set size of the file for tracking unlinked but still open inodes and inodes
+with truncate in progress. Larger file allows for better scalability, reserving
+a few blocks per cpu is ideal.
+.TP
 .B force_fsck
 Set a flag in the filesystem superblock indicating that errors have been found.
 This will force fsck to run at the next mount.
misc/tune2fs.c +87 -2

125 lines not on this page · raw message

Signature · 1 line
2.26.2
author
#13 lore raw

PATCH v3 8/9 mke2fs: Add orphan_file feature into mke2fs.conf

In reply to Jan Kara

Enable orphan_file feature by default in larger filesystems. Since the feature is COMPAT, older kernels will just ignore it and happily work with the filesystem as well.

---
 misc/mke2fs.conf.in | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
misc/mke2fs.conf.in +3 -1
index 01e35cf83150..d97d8d643d1d 100644
@@ -11,15 +11,17 @@
 		features = has_journal
 	}
 	ext4 = {
-		features = has_journal,extent,huge_file,flex_bg,metadata_csum,64bit,dir_nlink,extra_isize
+		features = has_journal,extent,huge_file,flex_bg,metadata_csum,64bit,dir_nlink,extra_isize,orphan_file
 		inode_size = 256
 	}
 	small = {
+		default_features = ^orphan_file
 		blocksize = 1024
 		inode_size = 128
 		inode_ratio = 4096
 	}
 	floppy = {
+		default_features = ^orphan_file
 		blocksize = 1024
 		inode_size = 128
 		inode_ratio = 8192
Signature · 1 line
2.26.2
author
#14 lore raw

PATCH v3 9/9 dumpe2fs, debugfs, e2image: Add support for orphan file

In reply to Jan Kara

Print inode number of orphan file in outputs, dump e2image file to filesystem image.

---
 debugfs/set_fields.c | 1 +
 lib/e2p/ls.c         | 3 +++
 misc/e2image.c       | 3 ++-
 3 files changed, 6 insertions(+), 1 deletion(-)
debugfs/set_fields.c +1 -0
index b00157940774..f916deab8cea 100644
@@ -183,6 +183,7 @@ static struct field_set_info super_fields[] = {
 	{ "lpf_ino", &set_sb.s_lpf_ino, NULL, 4, parse_uint },
 	{ "checksum_seed", &set_sb.s_checksum_seed, NULL, 4, parse_uint },
 	{ "encoding", &set_sb.s_encoding, NULL, 2, parse_encoding },
+	{ "orphan_file_inum", &set_sb.s_orphan_file_inum, NULL, 4, parse_uint },
 	{ 0, 0, 0, 0 }
 };
 
lib/e2p/ls.c +3 -0
index 176bee0fd19f..1762bc44cac4 100644
@@ -482,6 +482,9 @@ void list_super2(struct ext2_super_block * sb, FILE *f)
 	if (ext2fs_has_feature_casefold(sb))
 		fprintf(f, "Character encoding:       %s\n",
 			e2p_encoding2str(sb->s_encoding));
+	if (ext2fs_has_feature_orphan_file(sb))
+		fprintf(f, "Orphan file inode:        %u\n",
+			sb->s_orphan_file_inum);
 }
 
 void list_super (struct ext2_super_block * s)
misc/e2image.c +2 -1
index ac00827e4628..a9c64506d7cc 100644
@@ -1369,7 +1369,8 @@ static void write_raw_image_file(ext2_filsys fs, int fd, int type, int flags,
 		    ino == fs->super->s_journal_inum ||
 		    ino == quota_type2inum(USRQUOTA, fs->super) ||
 		    ino == quota_type2inum(GRPQUOTA, fs->super) ||
-		    ino == quota_type2inum(PRJQUOTA, fs->super)) {
+		    ino == quota_type2inum(PRJQUOTA, fs->super) ||
+		    ino == fs->super->s_orphan_file_inum) {
 			retval = ext2fs_block_iterate3(fs, ino,
 					BLOCK_FLAG_READ_ONLY, block_buf,
 					process_dir_block, &pb);
Signature · 1 line
2.26.2