Skip to contentSkip to conversation

PAGE_SIZE and zoned storage related improvements

PATCH In review Bart Van Assche 4 participants 15 messages Jun 23, 2022 – Aug 25, 2022 Library code

lore
Share & tools
lore
#0 lore raw

v2 cover PAGE_SIZE and zoned storage related improvements

Hi Jaegeuk,

This patch series fixes one issue reported by Peter Collingbourne and a few issues I discovered by reading the zoned block device source code. Please consider these patches for inclusion in the official f2fs-tools repository.

Thanks,

Bart.

Bart Van Assche (5):
Fix the struct f2fs_dentry_block definition
Fix f2fs_report_zone()
Improve compile-time type checking for f2fs_report_zone() Use F2FS_BLKSIZE for dev_read_block() buffers Use F2FS_BLKSIZE as the size of struct f2fs_summary_block

 fsck/mount.c        | 14 +++++++-------
 include/f2fs_fs.h   |  8 +++++---
 lib/libf2fs_zoned.c | 23 +++++++++++++++--------
 3 files changed, 27 insertions(+), 18 deletions(-)

_______________________________________________
Linux-f2fs-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

#1 lore raw

PATCH v2 1/5 Fix the struct f2fs_dentry_block definition

Fix the struct f2fs_dentry_block definition on systems for which PAGE_SIZE != 4096. This patch does not change the struct f2fs_dentry_block definition if PAGE_SIZE == 4096.

---
 include/f2fs_fs.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
include/f2fs_fs.h +2 -2
index 21a7e70d952d..fdbf7c7a0b35 100644
@@ -1317,7 +1317,7 @@ typedef __le32	f2fs_hash_t;
 #define SIZE_OF_DIR_ENTRY	11	/* by byte */
 #define SIZE_OF_DENTRY_BITMAP	((NR_DENTRY_IN_BLOCK + BITS_PER_BYTE - 1) / \
 					BITS_PER_BYTE)
-#define SIZE_OF_RESERVED	(PAGE_SIZE - ((SIZE_OF_DIR_ENTRY + \
+#define SIZE_OF_RESERVED	(F2FS_BLKSIZE - ((SIZE_OF_DIR_ENTRY + \
 				F2FS_SLOT_LEN) * \
 				NR_DENTRY_IN_BLOCK + SIZE_OF_DENTRY_BITMAP))
 #define MIN_INLINE_DENTRY_SIZE		40	/* just include '.' and '..' entries */
@@ -1341,7 +1341,7 @@ struct f2fs_dentry_block {
 	__u8 filename[NR_DENTRY_IN_BLOCK][F2FS_SLOT_LEN];
 };
 
-static_assert(sizeof(struct f2fs_dentry_block) == 4096, "");
+static_assert(sizeof(struct f2fs_dentry_block) == F2FS_BLKSIZE, "");
 
 /* for inline stuff */
 #define DEF_INLINE_RESERVED_SIZE	1

_______________________________________________
Linux-f2fs-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Reviewed-by Peter Collingbourne Tested-by Peter Collingbourne on v2 1/5
#2 lore raw

Re: [f2fs-dev] [PATCH v2 1/5] Fix the struct f2fs_dentry_block definition

Quoted 8 lines · Bart Van Assche

Fix the struct f2fs_dentry_block definition on systems for which PAGE_SIZE != 4096. This patch does not change the struct f2fs_dentry_block definition if PAGE_SIZE == 4096.

Thanks, this is what I had in mind and it fixes the build on my target with PAGE_SIZE != 4096. I also verified that a filesystem created on the PAGE_SIZE != 4096 target can be mounted on a machine with PAGE_SIZE == 4096.

Peter

_______________________________________________
Linux-f2fs-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Reviewed-by on v2 1/5
#3 lore raw

Re: [f2fs-dev] [PATCH v2 1/5] Fix the struct f2fs_dentry_block definition

In reply to Bart Van Assche
Quoted 7 lines · Bart Van Assche

Fix the struct f2fs_dentry_block definition on systems for which PAGE_SIZE != 4096. This patch does not change the struct f2fs_dentry_block definition if PAGE_SIZE == 4096.

Thanks,

_______________________________________________
Linux-f2fs-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

#4 lore raw

PATCH v2 2/5 Fix f2fs_report_zone()

In reply to Bart Van Assche

The definition of struct blk_zone_report is as follows:

	struct blk_zone_report {
		__u64		sector;
		__u32		nr_zones;
		__u32		flags;
		struct blk_zone zones[0];
	};

Since f2fs_report_zone() allocates the above data structure with malloc() and since f2fs_report_zone() only initializes the sector and nr_zones members, the flags member is not initialized. Modify f2fs_report_zone() such that 0 is passed as flags to the BLKREPORTZONE ioctl instead of a random value. This has been discovered by reading the source code.

Cc Shin'ichiro Kawasaki <[email protected]>
Fixes 6d7c7b785feb ("libf2fs_zoned: Introduce f2fs_report_zone() helper function")
---
 lib/libf2fs_zoned.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
lib/libf2fs_zoned.c +2 -1
index f383ce275342..d8de66b82029 100644
@@ -206,7 +206,8 @@ int f2fs_report_zone(int i, uint64_t sector, void *blkzone)
 	struct blk_zone_report *rep;
 	int ret = -1;
 
-	rep = malloc(sizeof(struct blk_zone_report) + sizeof(struct blk_zone));
+	rep = calloc(1, sizeof(struct blk_zone_report) +
+		     sizeof(struct blk_zone));
 	if (!rep) {
 		ERR_MSG("No memory for report zones\n");
 		return -ENOMEM;

_______________________________________________
Linux-f2fs-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Reviewed-by on v2 2/5
#5 lore raw

Re: [f2fs-dev] [PATCH v2 2/5] Fix f2fs_report_zone()

Quoted 16 lines · Bart Van Assche

The definition of struct blk_zone_report is as follows:

	struct blk_zone_report {
		__u64		sector;
		__u32		nr_zones;
		__u32		flags;
		struct blk_zone zones[0];
	};

Since f2fs_report_zone() allocates the above data structure with malloc() and since f2fs_report_zone() only initializes the sector and nr_zones members, the flags member is not initialized. Modify f2fs_report_zone() such that 0 is passed as flags to the BLKREPORTZONE ioctl instead of a random value. This has been discovered by reading the source code.

Quoted 3 lines
Cc Shin'ichiro Kawasaki <[email protected]>
Fixes 6d7c7b785feb ("libf2fs_zoned: Introduce f2fs_report_zone() helper function")

Thanks,

_______________________________________________
Linux-f2fs-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

#6 lore raw

PATCH v2 3/5 Improve compile-time type checking for f2fs_report_zone()

In reply to Bart Van Assche

Change the type of the third argument of f2fs_report_zone() from void * into struct blk_zone * to enable type checking.

---
 include/f2fs_fs.h   |  4 +++-
 lib/libf2fs_zoned.c | 24 +++++++++++++++---------
 2 files changed, 18 insertions(+), 10 deletions(-)
include/f2fs_fs.h +3 -1
index fdbf7c7a0b35..8125e9f8d082 100644
@@ -1560,9 +1560,11 @@ blk_zone_cond_str(struct blk_zone *blkz)
 
 #endif
 
+struct blk_zone;
+
 extern int f2fs_get_zoned_model(int);
 extern int f2fs_get_zone_blocks(int);
-extern int f2fs_report_zone(int, uint64_t, void *);
+extern int f2fs_report_zone(int, uint64_t, struct blk_zone *);
 typedef int (report_zones_cb_t)(int i, void *, void *);
 extern int f2fs_report_zones(int, report_zones_cb_t *, void *);
 extern int f2fs_check_zones(int);
lib/libf2fs_zoned.c +15 -9
index d8de66b82029..a0dd8bd269ef 100644
@@ -200,21 +200,26 @@ int f2fs_get_zone_blocks(int i)
 	return 0;
 }
 
-int f2fs_report_zone(int i, uint64_t sector, void *blkzone)
+int f2fs_report_zone(int i, uint64_t sector, struct blk_zone *blkzone)
 {
-	struct blk_zone *blkz = (struct blk_zone *)blkzone;
-	struct blk_zone_report *rep;
+	struct one_zone_report {
+		struct blk_zone_report	rep;
+		struct blk_zone		zone;
+	} *rep;
 	int ret = -1;
 
-	rep = calloc(1, sizeof(struct blk_zone_report) +
-		     sizeof(struct blk_zone));
+	static_assert(sizeof(*rep) == sizeof(rep->rep) + sizeof(rep->zone), "");
+
+	rep = calloc(1, sizeof(*rep));
 	if (!rep) {
 		ERR_MSG("No memory for report zones\n");
 		return -ENOMEM;
 	}
 
-	rep->sector = sector;
-	rep->nr_zones = 1;
+	rep->rep = (struct blk_zone_report){
+		.sector = sector,
+		.nr_zones = 1,
+	};
 	ret = ioctl(c.devices[i].fd, BLKREPORTZONE, rep);
 	if (ret != 0) {
 		ret = -errno;
@@ -222,7 +227,7 @@ int f2fs_report_zone(int i, uint64_t sector, void *blkzone)
 		goto out;
 	}
 
-	*blkz = *(struct blk_zone *)(rep + 1);
+	*blkzone = rep->zone;
 out:
 	free(rep);
 	return ret;
@@ -531,7 +536,8 @@ uint32_t f2fs_get_usable_segments(struct f2fs_super_block *sb)
 
 #else
 
-int f2fs_report_zone(int i, uint64_t UNUSED(sector), void *UNUSED(blkzone))
+int f2fs_report_zone(int i, uint64_t UNUSED(sector),
+		     struct blk_zone *UNUSED(blkzone))
 {
 	ERR_MSG("%d: Unsupported zoned block device\n", i);
 	return -1;

_______________________________________________
Linux-f2fs-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Reviewed-by on v2 3/5
#7 lore raw

Re: [f2fs-dev] [PATCH v2 3/5] Improve compile-time type checking for f2fs_report_zone()

Quoted 4 lines · Bart Van Assche

Change the type of the third argument of f2fs_report_zone() from void * into struct blk_zone * to enable type checking.

Thanks,

_______________________________________________
Linux-f2fs-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

#8 lore raw

PATCH v2 4/5 Use F2FS_BLKSIZE for dev_read_block() buffers

In reply to Bart Van Assche

Use F2FS_BLKSIZE instead of PAGE_SIZE for dev_read_block() buffers since dev_read_block() reads F2FS_BLKSIZE bytes.

---
 fsck/mount.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
fsck/mount.c +3 -3
index 108e1238493d..cc871fea5d10 100644
@@ -1115,7 +1115,7 @@ static void *get_checkpoint_version(block_t cp_addr)
 {
 	void *cp_page;
 
-	cp_page = malloc(PAGE_SIZE);
+	cp_page = malloc(F2FS_BLKSIZE);
 	ASSERT(cp_page);
 
 	if (dev_read_block(cp_page, cp_addr) < 0)
@@ -1821,7 +1821,7 @@ static void read_compacted_summaries(struct f2fs_sb_info *sbi)
 
 	start = start_sum_block(sbi);
 
-	kaddr = (char *)malloc(PAGE_SIZE);
+	kaddr = malloc(F2FS_BLKSIZE);
 	ASSERT(kaddr);
 
 	ret = dev_read_block(kaddr, start++);
@@ -1856,7 +1856,7 @@ static void read_compacted_summaries(struct f2fs_sb_info *sbi)
 			if (offset + SUMMARY_SIZE <=
 					PAGE_CACHE_SIZE - SUM_FOOTER_SIZE)
 				continue;
-			memset(kaddr, 0, PAGE_SIZE);
+			memset(kaddr, 0, F2FS_BLKSIZE);
 			ret = dev_read_block(kaddr, start++);
 			ASSERT(ret >= 0);
 			offset = 0;

_______________________________________________
Linux-f2fs-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Reviewed-by on v2 4/5
#9 lore raw

Re: [f2fs-dev] [PATCH v2 4/5] Use F2FS_BLKSIZE for dev_read_block() buffers

Quoted 4 lines · Bart Van Assche

Use F2FS_BLKSIZE instead of PAGE_SIZE for dev_read_block() buffers since dev_read_block() reads F2FS_BLKSIZE bytes.

Thanks,

_______________________________________________
Linux-f2fs-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

#10 lore raw

PATCH v2 5/5 Use F2FS_BLKSIZE as the size of struct f2fs_summary_block

In reply to Bart Van Assche

Since the size of struct f2fs_summary_block equals F2FS_BLKSIZE, use F2FS_BLKSIZE instead of PAGE_CACHE_SIZE as the size of struct f2fs_summary_block.

---
 fsck/mount.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
fsck/mount.c +4 -4
index cc871fea5d10..584e6d1370ae 100644
@@ -1854,7 +1854,7 @@ static void read_compacted_summaries(struct f2fs_sb_info *sbi)
 			curseg->sum_blk->entries[j] = *s;
 			offset += SUMMARY_SIZE;
 			if (offset + SUMMARY_SIZE <=
-					PAGE_CACHE_SIZE - SUM_FOOTER_SIZE)
+					F2FS_BLKSIZE - SUM_FOOTER_SIZE)
 				continue;
 			memset(kaddr, 0, F2FS_BLKSIZE);
 			ret = dev_read_block(kaddr, start++);
@@ -1914,7 +1914,7 @@ static void read_normal_summaries(struct f2fs_sb_info *sbi, int type)
 			blk_addr = GET_SUM_BLKADDR(sbi, segno);
 	}
 
-	sum_blk = (struct f2fs_summary_block *)malloc(PAGE_SIZE);
+	sum_blk = malloc(sizeof(*sum_blk));
 	ASSERT(sum_blk);
 
 	ret = dev_read_block(sum_blk, blk_addr);
@@ -1924,7 +1924,7 @@ static void read_normal_summaries(struct f2fs_sb_info *sbi, int type)
 		restore_node_summary(sbi, segno, sum_blk);
 
 	curseg = CURSEG_I(sbi, type);
-	memcpy(curseg->sum_blk, sum_blk, PAGE_CACHE_SIZE);
+	memcpy(curseg->sum_blk, sum_blk, sizeof(*sum_blk));
 	reset_curseg(sbi, type);
 	free(sum_blk);
 }
@@ -1990,7 +1990,7 @@ static int build_curseg(struct f2fs_sb_info *sbi)
 	SM_I(sbi)->curseg_array = array;
 
 	for (i = 0; i < NR_CURSEG_TYPE; i++) {
-		array[i].sum_blk = calloc(PAGE_CACHE_SIZE, 1);
+		array[i].sum_blk = calloc(sizeof(*(array[i].sum_blk)), 1);
 		if (!array[i].sum_blk) {
 			MSG(1, "\tError: Calloc failed for build_curseg!!\n");
 			goto seg_cleanup;

_______________________________________________
Linux-f2fs-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Reviewed-by on v2 5/5
#11 lore raw

Re: [f2fs-dev] [PATCH v2 5/5] Use F2FS_BLKSIZE as the size of struct f2fs_summary_block

Quoted 5 lines · Bart Van Assche

Since the size of struct f2fs_summary_block equals F2FS_BLKSIZE, use F2FS_BLKSIZE instead of PAGE_CACHE_SIZE as the size of struct f2fs_summary_block.

Thanks,

_______________________________________________
Linux-f2fs-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

#12 lore raw
In reply to Bart Van Assche
Quoted 3 lines · Bart Van Assche

This patch series fixes one issue reported by Peter Collingbourne and a few issues I discovered by reading the zoned block device source code. Please consider these patches for inclusion in the official f2fs-tools repository.

Hi Jaegeuk,

Please let me know if you want me to resend this patch series.

Thanks,

Bart.

_______________________________________________
Linux-f2fs-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

#13 lore raw
Quoted 8 lines · Bart Van Assche
Quoted 3 lines · Bart Van Assche

This patch series fixes one issue reported by Peter Collingbourne and a few issues I discovered by reading the zoned block device source code. Please consider these patches for inclusion in the official f2fs-tools repository.

Hi Jaegeuk,

Please let me know if you want me to resend this patch series.

Could you please check this branch?

https://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs-tools.git/log/?h=dev

Quoted 4 lines

Thanks,

Bart.

_______________________________________________
Linux-f2fs-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel