defrag.f2fs: return error for no space case

PATCH In review Jaegeuk Kim 2 participants 10 messages Nov 3, 2016

lore

Share & tools

lore

Jaegeuk Kim

author

2016-11-03 05:28 UTC #0 lore raw

PATCH 1/7 defrag.f2fs: return error for no space case

This returns error instead of assertion for resize.f2fs.

Signed-off-by Jaegeuk Kim <[email protected]>

---
 fsck/defrag.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

fsck/defrag.c +2 -2

index 079e7a7..bea0293 100644
@@ -78,8 +78,8 @@ int f2fs_defragment(struct f2fs_sb_info *sbi, u64 from, u64 len, u64 to, int lef
 			continue;
 
 		if (find_next_free_block(sbi, &target, left, se->type)) {
-			ASSERT_MSG("Not enough space to migrate blocks");
-			break;
+			MSG(0, "Not enough space to migrate blocks");
+			return -1;
 		}
 
 		if (migrate_block(sbi, idx, target)) {

Signature · 9 lines

2.8.3


------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi

Jaegeuk Kim

author

2016-11-03 05:29 UTC #1 lore raw

PATCH 2/7 resize.f2fs: fix wrong end_blkaddr

The end_blkaddr should add main area segments.

Signed-off-by Jaegeuk Kim <[email protected]>

---
 fsck/resize.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

fsck/resize.c +2 -2

index 6a645e3..5dc73a8 100644
@@ -563,8 +563,8 @@ int f2fs_resize(struct f2fs_sb_info *sbi)
 	old_main_blkaddr = get_sb(main_blkaddr);
 	new_main_blkaddr = get_newsb(main_blkaddr);
 	offset = new_main_blkaddr - old_main_blkaddr;
-	end_blkaddr = (get_sb(segment_count) << get_sb(log_blocks_per_seg)) +
-						get_sb(main_blkaddr);
+	end_blkaddr = (get_sb(segment_count_main) <<
+			get_sb(log_blocks_per_seg)) + get_sb(main_blkaddr);
 
 	if (old_main_blkaddr > new_main_blkaddr) {
 		MSG(0, "\tError: Support resize to expand only\n");

Signature · 9 lines

2.8.3


------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi

Jaegeuk Kim

author

2016-11-03 05:29 UTC #2 lore raw

PATCH 3/7 f2fs-tools: fix end block addres when finding free block

In reply to Jaegeuk Kim

This patch fixes wrong end_blkaddr condition for free block allocation.

Signed-off-by Jaegeuk Kim <[email protected]>

---
 fsck/mount.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

fsck/mount.c +4 -2

index 47bb529..879dd52 100644
@@ -1547,16 +1547,18 @@ void flush_sit_entries(struct f2fs_sb_info *sbi)
 
 int find_next_free_block(struct f2fs_sb_info *sbi, u64 *to, int left, int type)
 {
+	struct f2fs_super_block *sb = F2FS_RAW_SUPER(sbi);
 	struct seg_entry *se;
 	u32 segno;
 	u64 offset;
 	int not_enough = 0;
+	u64 end_blkaddr = (get_sb(segment_count_main) <<
+			get_sb(log_blocks_per_seg)) + get_sb(main_blkaddr);
 
 	if (get_free_segments(sbi) <= SM_I(sbi)->reserved_segments + 1)
 		not_enough = 1;
 
-	while (*to >= SM_I(sbi)->main_blkaddr &&
-			*to < F2FS_RAW_SUPER(sbi)->block_count) {
+	while (*to >= SM_I(sbi)->main_blkaddr && *to < end_blkaddr) {
 		segno = GET_SEGNO(sbi, *to);
 		offset = OFFSET_IN_SEG(sbi, *to);

Signature · 9 lines

2.8.3


------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi

Jaegeuk Kim

author

2016-11-03 05:29 UTC #3 lore raw

PATCH 4/7 resize.f2fs: get latest free segments

In reply to Jaegeuk Kim

The # of free segments can be changed by defragmentation.

Signed-off-by Jaegeuk Kim <[email protected]>

---
 fsck/resize.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

fsck/resize.c +1 -1

index 5dc73a8..4813de7 100644
@@ -424,7 +424,7 @@ static void rebuild_checkpoint(struct f2fs_sb_info *sbi,
 	set_cp(overprov_segment_count, get_cp(overprov_segment_count) +
 						get_cp(rsvd_segment_count));
 
-	free_segment_count = get_cp(free_segment_count);
+	free_segment_count = get_free_segments(sbi);
 	new_segment_count = get_newsb(segment_count_main) -
 					get_sb(segment_count_main);

Signature · 9 lines

2.8.3


------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi

heyunlei

2016-11-03 06:53 UTC #4 lore raw

Re: [PATCH 4/7] resize.f2fs: get latest free segments

Hi, Kim

Quoted 12 lines · Jaegeuk Kim

The # of free segments can be changed by defragmentation.
Signed-off-by Jaegeuk Kim <[email protected]>
---
 fsck/resize.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
fsck/resize.c +0 -0
index 5dc73a8..4813de7 100644

fsck/resize.c static void rebuild_checkpoint(struct f2fs_sb_info *sbi, · line 427

Quoted 5 lines

fsck/resize.c +1 -1

@@ -424,7 +424,7 @@ static void rebuild_checkpoint(struct f2fs_sb_info *sbi,
 	set_cp(overprov_segment_count, get_cp(overprov_segment_count) +
 						get_cp(rsvd_segment_count));
 
-	free_segment_count = get_cp(free_segment_count);
+	free_segment_count = get_free_segments(sbi);

f2fs_defragment will write a checkpoint and then update free_segment_count by get_free_segments(sbi)o<
Is it righto<

Thankso<

fsck/resize.c static void rebuild_checkpoint(struct f2fs_sb_info *sbi, · line 430

Quoted 4 lines

fsck/resize.c +0 -0

 	new_segment_count = get_newsb(segment_count_main) -
 					get_sb(segment_count_main);

------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi

Jaegeuk Kim

author

2016-11-03 17:44 UTC #5 lore raw

Re: [PATCH 4/7] resize.f2fs: get latest free segments

Quoted 18 lines · heyunlei

Hi, Kim
Quoted 15 lines · Jaegeuk Kim
The # of free segments can be changed by defragmentation.
Signed-off-by Jaegeuk Kim <[email protected]>
---
 fsck/resize.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
fsck/resize.c +0 -0
index 5dc73a8..4813de7 100644
@@ -424,7 +424,7 @@ static void rebuild_checkpoint(struct f2fs_sb_info *sbi,
 	set_cp(overprov_segment_count, get_cp(overprov_segment_count) +
 						get_cp(rsvd_segment_count));
 

Quoted 5 lines

fsck/resize.c static void rebuild_checkpoint(struct f2fs_sb_info *sbi, · line 427
Quoted 2 lines
fsck/resize.c +1 -1
-	free_segment_count = get_cp(free_segment_count);
+	free_segment_count = get_free_segments(sbi);
f2fs_defragment will write a checkpoint and then update free_segment_count by get_free_segments(sbi)o<

Is it righto<

In f2fs_defragment(),
- flush_journal_entries()
- migrate_blocks()
- move_curseg_info()
- zero-journal_entries()
- write_curseg_info()
- flush_sit_entries()
- write_checkpoint()
- free_segment_count = get_free_segments()
- write cp pack

Since I can't catch your point, could you please explain in more detail?

Quoted 7 lines

Thankso<

Quoted 4 lines

fsck/resize.c +0 -0

 	new_segment_count = get_newsb(segment_count_main) -
 					get_sb(segment_count_main);

Jaegeuk Kim

author

2016-11-03 17:44 UTC #6 lore raw

Re: [PATCH 4/7] resize.f2fs: get latest free segments

In reply to heyunlei

Quoted 18 lines · heyunlei

Hi, Kim
Quoted 15 lines · Jaegeuk Kim
The # of free segments can be changed by defragmentation.
Signed-off-by Jaegeuk Kim <[email protected]>
---
 fsck/resize.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
fsck/resize.c +0 -0
index 5dc73a8..4813de7 100644
@@ -424,7 +424,7 @@ static void rebuild_checkpoint(struct f2fs_sb_info *sbi,
 	set_cp(overprov_segment_count, get_cp(overprov_segment_count) +
 						get_cp(rsvd_segment_count));
 

Quoted 5 lines

fsck/resize.c static void rebuild_checkpoint(struct f2fs_sb_info *sbi, · line 427
Quoted 2 lines
fsck/resize.c +1 -1
-	free_segment_count = get_cp(free_segment_count);
+	free_segment_count = get_free_segments(sbi);
f2fs_defragment will write a checkpoint and then update free_segment_count by get_free_segments(sbi)o<

Is it righto<

Oh, I misread your point.
Yup, but, this is to handle f2fs_defragment incompletion. In that case, resize.f2fs would call migrate_main() and finally rebuild_checkpoint() with wrong free_segments.

Quoted 7 lines

Thankso<

Quoted 4 lines

fsck/resize.c +0 -0

 	new_segment_count = get_newsb(segment_count_main) -
 					get_sb(segment_count_main);

Jaegeuk Kim

author

2016-11-03 05:29 UTC #7 lore raw

PATCH 5/7 sload.f2fs: not support to find or add files in inline_dir directory

In reply to Jaegeuk Kim

Now, sload.f2fs doesn't support inline_dir.

Signed-off-by Jaegeuk Kim <[email protected]>

---
 fsck/dir.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

fsck/dir.c +13 -0

index 18b79bd..2009855 100644
@@ -185,6 +185,12 @@ static int f2fs_find_entry(struct f2fs_sb_info *sbi,
 	unsigned int max_depth;
 	unsigned int level;
 
+	if (dir->i.i_inline & F2FS_INLINE_DENTRY) {
+		ERR_MSG("Not support to find \"%s\" in inline_dir pino=%x\n",
+				de->name, de->pino);
+		return 0;
+	}
+
 	max_depth = dir->i.i_current_depth;
 	for (level = 0; level < max_depth; level ++) {
 		if (find_in_level(sbi, dir, level, de))

@@ -494,6 +500,13 @@ int f2fs_create(struct f2fs_sb_info *sbi, struct dentry *de)
 			de->ino = 0;
 		goto free_parent_dir;
 	}
+	if (parent->i.i_inline & F2FS_INLINE_DENTRY) {
+		ERR_MSG("Not support adding \"%s\" in inline_dir pino=%x\n",
+					de->name, de->pino);
+		if (de->file_type == F2FS_FT_REG_FILE)
+			de->ino = 0;
+		goto free_parent_dir;
+	}
 
 	child = calloc(BLOCK_SZ, 1);
 	ASSERT(child);

Signature · 9 lines

2.8.3


------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi

Jaegeuk Kim

author

2016-11-03 05:29 UTC #8 lore raw

PATCH 6/7 f2fs-tools: determine cp_addr by sbi->cur_cp

In reply to Jaegeuk Kim

We don't guarantee cp_addr is fixed by cp_ver due to sload.

Signed-off-by Jaegeuk Kim <[email protected]>

---
 fsck/f2fs.h | 13 ++-----------
 1 file changed, 2 insertions(+), 11 deletions(-)

fsck/f2fs.h +2 ~~-11~~

index 39ff161..fbb878a 100644
@@ -280,19 +280,10 @@ static inline bool is_set_ckpt_flags(struct f2fs_checkpoint *cp, unsigned int f)
 
 static inline block_t __start_cp_addr(struct f2fs_sb_info *sbi)
 {
-	block_t start_addr;
-	struct f2fs_checkpoint *ckpt = F2FS_CKPT(sbi);
-	unsigned long long ckpt_version = le64_to_cpu(ckpt->checkpoint_ver);
-
-	start_addr = le32_to_cpu(F2FS_RAW_SUPER(sbi)->cp_blkaddr);
+	block_t start_addr = le32_to_cpu(F2FS_RAW_SUPER(sbi)->cp_blkaddr);
 
-	/*
-	 * odd numbered checkpoint should at cp segment 0
-	 * and even segent must be at cp segment 1
-	 */
-	if (!(ckpt_version & 1))
+	if (sbi->cur_cp == 2)
 		start_addr += sbi->blocks_per_seg;
-
 	return start_addr;
 }

Signature · 9 lines

2.8.3


------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi

Jaegeuk Kim

author

2016-11-03 05:29 UTC #9 lore raw

PATCH 7/7 resize.f2fs: fill zeros for expanded ssa area

In reply to Jaegeuk Kim

This will avoid wrong behavior after expanding volume.

Signed-off-by Jaegeuk Kim <[email protected]>

---
 fsck/resize.c | 31 ++++++++++++++++++++++---------
 1 file changed, 22 insertions(+), 9 deletions(-)

fsck/resize.c +22 -9

index 4813de7..7691524 100644
@@ -206,24 +206,37 @@ static void migrate_ssa(struct f2fs_sb_info *sbi,
 	struct f2fs_super_block *sb = F2FS_RAW_SUPER(sbi);
 	block_t old_sum_blkaddr = get_sb(ssa_blkaddr);
 	block_t new_sum_blkaddr = get_newsb(ssa_blkaddr);
-	unsigned int segno;
+	block_t end_sum_blkaddr = get_newsb(main_blkaddr);
+	block_t blkaddr;
+	void *zero_block = calloc(BLOCK_SZ, 1);
+
+	ASSERT(zero_block);
 
 	if (offset && new_sum_blkaddr < old_sum_blkaddr + offset) {
-		segno = offset;
-		while (segno != TOTAL_SEGS(sbi)) {
-			move_ssa(sbi, segno, new_sum_blkaddr + segno - offset);
-			segno++;
+		blkaddr = new_sum_blkaddr;
+		while (blkaddr < end_sum_blkaddr) {
+			if (blkaddr - new_sum_blkaddr < TOTAL_SEGS(sbi))
+				move_ssa(sbi, offset, blkaddr);
+			else
+				dev_write_block(zero_block, blkaddr);
+			offset++;
+			blkaddr++;
 		}
 	} else {
-		segno = TOTAL_SEGS(sbi) - 1;
-		while (segno != offset - 1) {
-			move_ssa(sbi, segno, new_sum_blkaddr + segno - offset);
-			segno--;
+		blkaddr = end_sum_blkaddr - 1;
+		offset = TOTAL_SEGS(sbi) - 1;
+		while (blkaddr >= new_sum_blkaddr) {
+			if (blkaddr >= TOTAL_SEGS(sbi) + new_sum_blkaddr)
+				dev_write_block(zero_block, blkaddr);
+			else
+				move_ssa(sbi, offset--, blkaddr);
+			blkaddr--;
 		}
 	}
 
 	DBG(0, "Info: Done to migrate SSA blocks: sum_blkaddr = 0x%x -> 0x%x\n",
 				old_sum_blkaddr, new_sum_blkaddr);
+	free(zero_block);
 }
 
 static int shrink_nats(struct f2fs_sb_info *sbi,

Signature · 9 lines

2.8.3


------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi