Skip to contentSkip to conversation

f2fs: fix to update user block counts in block_operations()

PATCH Superseded Chao Yu 2 participants 3 messages Jun 6, 2024 – Jun 25, 2024 F2FS file system

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Chao Yu

author maintainer
#0 lore raw

PATCH 1/1 f2fs: fix to update user block counts in block_operations()

Commit 59c9081bc86e ("f2fs: allow write page cache when writting cp") allows write() to write data to page cache during checkpoint, so block count fields like .total_valid_block_count, .alloc_valid_block_count and .rf_node_block_count may encounter race condition as below:

CP Thread A
- write_checkpoint
- block_operations
- f2fs_down_write(&sbi->node_change)
- __prepare_cp_block
: ckpt->valid_block_count = .total_valid_block_count
- f2fs_up_write(&sbi->node_change)
- write
- f2fs_preallocate_blocks
- f2fs_map_blocks(,F2FS_GET_BLOCK_PRE_AIO)
- f2fs_map_lock
- f2fs_down_read(&sbi->node_change)
- f2fs_reserve_new_blocks
- inc_valid_block_count
: percpu_counter_add(&sbi->alloc_valid_block_count, count) : sbi->total_valid_block_count += count
- f2fs_up_read(&sbi->node_change)
- do_checkpoint
: sbi->last_valid_block_count = sbi->total_valid_block_count
: percpu_counter_set(&sbi->alloc_valid_block_count, 0)
: percpu_counter_set(&sbi->rf_node_block_count, 0)
- fsync
- need_do_checkpoint
- f2fs_space_for_roll_forward
: alloc_valid_block_count was reset to zero,
so, it may missed last data during checkpoint

Let's change to update .total_valid_block_count, .alloc_valid_block_count and .rf_node_block_count in block_operations(), then their access can be protected by .node_change and .cp_rwsem lock, so that it can avoid above race condition.

Fixes 59c9081bc86e ("f2fs: allow write page cache when writting cp")
Cc Yunlei He <[email protected]>
---
 fs/f2fs/checkpoint.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)
fs/f2fs/checkpoint.c +6 -5
index 66eaad591b60..010bbd5af211 100644
@@ -1298,6 +1298,12 @@ static int block_operations(struct f2fs_sb_info *sbi)
 	 * dirty node blocks and some checkpoint values by block allocation.
 	 */
 	__prepare_cp_block(sbi);
+
+	/* update user_block_counts */
+	sbi->last_valid_block_count = sbi->total_valid_block_count;
+	percpu_counter_set(&sbi->alloc_valid_block_count, 0);
+	percpu_counter_set(&sbi->rf_node_block_count, 0);
+
 	f2fs_up_write(&sbi->node_change);
 	return err;
 }
@@ -1575,11 +1581,6 @@ static int do_checkpoint(struct f2fs_sb_info *sbi, struct cp_control *cpc)
 		start_blk += NR_CURSEG_NODE_TYPE;
 	}
 
-	/* update user_block_counts */
-	sbi->last_valid_block_count = sbi->total_valid_block_count;
-	percpu_counter_set(&sbi->alloc_valid_block_count, 0);
-	percpu_counter_set(&sbi->rf_node_block_count, 0);
-
 	/* Here, we have one bio having CP pack except cp pack 2 page */
 	f2fs_sync_meta_pages(sbi, META, LONG_MAX, FS_CP_META_IO);
 	/* Wait for all dirty meta pages to be submitted for IO */
Signature · 1 line
2.40.1
maintainer
#1 lore raw
Quoted 49 lines · Chao Yu

Commit 59c9081bc86e ("f2fs: allow write page cache when writting cp") allows write() to write data to page cache during checkpoint, so block count fields like .total_valid_block_count, .alloc_valid_block_count and .rf_node_block_count may encounter race condition as below:

CP Thread A
- write_checkpoint
- block_operations
- f2fs_down_write(&sbi->node_change)
- __prepare_cp_block
: ckpt->valid_block_count = .total_valid_block_count
- f2fs_up_write(&sbi->node_change)
- write
- f2fs_preallocate_blocks
- f2fs_map_blocks(,F2FS_GET_BLOCK_PRE_AIO)
- f2fs_map_lock
- f2fs_down_read(&sbi->node_change)
- f2fs_reserve_new_blocks
- inc_valid_block_count
: percpu_counter_add(&sbi->alloc_valid_block_count, count) : sbi->total_valid_block_count += count
- f2fs_up_read(&sbi->node_change)
- do_checkpoint
: sbi->last_valid_block_count = sbi->total_valid_block_count
: percpu_counter_set(&sbi->alloc_valid_block_count, 0)
: percpu_counter_set(&sbi->rf_node_block_count, 0)
- fsync
- need_do_checkpoint
- f2fs_space_for_roll_forward
: alloc_valid_block_count was reset to zero,
so, it may missed last data during checkpoint

Let's change to update .total_valid_block_count, .alloc_valid_block_count and .rf_node_block_count in block_operations(), then their access can be protected by .node_change and .cp_rwsem lock, so that it can avoid above race condition.

Fixes 59c9081bc86e ("f2fs: allow write page cache when writting cp")
Cc Yunlei He <[email protected]>
---
 fs/f2fs/checkpoint.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)
fs/f2fs/checkpoint.c +0 -0
index 66eaad591b60..010bbd5af211 100644

fs/f2fs/checkpoint.c static int block_operations(struct f2fs_sb_info *sbi) · line 1305

Quoted 8 lines
fs/f2fs/checkpoint.c +5 -0
@@ -1298,6 +1298,12 @@ static int block_operations(struct f2fs_sb_info *sbi)
 	 * dirty node blocks and some checkpoint values by block allocation.
 	 */
 	__prepare_cp_block(sbi);
+
+	/* update user_block_counts */
+	sbi->last_valid_block_count = sbi->total_valid_block_count;
+	percpu_counter_set(&sbi->alloc_valid_block_count, 0);
+	percpu_counter_set(&sbi->rf_node_block_count, 0);

Need to add this in __prepare_cp_block()?

Quoted 19 lines
fs/f2fs/checkpoint.c +1 -5
+
 	f2fs_up_write(&sbi->node_change);
 	return err;
 }
@@ -1575,11 +1581,6 @@ static int do_checkpoint(struct f2fs_sb_info *sbi, struct cp_control *cpc)
 		start_blk += NR_CURSEG_NODE_TYPE;
 	}
 
-	/* update user_block_counts */
-	sbi->last_valid_block_count = sbi->total_valid_block_count;
-	percpu_counter_set(&sbi->alloc_valid_block_count, 0);
-	percpu_counter_set(&sbi->rf_node_block_count, 0);
-
 	/* Here, we have one bio having CP pack except cp pack 2 page */
 	f2fs_sync_meta_pages(sbi, META, LONG_MAX, FS_CP_META_IO);
 	/* Wait for all dirty meta pages to be submitted for IO */
Signature · 1 line
2.40.1
author maintainer
#2 lore raw
Quoted 45 lines · Jaegeuk Kim
Quoted 44 lines · Chao Yu

Commit 59c9081bc86e ("f2fs: allow write page cache when writting cp") allows write() to write data to page cache during checkpoint, so block count fields like .total_valid_block_count, .alloc_valid_block_count and .rf_node_block_count may encounter race condition as below:

CP Thread A
- write_checkpoint
- block_operations
- f2fs_down_write(&sbi->node_change)
- __prepare_cp_block
: ckpt->valid_block_count = .total_valid_block_count
- f2fs_up_write(&sbi->node_change)
- write
- f2fs_preallocate_blocks
- f2fs_map_blocks(,F2FS_GET_BLOCK_PRE_AIO)
- f2fs_map_lock
- f2fs_down_read(&sbi->node_change)
- f2fs_reserve_new_blocks
- inc_valid_block_count
: percpu_counter_add(&sbi->alloc_valid_block_count, count) : sbi->total_valid_block_count += count
- f2fs_up_read(&sbi->node_change)
- do_checkpoint
: sbi->last_valid_block_count = sbi->total_valid_block_count
: percpu_counter_set(&sbi->alloc_valid_block_count, 0)
: percpu_counter_set(&sbi->rf_node_block_count, 0)
- fsync
- need_do_checkpoint
- f2fs_space_for_roll_forward
: alloc_valid_block_count was reset to zero,
so, it may missed last data during checkpoint

Let's change to update .total_valid_block_count, .alloc_valid_block_count and .rf_node_block_count in block_operations(), then their access can be protected by .node_change and .cp_rwsem lock, so that it can avoid above race condition.

Fixes 59c9081bc86e ("f2fs: allow write page cache when writting cp")
Cc Yunlei He <[email protected]>
---
  fs/f2fs/checkpoint.c | 11 ++++++-----
  1 file changed, 6 insertions(+), 5 deletions(-)
Quoted 15 lines

fs/f2fs/checkpoint.c static int block_operations(struct f2fs_sb_info *sbi) · line 1305

Quoted 13 lines
fs/f2fs/checkpoint.c +5 -0
index 66eaad591b60..010bbd5af211 100644
@@ -1298,6 +1298,12 @@ static int block_operations(struct f2fs_sb_info *sbi)
  	 * dirty node blocks and some checkpoint values by block allocation.
  	 */
  	__prepare_cp_block(sbi);
+
+	/* update user_block_counts */
+	sbi->last_valid_block_count = sbi->total_valid_block_count;
+	percpu_counter_set(&sbi->alloc_valid_block_count, 0);
+	percpu_counter_set(&sbi->rf_node_block_count, 0);

Need to add this in __prepare_cp_block()?

Fine to me, will update in v2.

Thanks,

Quoted 20 lines
Quoted 20 lines
fs/f2fs/checkpoint.c +1 -5
+
  	f2fs_up_write(&sbi->node_change);
  	return err;
  }
@@ -1575,11 +1581,6 @@ static int do_checkpoint(struct f2fs_sb_info *sbi, struct cp_control *cpc)
  		start_blk += NR_CURSEG_NODE_TYPE;
  	}
 
-	/* update user_block_counts */
-	sbi->last_valid_block_count = sbi->total_valid_block_count;
-	percpu_counter_set(&sbi->alloc_valid_block_count, 0);
-	percpu_counter_set(&sbi->rf_node_block_count, 0);
-
  	/* Here, we have one bio having CP pack except cp pack 2 page */
  	f2fs_sync_meta_pages(sbi, META, LONG_MAX, FS_CP_META_IO);
  	/* Wait for all dirty meta pages to be submitted for IO */
Signature · 1 line
2.40.1