f2fs: fix not to drop mount options when retrying fill_super
Subsystems (4)
Jaegeuk Kim
maintainerPATCH 1/5 f2fs: fix not to drop mount options when retrying fill_super
If wrong mount option was requested, f2fs tries to fill_super again. But, during the next trial, f2fs has no valid mount options, since parse_options deleted all the separators in the original string.
--- fs/f2fs/super.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-)
fs/f2fs/super.c +12 -3
@@ -950,6 +950,7 @@ static int f2fs_fill_super(struct super_block *sb, void *data, int silent)struct inode *root;long err = -EINVAL ;bool retry = true; +char *options =NULL ;int i; try_onemore:
@@ -981,9 +982,15 @@ try_onemore:set_opt (sbi,POSIX_ACL );#endif /* parse mount options */ - err =parse_options (sb, (char *)data); -if (err) + options =kstrdup ((const char *)data,GFP_KERNEL ); +if (data && !options) { + err = -ENOMEM ; goto free_sb_buf; + } + + err =parse_options (sb, options); +if (err) + goto free_options; sb->s_maxbytes =max_file_size (le32_to_cpu (raw_super->log_blocksize)); sb->s_max_links =F2FS_LINK_MAX ;
@@ -1027,7 +1034,7 @@ try_onemore:if (IS_ERR (sbi->meta_inode)) {f2fs_msg (sb,KERN_ERR ,"Failed to read F2FS meta data inode" ); err =PTR_ERR (sbi->meta_inode); - goto free_sb_buf; + goto free_options; } err =get_valid_checkpoint (sbi);
@@ -1176,6 +1183,8 @@ free_cp: free_meta_inode:make_bad_inode (sbi->meta_inode);iput (sbi->meta_inode); +free_options: +kfree (options); free_sb_buf:brelse (raw_super_buf); free_sbi:
Signature · 1 line
2.1.1
Jaegeuk Kim
maintainerPATCH 2/5 f2fs: support norecovery mount option
This patch adds a mount option, norecovery, which is mostly same as disable_roll_forward. The only difference is that norecovery should be activated with read-only mount option.
This can be used when user wants to check whether f2fs is mountable or not without any recovery process. (e.g., xfstests/200)
--- fs/f2fs/super.c | 8 ++++++++ 1 file changed, 8 insertions(+)
fs/f2fs/super.c +8 -0
@@ -42,6 +42,7 @@ static struct kset *f2fs_kset;enum { Opt_gc_background, Opt_disable_roll_forward, + Opt_norecovery, Opt_discard, Opt_noheap, Opt_user_xattr,
@@ -62,6 +63,7 @@ enum {static match_table_t f2fs_tokens = { {Opt_gc_background,"background_gc=%s" }, {Opt_disable_roll_forward,"disable_roll_forward" }, + {Opt_norecovery,"norecovery" }, {Opt_discard,"discard" }, {Opt_noheap,"no_heap" }, {Opt_user_xattr,"user_xattr" },
@@ -287,6 +289,12 @@ static int parse_options(struct super_block *sb, char *options)case Opt_disable_roll_forward:set_opt (sbi,DISABLE_ROLL_FORWARD );break ; +case Opt_norecovery: +/* this option mounts f2fs with ro */ +set_opt (sbi,DISABLE_ROLL_FORWARD ); +if (!f2fs_readonly (sb)) +return -EINVAL ; +break ;case Opt_discard:set_opt (sbi,DISCARD );break ;
Signature · 1 line
2.1.1
RE: [PATCH 2/5] f2fs: support norecovery mount option
Hi Jaegeuk,
Quoted 34 lines
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Jaegeuk Kim
Sent: Wednesday, January 28, 2015 7:32 AM
To: [email protected]; [email protected]; [email protected]
Cc: Jaegeuk Kim
Subject: [PATCH 2/5] f2fs: support norecovery mount optionThis patch adds a mount option, norecovery, which is mostly same as
disable_roll_forward. The only difference is that norecovery should be activated with read-only mount option.This can be used when user wants to check whether f2fs is mountable or not without any recovery process. (e.g., xfstests/200)
Signed-off-by Jaegeuk Kim <[email protected]>--- fs/f2fs/super.c | 8 ++++++++ 1 file changed, 8 insertions(+)fs/f2fs/super.c +1
-0@@ -42,6 +42,7 @@ static struct kset *f2fs_kset;enum { Opt_gc_background, Opt_disable_roll_forward, + Opt_norecovery, Opt_discard, Opt_noheap, Opt_user_xattr,
fs/f2fs/super.c enum { · line 66
Quoted 4 lines
fs/f2fs/super.c +1
-0@@ -62,6 +63,7 @@ enum {static match_table_t f2fs_tokens = { {Opt_gc_background,"background_gc=%s" }, {Opt_disable_roll_forward,"disable_roll_forward" }, + {Opt_norecovery,"norecovery" },
IMO, it's better to show 'norecovery' in f2fs_show_options instead of showing 'disable_roll_forward', so user will know the details of mount option as we show.
fs/f2fs/super.c static int parse_options(struct super_block *sb, char *options) · line 294
Quoted 10 lines
fs/f2fs/super.c +3
-0{Opt_discard, "discard"}, {Opt_noheap, "no_heap"}, {Opt_user_xattr, "user_xattr"},@@ -287,6 +289,12 @@ static int parse_options(struct super_block *sb, char *options)case Opt_disable_roll_forward:set_opt (sbi,DISABLE_ROLL_FORWARD );break ; +case Opt_norecovery: +/* this option mounts f2fs with ro */ +set_opt (sbi,DISABLE_ROLL_FORWARD );
set_opt(sbi, NORECOVERY); and check this option before recover_fsync_data()?
It's better to add description for this "norecovery" option in Documentation.
Thanks,
Quoted 13 lines
fs/f2fs/super.c +3
-1+if (!f2fs_readonly (sb)) +return -EINVAL ; +break ;case Opt_discard:set_opt (sbi,DISCARD );break ; --2.1.1
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Jaegeuk Kim
maintainerRe: [PATCH 2/5] f2fs: support norecovery mount option
Hi Chao,
Quoted 35 lines · Chao Yu
Hi Jaegeuk,
Quoted 33 lines
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Jaegeuk Kim
Sent: Wednesday, January 28, 2015 7:32 AM
To: [email protected]; [email protected]; [email protected]
Cc: Jaegeuk Kim
Subject: [PATCH 2/5] f2fs: support norecovery mount optionThis patch adds a mount option, norecovery, which is mostly same as
disable_roll_forward. The only difference is that norecovery should be activated with read-only mount option.This can be used when user wants to check whether f2fs is mountable or not without any recovery process. (e.g., xfstests/200)
Signed-off-by Jaegeuk Kim <[email protected]>--- fs/f2fs/super.c | 8 ++++++++ 1 file changed, 8 insertions(+)fs/f2fs/super.c +1
-0@@ -42,6 +42,7 @@ static struct kset *f2fs_kset;enum { Opt_gc_background, Opt_disable_roll_forward, + Opt_norecovery, Opt_discard, Opt_noheap, Opt_user_xattr,
Quoted 8 lines
fs/f2fs/super.c enum { · line 66
Quoted 5 lines
fs/f2fs/super.c +1
-0@@ -62,6 +63,7 @@ enum {static match_table_t f2fs_tokens = { {Opt_gc_background,"background_gc=%s" }, {Opt_disable_roll_forward,"disable_roll_forward" }, + {Opt_norecovery,"norecovery" },IMO, it's better to show 'norecovery' in
f2fs_show_optionsinstead of showing 'disable_roll_forward', so user will know the details of mount option as we show.
Hmm, I did think like that at the first time.
But, as you guess, this option is totally same as ro,disable_roll_forward. So, I just wanted to avoid user's confusing for the behaviors of this option.
Quoted 14 lines
fs/f2fs/super.c static int parse_options(struct super_block *sb, char *options) · line 294
Quoted 11 lines
fs/f2fs/super.c +3
-0{Opt_discard, "discard"}, {Opt_noheap, "no_heap"}, {Opt_user_xattr, "user_xattr"},@@ -287,6 +289,12 @@ static int parse_options(struct super_block *sb, char *options)case Opt_disable_roll_forward:set_opt (sbi,DISABLE_ROLL_FORWARD );break ; +case Opt_norecovery: +/* this option mounts f2fs with ro */ +set_opt (sbi,DISABLE_ROLL_FORWARD );
set_opt(sbi, NORECOVERY); and check this option beforerecover_fsync_data()?
Quoted 1 line
It's better to add description for this "norecovery" option in Documentation.
I'll do that.
Thanks,
Quoted 16 lines
Thanks,
Quoted 13 lines
fs/f2fs/super.c +3
-1+if (!f2fs_readonly (sb)) +return -EINVAL ; +break ;case Opt_discard:set_opt (sbi,DISCARD );break ; --2.1.1
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Jaegeuk Kim
maintainerRe: [f2fs-dev] [PATCH 2/5 v2] f2fs: support norecovery mount option
Change log from v1:
o add description for the new mount option in Documentation/filesystems/f2fs.txt
Quoted 1 line
From
079dbb14c7d91d90863c9be4d9337b8ec086db7eMon Sep 17 00:00:00 2001
From: Jaegeuk Kim <[email protected]>
Date: Fri, 23 Jan 2015 18:33:46 -0800
Subject: [PATCH] f2fs: support norecovery mount option
This patch adds a mount option, norecovery, which is mostly same as disable_roll_forward. The only difference is that norecovery should be activated with read-only mount option.
This can be used when user wants to check whether f2fs is mountable or not without any recovery process. (e.g., xfstests/200)
--- Documentation/filesystems/f2fs.txt | 2 ++ fs/f2fs/super.c | 8 ++++++++ 2 files changed, 10 insertions(+)
Documentation/filesystems/f2fs.txt +2 -0
@@ -106,6 +106,8 @@ background_gc=%s Turn on/off cleaning operations, namely garbage Default value for this option is on. So garbage collection is on by default. disable_roll_forward Disable the roll-forward recovery routine +norecovery Disable the roll-forward recovery routine, mounted read- + only (i.e., -o ro,disable_roll_forward) discard Issue discard/TRIM commands when a segment is cleaned. no_heap Disable heap-style segment allocation which finds free segments for data from the beginning of main area, while
fs/f2fs/super.c +8 -0
@@ -42,6 +42,7 @@ static struct kset *f2fs_kset;enum { Opt_gc_background, Opt_disable_roll_forward, + Opt_norecovery, Opt_discard, Opt_noheap, Opt_user_xattr,
@@ -62,6 +63,7 @@ enum {static match_table_t f2fs_tokens = { {Opt_gc_background,"background_gc=%s" }, {Opt_disable_roll_forward,"disable_roll_forward" }, + {Opt_norecovery,"norecovery" }, {Opt_discard,"discard" }, {Opt_noheap,"no_heap" }, {Opt_user_xattr,"user_xattr" },
@@ -287,6 +289,12 @@ static int parse_options(struct super_block *sb, char *options)case Opt_disable_roll_forward:set_opt (sbi,DISABLE_ROLL_FORWARD );break ; +case Opt_norecovery: +/* this option mounts f2fs with ro */ +set_opt (sbi,DISABLE_ROLL_FORWARD ); +if (!f2fs_readonly (sb)) +return -EINVAL ; +break ;case Opt_discard:set_opt (sbi,DISCARD );break ;
Signature · 1 line
2.1.1
Chao Yu
Reviewed-byRE: [f2fs-dev] [PATCH 2/5 v2] f2fs: support norecovery mount option
Quoted 16 lines
-----Original Message-----
From: Jaegeuk Kim [mailto:[email protected]]
Sent: Friday, January 30, 2015 2:31 AM
To: Chao Yu
Cc: [email protected]; [email protected]; [email protected]
Subject: Re: [f2fs-dev] [PATCH 2/5 v2] f2fs: support norecovery mount optionChange log from v1:
o add description for the new mount option inDocumentation/filesystems/f2fs.txtFrom
079dbb14c7d91d90863c9be4d9337b8ec086db7eMon Sep 17 00:00:00 2001
From: Jaegeuk Kim <[email protected]>
Date: Fri, 23 Jan 2015 18:33:46 -0800
Subject: [PATCH] f2fs: support norecovery mount option
Quoted 8 lines
This patch adds a mount option, norecovery, which is mostly same as
disable_roll_forward. The only difference is that norecovery should be activated with read-only mount option.This can be used when user wants to check whether f2fs is mountable or not without any recovery process. (e.g., xfstests/200)
Signed-off-by Jaegeuk Kim <[email protected]>
RE: [PATCH 2/5] f2fs: support norecovery mount option
In reply to Jaegeuk KimHi Jaegeuk,
Quoted 55 lines
-----Original Message-----
From: Jaegeuk Kim [mailto:[email protected]]
Sent: Friday, January 30, 2015 2:28 AM
To: Chao Yu
Cc: [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH 2/5] f2fs: support norecovery mount optionHi Chao,
Quoted 35 lines · Chao Yu
Hi Jaegeuk,
Quoted 33 lines
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Jaegeuk Kim
Sent: Wednesday, January 28, 2015 7:32 AM
To: [email protected]; [email protected]; [email protected]
Cc: Jaegeuk Kim
Subject: [PATCH 2/5] f2fs: support norecovery mount optionThis patch adds a mount option, norecovery, which is mostly same as
disable_roll_forward. The only difference is that norecovery should be activated with read-only mount option.This can be used when user wants to check whether f2fs is mountable or not without any recovery process. (e.g., xfstests/200)
Signed-off-by Jaegeuk Kim <[email protected]>--- fs/f2fs/super.c | 8 ++++++++ 1 file changed, 8 insertions(+)fs/f2fs/super.c +1
-0@@ -42,6 +42,7 @@ static struct kset *f2fs_kset;enum { Opt_gc_background, Opt_disable_roll_forward, + Opt_norecovery, Opt_discard, Opt_noheap, Opt_user_xattr,Quoted 8 lines
fs/f2fs/super.c enum { · line 66
Quoted 5 lines
fs/f2fs/super.c +1
-0@@ -62,6 +63,7 @@ enum {static match_table_t f2fs_tokens = { {Opt_gc_background,"background_gc=%s" }, {Opt_disable_roll_forward,"disable_roll_forward" }, + {Opt_norecovery,"norecovery" },IMO, it's better to show 'norecovery' in
f2fs_show_optionsinstead of showing 'disable_roll_forward', so user will know the details of mount option as we show.
Quoted 3 lines
Hmm, I did think like that at the first time.
But, as you guess, this option is totally same as ro,disable_roll_forward. So, I just wanted to avoid user's confusing for the behaviors of this option.
All right, I just worry about that if users could not find the key word 'norecovery' in /etc/mtab as they think it should be there because they mount with it before. Not sure this will happen.
Quoted 15 lines
Quoted 15 lines
fs/f2fs/super.c static int parse_options(struct super_block *sb, char *options) · line 294
Quoted 12 lines
fs/f2fs/super.c +3
-0{Opt_discard, "discard"}, {Opt_noheap, "no_heap"}, {Opt_user_xattr, "user_xattr"},@@ -287,6 +289,12 @@ static int parse_options(struct super_block *sb, char *options)case Opt_disable_roll_forward:set_opt (sbi,DISABLE_ROLL_FORWARD );break ; +case Opt_norecovery: +/* this option mounts f2fs with ro */ +set_opt (sbi,DISABLE_ROLL_FORWARD );
set_opt(sbi, NORECOVERY); and check this option beforerecover_fsync_data()?
Quoted 3 lines
Quoted 1 line
It's better to add description for this "norecovery" option in Documentation.
I'll do that.
It will really help our users.
Thank you. :)
Regards,
Quoted 19 lines
Thanks,
Quoted 16 lines
Thanks,
Quoted 13 lines
fs/f2fs/super.c +3
-1+if (!f2fs_readonly (sb)) +return -EINVAL ; +break ;case Opt_discard:set_opt (sbi,DISCARD );break ; --2.1.1
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Jaegeuk Kim
maintainerPATCH 3/5 f2fs: avoid write_checkpoint if f2fs is mounted readonly
In reply to Jaegeuk KimDo not change any partition when f2fs is changed to readonly mode.
--- fs/f2fs/checkpoint.c | 2 ++ 1 file changed, 2 insertions(+)
fs/f2fs/checkpoint.c +2 -0
@@ -1067,6 +1067,8 @@ void write_checkpoint(struct f2fs_sb_info *sbi, struct cp_control *cpc) goto out;if (unlikely (f2fs_cp_error (sbi))) goto out; +if (f2fs_readonly (sbi->sb)) + goto out;if (block_operations (sbi)) goto out;
Signature · 9 lines
2.1.1 ------------------------------------------------------------------------------ Dive into the World of Parallel Programming. The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/
RE: [PATCH 3/5] f2fs: avoid write_checkpoint if f2fs is mounted readonly
Quoted 9 lines
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Jaegeuk Kim
Sent: Wednesday, January 28, 2015 7:32 AM
To: [email protected]; [email protected]; [email protected]
Cc: Jaegeuk Kim
Subject: [PATCH 3/5] f2fs: avoidwrite_checkpointif f2fs is mounted readonly
Quoted 3 lines
Do not change any partition when f2fs is changed to readonly mode.
Signed-off-by Jaegeuk Kim <[email protected]>
Jaegeuk Kim
maintainerPATCH 4/5 f2fs: should fail mount when trying to recover data on read-only dev
In reply to Jaegeuk KimIf device is read-only, we should not proceed data recovery.
But, if the previous checkpoint was done by normal clean shutdown, it's safe to proceed the recovery, since there will be no data to be recovered.
--- fs/f2fs/super.c | 9 +++++++++ 1 file changed, 9 insertions(+)
fs/f2fs/super.c +9 -0
@@ -1149,6 +1149,15 @@ try_onemore:/* recover fsynced data */ if (!test_opt (sbi,DISABLE_ROLL_FORWARD )) { +/* +* mount should be failed, when device has readonly mode, and +* previous checkpoint was not done by clean system shutdown. +*/ +if (bdev_read_only (sb->s_bdev) && + !is_set_ckpt_flags (sbi->ckpt,CP_UMOUNT_FLAG )) { + err = -EROFS ; + goto free_kobj; + } err =recover_fsync_data (sbi);if (err) { f2fs_msg(sb, KERN_ERR,
Signature · 9 lines
2.1.1 ------------------------------------------------------------------------------ Dive into the World of Parallel Programming. The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/
RE: [PATCH 4/5] f2fs: should fail mount when trying to recover data on read-only dev
Hi Jaegeuk,
Quoted 9 lines
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Jaegeuk Kim
Sent: Wednesday, January 28, 2015 7:32 AM
To: [email protected]; [email protected]; [email protected]
Cc: Jaegeuk Kim
Subject: [PATCH 4/5] f2fs: should fail mount when trying to recover data on read-only dev
Quoted 3 lines
If device is read-only, we should not proceed data recovery.
But, if the previous checkpoint was done by normal clean shutdown, it's safe to proceed the recovery, since there will be no data to be recovered.
Now, with 'fastboot' option, f2fs will write cp with CP_UMOUNT_FLAG, but we count not guarantee there is no abnormally power cut after data fsynced.
Thanks,
Quoted 33 lines
Signed-off-by Jaegeuk Kim <[email protected]>--- fs/f2fs/super.c | 9 +++++++++ 1 file changed, 9 insertions(+)fs/f2fs/super.c +9
-1@@ -1149,6 +1149,15 @@ try_onemore:/* recover fsynced data */ if (!test_opt (sbi,DISABLE_ROLL_FORWARD )) { +/* +* mount should be failed, when device has readonly mode, and +* previous checkpoint was not done by clean system shutdown. +*/ +if (bdev_read_only (sb->s_bdev) && + !is_set_ckpt_flags (sbi->ckpt,CP_UMOUNT_FLAG )) { + err = -EROFS ; + goto free_kobj; + } err =recover_fsync_data (sbi);if (err) { f2fs_msg(sb, KERN_ERR, --2.1.1
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Jaegeuk Kim
maintainerRe: [PATCH 4/5] f2fs: should fail mount when trying to recover data on read-only dev
Hi Chao,
This is another patch to fix that.
Quoted 1 line
From
d241924043778d0fe01e9020d5771cc42cf246e6Mon Sep 17 00:00:00 2001
From: Jaegeuk Kim <[email protected]>
Date: Thu, 29 Jan 2015 11:45:33 -0800
Subject: [PATCH] f2fs: split UMOUNT and FASTBOOT flags
This patch adds FASTBOOT flag into checkpoint as follows.
- CP_UMOUNT_FLAG is set when system is umounted.
- CP_FASTBOOT_FLAG is set when intermediate checkpoint having node summaries was done.
So, if you get CP_UMOUNT_FLAG from checkpoint, the system was umounted cleanly. Instead, if there was sudden-power-off, you can get CP_FASTBOOT_FLAG or nothing.
--- fs/f2fs/checkpoint.c | 19 +++++++++++++------ fs/f2fs/f2fs.h | 23 +++++++++++++++++++++++ fs/f2fs/gc.c | 3 +-- fs/f2fs/segment.c | 11 +++++------ fs/f2fs/super.c | 5 ++--- include/linux/f2fs_fs.h | 1 + include/trace/events/f2fs.h | 1 + 7 files changed, 46 insertions(+), 17 deletions(-)
fs/f2fs/checkpoint.c +13 -6
@@ -956,17 +956,24 @@ static void do_checkpoint(struct f2fs_sb_info *sbi, struct cp_control *cpc) ckpt->cp_pack_start_sum =cpu_to_le32 (1 + cp_payload_blks + orphan_blocks); -if (cpc->reason ==CP_UMOUNT ) { -set_ckpt_flags (ckpt,CP_UMOUNT_FLAG ); +if (__remain_node_summaries (cpc->reason)) ckpt->cp_pack_total_block_count =cpu_to_le32 (F2FS_CP_PACKS + cp_payload_blks + data_sum_blocks + orphan_blocks +NR_CURSEG_NODE_TYPE ); - }else { -clear_ckpt_flags (ckpt,CP_UMOUNT_FLAG ); +else ckpt->cp_pack_total_block_count =cpu_to_le32 (F2FS_CP_PACKS + cp_payload_blks + data_sum_blocks + orphan_blocks); - } + +if (cpc->reason ==CP_UMOUNT ) +set_ckpt_flags (ckpt,CP_UMOUNT_FLAG ); +else +clear_ckpt_flags (ckpt,CP_UMOUNT_FLAG ); + +if (cpc->reason ==CP_FASTBOOT ) +set_ckpt_flags (ckpt,CP_FASTBOOT_FLAG ); +else +clear_ckpt_flags (ckpt,CP_FASTBOOT_FLAG );if (orphan_num)set_ckpt_flags (ckpt,CP_ORPHAN_PRESENT_FLAG );
@@ -1010,7 +1017,7 @@ static void do_checkpoint(struct f2fs_sb_info *sbi, struct cp_control *cpc)write_data_summaries (sbi, start_blk); start_blk += data_sum_blocks; -if (cpc->reason ==CP_UMOUNT ) { +if (__remain_node_summaries (cpc->reason)) {write_node_summaries (sbi, start_blk); start_blk +=NR_CURSEG_NODE_TYPE ; }
fs/f2fs/f2fs.h +23 -0
@@ -100,6 +100,7 @@ enum {enum {CP_UMOUNT , +CP_FASTBOOT ,CP_SYNC ,CP_DISCARD , };
@@ -788,6 +789,28 @@ static inline void f2fs_unlock_all(struct f2fs_sb_info *sbi)up_write (&sbi->cp_rwsem); } +static inline int __get_cp_reason (struct f2fs_sb_info *sbi) +{ +int reason =CP_SYNC ; + +if (test_opt (sbi,FASTBOOT )) + reason =CP_FASTBOOT ; +if (is_sbi_flag_set (sbi,SBI_IS_CLOSE )) + reason =CP_UMOUNT ; +return reason; +} + +static inline bool __remain_node_summaries (int reason) +{ +return (reason ==CP_UMOUNT || reason ==CP_FASTBOOT ); +} + +static inline bool __exist_node_summaries (struct f2fs_sb_info *sbi) +{ +return (is_set_ckpt_flags (F2FS_CKPT (sbi),CP_UMOUNT_FLAG ) || +is_set_ckpt_flags (F2FS_CKPT (sbi),CP_FASTBOOT_FLAG )); +} +/* * Check whether the given nid is within node id range. */
fs/f2fs/gc.c +1 -2
@@ -698,8 +698,7 @@ int f2fs_gc(struct f2fs_sb_info *sbi) .iroot = RADIX_TREE_INIT(GFP_NOFS), }; - cpc.reason =test_opt (sbi,FASTBOOT ) ?CP_UMOUNT :CP_SYNC ; - + cpc.reason =__get_cp_reason (sbi); gc_more:if (unlikely (!(sbi->sb->s_flags &MS_ACTIVE ))) goto stop;
fs/f2fs/segment.c +5 -6
@@ -1401,7 +1401,7 @@ static int read_normal_summaries(struct f2fs_sb_info *sbi, int type) segno =le32_to_cpu (ckpt->cur_data_segno[type]); blk_off =le16_to_cpu (ckpt->cur_data_blkoff[type -CURSEG_HOT_DATA ]); -if (is_set_ckpt_flags (ckpt,CP_UMOUNT_FLAG )) +if (__exist_node_summaries (sbi)) blk_addr =sum_blk_addr (sbi,NR_CURSEG_TYPE , type);else blk_addr =sum_blk_addr (sbi,NR_CURSEG_DATA_TYPE , type);
@@ -1410,7 +1410,7 @@ static int read_normal_summaries(struct f2fs_sb_info *sbi, int type) CURSEG_HOT_NODE]); blk_off =le16_to_cpu (ckpt->cur_node_blkoff[type -CURSEG_HOT_NODE ]); -if (is_set_ckpt_flags (ckpt,CP_UMOUNT_FLAG )) +if (__exist_node_summaries (sbi)) blk_addr =sum_blk_addr (sbi,NR_CURSEG_NODE_TYPE , type -CURSEG_HOT_NODE );else
@@ -1421,7 +1421,7 @@ static int read_normal_summaries(struct f2fs_sb_info *sbi, int type) sum = (struct f2fs_summary_block *)page_address (new);if (IS_NODESEG (type)) { -if (is_set_ckpt_flags (ckpt,CP_UMOUNT_FLAG )) { +if (__exist_node_summaries (sbi)) {struct f2fs_summary *ns = &sum->entries[0 ];int i;for (i =0 ; i < sbi->blocks_per_seg; i++, ns++) {
@@ -1470,7 +1470,7 @@ static int restore_curseg_summaries(struct f2fs_sb_info *sbi) type =CURSEG_HOT_NODE ; } -if (is_set_ckpt_flags (F2FS_CKPT (sbi),CP_UMOUNT_FLAG )) +if (__exist_node_summaries (sbi))ra_meta_pages (sbi,sum_blk_addr (sbi,NR_CURSEG_TYPE , type),NR_CURSEG_TYPE - type,META_CP );
@@ -1567,8 +1567,7 @@ void write_data_summaries(struct f2fs_sb_info *sbi, block_t start_blk)void write_node_summaries (struct f2fs_sb_info *sbi,block_t start_blk) { -if (is_set_ckpt_flags (F2FS_CKPT (sbi),CP_UMOUNT_FLAG )) -write_normal_summaries (sbi, start_blk,CURSEG_HOT_NODE ); +write_normal_summaries (sbi, start_blk,CURSEG_HOT_NODE ); } int lookup_journal_in_cursum(struct f2fs_summary_block *sum, int type,
fs/f2fs/super.c +2 -3
@@ -500,9 +500,8 @@ int f2fs_sync_fs(struct super_block *sb, int sync)if (sync) {struct cp_control cpc; - cpc.reason = (test_opt (sbi,FASTBOOT ) || -is_sbi_flag_set (sbi,SBI_IS_CLOSE )) ? -CP_UMOUNT :CP_SYNC ; + cpc.reason =__get_cp_reason (sbi); +mutex_lock (&sbi->gc_mutex);write_checkpoint (sbi, &cpc);mutex_unlock (&sbi->gc_mutex);
include/linux/f2fs_fs.h +1 -0
@@ -87,6 +87,7 @@ struct f2fs_super_block {/* * For checkpoint */ +#define CP_FASTBOOT_FLAG 0x00000020#define CP_FSCK_FLAG 0x00000010#define CP_ERROR_FLAG 0x00000008#define CP_COMPACT_SUM_FLAG 0x00000004
include/trace/events/f2fs.h +1 -0
@@ -72,6 +72,7 @@#define show_cpreason (type) \ __print_symbolic(type, \ { CP_UMOUNT, "Umount" }, \ + { CP_FASTBOOT, "Fastboot" }, \ { CP_SYNC, "Sync" }, \ { CP_DISCARD, "Discard" })
Signature · 1 line
2.1.1
Chao Yu
Reviewed-byRE: [PATCH 4/5] f2fs: should fail mount when trying to recover data on read-only dev
Hi Jaegeuk,
Quoted 17 lines
-----Original Message-----
From: Jaegeuk Kim [mailto:[email protected]]
Sent: Friday, January 30, 2015 5:40 AM
To: Chao Yu
Cc: [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH 4/5] f2fs: should fail mount when trying to recover data on read-only devHi Chao,
This is another patch to fix that.
From
d241924043778d0fe01e9020d5771cc42cf246e6Mon Sep 17 00:00:00 2001
From: Jaegeuk Kim <[email protected]>
Date: Thu, 29 Jan 2015 11:45:33 -0800
Subject: [PATCH] f2fs: split UMOUNT and FASTBOOT flags
Quoted 8 lines
This patch adds FASTBOOT flag into checkpoint as follows.
-
CP_UMOUNT_FLAGis set when system is umounted.
-CP_FASTBOOT_FLAGis set when intermediate checkpoint having node summaries was done.So, if you get
CP_UMOUNT_FLAGfrom checkpoint, the system was umounted cleanly. Instead, if there was sudden-power-off, you can getCP_FASTBOOT_FLAGor nothing.
Nice work, it looks good to me, and this can fix that issue. :)
Quoted 2 lines
Signed-off-by Jaegeuk Kim <[email protected]>
RE: [PATCH 4/5] f2fs: should fail mount when trying to recover data on read-only dev
In reply to Jaegeuk KimQuoted 9 lines
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Jaegeuk Kim
Sent: Wednesday, January 28, 2015 7:32 AM
To: [email protected]; [email protected]; [email protected]
Cc: Jaegeuk Kim
Subject: [PATCH 4/5] f2fs: should fail mount when trying to recover data on read-only dev
Quoted 5 lines
If device is read-only, we should not proceed data recovery.
But, if the previous checkpoint was done by normal clean shutdown, it's safe to proceed the recovery, since there will be no data to be recovered.Signed-off-by Jaegeuk Kim <[email protected]>
Jaegeuk Kim
maintainerPATCH 5/5 f2fs: introduce a batched trim
In reply to Jaegeuk KimThis patch introduces a batched trimming feature, which submits split discard commands.
--- fs/f2fs/f2fs.h | 1 + fs/f2fs/segment.c | 15 ++++++++++----- 2 files changed, 11 insertions(+), 5 deletions(-)
fs/f2fs/f2fs.h +1 -0
@@ -104,6 +104,7 @@ enum { CP_DISCARD, }; +#define BATCHED_TRIM_SEGMENTS 10struct cp_control {int reason;__u64 trim_start;
fs/f2fs/segment.c +10 -5
@@ -1066,14 +1066,19 @@ int f2fs_trim_fs(struct f2fs_sb_info *sbi, struct fstrim_range *range) end_segno = (end >=MAX_BLKADDR (sbi)) ?MAIN_SEGS (sbi) -1 :GET_SEGNO (sbi, end); cpc.reason =CP_DISCARD ; - cpc.trim_start = start_segno; - cpc.trim_end = end_segno; cpc.trim_minlen = range->minlen >> sbi->log_blocksize;/* do checkpoint to issue discard commands safely */ -mutex_lock (&sbi->gc_mutex); -write_checkpoint (sbi, &cpc); -mutex_unlock (&sbi->gc_mutex); +for (; start_segno <= end_segno; + start_segno +=BATCHED_TRIM_SEGMENTS +1 ) { + cpc.trim_start = start_segno; + cpc.trim_end = min_t(unsigned int, + start_segno +BATCHED_TRIM_SEGMENTS , end_segno); + +mutex_lock (&sbi->gc_mutex); +write_checkpoint (sbi, &cpc); +mutex_unlock (&sbi->gc_mutex); + } out: range->len = cpc.trimmed << sbi->log_blocksize;return 0 ;
Signature · 9 lines
2.1.1 ------------------------------------------------------------------------------ Dive into the World of Parallel Programming. The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/
RE: [PATCH 5/5] f2fs: introduce a batched trim
Hi Jaegeuk,
Quoted 11 lines
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Jaegeuk Kim
Sent: Wednesday, January 28, 2015 7:32 AM
To: [email protected]; [email protected]; [email protected]
Cc: Jaegeuk Kim
Subject: [PATCH 5/5] f2fs: introduce a batched trimThis patch introduces a batched trimming feature, which submits split discard commands.
I didn't get it, why we should split discard commands. :(
Does smaller discarding for flash shows better performance or effect or safety? Can you please explain more about this patch?
Thanks,
Quoted 55 lines
Signed-off-by Jaegeuk Kim <[email protected]>--- fs/f2fs/f2fs.h | 1 + fs/f2fs/segment.c | 15 ++++++++++----- 2 files changed, 11 insertions(+), 5 deletions(-)fs/f2fs/f2fs.h +1
-0@@ -104,6 +104,7 @@ enum { CP_DISCARD, }; +#define BATCHED_TRIM_SEGMENTS 10struct cp_control {int reason;__u64 trim_start;fs/f2fs/segment.c +10
-6@@ -1066,14 +1066,19 @@ int f2fs_trim_fs(struct f2fs_sb_info *sbi, struct fstrim_range *range) end_segno = (end >=MAX_BLKADDR (sbi)) ?MAIN_SEGS (sbi) -1 :GET_SEGNO (sbi, end); cpc.reason =CP_DISCARD ; - cpc.trim_start = start_segno; - cpc.trim_end = end_segno; cpc.trim_minlen = range->minlen >> sbi->log_blocksize;/* do checkpoint to issue discard commands safely */ -mutex_lock (&sbi->gc_mutex); -write_checkpoint (sbi, &cpc); -mutex_unlock (&sbi->gc_mutex); +for (; start_segno <= end_segno; + start_segno +=BATCHED_TRIM_SEGMENTS +1 ) { + cpc.trim_start = start_segno; + cpc.trim_end = min_t(unsigned int, + start_segno +BATCHED_TRIM_SEGMENTS , end_segno); + +mutex_lock (&sbi->gc_mutex); +write_checkpoint (sbi, &cpc); +mutex_unlock (&sbi->gc_mutex); + } out: range->len = cpc.trimmed << sbi->log_blocksize;return 0 ; --2.1.1
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Jaegeuk Kim
maintainerRe: [PATCH 5/5] f2fs: introduce a batched trim
Hi Chao,
Quoted 11 lines · Chao Yu
Hi Jaegeuk,
Quoted 9 lines
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Jaegeuk Kim
Sent: Wednesday, January 28, 2015 7:32 AM
To: [email protected]; [email protected]; [email protected]
Cc: Jaegeuk Kim
Subject: [PATCH 5/5] f2fs: introduce a batched trim
Quoted 7 lines
Quoted 2 lines
This patch introduces a batched trimming feature, which submits split discard commands.
I didn't get it, why we should split discard commands. :(
Does smaller discarding for flash shows better performance or effect or safety? Can you please explain more about this patch?
This is to avoid long latency due to huge trim commands.
If fstrim was triggered ranging from 0 to the end of device, we should lock all the checkpoint-related mutexes, resulting in very long latency.
Thanks,
Quoted 58 lines
Thanks,
Quoted 55 lines
Signed-off-by Jaegeuk Kim <[email protected]>--- fs/f2fs/f2fs.h | 1 + fs/f2fs/segment.c | 15 ++++++++++----- 2 files changed, 11 insertions(+), 5 deletions(-)fs/f2fs/f2fs.h +1
-0@@ -104,6 +104,7 @@ enum { CP_DISCARD, }; +#define BATCHED_TRIM_SEGMENTS 10struct cp_control {int reason;__u64 trim_start;fs/f2fs/segment.c +10
-6@@ -1066,14 +1066,19 @@ int f2fs_trim_fs(struct f2fs_sb_info *sbi, struct fstrim_range *range) end_segno = (end >=MAX_BLKADDR (sbi)) ?MAIN_SEGS (sbi) -1 :GET_SEGNO (sbi, end); cpc.reason =CP_DISCARD ; - cpc.trim_start = start_segno; - cpc.trim_end = end_segno; cpc.trim_minlen = range->minlen >> sbi->log_blocksize;/* do checkpoint to issue discard commands safely */ -mutex_lock (&sbi->gc_mutex); -write_checkpoint (sbi, &cpc); -mutex_unlock (&sbi->gc_mutex); +for (; start_segno <= end_segno; + start_segno +=BATCHED_TRIM_SEGMENTS +1 ) { + cpc.trim_start = start_segno; + cpc.trim_end = min_t(unsigned int, + start_segno +BATCHED_TRIM_SEGMENTS , end_segno); + +mutex_lock (&sbi->gc_mutex); +write_checkpoint (sbi, &cpc); +mutex_unlock (&sbi->gc_mutex); + } out: range->len = cpc.trimmed << sbi->log_blocksize;return 0 ; --2.1.1
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH 5/5] f2fs: introduce a batched trim
Hi Jaegeuk,
Quoted 25 lines
-----Original Message-----
From: Jaegeuk Kim [mailto:[email protected]]
Sent: Friday, January 30, 2015 5:41 AM
To: Chao Yu
Cc: [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH 5/5] f2fs: introduce a batched trimHi Chao,
Quoted 14 lines · Chao Yu
Hi Jaegeuk,
Quoted 11 lines
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Jaegeuk Kim
Sent: Wednesday, January 28, 2015 7:32 AM
To: [email protected]; [email protected]; [email protected]
Cc: Jaegeuk Kim
Subject: [PATCH 5/5] f2fs: introduce a batched trimThis patch introduces a batched trimming feature, which submits split discard commands.
Quoted 8 lines
Quoted 4 lines
I didn't get it, why we should split discard commands. :(
Does smaller discarding for flash shows better performance or effect or safety? Can you please explain more about this patch?
This is to avoid long latency due to huge trim commands.
If fstrim was triggered ranging from 0 to the end of device, we should lock all the checkpoint-related mutexes, resulting in very long latency.
Ah.. thanks for your explanation, how about adding above description into commit log?
Quoted 21 lines
Thanks,
Quoted 18 lines
Thanks,
Quoted 15 lines
Signed-off-by Jaegeuk Kim <[email protected]>--- fs/f2fs/f2fs.h | 1 + fs/f2fs/segment.c | 15 ++++++++++----- 2 files changed, 11 insertions(+), 5 deletions(-)fs/f2fs/f2fs.h +0
-0@@ -104,6 +104,7 @@ enum { CP_DISCARD, };
Quoted 1 line
Quoted 1 line
fs/f2fs/f2fs.h enum { · line 107
Quoted 1 line
fs/f2fs/f2fs.h +1
-0+#define BATCHED_TRIM_SEGMENTS 10
Our macro here shows one batched trim discard 10 segments, but actually we discard 11 segments each time in f2fs_trim_fs, how about making them consistent?
One other point is that, I guess gc in FTL will be triggered each time after discard command was issued, but if our basic size of one batched trim is not multiple of gc unit size in FTL, efficiency of gc in FTL will be lower.
Now that f2fs has nature advantage that we can align section size to gc unit size in FTL. Why not trying to op flash with section size as much as possible?
So how about use this:
#define BATCHED_TRIM_SECTIONS_SHIFT 4
our trim size: section size << BATCHED_TRIM_SECTIONS_SHIFT
Thanks,
Quoted 40 lines
Quoted 8 lines
fs/f2fs/segment.c int f2fs_trim_fs(struct f2fs_sb_info *sbi, struct fstrim_range
Quoted 8 lines
fs/f2fs/f2fs.h +0
-0struct cp_control {int reason;__u64 trim_start;fs/f2fs/segment.c +0
-0@@ -1066,14 +1066,19 @@ int f2fs_trim_fs(struct f2fs_sb_info *sbi, struct fstrim_range*range)
Quoted 31 lines
Quoted 31 lines
fs/f2fs/segment.c +10
-6end_segno = (end >=MAX_BLKADDR (sbi)) ?MAIN_SEGS (sbi) -1 :GET_SEGNO (sbi, end); cpc.reason =CP_DISCARD ; - cpc.trim_start = start_segno; - cpc.trim_end = end_segno; cpc.trim_minlen = range->minlen >> sbi->log_blocksize;/* do checkpoint to issue discard commands safely */ -mutex_lock (&sbi->gc_mutex); -write_checkpoint (sbi, &cpc); -mutex_unlock (&sbi->gc_mutex); +for (; start_segno <= end_segno; + start_segno +=BATCHED_TRIM_SEGMENTS +1 ) { + cpc.trim_start = start_segno; + cpc.trim_end = min_t(unsigned int, + start_segno +BATCHED_TRIM_SEGMENTS , end_segno); + +mutex_lock (&sbi->gc_mutex); +write_checkpoint (sbi, &cpc); +mutex_unlock (&sbi->gc_mutex); + } out: range->len = cpc.trimmed << sbi->log_blocksize;return 0 ; --2.1.1
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Jaegeuk Kim
maintainerRe: [PATCH 5/5 v2] f2fs: introduce a batched trim
Change long from v1:
o add description
o change the # of batched segments suggested by Chao
o make consistent for # of batched segments
This patch introduces a batched trimming feature, which submits split discard commands.
This patch introduces a batched trimming feature, which submits split discard commands.
This is to avoid long latency due to huge trim commands.
If fstrim was triggered ranging from 0 to the end of device, we should lock all the checkpoint-related mutexes, resulting in very long latency.
--- fs/f2fs/f2fs.h | 2 ++ fs/f2fs/segment.c | 16 +++++++++++----- 2 files changed, 13 insertions(+), 5 deletions(-)
fs/f2fs/f2fs.h +2 -0
@@ -105,6 +105,8 @@ enum { CP_DISCARD, }; +#define BATCHED_TRIM_SEGMENTS (sbi) (((sbi)->segs_per_sec) << 5) +struct cp_control {int reason;__u64 trim_start;
fs/f2fs/segment.c +11 -5
@@ -1066,14 +1066,20 @@ int f2fs_trim_fs(struct f2fs_sb_info *sbi, struct fstrim_range *range) end_segno = (end >=MAX_BLKADDR (sbi)) ?MAIN_SEGS (sbi) -1 :GET_SEGNO (sbi, end); cpc.reason =CP_DISCARD ; - cpc.trim_start = start_segno; - cpc.trim_end = end_segno; cpc.trim_minlen = range->minlen >> sbi->log_blocksize;/* do checkpoint to issue discard commands safely */ -mutex_lock (&sbi->gc_mutex); -write_checkpoint (sbi, &cpc); -mutex_unlock (&sbi->gc_mutex); +for (; start_segno <= end_segno; + start_segno +=BATCHED_TRIM_SEGMENTS (sbi)) { + cpc.trim_start = start_segno; + cpc.trim_end = min_t(unsigned int, + start_segno +BATCHED_TRIM_SEGMENTS (sbi) -1 , + end_segno); + +mutex_lock (&sbi->gc_mutex); +write_checkpoint (sbi, &cpc); +mutex_unlock (&sbi->gc_mutex); + } out: range->len = cpc.trimmed << sbi->log_blocksize;return 0 ;
Signature · 1 line
2.1.1
Re: [f2fs-dev] [PATCH 5/5 v2] f2fs: introduce a batched trim
Hi Jaegeuk,
IMHO, it looks better user could decide the size of trim considering latency of trim. Otherwise, additional checkpoints user doesn't want will occur.
Regards,
Changman
Quoted 78 lines · Jaegeuk Kim
Change long from v1:
o add description
o change the # of batched segments suggested by Chao
o make consistent for # of batched segmentsThis patch introduces a batched trimming feature, which submits split discard commands.
This patch introduces a batched trimming feature, which submits split discard commands.
This is to avoid long latency due to huge trim commands.
If fstrim was triggered ranging from 0 to the end of device, we should lock all the checkpoint-related mutexes, resulting in very long latency.Signed-off-by Jaegeuk Kim <[email protected]>--- fs/f2fs/f2fs.h | 2 ++ fs/f2fs/segment.c | 16 +++++++++++----- 2 files changed, 13 insertions(+), 5 deletions(-)fs/f2fs/f2fs.h +2
-0@@ -105,6 +105,8 @@ enum { CP_DISCARD, }; +#define BATCHED_TRIM_SEGMENTS (sbi) (((sbi)->segs_per_sec) << 5) +struct cp_control {int reason;__u64 trim_start;fs/f2fs/segment.c +11
-5@@ -1066,14 +1066,20 @@ int f2fs_trim_fs(struct f2fs_sb_info *sbi, struct fstrim_range *range) end_segno = (end >=MAX_BLKADDR (sbi)) ?MAIN_SEGS (sbi) -1 :GET_SEGNO (sbi, end); cpc.reason =CP_DISCARD ; - cpc.trim_start = start_segno; - cpc.trim_end = end_segno; cpc.trim_minlen = range->minlen >> sbi->log_blocksize;/* do checkpoint to issue discard commands safely */ -mutex_lock (&sbi->gc_mutex); -write_checkpoint (sbi, &cpc); -mutex_unlock (&sbi->gc_mutex); +for (; start_segno <= end_segno; + start_segno +=BATCHED_TRIM_SEGMENTS (sbi)) { + cpc.trim_start = start_segno; + cpc.trim_end = min_t(unsigned int, + start_segno +BATCHED_TRIM_SEGMENTS (sbi) -1 , + end_segno); + +mutex_lock (&sbi->gc_mutex); +write_checkpoint (sbi, &cpc); +mutex_unlock (&sbi->gc_mutex); + } out: range->len = cpc.trimmed << sbi->log_blocksize;return 0 ;Signature · 13 lines
2.1.1 ------------------------------------------------------------------------------ Dive into the World of Parallel Programming. The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ _______________________________________________ Linux-f2fs-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
Jaegeuk Kim
maintainerRe: [f2fs-dev] [PATCH 5/5 v3] f2fs: introduce a batched trim
Hi Changman,
Good idea!
Change log from v2:
o add sysfs to change the # of sections for trimming.
Change log from v1:
o add description
o change the # of batched segments suggested by Chao
o make consistent for # of batched segments
This patch introduces a batched trimming feature, which submits split discard commands.
This is to avoid long latency due to huge trim commands.
If fstrim was triggered ranging from 0 to the end of device, we should lock all the checkpoint-related mutexes, resulting in very long latency.
--- Documentation/ABI/testing/sysfs-fs-f2fs | 6 ++++++ Documentation/filesystems/f2fs.txt | 4 ++++ fs/f2fs/f2fs.h | 7 +++++++ fs/f2fs/segment.c | 18 +++++++++++++----- fs/f2fs/super.c | 2 ++ 5 files changed, 32 insertions(+), 5 deletions(-)
Documentation/ABI/testing/sysfs-fs-f2fs +6 -0
@@ -74,3 +74,9 @@ Date: March 2014 Contact: "Jaegeuk Kim" <[email protected]> Description: Controls the memory footprint used by f2fs. + +What: /sys/fs/f2fs/<disk>/trim_sections +Date: February 2015 +Contact: "Jaegeuk Kim" <[email protected]> +Description: + Controls the trimming rate in batch mode.
Documentation/filesystems/f2fs.txt +4 -0
@@ -199,6 +199,10 @@ Files in /sys/fs/f2fs/<devname> checkpoint is triggered, and issued during the checkpoint. By default, it is disabled with 0. + trim_sections This parameter controls the number of sections + to be trimmed out in batch mode when FITRIM + conducts. 32 sections is set by default. + ipu_policy This parameter controls the policy of in-place updates in f2fs. There are five policies: 0x01: F2FS_IPU_FORCE, 0x02: F2FS_IPU_SSR,
fs/f2fs/f2fs.h +7 -0
@@ -105,6 +105,10 @@ enum { CP_DISCARD, }; +#define DEF_BATCHED_TRIM_SECTIONS 32 +#define BATCHED_TRIM_SEGMENTS (sbi) \ + (SM_I(sbi)->trim_sections * (sbi)->segs_per_sec) +struct cp_control {int reason;__u64 trim_start;
@@ -448,6 +452,9 @@ struct f2fs_sm_info {int nr_discards;/* # of discards in the list */ int max_discards;/* max. discards to be issued */ +/* for batched trimming */ +int trim_sections;/* # of sections to trim */ +struct list_head sit_entry_set;/* sit entry set list */ unsigned int ipu_policy;/* in-place-update policy */
fs/f2fs/segment.c +13 -5
@@ -1066,14 +1066,20 @@ int f2fs_trim_fs(struct f2fs_sb_info *sbi, struct fstrim_range *range) end_segno = (end >=MAX_BLKADDR (sbi)) ?MAIN_SEGS (sbi) -1 :GET_SEGNO (sbi, end); cpc.reason =CP_DISCARD ; - cpc.trim_start = start_segno; - cpc.trim_end = end_segno; cpc.trim_minlen = range->minlen >> sbi->log_blocksize;/* do checkpoint to issue discard commands safely */ -mutex_lock (&sbi->gc_mutex); -write_checkpoint (sbi, &cpc); -mutex_unlock (&sbi->gc_mutex); +for (; start_segno <= end_segno; + start_segno +=BATCHED_TRIM_SEGMENTS (sbi)) { + cpc.trim_start = start_segno; + cpc.trim_end = min_t(unsigned int, + start_segno +BATCHED_TRIM_SEGMENTS (sbi) -1 , + end_segno); + +mutex_lock (&sbi->gc_mutex); +write_checkpoint (sbi, &cpc); +mutex_unlock (&sbi->gc_mutex); + } out: range->len = cpc.trimmed << sbi->log_blocksize;return 0 ;
@@ -2127,6 +2133,8 @@ int build_segment_manager(struct f2fs_sb_info *sbi) sm_info->nr_discards =0 ; sm_info->max_discards =0 ; + sm_info->trim_sections =DEF_BATCHED_TRIM_SECTIONS ; +INIT_LIST_HEAD (&sm_info->sit_entry_set);if (test_opt (sbi,FLUSH_MERGE ) && !f2fs_readonly (sbi->sb)) {
fs/f2fs/super.c +2 -0
@@ -195,6 +195,7 @@ F2FS_RW_ATTR(GC_THREAD, f2fs_gc_kthread, gc_no_gc_sleep_time, no_gc_sleep_time);F2FS_RW_ATTR (GC_THREAD , f2fs_gc_kthread, gc_idle, gc_idle);F2FS_RW_ATTR (SM_INFO , f2fs_sm_info, reclaim_segments, rec_prefree_segments);F2FS_RW_ATTR (SM_INFO , f2fs_sm_info, max_small_discards, max_discards); +F2FS_RW_ATTR (SM_INFO , f2fs_sm_info, batched_trim_sections, trim_sections);F2FS_RW_ATTR (SM_INFO , f2fs_sm_info, ipu_policy, ipu_policy);F2FS_RW_ATTR (SM_INFO , f2fs_sm_info, min_ipu_util, min_ipu_util);F2FS_RW_ATTR (SM_INFO , f2fs_sm_info, min_fsync_blocks, min_fsync_blocks);
@@ -210,6 +211,7 @@ static struct attribute *f2fs_attrs[] = {ATTR_LIST (gc_idle),ATTR_LIST (reclaim_segments),ATTR_LIST (max_small_discards), +ATTR_LIST (batched_trim_sections),ATTR_LIST (ipu_policy),ATTR_LIST (min_ipu_util), ATTR_LIST(min_fsync_blocks),
Signature · 1 line
2.1.1
RE: [f2fs-dev] [PATCH 5/5 v3] f2fs: introduce a batched trim
Hi Jaegeuk, Changman,
Quoted 8 lines
-----Original Message-----
From: Jaegeuk Kim [mailto:[email protected]]
Sent: Wednesday, February 04, 2015 4:11 AM
To: Changman Lee
Cc: Chao Yu; [email protected]; [email protected]; [email protected]
Subject: Re: [f2fs-dev] [PATCH 5/5 v3] f2fs: introduce a batched trim
Quoted 6 lines
Hi Changman,
Good idea!
Change log from v2:
o add sysfs to change the # of sections for trimming.
Good idea!
Quoted 88 lines
Change log from v1:
o add description
o change the # of batched segments suggested by Chao
o make consistent for # of batched segmentsThis patch introduces a batched trimming feature, which submits split discard commands.
This is to avoid long latency due to huge trim commands.
If fstrim was triggered ranging from 0 to the end of device, we should lock all the checkpoint-related mutexes, resulting in very long latency.Signed-off-by Jaegeuk Kim <[email protected]>--- Documentation/ABI/testing/sysfs-fs-f2fs | 6 ++++++ Documentation/filesystems/f2fs.txt | 4 ++++ fs/f2fs/f2fs.h | 7 +++++++ fs/f2fs/segment.c | 18 +++++++++++++----- fs/f2fs/super.c | 2 ++ 5 files changed, 32 insertions(+), 5 deletions(-)Documentation/ABI/testing/sysfs-fs-f2fs +6
-0@@ -74,3 +74,9 @@ Date: March 2014 Contact: "Jaegeuk Kim" <[email protected]> Description: Controls the memory footprint used by f2fs. + +What: /sys/fs/f2fs/<disk>/trim_sections +Date: February 2015 +Contact: "Jaegeuk Kim" <[email protected]> +Description: + Controls the trimming rate in batch mode.Documentation/filesystems/f2fs.txt +4
-0@@ -199,6 +199,10 @@ Files in /sys/fs/f2fs/<devname> checkpoint is triggered, and issued during the checkpoint. By default, it is disabled with 0. + trim_sections This parameter controls the number of sections + to be trimmed out in batch mode when FITRIM + conducts. 32 sections is set by default. + ipu_policy This parameter controls the policy of in-place updates in f2fs. There are five policies: 0x01: F2FS_IPU_FORCE, 0x02: F2FS_IPU_SSR,fs/f2fs/f2fs.h +7
-0@@ -105,6 +105,10 @@ enum { CP_DISCARD, }; +#define DEF_BATCHED_TRIM_SECTIONS 32 +#define BATCHED_TRIM_SEGMENTS (sbi) \ + (SM_I(sbi)->trim_sections * (sbi)->segs_per_sec) +struct cp_control {int reason;__u64 trim_start;@@ -448,6 +452,9 @@ struct f2fs_sm_info {int nr_discards;/* # of discards in the list */ int max_discards;/* max. discards to be issued */ +/* for batched trimming */ +int trim_sections;/* # of sections to trim */ +struct list_head sit_entry_set;/* sit entry set list */ unsigned int ipu_policy;/* in-place-update policy */ fs/f2fs/segment.c +0
-2@@ -1066,14 +1066,20 @@ int f2fs_trim_fs(struct f2fs_sb_info *sbi, struct fstrim_range *range) end_segno = (end >=MAX_BLKADDR (sbi)) ?MAIN_SEGS (sbi) -1 :GET_SEGNO (sbi, end); cpc.reason =CP_DISCARD ; - cpc.trim_start = start_segno; - cpc.trim_end = end_segno; cpc.trim_minlen = range->minlen >> sbi->log_blocksize;
fs/f2fs/segment.c int f2fs_trim_fs(struct f2fs_sb_info *sbi, struct fstrim_range *range) · line 1076
Quoted 10 lines
fs/f2fs/segment.c +5
-3/* do checkpoint to issue discard commands safely */ -mutex_lock (&sbi->gc_mutex); -write_checkpoint (sbi, &cpc); -mutex_unlock (&sbi->gc_mutex); +for (; start_segno <= end_segno; + start_segno +=BATCHED_TRIM_SEGMENTS (sbi)) { + cpc.trim_start = start_segno; + cpc.trim_end = min_t(unsigned int, + start_segno + BATCHED_TRIM_SEGMENTS (sbi) - 1,
Actually what I mean is that in each trim we try to align start segment to first segment of section as much as possible.
How about using:
cpc.trim_end = min_t(unsigned int, rounddown(start_segno + BATCHED_TRIM_SEGMENTS(sbi), (sbi)->segs_per_sec) - 1, end_segno);
Thanks,
Quoted 41 lines
fs/f2fs/segment.c +8
-0+ end_segno); + +mutex_lock (&sbi->gc_mutex); +write_checkpoint (sbi, &cpc); +mutex_unlock (&sbi->gc_mutex); + } out: range->len = cpc.trimmed << sbi->log_blocksize;return 0 ;@@ -2127,6 +2133,8 @@ int build_segment_manager(struct f2fs_sb_info *sbi) sm_info->nr_discards =0 ; sm_info->max_discards =0 ; + sm_info->trim_sections =DEF_BATCHED_TRIM_SECTIONS ; +INIT_LIST_HEAD (&sm_info->sit_entry_set);if (test_opt (sbi,FLUSH_MERGE ) && !f2fs_readonly (sbi->sb)) {fs/f2fs/super.c +1
-0@@ -195,6 +195,7 @@ F2FS_RW_ATTR(GC_THREAD, f2fs_gc_kthread, gc_no_gc_sleep_time, no_gc_sleep_time);F2FS_RW_ATTR (GC_THREAD , f2fs_gc_kthread, gc_idle, gc_idle);F2FS_RW_ATTR (SM_INFO , f2fs_sm_info, reclaim_segments, rec_prefree_segments);F2FS_RW_ATTR (SM_INFO , f2fs_sm_info, max_small_discards, max_discards); +F2FS_RW_ATTR (SM_INFO , f2fs_sm_info, batched_trim_sections, trim_sections);F2FS_RW_ATTR (SM_INFO , f2fs_sm_info, ipu_policy, ipu_policy);F2FS_RW_ATTR (SM_INFO , f2fs_sm_info, min_ipu_util, min_ipu_util);
F2FS_RW_ATTR(SM_INFO, f2fs_sm_info, min_fsync_blocks, min_fsync_blocks);fs/f2fs/super.c +1
-1@@ -210,6 +211,7 @@ static struct attribute *f2fs_attrs[] = {ATTR_LIST (gc_idle),ATTR_LIST (reclaim_segments),ATTR_LIST (max_small_discards), +ATTR_LIST (batched_trim_sections),ATTR_LIST (ipu_policy),ATTR_LIST (min_ipu_util), ATTR_LIST(min_fsync_blocks), --2.1.1
Jaegeuk Kim
maintainerRe: [f2fs-dev] [PATCH 5/5 v4] f2fs: introduce a batched trim
Hi Chao,
Something like this?
Change log from v3:
o align to section size
This patch introduces a batched trimming feature, which submits split discard commands.
This is to avoid long latency due to huge trim commands.
If fstrim was triggered ranging from 0 to the end of device, we should lock all the checkpoint-related mutexes, resulting in very long latency.
--- Documentation/ABI/testing/sysfs-fs-f2fs | 6 ++++++ Documentation/filesystems/f2fs.txt | 4 ++++ fs/f2fs/f2fs.h | 7 +++++++ fs/f2fs/segment.c | 17 ++++++++++++----- fs/f2fs/super.c | 2 ++ 5 files changed, 31 insertions(+), 5 deletions(-)
Documentation/ABI/testing/sysfs-fs-f2fs +6 -0
@@ -74,3 +74,9 @@ Date: March 2014 Contact: "Jaegeuk Kim" <[email protected]> Description: Controls the memory footprint used by f2fs. + +What: /sys/fs/f2fs/<disk>/trim_sections +Date: February 2015 +Contact: "Jaegeuk Kim" <[email protected]> +Description: + Controls the trimming rate in batch mode.
Documentation/filesystems/f2fs.txt +4 -0
@@ -199,6 +199,10 @@ Files in /sys/fs/f2fs/<devname> checkpoint is triggered, and issued during the checkpoint. By default, it is disabled with 0. + trim_sections This parameter controls the number of sections + to be trimmed out in batch mode when FITRIM + conducts. 32 sections is set by default. + ipu_policy This parameter controls the policy of in-place updates in f2fs. There are five policies: 0x01: F2FS_IPU_FORCE, 0x02: F2FS_IPU_SSR,
fs/f2fs/f2fs.h +7 -0
@@ -105,6 +105,10 @@ enum { CP_DISCARD, }; +#define DEF_BATCHED_TRIM_SECTIONS 32 +#define BATCHED_TRIM_SEGMENTS (sbi) \ + (SM_I(sbi)->trim_sections * (sbi)->segs_per_sec) +struct cp_control {int reason;__u64 trim_start;
@@ -448,6 +452,9 @@ struct f2fs_sm_info {int nr_discards;/* # of discards in the list */ int max_discards;/* max. discards to be issued */ +/* for batched trimming */ +int trim_sections;/* # of sections to trim */ +struct list_head sit_entry_set;/* sit entry set list */ unsigned int ipu_policy;/* in-place-update policy */
fs/f2fs/segment.c +12 -5
@@ -1066,14 +1066,19 @@ int f2fs_trim_fs(struct f2fs_sb_info *sbi, struct fstrim_range *range) end_segno = (end >=MAX_BLKADDR (sbi)) ?MAIN_SEGS (sbi) -1 :GET_SEGNO (sbi, end); cpc.reason =CP_DISCARD ; - cpc.trim_start = start_segno; - cpc.trim_end = end_segno; cpc.trim_minlen = range->minlen >> sbi->log_blocksize;/* do checkpoint to issue discard commands safely */ -mutex_lock (&sbi->gc_mutex); -write_checkpoint (sbi, &cpc); -mutex_unlock (&sbi->gc_mutex); +for (; start_segno <= end_segno; start_segno = cpc.trim_end +1 ) { + cpc.trim_start = start_segno; + cpc.trim_end = min_t(unsigned int, rounddown(start_segno + +BATCHED_TRIM_SEGMENTS (sbi), + sbi->segs_per_sec) -1 , end_segno); + +mutex_lock (&sbi->gc_mutex); +write_checkpoint (sbi, &cpc); +mutex_unlock (&sbi->gc_mutex); + } out: range->len = cpc.trimmed << sbi->log_blocksize;return 0 ;
@@ -2127,6 +2132,8 @@ int build_segment_manager(struct f2fs_sb_info *sbi) sm_info->nr_discards =0 ; sm_info->max_discards =0 ; + sm_info->trim_sections =DEF_BATCHED_TRIM_SECTIONS ; +INIT_LIST_HEAD (&sm_info->sit_entry_set);if (test_opt (sbi,FLUSH_MERGE ) && !f2fs_readonly (sbi->sb)) {
fs/f2fs/super.c +2 -0
@@ -195,6 +195,7 @@ F2FS_RW_ATTR(GC_THREAD, f2fs_gc_kthread, gc_no_gc_sleep_time, no_gc_sleep_time);F2FS_RW_ATTR (GC_THREAD , f2fs_gc_kthread, gc_idle, gc_idle);F2FS_RW_ATTR (SM_INFO , f2fs_sm_info, reclaim_segments, rec_prefree_segments);F2FS_RW_ATTR (SM_INFO , f2fs_sm_info, max_small_discards, max_discards); +F2FS_RW_ATTR (SM_INFO , f2fs_sm_info, batched_trim_sections, trim_sections);F2FS_RW_ATTR (SM_INFO , f2fs_sm_info, ipu_policy, ipu_policy);F2FS_RW_ATTR (SM_INFO , f2fs_sm_info, min_ipu_util, min_ipu_util);F2FS_RW_ATTR (SM_INFO , f2fs_sm_info, min_fsync_blocks, min_fsync_blocks);
@@ -210,6 +211,7 @@ static struct attribute *f2fs_attrs[] = {ATTR_LIST (gc_idle),ATTR_LIST (reclaim_segments),ATTR_LIST (max_small_discards), +ATTR_LIST (batched_trim_sections),ATTR_LIST (ipu_policy),ATTR_LIST (min_ipu_util), ATTR_LIST(min_fsync_blocks),
Signature · 1 line
2.1.1
Chao Yu
Reviewed-byRE: [f2fs-dev] [PATCH 5/5 v4] f2fs: introduce a batched trim
Hi Jaegeuk,
Quoted 15 lines
-----Original Message-----
From: Jaegeuk Kim [mailto:[email protected]]
Sent: Friday, February 06, 2015 2:19 PM
To: Chao Yu
Cc: 'Changman Lee'; [email protected]; [email protected]; [email protected]
Subject: Re: [f2fs-dev] [PATCH 5/5 v4] f2fs: introduce a batched trimHi Chao,
Something like this?
Change log from v3:
o align to section size
Quoted 8 lines
This patch introduces a batched trimming feature, which submits split discard commands.
This is to avoid long latency due to huge trim commands.
If fstrim was triggered ranging from 0 to the end of device, we should lock all the checkpoint-related mutexes, resulting in very long latency.Signed-off-by Jaegeuk Kim <[email protected]>
Looks good to me.
Re: [f2fs-dev] [PATCH 5/5 v4] f2fs: introduce a batched trim
In reply to Jaegeuk KimHi Jaegeuk,
fs/f2fs/f2fs.h struct f2fs_sm_info { · line 456
Quoted 2 lines
fs/f2fs/f2fs.h +2
-0+/* for batched trimming */ +int trim_sections;/* # of sections to trim */
I would like to suggest to declare trim_sections variable as an unsigned int and not int, since it can't be negative by definition. What do you think about it?
Jaegeuk Kim
maintainerRe: [f2fs-dev] [PATCH 5/5 v4] f2fs: introduce a batched trim
Quoted 7 lines · Leon Romanovsky
Hi Jaegeuk,
fs/f2fs/f2fs.h struct f2fs_sm_info { · line 456
Quoted 2 lines
fs/f2fs/f2fs.h +2
-0+/* for batched trimming */ +int trim_sections;/* # of sections to trim */ I would like to suggest to declare
trim_sectionsvariable as an unsigned int and not int, since it can't be negative by definition. What do you think about it?
Hi Leon,
No problem. :)
I'll change that and merge the patch.
Thanks,
Hi Jaegeuk,
Quoted 9 lines
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Jaegeuk Kim
Sent: Wednesday, January 28, 2015 7:32 AM
To: [email protected]; [email protected]; [email protected]
Cc: Jaegeuk Kim
Subject: [PATCH 1/5] f2fs: fix not to drop mount options when retryingfill_super
Quoted 3 lines
If wrong mount option was requested, f2fs tries to
fill_superagain. But, during the next trial, f2fs has no valid mount options, sinceparse_optionsdeleted all the separators in the original string.
Nice catch!
But couldn't we encounter memory leak for options if we did not fail to fill super.
Thanks,
------------------------------------------------------------------------------
Dive into the World of Parallel Programming. The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/
Jaegeuk Kim
maintainerChang log from v1:
o fix memory leak pointed by Chao
Quoted 1 line
From
82760814db6a01539ad15026b0469686110d92bcMon Sep 17 00:00:00 2001
From: Jaegeuk Kim <[email protected]>
Date: Fri, 23 Jan 2015 17:41:39 -0800
Subject: [PATCH] f2fs: fix not to drop mount options when retrying fill_super
If wrong mount option was requested, f2fs tries to fill_super again. But, during the next trial, f2fs has no valid mount options, since parse_options deleted all the separators in the original string.
--- fs/f2fs/super.c | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-)
fs/f2fs/super.c +13 -3
@@ -951,6 +951,7 @@ static int f2fs_fill_super(struct super_block *sb, void *data, int silent)struct inode *root;long err = -EINVAL ;bool retry = true; +char *options =NULL ;int i; try_onemore:
@@ -982,9 +983,15 @@ try_onemore:set_opt (sbi,POSIX_ACL );#endif /* parse mount options */ - err =parse_options (sb, (char *)data); -if (err) + options =kstrdup ((const char *)data,GFP_KERNEL ); +if (data && !options) { + err = -ENOMEM ; goto free_sb_buf; + } + + err =parse_options (sb, options); +if (err) + goto free_options; sb->s_maxbytes =max_file_size (le32_to_cpu (raw_super->log_blocksize)); sb->s_max_links =F2FS_LINK_MAX ;
@@ -1028,7 +1035,7 @@ try_onemore:if (IS_ERR (sbi->meta_inode)) {f2fs_msg (sb,KERN_ERR ,"Failed to read F2FS meta data inode" ); err =PTR_ERR (sbi->meta_inode); - goto free_sb_buf; + goto free_options; } err =get_valid_checkpoint (sbi);
@@ -1153,6 +1160,7 @@ try_onemore:if (err) goto free_kobj; } +kfree (options);return 0 ; free_kobj:
@@ -1177,6 +1185,8 @@ free_cp: free_meta_inode:make_bad_inode (sbi->meta_inode);iput (sbi->meta_inode); +free_options: +kfree (options); free_sb_buf:brelse (raw_super_buf); free_sbi:
Signature · 1 line
2.1.1
Chao Yu
Reviewed-byQuoted 16 lines
-----Original Message-----
From: Jaegeuk Kim [mailto:[email protected]]
Sent: Friday, January 30, 2015 2:21 AM
To: Chao Yu
Cc: [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH 1/5 v2] f2fs: fix not to drop mount options when retryingfill_superChang log from v1:
o fix memory leak pointed by ChaoFrom
82760814db6a01539ad15026b0469686110d92bcMon Sep 17 00:00:00 2001
From: Jaegeuk Kim <[email protected]>
Date: Fri, 23 Jan 2015 17:41:39 -0800
Subject: [PATCH] f2fs: fix not to drop mount options when retryingfill_super
Quoted 5 lines
If wrong mount option was requested, f2fs tries to
fill_superagain. But, during the next trial, f2fs has no valid mount options, sinceparse_optionsdeleted all the separators in the original string.Signed-off-by Jaegeuk Kim <[email protected]>