f2fs: fix to force keeping write barrier for strict fsync mode
RFC PATCH v2 1/1 f2fs: fix to force keeping write barrier for strict fsync mode
[1] https://www.mail-archive.com/[email protected]/msg15126.html
As [1] reported, if lower device doesn't support write barrier, in below case:
- write page #0; persist
- overwrite page #0
- fsync
- write data page #0 OPU into device's cache
- write inode page into device's cache
- issue flush
If SPO is triggered during flush command, inode page can be persisted before data page #0, so that after recovery, inode page can be recovered with new physical block address of data page #0, however there may contains dummy data in new physical block address.
Then what user will see is: after overwrite & fsync + SPO, old data in file was corrupted, if any user do care about such case, we can suggest user to use STRICT fsync mode, in this mode, we will force to trigger preflush command to persist data in device cache in prior to node writeback, it avoids potential data corruption during fsync().
---
v2:
- fix this by adding additional preflush command rather than using atomic write flow.
fs/f2fs/file.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
fs/f2fs/file.c +14 -0
@@ -301,6 +301,20 @@ static int f2fs_do_sync_file(struct file *file, loff_t start, loff_t end, f2fs_exist_written_data(sbi, ino, UPDATE_INO)) goto flush_out; goto out; + }else { +/* +* for OPU case, during fsync(), node can be persisted before +* data when lower device doesn't support write barrier, result +* in data corruption after SPO. +* So for strict fsync mode, force to trigger preflush to keep +* data/node write order to avoid potential data corruption. +*/ +if (F2FS_OPTION (sbi).fsync_mode ==FSYNC_MODE_STRICT && + !atomic) { + ret =f2fs_issue_flush (sbi, inode->i_ino); +if (ret) + goto out; + } } go_write:/*
Signature · 1 line
2.29.2
Chao Yu
maintainerJaegeuk,
Any comments on this patch?
Quoted 57 lines · Chao Yu
[1] https://www.mail-archive.com/[email protected]/msg15126.html
As [1] reported, if lower device doesn't support write barrier, in below case:
- write page #0; persist
- overwrite page #0
- fsync
- write data page #0 OPU into device's cache
- write inode page into device's cache
- issue flushIf SPO is triggered during flush command, inode page can be persisted before data page #0, so that after recovery, inode page can be recovered with new physical block address of data page #0, however there may contains dummy data in new physical block address.
Then what user will see is: after overwrite & fsync + SPO, old data in file was corrupted, if any user do care about such case, we can suggest user to use STRICT fsync mode, in this mode, we will force to trigger preflush command to persist data in device cache in prior to node writeback, it avoids potential data corruption during
fsync().Signed-off-by Chao Yu <[email protected]>---
v2:
- fix this by adding additional preflush command rather than using atomic write flow.fs/f2fs/file.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)fs/f2fs/file.c +14
-0@@ -301,6 +301,20 @@ static int f2fs_do_sync_file(struct file *file, loff_t start, loff_t end, f2fs_exist_written_data(sbi, ino, UPDATE_INO)) goto flush_out; goto out; + }else { +/* +* for OPU case, during fsync(), node can be persisted before +* data when lower device doesn't support write barrier, result +* in data corruption after SPO. +* So for strict fsync mode, force to trigger preflush to keep +* data/node write order to avoid potential data corruption. +*/ +if (F2FS_OPTION (sbi).fsync_mode ==FSYNC_MODE_STRICT && + !atomic) { + ret =f2fs_issue_flush (sbi, inode->i_ino); +if (ret) + goto out; + } } go_write:/*
Chao Yu
maintainerStill no time to check this?
Thanks,
Quoted 68 lines · Chao Yu
Jaegeuk,
Any comments on this patch?
Quoted 54 lines · Chao Yu
[1] https://www.mail-archive.com/[email protected]/msg15126.html
As [1] reported, if lower device doesn't support write barrier, in below case:
- write page #0; persist
- overwrite page #0
- fsync
- write data page #0 OPU into device's cache
- write inode page into device's cache
- issue flushIf SPO is triggered during flush command, inode page can be persisted before data page #0, so that after recovery, inode page can be recovered with new physical block address of data page #0, however there may contains dummy data in new physical block address.
Then what user will see is: after overwrite & fsync + SPO, old data in file was corrupted, if any user do care about such case, we can suggest user to use STRICT fsync mode, in this mode, we will force to trigger preflush command to persist data in device cache in prior to node writeback, it avoids potential data corruption during
fsync().Signed-off-by Chao Yu <[email protected]>---
v2:
- fix this by adding additional preflush command rather than using atomic write flow.
fs/f2fs/file.c| 14 ++++++++++++++1 file changed, 14 insertions(+)fs/f2fs/file.c +14
-0@@ -301,6 +301,20 @@ static int f2fs_do_sync_file(struct file *file, loff_t start, loff_t end, f2fs_exist_written_data(sbi, ino, UPDATE_INO)) goto flush_out; goto out; + } else { + /* + * for OPU case, during fsync(), node can be persisted before + * data when lower device doesn't support write barrier, result + * in data corruption after SPO. + * So for strict fsync mode, force to trigger preflush to keep + * data/node write order to avoid potential data corruption. + */ + if (F2FS_OPTION(sbi).fsync_mode == FSYNC_MODE_STRICT && + !atomic) { + ret = f2fs_issue_flush(sbi, inode->i_ino); + if (ret) + goto out; + } }Quoted 3 lines
go_write:
/*_______________________________________________
Linux-f2fs-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
Jaegeuk Kim
maintainerQuoted 11 lines · Chao Yu
[1] https://www.mail-archive.com/[email protected]/msg15126.html
As [1] reported, if lower device doesn't support write barrier, in below case:
- write page #0; persist
- overwrite page #0
- fsync
- write data page #0 OPU into device's cache
- write inode page into device's cache
- issue flush
Well, we have preflush for node writes, so I don't think this is the case.
fio.op_flags |= REQ_PREFLUSH | REQ_FUA;
Quoted 47 lines
If SPO is triggered during flush command, inode page can be persisted before data page #0, so that after recovery, inode page can be recovered with new physical block address of data page #0, however there may contains dummy data in new physical block address.
Then what user will see is: after overwrite & fsync + SPO, old data in file was corrupted, if any user do care about such case, we can suggest user to use STRICT fsync mode, in this mode, we will force to trigger preflush command to persist data in device cache in prior to node writeback, it avoids potential data corruption during
fsync().Signed-off-by Chao Yu <[email protected]>---
v2:
- fix this by adding additional preflush command rather than using atomic write flow.fs/f2fs/file.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)fs/f2fs/file.c +14
-0@@ -301,6 +301,20 @@ static int f2fs_do_sync_file(struct file *file, loff_t start, loff_t end, f2fs_exist_written_data(sbi, ino, UPDATE_INO)) goto flush_out; goto out; + }else { +/* +* for OPU case, during fsync(), node can be persisted before +* data when lower device doesn't support write barrier, result +* in data corruption after SPO. +* So for strict fsync mode, force to trigger preflush to keep +* data/node write order to avoid potential data corruption. +*/ +if (F2FS_OPTION (sbi).fsync_mode ==FSYNC_MODE_STRICT && + !atomic) { + ret =f2fs_issue_flush (sbi, inode->i_ino); +if (ret) + goto out; + } } go_write:/* Signature · 1 line
2.29.2
Chao Yu
maintainerQuoted 13 lines · Jaegeuk Kim
Quoted 11 lines · Chao Yu
[1] https://www.mail-archive.com/[email protected]/msg15126.html
As [1] reported, if lower device doesn't support write barrier, in below case:
- write page #0; persist
- overwrite page #0
- fsync
- write data page #0 OPU into device's cache
- write inode page into device's cache
- issue flush
Quoted 3 lines
Well, we have preflush for node writes, so I don't think this is the case.
fio.op_flags |= REQ_PREFLUSH | REQ_FUA;
This is only used for atomic write case, right?
I mean the common case which is called from f2fs_issue_flush() in f2fs_do_sync_file().
And please see do_checkpoint(), we call f2fs_flush_device_cache() and commit_checkpoint() separately to keep persistence order of CP datas.
See commit 46706d5917f4 ("f2fs: flush cp pack except cp pack 2 page at first") for details.
Thanks,
Quoted 48 lines
Quoted 48 lines
If SPO is triggered during flush command, inode page can be persisted before data page #0, so that after recovery, inode page can be recovered with new physical block address of data page #0, however there may contains dummy data in new physical block address.
Then what user will see is: after overwrite & fsync + SPO, old data in file was corrupted, if any user do care about such case, we can suggest user to use STRICT fsync mode, in this mode, we will force to trigger preflush command to persist data in device cache in prior to node writeback, it avoids potential data corruption during
fsync().Signed-off-by Chao Yu <[email protected]>---
v2:
- fix this by adding additional preflush command rather than using atomic write flow.fs/f2fs/file.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)fs/f2fs/file.c +14
-0@@ -301,6 +301,20 @@ static int f2fs_do_sync_file(struct file *file, loff_t start, loff_t end, f2fs_exist_written_data(sbi, ino, UPDATE_INO)) goto flush_out; goto out; + }else { +/* +* for OPU case, during fsync(), node can be persisted before +* data when lower device doesn't support write barrier, result +* in data corruption after SPO. +* So for strict fsync mode, force to trigger preflush to keep +* data/node write order to avoid potential data corruption. +*/ +if (F2FS_OPTION (sbi).fsync_mode ==FSYNC_MODE_STRICT && + !atomic) { + ret =f2fs_issue_flush (sbi, inode->i_ino); +if (ret) + goto out; + } } go_write:/* Signature · 1 line
2.29.2
Jaegeuk Kim
maintainerQuoted 14 lines · Chao Yu
Quoted 13 lines · Jaegeuk Kim
Quoted 11 lines · Chao Yu
[1] https://www.mail-archive.com/[email protected]/msg15126.html
As [1] reported, if lower device doesn't support write barrier, in below case:
- write page #0; persist
- overwrite page #0
- fsync
- write data page #0 OPU into device's cache
- write inode page into device's cache
- issue flush
Quoted 8 lines
Quoted 3 lines
Well, we have preflush for node writes, so I don't think this is the case.
fio.op_flags |= REQ_PREFLUSH | REQ_FUA;This is only used for atomic write case, right?
I mean the common case which is called from
f2fs_issue_flush()inf2fs_do_sync_file().
How about adding PREFLUSH when writing node blocks aligned to the above set?
Quoted 57 lines
And please see
do_checkpoint(), we callf2fs_flush_device_cache()andcommit_checkpoint()separately to keep persistence order of CP datas.See commit
46706d5917f4("f2fs: flush cp pack except cp pack 2 page at first") for details.Thanks,
Quoted 48 lines
Quoted 48 lines
If SPO is triggered during flush command, inode page can be persisted before data page #0, so that after recovery, inode page can be recovered with new physical block address of data page #0, however there may contains dummy data in new physical block address.
Then what user will see is: after overwrite & fsync + SPO, old data in file was corrupted, if any user do care about such case, we can suggest user to use STRICT fsync mode, in this mode, we will force to trigger preflush command to persist data in device cache in prior to node writeback, it avoids potential data corruption during
fsync().Signed-off-by Chao Yu <[email protected]>---
v2:
- fix this by adding additional preflush command rather than using atomic write flow.fs/f2fs/file.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)fs/f2fs/file.c +14
-0@@ -301,6 +301,20 @@ static int f2fs_do_sync_file(struct file *file, loff_t start, loff_t end, f2fs_exist_written_data(sbi, ino, UPDATE_INO)) goto flush_out; goto out; + }else { +/* +* for OPU case, during fsync(), node can be persisted before +* data when lower device doesn't support write barrier, result +* in data corruption after SPO. +* So for strict fsync mode, force to trigger preflush to keep +* data/node write order to avoid potential data corruption. +*/ +if (F2FS_OPTION (sbi).fsync_mode ==FSYNC_MODE_STRICT && + !atomic) { + ret =f2fs_issue_flush (sbi, inode->i_ino); +if (ret) + goto out; + } } go_write:/* Signature · 1 line
2.29.2
Chao Yu
maintainerQuoted 17 lines · Jaegeuk Kim
Quoted 16 lines · Chao Yu
Quoted 15 lines · Jaegeuk Kim
Quoted 11 lines · Chao Yu
[1] https://www.mail-archive.com/[email protected]/msg15126.html
As [1] reported, if lower device doesn't support write barrier, in below case:
- write page #0; persist
- overwrite page #0
- fsync
- write data page #0 OPU into device's cache
- write inode page into device's cache
- issue flushWell, we have preflush for node writes, so I don't think this is the case.
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
fio.op_flags |= REQ_PREFLUSH | REQ_FUA;This is only used for atomic write case, right?
I mean the common case which is called from
f2fs_issue_flush()inf2fs_do_sync_file().How about adding PREFLUSH when writing node blocks aligned to the above set?
You mean implementation like v1 as below?
https://lore.kernel.org/linux-f2fs-devel/[email protected]/
Thanks,
Quoted 58 lines
Quoted 58 lines
And please see
do_checkpoint(), we callf2fs_flush_device_cache()andcommit_checkpoint()separately to keep persistence order of CP datas.See commit
46706d5917f4("f2fs: flush cp pack except cp pack 2 page at first") for details.Thanks,
Quoted 48 lines
Quoted 48 lines
If SPO is triggered during flush command, inode page can be persisted before data page #0, so that after recovery, inode page can be recovered with new physical block address of data page #0, however there may contains dummy data in new physical block address.
Then what user will see is: after overwrite & fsync + SPO, old data in file was corrupted, if any user do care about such case, we can suggest user to use STRICT fsync mode, in this mode, we will force to trigger preflush command to persist data in device cache in prior to node writeback, it avoids potential data corruption during
fsync().Signed-off-by Chao Yu <[email protected]>---
v2:
- fix this by adding additional preflush command rather than using atomic write flow.fs/f2fs/file.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)fs/f2fs/file.c +14
-0@@ -301,6 +301,20 @@ static int f2fs_do_sync_file(struct file *file, loff_t start, loff_t end, f2fs_exist_written_data(sbi, ino, UPDATE_INO)) goto flush_out; goto out; + }else { +/* +* for OPU case, during fsync(), node can be persisted before +* data when lower device doesn't support write barrier, result +* in data corruption after SPO. +* So for strict fsync mode, force to trigger preflush to keep +* data/node write order to avoid potential data corruption. +*/ +if (F2FS_OPTION (sbi).fsync_mode ==FSYNC_MODE_STRICT && + !atomic) { + ret =f2fs_issue_flush (sbi, inode->i_ino); +if (ret) + goto out; + } } go_write:/* Signature · 1 line
2.29.2
Jaegeuk Kim
maintainerQuoted 22 lines · Chao Yu
Quoted 21 lines · Jaegeuk Kim
Quoted 20 lines · Chao Yu
Quoted 13 lines · Jaegeuk Kim
Quoted 11 lines · Chao Yu
[1] https://www.mail-archive.com/[email protected]/msg15126.html
As [1] reported, if lower device doesn't support write barrier, in below case:
- write page #0; persist
- overwrite page #0
- fsync
- write data page #0 OPU into device's cache
- write inode page into device's cache
- issue flushQuoted 3 lines
Well, we have preflush for node writes, so I don't think this is the case.
fio.op_flags |= REQ_PREFLUSH | REQ_FUA;This is only used for atomic write case, right?
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
I mean the common case which is called from
f2fs_issue_flush()inf2fs_do_sync_file().How about adding PREFLUSH when writing node blocks aligned to the above set?
You mean implementation like v1 as below?
Yea, I think so. :P
Quoted 61 lines
Thanks,
Quoted 58 lines
Quoted 58 lines
And please see
do_checkpoint(), we callf2fs_flush_device_cache()andcommit_checkpoint()separately to keep persistence order of CP datas.See commit
46706d5917f4("f2fs: flush cp pack except cp pack 2 page at first") for details.Thanks,
Quoted 48 lines
Quoted 48 lines
If SPO is triggered during flush command, inode page can be persisted before data page #0, so that after recovery, inode page can be recovered with new physical block address of data page #0, however there may contains dummy data in new physical block address.
Then what user will see is: after overwrite & fsync + SPO, old data in file was corrupted, if any user do care about such case, we can suggest user to use STRICT fsync mode, in this mode, we will force to trigger preflush command to persist data in device cache in prior to node writeback, it avoids potential data corruption during
fsync().Signed-off-by Chao Yu <[email protected]>---
v2:
- fix this by adding additional preflush command rather than using atomic write flow.fs/f2fs/file.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)fs/f2fs/file.c +14
-0@@ -301,6 +301,20 @@ static int f2fs_do_sync_file(struct file *file, loff_t start, loff_t end, f2fs_exist_written_data(sbi, ino, UPDATE_INO)) goto flush_out; goto out; + }else { +/* +* for OPU case, during fsync(), node can be persisted before +* data when lower device doesn't support write barrier, result +* in data corruption after SPO. +* So for strict fsync mode, force to trigger preflush to keep +* data/node write order to avoid potential data corruption. +*/ +if (F2FS_OPTION (sbi).fsync_mode ==FSYNC_MODE_STRICT && + !atomic) { + ret =f2fs_issue_flush (sbi, inode->i_ino); +if (ret) + goto out; + } } go_write:/* Signature · 1 line
2.29.2
Chao Yu
maintainerQuoted 26 lines · Jaegeuk Kim
Quoted 25 lines · Chao Yu
Quoted 24 lines · Jaegeuk Kim
Quoted 22 lines · Chao Yu
Quoted 13 lines · Jaegeuk Kim
Quoted 11 lines · Chao Yu
[1] https://www.mail-archive.com/[email protected]/msg15126.html
As [1] reported, if lower device doesn't support write barrier, in below case:
- write page #0; persist
- overwrite page #0
- fsync
- write data page #0 OPU into device's cache
- write inode page into device's cache
- issue flushQuoted 3 lines
Well, we have preflush for node writes, so I don't think this is the case.
fio.op_flags |= REQ_PREFLUSH | REQ_FUA;This is only used for atomic write case, right?
I mean the common case which is called from
f2fs_issue_flush()inf2fs_do_sync_file().
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
How about adding PREFLUSH when writing node blocks aligned to the above set?
You mean implementation like v1 as below?
Yea, I think so. :P
I prefer v2, we may have several schemes to improve performance with v2, e.g.
- use inplace IO to avoid newly added preflush
- use flush_merge option to avoid redundant preflush
- if lower device supports barrier IO, we can avoid newly added preflush
Thanks,
Quoted 62 lines
Quoted 62 lines
Thanks,
Quoted 58 lines
Quoted 58 lines
And please see
do_checkpoint(), we callf2fs_flush_device_cache()andcommit_checkpoint()separately to keep persistence order of CP datas.See commit
46706d5917f4("f2fs: flush cp pack except cp pack 2 page at first") for details.Thanks,
Quoted 48 lines
Quoted 48 lines
If SPO is triggered during flush command, inode page can be persisted before data page #0, so that after recovery, inode page can be recovered with new physical block address of data page #0, however there may contains dummy data in new physical block address.
Then what user will see is: after overwrite & fsync + SPO, old data in file was corrupted, if any user do care about such case, we can suggest user to use STRICT fsync mode, in this mode, we will force to trigger preflush command to persist data in device cache in prior to node writeback, it avoids potential data corruption during
fsync().Signed-off-by Chao Yu <[email protected]>---
v2:
- fix this by adding additional preflush command rather than using atomic write flow.fs/f2fs/file.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)fs/f2fs/file.c +14
-0@@ -301,6 +301,20 @@ static int f2fs_do_sync_file(struct file *file, loff_t start, loff_t end, f2fs_exist_written_data(sbi, ino, UPDATE_INO)) goto flush_out; goto out; + }else { +/* +* for OPU case, during fsync(), node can be persisted before +* data when lower device doesn't support write barrier, result +* in data corruption after SPO. +* So for strict fsync mode, force to trigger preflush to keep +* data/node write order to avoid potential data corruption. +*/ +if (F2FS_OPTION (sbi).fsync_mode ==FSYNC_MODE_STRICT && + !atomic) { + ret =f2fs_issue_flush (sbi, inode->i_ino); +if (ret) + goto out; + } } go_write:/* Signature · 1 line
2.29.2
Jaegeuk Kim
maintainerQuoted 31 lines · Chao Yu
Quoted 30 lines · Jaegeuk Kim
Quoted 29 lines · Chao Yu
Quoted 17 lines · Jaegeuk Kim
Quoted 16 lines · Chao Yu
Quoted 15 lines · Jaegeuk Kim
Quoted 11 lines · Chao Yu
[1] https://www.mail-archive.com/[email protected]/msg15126.html
As [1] reported, if lower device doesn't support write barrier, in below case:
- write page #0; persist
- overwrite page #0
- fsync
- write data page #0 OPU into device's cache
- write inode page into device's cache
- issue flushWell, we have preflush for node writes, so I don't think this is the case.
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
fio.op_flags |= REQ_PREFLUSH | REQ_FUA;This is only used for atomic write case, right?
I mean the common case which is called from
f2fs_issue_flush()inf2fs_do_sync_file().How about adding PREFLUSH when writing node blocks aligned to the above set?
You mean implementation like v1 as below?
Quoted 8 lines
Quoted 3 lines
Yea, I think so. :P
I prefer v2, we may have several schemes to improve performance with v2, e.g.
- use inplace IO to avoid newly added preflush
- useflush_mergeoption to avoid redundant preflush
- if lower device supports barrier IO, we can avoid newly added preflush
Doesn't v2 give one more flush than v1? Why do you want to take worse one and try to improve back? Not clear the benefit on v2.
Quoted 65 lines
Thanks,
Quoted 62 lines
Quoted 62 lines
Thanks,
Quoted 58 lines
Quoted 58 lines
And please see
do_checkpoint(), we callf2fs_flush_device_cache()andcommit_checkpoint()separately to keep persistence order of CP datas.See commit
46706d5917f4("f2fs: flush cp pack except cp pack 2 page at first") for details.Thanks,
Quoted 48 lines
Quoted 48 lines
If SPO is triggered during flush command, inode page can be persisted before data page #0, so that after recovery, inode page can be recovered with new physical block address of data page #0, however there may contains dummy data in new physical block address.
Then what user will see is: after overwrite & fsync + SPO, old data in file was corrupted, if any user do care about such case, we can suggest user to use STRICT fsync mode, in this mode, we will force to trigger preflush command to persist data in device cache in prior to node writeback, it avoids potential data corruption during
fsync().Signed-off-by Chao Yu <[email protected]>---
v2:
- fix this by adding additional preflush command rather than using atomic write flow.fs/f2fs/file.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)fs/f2fs/file.c +14
-0@@ -301,6 +301,20 @@ static int f2fs_do_sync_file(struct file *file, loff_t start, loff_t end, f2fs_exist_written_data(sbi, ino, UPDATE_INO)) goto flush_out; goto out; + }else { +/* +* for OPU case, during fsync(), node can be persisted before +* data when lower device doesn't support write barrier, result +* in data corruption after SPO. +* So for strict fsync mode, force to trigger preflush to keep +* data/node write order to avoid potential data corruption. +*/ +if (F2FS_OPTION (sbi).fsync_mode ==FSYNC_MODE_STRICT && + !atomic) { + ret =f2fs_issue_flush (sbi, inode->i_ino); +if (ret) + goto out; + } } go_write:/* Signature · 1 line
2.29.2
Chao Yu
maintainerQuoted 34 lines · Jaegeuk Kim
Quoted 33 lines · Chao Yu
Quoted 32 lines · Jaegeuk Kim
Quoted 30 lines · Chao Yu
Quoted 17 lines · Jaegeuk Kim
Quoted 16 lines · Chao Yu
Quoted 15 lines · Jaegeuk Kim
Quoted 11 lines · Chao Yu
[1] https://www.mail-archive.com/[email protected]/msg15126.html
As [1] reported, if lower device doesn't support write barrier, in below case:
- write page #0; persist
- overwrite page #0
- fsync
- write data page #0 OPU into device's cache
- write inode page into device's cache
- issue flushWell, we have preflush for node writes, so I don't think this is the case.
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
fio.op_flags |= REQ_PREFLUSH | REQ_FUA;This is only used for atomic write case, right?
I mean the common case which is called from
f2fs_issue_flush()inf2fs_do_sync_file().How about adding PREFLUSH when writing node blocks aligned to the above set?
You mean implementation like v1 as below?
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
Yea, I think so. :P
I prefer v2, we may have several schemes to improve performance with v2, e.g.
- use inplace IO to avoid newly added preflush
- useflush_mergeoption to avoid redundant preflush
- if lower device supports barrier IO, we can avoid newly added preflushDoesn't v2 give one more flush than v1? Why do you want to take worse one and
FUA implies an extra preflush command or similar mechanism in lower device to keep data in bio being persistent before this command's completion.
Also if lower device doesn't support FUA natively, block layer turns it into an empty PREFLUSH command.
So, it's hard to say which one will win the benchmark game, maybe we need some performance data before making the choice, but you know, it depends on device's character.
Quoted 1 line
try to improve back? Not clear the benefit on v2.
Well, if user suffer and complain performance regression with v1, any plan to improve it?
I just thought about plan B/C/D for no matter v1 or v2.
Thanks,
Quoted 66 lines
Quoted 66 lines
Thanks,
Quoted 62 lines
Quoted 62 lines
Thanks,
Quoted 58 lines
Quoted 58 lines
And please see
do_checkpoint(), we callf2fs_flush_device_cache()andcommit_checkpoint()separately to keep persistence order of CP datas.See commit
46706d5917f4("f2fs: flush cp pack except cp pack 2 page at first") for details.Thanks,
Quoted 48 lines
Quoted 48 lines
If SPO is triggered during flush command, inode page can be persisted before data page #0, so that after recovery, inode page can be recovered with new physical block address of data page #0, however there may contains dummy data in new physical block address.
Then what user will see is: after overwrite & fsync + SPO, old data in file was corrupted, if any user do care about such case, we can suggest user to use STRICT fsync mode, in this mode, we will force to trigger preflush command to persist data in device cache in prior to node writeback, it avoids potential data corruption during
fsync().Signed-off-by Chao Yu <[email protected]>---
v2:
- fix this by adding additional preflush command rather than using atomic write flow.fs/f2fs/file.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)fs/f2fs/file.c +14
-0@@ -301,6 +301,20 @@ static int f2fs_do_sync_file(struct file *file, loff_t start, loff_t end, f2fs_exist_written_data(sbi, ino, UPDATE_INO)) goto flush_out; goto out; + }else { +/* +* for OPU case, during fsync(), node can be persisted before +* data when lower device doesn't support write barrier, result +* in data corruption after SPO. +* So for strict fsync mode, force to trigger preflush to keep +* data/node write order to avoid potential data corruption. +*/ +if (F2FS_OPTION (sbi).fsync_mode ==FSYNC_MODE_STRICT && + !atomic) { + ret =f2fs_issue_flush (sbi, inode->i_ino); +if (ret) + goto out; + } } go_write:/* Signature · 1 line
2.29.2
Jaegeuk Kim
maintainerQuoted 47 lines · Chao Yu
Quoted 34 lines · Jaegeuk Kim
Quoted 33 lines · Chao Yu
Quoted 32 lines · Jaegeuk Kim
Quoted 30 lines · Chao Yu
Quoted 17 lines · Jaegeuk Kim
Quoted 16 lines · Chao Yu
Quoted 15 lines · Jaegeuk Kim
Quoted 11 lines · Chao Yu
[1] https://www.mail-archive.com/[email protected]/msg15126.html
As [1] reported, if lower device doesn't support write barrier, in below case:
- write page #0; persist
- overwrite page #0
- fsync
- write data page #0 OPU into device's cache
- write inode page into device's cache
- issue flushWell, we have preflush for node writes, so I don't think this is the case.
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
fio.op_flags |= REQ_PREFLUSH | REQ_FUA;This is only used for atomic write case, right?
I mean the common case which is called from
f2fs_issue_flush()inf2fs_do_sync_file().How about adding PREFLUSH when writing node blocks aligned to the above set?
You mean implementation like v1 as below?
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
Yea, I think so. :P
I prefer v2, we may have several schemes to improve performance with v2, e.g.
- use inplace IO to avoid newly added preflush
- useflush_mergeoption to avoid redundant preflush
- if lower device supports barrier IO, we can avoid newly added preflushDoesn't v2 give one more flush than v1? Why do you want to take worse one and
FUA implies an extra preflush command or similar mechanism in lower device to keep data in bio being persistent before this command's completion.
Quoted 6 lines
Also if lower device doesn't support FUA natively, block layer turns it into an empty PREFLUSH command.
So, it's hard to say which one will win the benchmark game, maybe we need some performance data before making the choice, but you know, it depends on device's character.
I was looking at # of bios.
Quoted 6 lines
Quoted 2 lines
try to improve back? Not clear the benefit on v2.
Well, if user suffer and complain performance regression with v1, any plan to improve it?
I just thought about plan B/C/D for no matter v1 or v2.
I assumed you wanted v2 since it might be used for B/C/D improvements. But, it seems it wasn't. My point is to save one bio, but piggyback the flag to the device driver.
Quoted 69 lines
Thanks,
Quoted 66 lines
Quoted 66 lines
Thanks,
Quoted 62 lines
Quoted 62 lines
Thanks,
Quoted 58 lines
Quoted 58 lines
And please see
do_checkpoint(), we callf2fs_flush_device_cache()andcommit_checkpoint()separately to keep persistence order of CP datas.See commit
46706d5917f4("f2fs: flush cp pack except cp pack 2 page at first") for details.Thanks,
Quoted 48 lines
Quoted 48 lines
If SPO is triggered during flush command, inode page can be persisted before data page #0, so that after recovery, inode page can be recovered with new physical block address of data page #0, however there may contains dummy data in new physical block address.
Then what user will see is: after overwrite & fsync + SPO, old data in file was corrupted, if any user do care about such case, we can suggest user to use STRICT fsync mode, in this mode, we will force to trigger preflush command to persist data in device cache in prior to node writeback, it avoids potential data corruption during
fsync().Signed-off-by Chao Yu <[email protected]>---
v2:
- fix this by adding additional preflush command rather than using atomic write flow.fs/f2fs/file.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)fs/f2fs/file.c +14
-0@@ -301,6 +301,20 @@ static int f2fs_do_sync_file(struct file *file, loff_t start, loff_t end, f2fs_exist_written_data(sbi, ino, UPDATE_INO)) goto flush_out; goto out; + }else { +/* +* for OPU case, during fsync(), node can be persisted before +* data when lower device doesn't support write barrier, result +* in data corruption after SPO. +* So for strict fsync mode, force to trigger preflush to keep +* data/node write order to avoid potential data corruption. +*/ +if (F2FS_OPTION (sbi).fsync_mode ==FSYNC_MODE_STRICT && + !atomic) { + ret =f2fs_issue_flush (sbi, inode->i_ino); +if (ret) + goto out; + } } go_write:/* Signature · 1 line
2.29.2
Chao Yu
maintainerQuoted 60 lines · Jaegeuk Kim
Quoted 47 lines · Chao Yu
Quoted 34 lines · Jaegeuk Kim
Quoted 33 lines · Chao Yu
Quoted 32 lines · Jaegeuk Kim
Quoted 30 lines · Chao Yu
Quoted 17 lines · Jaegeuk Kim
Quoted 16 lines · Chao Yu
Quoted 15 lines · Jaegeuk Kim
Quoted 11 lines · Chao Yu
[1] https://www.mail-archive.com/[email protected]/msg15126.html
As [1] reported, if lower device doesn't support write barrier, in below case:
- write page #0; persist
- overwrite page #0
- fsync
- write data page #0 OPU into device's cache
- write inode page into device's cache
- issue flushWell, we have preflush for node writes, so I don't think this is the case.
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
fio.op_flags |= REQ_PREFLUSH | REQ_FUA;This is only used for atomic write case, right?
I mean the common case which is called from
f2fs_issue_flush()inf2fs_do_sync_file().How about adding PREFLUSH when writing node blocks aligned to the above set?
You mean implementation like v1 as below?
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
Yea, I think so. :P
I prefer v2, we may have several schemes to improve performance with v2, e.g.
- use inplace IO to avoid newly added preflush
- useflush_mergeoption to avoid redundant preflush
- if lower device supports barrier IO, we can avoid newly added preflushDoesn't v2 give one more flush than v1? Why do you want to take worse one and
FUA implies an extra preflush command or similar mechanism in lower device to keep data in bio being persistent before this command's completion.
Quoted 6 lines
Also if lower device doesn't support FUA natively, block layer turns it into an empty PREFLUSH command.
So, it's hard to say which one will win the benchmark game, maybe we need some performance data before making the choice, but you know, it depends on device's character.
I was looking at # of bios.
Quoted 3 lines
Quoted 2 lines
try to improve back? Not clear the benefit on v2.
Quoted 7 lines
Quoted 3 lines
Well, if user suffer and complain performance regression with v1, any plan to improve it?
I just thought about plan B/C/D for no matter v1 or v2.
I assumed you wanted v2 since it might be used for B/C/D improvements. But, it seems it wasn't. My point is to save one bio, but piggyback the flag to the device driver.
I doubt the conclusion...but it needs to get some data to prove it.
I think the right way is merging v1 now to fix the bug firstly, and let me do the comparison on them a little bit later to see whether we need another implementation... thoughts?
Thanks,
Quoted 70 lines
Quoted 70 lines
Thanks,
Quoted 66 lines
Quoted 66 lines
Thanks,
Quoted 62 lines
Quoted 62 lines
Thanks,
Quoted 58 lines
Quoted 58 lines
And please see
do_checkpoint(), we callf2fs_flush_device_cache()andcommit_checkpoint()separately to keep persistence order of CP datas.See commit
46706d5917f4("f2fs: flush cp pack except cp pack 2 page at first") for details.Thanks,
Quoted 48 lines
Quoted 48 lines
If SPO is triggered during flush command, inode page can be persisted before data page #0, so that after recovery, inode page can be recovered with new physical block address of data page #0, however there may contains dummy data in new physical block address.
Then what user will see is: after overwrite & fsync + SPO, old data in file was corrupted, if any user do care about such case, we can suggest user to use STRICT fsync mode, in this mode, we will force to trigger preflush command to persist data in device cache in prior to node writeback, it avoids potential data corruption during
fsync().Signed-off-by Chao Yu <[email protected]>---
v2:
- fix this by adding additional preflush command rather than using atomic write flow.fs/f2fs/file.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)fs/f2fs/file.c +14
-0@@ -301,6 +301,20 @@ static int f2fs_do_sync_file(struct file *file, loff_t start, loff_t end, f2fs_exist_written_data(sbi, ino, UPDATE_INO)) goto flush_out; goto out; + }else { +/* +* for OPU case, during fsync(), node can be persisted before +* data when lower device doesn't support write barrier, result +* in data corruption after SPO. +* So for strict fsync mode, force to trigger preflush to keep +* data/node write order to avoid potential data corruption. +*/ +if (F2FS_OPTION (sbi).fsync_mode ==FSYNC_MODE_STRICT && + !atomic) { + ret =f2fs_issue_flush (sbi, inode->i_ino); +if (ret) + goto out; + } } go_write:/* Signature · 1 line
2.29.2
Jaegeuk Kim
maintainerQuoted 69 lines · Chao Yu
Quoted 60 lines · Jaegeuk Kim
Quoted 47 lines · Chao Yu
Quoted 34 lines · Jaegeuk Kim
Quoted 33 lines · Chao Yu
Quoted 32 lines · Jaegeuk Kim
Quoted 30 lines · Chao Yu
Quoted 17 lines · Jaegeuk Kim
Quoted 16 lines · Chao Yu
Quoted 15 lines · Jaegeuk Kim
Quoted 11 lines · Chao Yu
[1] https://www.mail-archive.com/[email protected]/msg15126.html
As [1] reported, if lower device doesn't support write barrier, in below case:
- write page #0; persist
- overwrite page #0
- fsync
- write data page #0 OPU into device's cache
- write inode page into device's cache
- issue flushWell, we have preflush for node writes, so I don't think this is the case.
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
fio.op_flags |= REQ_PREFLUSH | REQ_FUA;This is only used for atomic write case, right?
I mean the common case which is called from
f2fs_issue_flush()inf2fs_do_sync_file().How about adding PREFLUSH when writing node blocks aligned to the above set?
You mean implementation like v1 as below?
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
Yea, I think so. :P
I prefer v2, we may have several schemes to improve performance with v2, e.g.
- use inplace IO to avoid newly added preflush
- useflush_mergeoption to avoid redundant preflush
- if lower device supports barrier IO, we can avoid newly added preflushDoesn't v2 give one more flush than v1? Why do you want to take worse one and
FUA implies an extra preflush command or similar mechanism in lower device to keep data in bio being persistent before this command's completion.
Quoted 6 lines
Also if lower device doesn't support FUA natively, block layer turns it into an empty PREFLUSH command.
So, it's hard to say which one will win the benchmark game, maybe we need some performance data before making the choice, but you know, it depends on device's character.
I was looking at # of bios.
Quoted 3 lines
Quoted 2 lines
try to improve back? Not clear the benefit on v2.
Quoted 7 lines
Quoted 3 lines
Well, if user suffer and complain performance regression with v1, any plan to improve it?
I just thought about plan B/C/D for no matter v1 or v2.
I assumed you wanted v2 since it might be used for B/C/D improvements. But, it seems it wasn't. My point is to save one bio, but piggyback the flag to the device driver.
Quoted 5 lines
I doubt the conclusion...but it needs to get some data to prove it.
I think the right way is merging v1 now to fix the bug firstly, and let me do the comparison on them a little bit later to see whether we need another implementation... thoughts?
Chao, could you please post v1 with an updated description?
Quoted 65 lines
Thanks,
Quoted 62 lines
Quoted 62 lines
Thanks,
Quoted 58 lines
Quoted 58 lines
Thanks,
Quoted 54 lines
Quoted 54 lines
Thanks,
Quoted 50 lines
Quoted 50 lines
And please see
do_checkpoint(), we callf2fs_flush_device_cache()andcommit_checkpoint()separately to keep persistence order of CP datas.See commit
46706d5917f4("f2fs: flush cp pack except cp pack 2 page at first") for details.Thanks,
Quoted 40 lines
Quoted 40 lines
If SPO is triggered during flush command, inode page can be persisted before data page #0, so that after recovery, inode page can be recovered with new physical block address of data page #0, however there may contains dummy data in new physical block address.
Then what user will see is: after overwrite & fsync + SPO, old data in file was corrupted, if any user do care about such case, we can suggest user to use STRICT fsync mode, in this mode, we will force to trigger preflush command to persist data in device cache in prior to node writeback, it avoids potential data corruption during
fsync().Signed-off-by Chao Yu <[email protected]>---
v2:
- fix this by adding additional preflush command rather than using atomic write flow.fs/f2fs/file.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)fs/f2fs/file.c +11
-0@@ -301,6 +301,20 @@ static int f2fs_do_sync_file(struct file *file, loff_t start, loff_t end, f2fs_exist_written_data(sbi, ino, UPDATE_INO)) goto flush_out; goto out; + }else { +/* +* for OPU case, during fsync(), node can be persisted before +* data when lower device doesn't support write barrier, result +* in data corruption after SPO. +* So for strict fsync mode, force to trigger preflush to keep +* data/node write order to avoid potential data corruption. +*/ +if (F2FS_OPTION (sbi).fsync_mode ==FSYNC_MODE_STRICT && + !atomic) { + ret =f2fs_issue_flush (sbi, inode->i_ino);
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
fs/f2fs/file.c +3
-0+if (ret) + goto out; + } } go_write:/* Signature · 1 line
2.29.2
_______________________________________________
Linux-f2fs-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel