ocfs2: support append O_DIRECT write
v6 cover ocfs2: support append O_DIRECT write
Currently in case of append O_DIRECT write (block not allocated yet), ocfs2 will fall back to buffered I/O. This has some disadvantages. Firstly, it is not the behavior as expected.
Secondly, it will consume huge page cache, e.g. in mass backup scenario. Thirdly, modern filesystems such as ext4 support this feature.
In this patch set, the direct I/O write doesn't fallback to buffer I/O write any more because the allocate blocks are enabled in direct I/O now.
changelog:
v6 <- v5:
-- Take Mark's advice to use prefix "dio-" to distinguish dio orphan
entry from unlink/rename.
-- Take Mark's advice to treat this feature as a ro compat feature. -- Fix a bug in case of not cluster aligned io, cluster_align should
be !zero_len, not !!zero_len.
-- Fix a bug in case of fallocate with FALLOC_FL_KEEP_SIZE. -- Fix the wrong *ppos and written when completing the rest request
using buffer io.
Corresponding ocfs2 tools (mkfs.ocfs2, tunefs.ocfs2, fsck.ocfs2, etc.) will be updated later.
v6 cover ocfs2: support append O_DIRECT write
Hi Joseph,
Did this version make any performance improvement with v5? I tested v5, and it didn't improve performance with original buffer write + sync.
Thanks,
Junxiao.
Quoted 30 lines · Joseph Qi
Currently in case of append
O_DIRECTwrite (block not allocated yet), ocfs2 will fall back to buffered I/O. This has some disadvantages. Firstly, it is not the behavior as expected.
Secondly, it will consume huge page cache, e.g. in mass backup scenario. Thirdly, modern filesystems such as ext4 support this feature.In this patch set, the direct I/O write doesn't fallback to buffer I/O write any more because the allocate blocks are enabled in direct I/O now.
changelog:
v6 <- v5:
-- Take Mark's advice to use prefix "dio-" to distinguish dio orphan
entry from unlink/rename.
-- Take Mark's advice to treat this feature as a ro compat feature. -- Fix a bug in case of not cluster aligned io,cluster_alignshould
be !zero_len, not !!zero_len.
-- Fix a bug in case of fallocate withFALLOC_FL_KEEP_SIZE. -- Fix the wrong *ppos and written when completing the rest request
using buffer io.Corresponding ocfs2 tools (mkfs.ocfs2, tunefs.ocfs2, fsck.ocfs2, etc.) will be updated later.
_______________________________________________
Ocfs2-devel mailing list
Ocfs2-devel at oss.oracle.com
https://oss.oracle.com/mailman/listinfo/ocfs2-devel
v6 cover ocfs2: support append O_DIRECT write
Hi Junxiao,
Quoted 4 lines · Junxiao Bi
Hi Joseph,
Did this version make any performance improvement with v5? I tested v5, and it didn't improve performance with original buffer write + sync.
No performance difference between these two versions.
But we have tested with fio before, it shows about 5% performance improvement with normal buffer write (without sync).
As I described, this feature is not truly for performance improvement. We aim to reduce the host page cache consumption. For example, dom0 in virtualization case which won't be configured too much memory.
--
Joseph
Quoted 38 lines
Thanks,
Junxiao.Quoted 30 lines · Joseph Qi
Currently in case of append
O_DIRECTwrite (block not allocated yet), ocfs2 will fall back to buffered I/O. This has some disadvantages. Firstly, it is not the behavior as expected.
Secondly, it will consume huge page cache, e.g. in mass backup scenario. Thirdly, modern filesystems such as ext4 support this feature.In this patch set, the direct I/O write doesn't fallback to buffer I/O write any more because the allocate blocks are enabled in direct I/O now.
changelog:
v6 <- v5:
-- Take Mark's advice to use prefix "dio-" to distinguish dio orphan
entry from unlink/rename.
-- Take Mark's advice to treat this feature as a ro compat feature. -- Fix a bug in case of not cluster aligned io,cluster_alignshould
be !zero_len, not !!zero_len.
-- Fix a bug in case of fallocate withFALLOC_FL_KEEP_SIZE. -- Fix the wrong *ppos and written when completing the rest request
using buffer io.Corresponding ocfs2 tools (mkfs.ocfs2, tunefs.ocfs2, fsck.ocfs2, etc.) will be updated later.
_______________________________________________
Ocfs2-devel mailing list
Ocfs2-devel at oss.oracle.com
https://oss.oracle.com/mailman/listinfo/ocfs2-devel
v6 cover ocfs2: support append O_DIRECT write
Quoted 5 lines · Joseph Qi
Hi Junxiao,
Quoted 2 lines · Junxiao Bi
Hi Joseph,
Quoted 8 lines
Quoted 2 lines
Did this version make any performance improvement with v5? I tested v5, and it didn't improve performance with original buffer write + sync.
No performance difference between these two versions.
But we have tested with fio before, it shows about 5% performance improvement with normal buffer write (without sync).
As I described, this feature is not truly for performance improvement. We aim to reduce the host page cache consumption. For example, dom0 in virtualization case which won't be configured too much memory.
I cared the direct-io performance because recently i got a bug that ocfs2 appending write direct-io performance was two times less than ext4. Since you followed some idea from ext4, do you know why ext4 is faster and any advise to improve ocfs2 performance?
Thanks,
Junxiao.
Quoted 43 lines
--
JosephQuoted 38 lines
Thanks,
Junxiao.Quoted 30 lines · Joseph Qi
Currently in case of append
O_DIRECTwrite (block not allocated yet), ocfs2 will fall back to buffered I/O. This has some disadvantages. Firstly, it is not the behavior as expected.
Secondly, it will consume huge page cache, e.g. in mass backup scenario. Thirdly, modern filesystems such as ext4 support this feature.In this patch set, the direct I/O write doesn't fallback to buffer I/O write any more because the allocate blocks are enabled in direct I/O now.
changelog:
v6 <- v5:
-- Take Mark's advice to use prefix "dio-" to distinguish dio orphan
entry from unlink/rename.
-- Take Mark's advice to treat this feature as a ro compat feature. -- Fix a bug in case of not cluster aligned io,cluster_alignshould
be !zero_len, not !!zero_len.
-- Fix a bug in case of fallocate withFALLOC_FL_KEEP_SIZE. -- Fix the wrong *ppos and written when completing the rest request
using buffer io.Corresponding ocfs2 tools (mkfs.ocfs2, tunefs.ocfs2, fsck.ocfs2, etc.) will be updated later.
_______________________________________________
Ocfs2-devel mailing list
Ocfs2-devel at oss.oracle.com
https://oss.oracle.com/mailman/listinfo/ocfs2-devel
v6 cover ocfs2: support append O_DIRECT write
Quoted 14 lines · Junxiao Bi
Quoted 5 lines · Joseph Qi
Hi Junxiao,
Quoted 2 lines · Junxiao Bi
Hi Joseph,
Quoted 8 lines
Quoted 2 lines
Did this version make any performance improvement with v5? I tested v5, and it didn't improve performance with original buffer write + sync.
No performance difference between these two versions.
But we have tested with fio before, it shows about 5% performance improvement with normal buffer write (without sync).
As I described, this feature is not truly for performance improvement. We aim to reduce the host page cache consumption. For example, dom0 in virtualization case which won't be configured too much memory.
Quoted 4 lines
I cared the direct-io performance because recently i got a bug that ocfs2 appending write direct-io performance was two times less than ext4. Since you followed some idea from ext4, do you know why ext4 is faster and any advise to improve ocfs2 performance?
I have sent an related email "[RFD] ocfs2: poor performance on append write/punch hole" before to discuss this topic but unfortunately no reply.
https://oss.oracle.com/pipermail/ocfs2-devel/2014-January/009585.html
Current ocfs2 append O_DIRECT write will fall back to buffer io and then do journal force commit. And my test shows jbd2_journal_force_commit consumes about 90% out of the total time.
After this patch set, io will go directly to disk and no longer need page cache and force commit. But block allocation, meta data update, as well as other necessary steps are still there.
Yes, I have followed ext4 when implementing this feature. And in my opinion, the flow is almost the same with ext4. So I am not sure if the difference is caused by disk layout.
--
Joseph
Quoted 50 lines
Thanks,
Junxiao.Quoted 43 lines
--
JosephQuoted 38 lines
Thanks,
Junxiao.Quoted 30 lines · Joseph Qi
Currently in case of append
O_DIRECTwrite (block not allocated yet), ocfs2 will fall back to buffered I/O. This has some disadvantages. Firstly, it is not the behavior as expected.
Secondly, it will consume huge page cache, e.g. in mass backup scenario. Thirdly, modern filesystems such as ext4 support this feature.In this patch set, the direct I/O write doesn't fallback to buffer I/O write any more because the allocate blocks are enabled in direct I/O now.
changelog:
v6 <- v5:
-- Take Mark's advice to use prefix "dio-" to distinguish dio orphan
entry from unlink/rename.
-- Take Mark's advice to treat this feature as a ro compat feature. -- Fix a bug in case of not cluster aligned io,cluster_alignshould
be !zero_len, not !!zero_len.
-- Fix a bug in case of fallocate withFALLOC_FL_KEEP_SIZE. -- Fix the wrong *ppos and written when completing the rest request
using buffer io.Corresponding ocfs2 tools (mkfs.ocfs2, tunefs.ocfs2, fsck.ocfs2, etc.) will be updated later.
_______________________________________________
Ocfs2-devel mailing list
Ocfs2-devel at oss.oracle.com
https://oss.oracle.com/mailman/listinfo/ocfs2-devel.
v6 cover ocfs2: support append O_DIRECT write
Quoted 24 lines · Joseph Qi
Quoted 14 lines · Junxiao Bi
Quoted 5 lines · Joseph Qi
Hi Junxiao,
Quoted 2 lines · Junxiao Bi
Hi Joseph,
Quoted 8 lines
Quoted 2 lines
Did this version make any performance improvement with v5? I tested v5, and it didn't improve performance with original buffer write + sync.
No performance difference between these two versions.
But we have tested with fio before, it shows about 5% performance improvement with normal buffer write (without sync).
As I described, this feature is not truly for performance improvement. We aim to reduce the host page cache consumption. For example, dom0 in virtualization case which won't be configured too much memory.Quoted 4 lines
I cared the direct-io performance because recently i got a bug that ocfs2 appending write direct-io performance was two times less than ext4. Since you followed some idea from ext4, do you know why ext4 is faster and any advise to improve ocfs2 performance?
I have sent an related email "[RFD] ocfs2: poor performance on append write/punch hole" before to discuss this topic but unfortunately no reply.
https://oss.oracle.com/pipermail/ocfs2-devel/2014-January/009585.html
Quoted 7 lines
Current ocfs2 append
O_DIRECTwrite will fall back to buffer io and then do journal force commit. And my test showsjbd2_journal_force_commitconsumes about 90% out of the total time.After this patch set, io will go directly to disk and no longer need page cache and force commit. But block allocation, meta data update, as well as other necessary steps are still there.
Yes, i think these journal operations may be the key to the performance, I traced the direct io for ext4 when I do dd test, I found there was no journal operation during every write, but after dd done, about 5s, there were some journal flush to disk, looks like ext4 cached the journal. If ocfs2 can do this, maybe performance will be good.
Thanks,
Junxiao.
Quoted 59 lines
Yes, I have followed ext4 when implementing this feature. And in my opinion, the flow is almost the same with ext4. So I am not sure if the difference is caused by disk layout.
--
JosephQuoted 50 lines
Thanks,
Junxiao.Quoted 43 lines
--
JosephQuoted 38 lines
Thanks,
Junxiao.Quoted 30 lines · Joseph Qi
Currently in case of append
O_DIRECTwrite (block not allocated yet), ocfs2 will fall back to buffered I/O. This has some disadvantages. Firstly, it is not the behavior as expected.
Secondly, it will consume huge page cache, e.g. in mass backup scenario. Thirdly, modern filesystems such as ext4 support this feature.In this patch set, the direct I/O write doesn't fallback to buffer I/O write any more because the allocate blocks are enabled in direct I/O now.
changelog:
v6 <- v5:
-- Take Mark's advice to use prefix "dio-" to distinguish dio orphan
entry from unlink/rename.
-- Take Mark's advice to treat this feature as a ro compat feature. -- Fix a bug in case of not cluster aligned io,cluster_alignshould
be !zero_len, not !!zero_len.
-- Fix a bug in case of fallocate withFALLOC_FL_KEEP_SIZE. -- Fix the wrong *ppos and written when completing the rest request
using buffer io.Corresponding ocfs2 tools (mkfs.ocfs2, tunefs.ocfs2, fsck.ocfs2, etc.) will be updated later.
_______________________________________________
Ocfs2-devel mailing list
Ocfs2-devel at oss.oracle.com
https://oss.oracle.com/mailman/listinfo/ocfs2-devel.