btrfs: optimize barrier usage for Rmw atomics
PATCH 1/1 btrfs: optimize barrier usage for Rmw atomics
Use smp_mb__after_atomic() instead of smp_mb() and avoid the unnecessary barrier for non LL/SC architectures, such as x86.
--- fs/btrfs/btrfs_inode.h | 2 +- fs/btrfs/file.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
fs/btrfs/btrfs_inode.h +1 -1
@@ -325,7 +325,7 @@ struct btrfs_dio_private {static inline void btrfs_inode_block_unlocked_dio (struct btrfs_inode *inode) {set_bit (BTRFS_INODE_READDIO_NEED_LOCK , &inode->runtime_flags); -smp_mb (); +smp_mb__after_atomic (); }static inline void btrfs_inode_resume_unlocked_dio (struct btrfs_inode *inode)
fs/btrfs/file.c +1 -1
@@ -2143,7 +2143,7 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync) }atomic_inc (&root->log_batch); -smp_mb (); +smp_mb__after_atomic ();if (btrfs_inode_in_log (BTRFS_I (inode), fs_info->generation) ||BTRFS_I (inode)->last_trans <= fs_info->last_trans_committed) {/*
Signature · 1 line
2.16.4
Quoted 4 lines · Davidlohr Bueso
Use
smp_mb__after_atomic()instead ofsmp_mb()and avoid the unnecessary barrier for non LL/SC architectures, such as x86.Signed-off-by Davidlohr Bueso <[email protected]>
While on the topic of this I've been sitting on the following local patch for about a year, care to review the barriers:
From e659e5db649be01aec20515aef8ca48143e10c0b Mon Sep 17 00:00:00 2001
From: Nikolay Borisov <[email protected]>
Date: Wed, 7 Mar 2018 17:19:12 +0200
Subject: [PATCH] btrfs: Fix memory ordering of unlocked dio reads vs truncate
--- fs/btrfs/btrfs_inode.h | 17 ----------------- fs/btrfs/inode.c | 41 ++++++++++++++++++++++++++++++++++------- 2 files changed, 34 insertions(+), 24 deletions(-)
fs/btrfs/btrfs_inode.h +0 -17
@@ -317,23 +317,6 @@ struct btrfs_dio_private { blk_status_t); }; -/* -* Disable DIO read nolock optimization, so new dio readers will be forced -* to grab i_mutex. It is used to avoid the endless truncate due to -* nonlocked dio read. -*/ -static inline void btrfs_inode_block_unlocked_dio (struct btrfs_inode *inode) -{ -set_bit (BTRFS_INODE_READDIO_NEED_LOCK , &inode->runtime_flags); -smp_mb (); -} - -static inline void btrfs_inode_resume_unlocked_dio (struct btrfs_inode *inode) -{ -smp_mb__before_atomic (); -clear_bit (BTRFS_INODE_READDIO_NEED_LOCK , &inode->runtime_flags); -} -/* Array of bytes with variable length, hexadecimal format 0x1234 */ #define CSUM_FMT "0x%*phN"#define CSUM_FMT_VALUE (size, bytes) size, bytes
fs/btrfs/inode.c +34 -7
55 lines not on this page · raw message
Signature · 1 line
2.17.1
Quoted 2 lines · Davidlohr Bueso
Use
smp_mb__after_atomic()instead ofsmp_mb()and avoid the unnecessary barrier for non LL/SC architectures, such as x86.
So that's a conflicting advice from what we got when discussing wich barriers to use in 6282675e6708ec78518cc0e9ad1f1f73d7c5c53d and the memory is still fresh. My first idea was to take the smp_mb__after_atomic and __before_atomic variants and after discussion with various people the plain smp_wmb/smp_rmb were suggested and used in the end.
I can dig the email threads and excerpts from irc conversations, maybe Nik has them at hand too. We do want to get rid of all unnecessary and uncommented barriers in btrfs code, so I appreciate your patch.
Quoted 11 lines
Signed-off-by Davidlohr Bueso <[email protected]>--- fs/btrfs/btrfs_inode.h | 2 +- fs/btrfs/file.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)fs/btrfs/btrfs_inode.h +0
-0
fs/btrfs/btrfs_inode.h struct btrfs_dio_private { · line 328
Quoted 5 lines
fs/btrfs/btrfs_inode.h +1
-1@@ -325,7 +325,7 @@ struct btrfs_dio_private {static inline void btrfs_inode_block_unlocked_dio (struct btrfs_inode *inode) {set_bit (BTRFS_INODE_READDIO_NEED_LOCK , &inode->runtime_flags); -smp_mb (); +smp_mb__after_atomic ();
In this case I think we should use the smp_wmb/smp_rmb pattern rather than the full barrier.
fs/btrfs/file.c int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync) · line 2146
Quoted 13 lines
fs/btrfs/btrfs_inode.h +0
-0}static inline void btrfs_inode_resume_unlocked_dio (struct btrfs_inode *inode)fs/btrfs/file.c +1
-1@@ -2143,7 +2143,7 @@ int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync) }atomic_inc (&root->log_batch); -smp_mb (); +smp_mb__after_atomic ();
That's the problem with uncommented barriers that it's not clear what are they related to. In this case it's not the atomic_inc above that would justify __after_atomic. The patch that added it is years old so any change to that barrier would require deeper analysis.
Quoted 3 lines
fs/btrfs/file.c +0
-0if (btrfs_inode_in_log (BTRFS_I (inode), fs_info->generation) ||BTRFS_I (inode)->last_trans <= fs_info->last_trans_committed) {/*
Quoted 10 lines · David Sterba
Quoted 2 lines · Davidlohr Bueso
Use
smp_mb__after_atomic()instead ofsmp_mb()and avoid the unnecessary barrier for non LL/SC architectures, such as x86.So that's a conflicting advice from what we got when discussing wich barriers to use in
6282675e6708ec78518cc0e9ad1f1f73d7c5c53dand the memory is still fresh. My first idea was to take thesmp_mb__after_atomicand__before_atomicvariants and after discussion with various people the plain smp_wmb/smp_rmb were suggested and used in the end.
So the patch you mention deals with test_bit(), which is out of the scope of smp_mb__{before,after}_atomic() as it's not a RMW operation. atomic_inc() and set_bit() are, however, meant to use these barriers.
Quoted 4 lines
I can dig the email threads and excerpts from irc conversations, maybe Nik has them at hand too. We do want to get rid of all unnecessary and uncommented barriers in btrfs code, so I appreciate your patch.
Yeah, I struggled with the amount of undocumented barriers, and decided not to go down that rabbit hole. This patch is only an equivalent of what is currently there. When possible, getting rid of barriers is of course better.
Thanks,
Davidlohr
Quoted 13 lines · Davidlohr Bueso
Quoted 10 lines · David Sterba
Quoted 2 lines · Davidlohr Bueso
Use
smp_mb__after_atomic()instead ofsmp_mb()and avoid the unnecessary barrier for non LL/SC architectures, such as x86.So that's a conflicting advice from what we got when discussing wich barriers to use in
6282675e6708ec78518cc0e9ad1f1f73d7c5c53dand the memory is still fresh. My first idea was to take thesmp_mb__after_atomicand__before_atomicvariants and after discussion with various people the plain smp_wmb/smp_rmb were suggested and used in the end.
Quoted 4 lines
So the patch you mention deals with
test_bit(), which is out of the scope of smp_mb__{before,after}_atomic() as it's not a RMW operation.atomic_inc()
andset_bit()are, however, meant to use these barriers.
Exactly!
I'm still not convinced to use full barrier for test_bit() and I see no reason to use any barrier for test_bit().
All mb should only be needed between two or more memory access, thus mb should sit between set/clear_bit() and other operations, not around test_bit().
Quoted 10 lines
Quoted 5 lines
I can dig the email threads and excerpts from irc conversations, maybe Nik has them at hand too. We do want to get rid of all unnecessary and uncommented barriers in btrfs code, so I appreciate your patch.
Yeah, I struggled with the amount of undocumented barriers, and decided not to go down that rabbit hole. This patch is only an equivalent of what is currently there. When possible, getting rid of barriers is of course better.
BTW, is there any convincing method to do proper mb examination?
I really found it hard to convince others or even myself when mb is involved.
Thanks,
Qu
Quoted 3 lines
Thanks,
Davidlohr
Quoted 39 lines · Qu Wenruo
Quoted 13 lines · Davidlohr Bueso
Quoted 10 lines · David Sterba
Quoted 2 lines · Davidlohr Bueso
Use
smp_mb__after_atomic()instead ofsmp_mb()and avoid the unnecessary barrier for non LL/SC architectures, such as x86.So that's a conflicting advice from what we got when discussing wich barriers to use in
6282675e6708ec78518cc0e9ad1f1f73d7c5c53dand the memory is still fresh. My first idea was to take thesmp_mb__after_atomicand__before_atomicvariants and after discussion with various people the plain smp_wmb/smp_rmb were suggested and used in the end.Quoted 4 lines
So the patch you mention deals with
test_bit(), which is out of the scope of smp_mb__{before,after}_atomic() as it's not a RMW operation.atomic_inc()
andset_bit()are, however, meant to use these barriers.Exactly!
I'm still not convinced to use full barrier fortest_bit()and I see no reason to use any barrier fortest_bit().
All mb should only be needed between two or more memory access, thus mb should sit between set/clear_bit() and other operations, not aroundtest_bit().Quoted 10 lines
Quoted 5 lines
I can dig the email threads and excerpts from irc conversations, maybe Nik has them at hand too. We do want to get rid of all unnecessary and uncommented barriers in btrfs code, so I appreciate your patch.
Yeah, I struggled with the amount of undocumented barriers, and decided not to go down that rabbit hole. This patch is only an equivalent of what is currently there. When possible, getting rid of barriers is of course better.
Quoted 4 lines
BTW, is there any convincing method to do proper mb examination?
I really found it hard to convince others or even myself when mb is involved.
Yes there is - the LKMM, you can write a litmus test. Check out tootls/memory-model
Quoted 7 lines
Thanks,
QuQuoted 3 lines
Thanks,
Davidlohr