f2fs: optimize fs_lock for better performance
Subsystems (2)
Hi Kim,
I did some tests as you mention of using random instead of spin_lock. The test model is as following:
eight threads race to grab one of eight locks for one thousand times, and I used four methods to generate lock num:
1.atomic_add_return(1, &sbi->next_lock_num) % NR_GLOBAL_LOCKS; 2.spin_lock(); next_lock_num++ % NR_GLOBAL_LOCKS; spin_unlock(); 3.ktime_get().tv64 % NR_GLOBAL_LOCKS; 4.get_random_bytes(&next_lock, sizeof(unsigned int));
the result indicate that:
max count of collide continuously: 4 > 3 > 2 = 1
max-min count of lock is grabbed: 4 > 3 > 2 = 1
elapsed time of generating: 3 > 2 > 4 > 1
So I think it's better to use atomic_add_return in round-robin method to cost less time and reduce collide.
What's your opinion?
thanks
------- Original Message -------
Sender : ???<[email protected]> S5(??)/??/?????????(???)/???? Date : 九月 10, 2013 09:52 (GMT+09:00)
Title : Re: [f2fs-dev][PATCH] f2fs: optimize fs_lock for better performance
Hi,
At first, thank you for the report and please follow the email writing rules. :)
Anyway, I agree to the below issue.
One thing that I can think of is that we don't need to use the spin_lock, since we don't care about the exact lock number, but just need to get any not-collided number.
So, how about removing the spin_lock?
And how about using a random number?
Thanks,
2013-09-06 (?), 09:48 +0000, Chao Yu:
Quoted 138 lines
Hi Kim:
I think there is a performance problem: when all sbi->fs_lock isholded,
then all other threads may get the same
next_lockvalue fromsbi->next_lock_numin functionmutex_lock_op,and wait to get the same lock at position fs_lock[next_lock], it unbalance the
fs_lockusage.It may lost performance when we do the multithread test.
Here is the patch to fix this problem:
Signed-off-by Yu Chaofs/f2fs/f2fs.h +0
-0old mode 100644
new mode 100755
index 467d42d..983bb45
---
a/fs/f2fs/f2fs.h+++
b/fs/f2fs/f2fs.hfs/f2fs/f2fs.h +0
-0@@ -371,6 +371,7 @@ struct f2fs_sb_info {struct mutex fs_lock[NR_GLOBAL_LOCKS ];/* blocking FS operations */
struct mutex
node_write; /* locking node writes
*/struct mutex writepages; /* mutex for
writepages()*/+
spinlock_tspin_lock; /* lock fornext_lock_num*/unsigned char
next_lock_num; /* round-robin global
locks */int
por_doing; /* recovery is doing
or not */int
on_build_free_nids; /*build_free_nidsis
doing */fs/f2fs/f2fs.h +6
-2@@ -533,15 +534,19 @@ static inline void mutex_unlock_all(struct f2fs_sb_info *sbi)static inline int mutex_lock_op (struct f2fs_sb_info *sbi) { -unsigned char next_lock = sbi->next_lock_num %NR_GLOBAL_LOCKS ; +unsigned char next_lock;int i =0 ;for (; i <NR_GLOBAL_LOCKS ; i++)if (mutex_trylock (&sbi->fs_lock[i]))return i; -mutex_lock (&sbi->fs_lock[next_lock]); +spin_lock (&sbi->spin_lock); + next_lock = sbi->next_lock_num %NR_GLOBAL_LOCKS ; sbi->next_lock_num++; +spin_unlock (&sbi->spin_lock); + +mutex_lock (&sbi->fs_lock[next_lock]);return next_lock; }fs/f2fs/super.c +0
-0old mode 100644
new mode 100755
index 75c7dc3..4f27596
---
a/fs/f2fs/super.c+++
b/fs/f2fs/super.cfs/f2fs/super.c +1
-0@@ -657,6 +657,7 @@ static int f2fs_fill_super(struct super_block *sb, void *data, int silent) mutex_init(&sbi->cp_mutex);for (i =0 ; i <NR_GLOBAL_LOCKS ; i++)mutex_init (&sbi->fs_lock[i]); +spin_lock_init (&sbi->spin_lock);mutex_init (&sbi->node_write); sbi->por_doing =0 ;spin_lock_init (&sbi->stat_lock);(END)
Signature · 2 lines
Jaegeuk Kim Samsung
Hi,
2013/9/11 Chao Yu <[email protected]>
Quoted 13 lines
Hi Kim,
I did some tests as you mention of using random instead of
spin_lock. The test model is as following:
eight threads race to grab one of eight locks for one thousand times, and I used four methods to generate lock num:1.atomic_add_return(1,
&sbi->next_lock_num) %NR_GLOBAL_LOCKS; 2.spin_lock(); next_lock_num++ %NR_GLOBAL_LOCKS;spin_unlock(); 3.ktime_get().tv64 %NR_GLOBAL_LOCKS; 4.get_random_bytes(&next_lock, sizeof(unsigned int));
Quoted 8 lines
the result indicate that:
max count of collide continuously: 4 > 3 > 2 = 1
max-min count of lock is grabbed: 4 > 3 > 2 = 1
elapsed time of generating: 3 > 2 > 4 > 1So I think it's better to use
atomic_add_returnin round-robin method to cost less time and reduce collide.
What's your opinion?
Could you test with sbi->next_lock_num++ only instead of using atomic_add_return?
IMO, this is just an integer value and still I don't think this value should be covered by any kind of locks.
Thanks,
Quoted 154 lines
thanks
------- Original Message -------
Sender : ???<[email protected]> S5(??)/??/?????????(???)/???? Date : 九月 10, 2013 09:52 (GMT+09:00)
Title : Re: [f2fs-dev][PATCH] f2fs: optimizefs_lockfor better performanceHi,
At first, thank you for the report and please follow the email writing rules. :)
Anyway, I agree to the below issue.
One thing that I can think of is that we don't need to use thespin_lock, since we don't care about the exact lock number, but just need to get any not-collided number.So, how about removing the
spin_lock?
And how about using a random number?
Thanks,2013-09-06 (?), 09:48 +0000, Chao Yu:
Quoted 131 lines
Hi Kim:
I think there is a performance problem: when all sbi->fs_lock isholded,
then all other threads may get the same
next_lockvalue fromsbi->next_lock_numin functionmutex_lock_op,and wait to get the same lock at position fs_lock[next_lock], it unbalance the
fs_lockusage.It may lost performance when we do the multithread test.
Here is the patch to fix this problem:
Signed-off-by Yu Chaofs/f2fs/f2fs.h +0
-0old mode 100644
new mode 100755
index 467d42d..983bb45
---
a/fs/f2fs/f2fs.h+++
b/fs/f2fs/f2fs.hfs/f2fs/f2fs.h +0
-0@@ -371,6 +371,7 @@ struct f2fs_sb_info {struct mutex fs_lock[NR_GLOBAL_LOCKS ];/* blocking FS operations */
struct mutex
node_write; /* locking node writes
*/struct mutex writepages; /* mutex for
writepages()*/+
spinlock_tspin_lock; /* lock fornext_lock_num*/unsigned char
next_lock_num; /* round-robin global
locks */int
por_doing; /* recovery is doing
or not */int
on_build_free_nids; /*build_free_nidsis
doing */fs/f2fs/f2fs.h +6
-2@@ -533,15 +534,19 @@ static inline void mutex_unlock_all(struct f2fs_sb_info *sbi)static inline int mutex_lock_op (struct f2fs_sb_info *sbi) { -unsigned char next_lock = sbi->next_lock_num %NR_GLOBAL_LOCKS ; +unsigned char next_lock;int i =0 ;for (; i <NR_GLOBAL_LOCKS ; i++)if (mutex_trylock (&sbi->fs_lock[i]))return i; -mutex_lock (&sbi->fs_lock[next_lock]); +spin_lock (&sbi->spin_lock); + next_lock = sbi->next_lock_num %NR_GLOBAL_LOCKS ; sbi->next_lock_num++; +spin_unlock (&sbi->spin_lock); + +mutex_lock (&sbi->fs_lock[next_lock]);return next_lock; }fs/f2fs/super.c +0
-0old mode 100644
new mode 100755
index 75c7dc3..4f27596
---
a/fs/f2fs/super.c+++
b/fs/f2fs/super.cfs/f2fs/super.c +1
-0@@ -657,6 +657,7 @@ static int f2fs_fill_super(struct super_block *sb, void *data, int silent) mutex_init(&sbi->cp_mutex);for (i =0 ; i <NR_GLOBAL_LOCKS ; i++)mutex_init (&sbi->fs_lock[i]); +spin_lock_init (&sbi->spin_lock);mutex_init (&sbi->node_write); sbi->por_doing =0 ;spin_lock_init (&sbi->stat_lock);
Quoted 11 lines
Quoted 7 lines
(END)
--
Jaegeuk Kim
Samsung
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi Kim
Quoted 5 lines
-----Original Message-----
From: Kim Jaegeuk [mailto:[email protected]]
Sent: Wednesday, September 11, 2013 9:15 PM
To: [email protected]
Cc: ???; 谭姝; [email protected];
Quoted 2 lines
[email protected]
Subject: Re: Re: [f2fs-dev][PATCH] f2fs: optimizefs_lockfor better
performance
Quoted 21 lines
Hi,
2013/9/11 Chao Yu <[email protected]>
Quoted 17 lines
Hi Kim,
I did some tests as you mention of using random instead of
spin_lock. The test model is as following:
eight threads race to grab one of eight locks for one thousand times, and I used four methods to generate lock num:1.atomic_add_return(1,
&sbi->next_lock_num) %NR_GLOBAL_LOCKS; 2.spin_lock(); next_lock_num++ %NR_GLOBAL_LOCKS;spin_unlock(); 3.ktime_get().tv64 %NR_GLOBAL_LOCKS; 4.get_random_bytes(&next_lock, sizeof(unsigned int));the result indicate that:
max count of collide continuously: 4 > 3 > 2 = 1 max-min count of lock is grabbed: 4 > 3 > 2 = 1 elapsed time of generating: 3 > 2 > 4 > 1
Quoted 7 lines
Quoted 3 lines
So I think it's better to use
atomic_add_returnin round-robin method to cost less time and reduce collide.
What's your opinion?Could you test with sbi->next_lock_num++ only instead of using
atomic_add_return?
IMO, this is just an integer value and still I don't think this value
should be
Quoted 2 lines
covered by any kind of locks.
Thanks,
Thanks for the advice, I have tested sbi->next_lock_num++.
The time cost of it is a little bit lower than the atomic one's. for running 8 thread for 1000000 times test.
the performance of it's balance and collide play quit well than I expected.
Can we modify this patch as following?
root@virtaulmachine:/home/yuchao/git/linux-next/fs/f2fs# git diff --stat fs/f2fs/f2fs.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) root@virtaulmachine:/home/yuchao/git/linux-next/fs/f2fs# git diff
fs/f2fs/f2fs.h +0 -0
@@ -544,15 +544,15 @@ static inline void mutex_unlock_all(struct
f2fs_sb_info *sbi)
static inline int mutex_lock_op(struct f2fs_sb_info *sbi) {
- unsigned char next_lock = sbi->next_lock_num % NR_GLOBAL_LOCKS;
+ unsigned char next_lock;
int i = 0;
for (; i < NR_GLOBAL_LOCKS; i++)
if (mutex_trylock(&sbi->fs_lock[i]))
return i;+ next_lock = sbi->next_lock_num++ % NR_GLOBAL_LOCKS;
mutex_lock(&sbi->fs_lock[next_lock]);
- sbi->next_lock_num++;
return next_lock;}
Quoted 170 lines
Quoted 56 lines
thanks
------- Original Message -------
Sender : ???<[email protected]> S5(??)/??/?????????(???)/???? Date : 九月 10, 2013 09:52 (GMT+09:00)
Title : Re: [f2fs-dev][PATCH] f2fs: optimizefs_lockfor better performanceHi,
At first, thank you for the report and please follow the email writing rules. :)
Anyway, I agree to the below issue.
One thing that I can think of is that we don't need to use thespin_lock, since we don't care about the exact lock number, but just need to get any not-collided number.So, how about removing the
spin_lock?
And how about using a random number?
Thanks,2013-09-06 (?), 09:48 +0000, Chao Yu:
Quoted 31 lines
Hi Kim:
I think there is a performance problem: when all sbi->fs_lockis holded,
then all other threads may get the same
next_lockvalue fromsbi->next_lock_numin functionmutex_lock_op,and wait to get the same lock at position fs_lock[next_lock], it unbalance the
fs_lockusage.It may lost performance when we do the multithread test.
Here is the patch to fix this problem:
Signed-off-by Yu Chaofs/f2fs/f2fs.h +0
-0old mode 100644
new mode 100755
index 467d42d..983bb45
---
a/fs/f2fs/f2fs.hQuoted 8 lines
Quoted 8 lines
+++
b/fs/f2fs/f2fs.hfs/f2fs/f2fs.h +0
-0@@ -371,6 +371,7 @@ struct f2fs_sb_info {struct mutex fs_lock[NR_GLOBAL_LOCKS ];/* blocking FS operations */
struct mutex
node_write; /* locking nodewrites
Quoted 9 lines
Quoted 9 lines
*/
struct mutex writepages; /* mutex for
writepages()*/+
spinlock_tspin_lock; /* lock fornext_lock_num*/unsigned char
next_lock_num; /* round-robinglobal
Quoted 3 lines
Quoted 3 lines
locks */
int
por_doing; /* recovery isdoing
Quoted 91 lines
Quoted 80 lines
or not */
int
on_build_free_nids; /*build_free_nidsis
doing */fs/f2fs/f2fs.h +6
-2@@ -533,15 +534,19 @@ static inline void mutex_unlock_all(struct f2fs_sb_info *sbi)static inline int mutex_lock_op (struct f2fs_sb_info *sbi) { -unsigned char next_lock = sbi->next_lock_num %NR_GLOBAL_LOCKS ; +unsigned char next_lock;int i =0 ;for (; i <NR_GLOBAL_LOCKS ; i++)if (mutex_trylock (&sbi->fs_lock[i]))return i; -mutex_lock (&sbi->fs_lock[next_lock]); +spin_lock (&sbi->spin_lock); + next_lock = sbi->next_lock_num %NR_GLOBAL_LOCKS ; sbi->next_lock_num++; +spin_unlock (&sbi->spin_lock); + +mutex_lock (&sbi->fs_lock[next_lock]);return next_lock; }fs/f2fs/super.c +0
-0old mode 100644
new mode 100755
index 75c7dc3..4f27596
---
a/fs/f2fs/super.c+++
b/fs/f2fs/super.cfs/f2fs/super.c +1
-0@@ -657,6 +657,7 @@ static int f2fs_fill_super(struct super_block *sb, void *data, int silent)mutex_init (&sbi->cp_mutex);for (i =0 ; i <NR_GLOBAL_LOCKS ; i++)mutex_init (&sbi->fs_lock[i]); +spin_lock_init (&sbi->spin_lock);mutex_init (&sbi->node_write); sbi->por_doing =0 ;spin_lock_init (&sbi->stat_lock);Quoted 7 lines
(END)
--
Jaegeuk Kim
Samsung