Skip to contentSkip to conversation

f2fs: optimize fs_lock for better performance

PATCH New Jaegeuk Kim 4 participants 7 messages Sep 10, 2013 – Sep 12, 2013 1 message not in the archive

Subsystems (2)
lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

#1 lore raw

Hi,

At first, thank you for the report and please follow the email writing rules. :)

Anyway, I agree to the below issue.
One thing that I can think of is that we don't need to use the spin_lock, since we don't care about the exact lock number, but just need to get any not-collided number.

So, how about removing the spin_lock?
And how about using a random number?
Thanks,

2013-09-06 (금), 09:48 +0000, Chao Yu:

Quoted 138 lines

Hi Kim:

     I think there is a performance problem: when all sbi->fs_lock is

holded,

then all other threads may get the same next_lock value from sbi->next_lock_num in function mutex_lock_op,

and wait to get the same lock at position fs_lock[next_lock], it unbalance the fs_lock usage.

It may lost performance when we do the multithread test.

Here is the patch to fix this problem:

fs/f2fs/f2fs.h +0 -0

old mode 100644

new mode 100755

index 467d42d..983bb45

--- a/fs/f2fs/f2fs.h

+++ b/fs/f2fs/f2fs.h

fs/f2fs/f2fs.h +0 -0
@@ -371,6 +371,7 @@ struct f2fs_sb_info {
 
        struct mutex fs_lock[NR_GLOBAL_LOCKS];  /* blocking FS

operations */

struct mutex node_write; /* locking node writes
*/

struct mutex writepages; /* mutex for
writepages() */

+ spinlock_t spin_lock; /* lock for
next_lock_num */

unsigned char next_lock_num; /* round-robin global
locks */

int por_doing; /* recovery is doing
or not */

int on_build_free_nids; /* build_free_nids is
doing */

fs/f2fs/f2fs.h +6 -2
@@ -533,15 +534,19 @@ static inline void mutex_unlock_all(struct
 f2fs_sb_info *sbi)
 
 
 
 static inline int mutex_lock_op(struct f2fs_sb_info *sbi)
 
 {
 
-       unsigned char next_lock = sbi->next_lock_num %
 NR_GLOBAL_LOCKS;
 
+       unsigned char next_lock;
 
        int i = 0;
 
 
 
        for (; i < NR_GLOBAL_LOCKS; i++)
 
                if (mutex_trylock(&sbi->fs_lock[i]))
 
                        return i;
 
 
 
-       mutex_lock(&sbi->fs_lock[next_lock]);
 
+       spin_lock(&sbi->spin_lock);
 
+       next_lock = sbi->next_lock_num % NR_GLOBAL_LOCKS;
 
        sbi->next_lock_num++;
 
+       spin_unlock(&sbi->spin_lock);
 
+
 
+       mutex_lock(&sbi->fs_lock[next_lock]);
 
        return next_lock;
 
 }
fs/f2fs/super.c +0 -0

old mode 100644

new mode 100755

index 75c7dc3..4f27596

--- a/fs/f2fs/super.c

+++ b/fs/f2fs/super.c

fs/f2fs/super.c +1 -0
@@ -657,6 +657,7 @@ static int f2fs_fill_super(struct super_block *sb,
 void *data, int silent)
 
        mutex_init(&sbi->cp_mutex);
 
        for (i = 0; i < NR_GLOBAL_LOCKS; i++)
 
                mutex_init(&sbi->fs_lock[i]);
 
+       spin_lock_init(&sbi->spin_lock);
 
        mutex_init(&sbi->node_write);
 
        sbi->por_doing = 0;
 
        spin_lock_init(&sbi->stat_lock);

(END)

Signature · 2 lines
Jaegeuk Kim
Samsung
#2 lore raw

Hi Jaegeuk,

Quoted 9 lines · Jaegeuk Kim

Hi,

At first, thank you for the report and please follow the email writing rules. :)

Anyway, I agree to the below issue.
One thing that I can think of is that we don't need to use the spin_lock, since we don't care about the exact lock number, but just need to get any not-collided number.

Agree, but if all the locks are held, IMO, we need to balance the following threads to wait for each not-collided number lock, though complete balance is unreachable.

Quoted 2 lines

So, how about removing the spin_lock?

Yeah, in this case, spin_lock is a bit heavy cost.

Quoted 1 line

And how about using a random number?

Now NR_GLOBAL_LOCKS is 8, it seems that random can not offer an balance number as we expected.

Regards,
Gu

Quoted 134 lines

Thanks,

2013-09-06 (금), 09:48 +0000, Chao Yu:

Quoted 131 lines

Hi Kim:

     I think there is a performance problem: when all sbi->fs_lock is

holded,

then all other threads may get the same next_lock value from sbi->next_lock_num in function mutex_lock_op,

and wait to get the same lock at position fs_lock[next_lock], it unbalance the fs_lock usage.

It may lost performance when we do the multithread test.

Here is the patch to fix this problem:

fs/f2fs/f2fs.h +0 -0

old mode 100644

new mode 100755

index 467d42d..983bb45

--- a/fs/f2fs/f2fs.h

+++ b/fs/f2fs/f2fs.h

fs/f2fs/f2fs.h +0 -0
@@ -371,6 +371,7 @@ struct f2fs_sb_info {
 
        struct mutex fs_lock[NR_GLOBAL_LOCKS];  /* blocking FS

operations */

struct mutex node_write; /* locking node writes
*/

struct mutex writepages; /* mutex for
writepages() */

+ spinlock_t spin_lock; /* lock for
next_lock_num */

unsigned char next_lock_num; /* round-robin global
locks */

int por_doing; /* recovery is doing
or not */

int on_build_free_nids; /* build_free_nids is
doing */

fs/f2fs/f2fs.h +6 -2
@@ -533,15 +534,19 @@ static inline void mutex_unlock_all(struct
 f2fs_sb_info *sbi)
 
 
 
 static inline int mutex_lock_op(struct f2fs_sb_info *sbi)
 
 {
 
-       unsigned char next_lock = sbi->next_lock_num %
 NR_GLOBAL_LOCKS;
 
+       unsigned char next_lock;
 
        int i = 0;
 
 
 
        for (; i < NR_GLOBAL_LOCKS; i++)
 
                if (mutex_trylock(&sbi->fs_lock[i]))
 
                        return i;
 
 
 
-       mutex_lock(&sbi->fs_lock[next_lock]);
 
+       spin_lock(&sbi->spin_lock);
 
+       next_lock = sbi->next_lock_num % NR_GLOBAL_LOCKS;
 
        sbi->next_lock_num++;
 
+       spin_unlock(&sbi->spin_lock);
 
+
 
+       mutex_lock(&sbi->fs_lock[next_lock]);
 
        return next_lock;
 
 }
fs/f2fs/super.c +0 -0

old mode 100644

new mode 100755

index 75c7dc3..4f27596

--- a/fs/f2fs/super.c

+++ b/fs/f2fs/super.c

fs/f2fs/super.c +1 -0
@@ -657,6 +657,7 @@ static int f2fs_fill_super(struct super_block *sb,
 void *data, int silent)
 
        mutex_init(&sbi->cp_mutex);
 
        for (i = 0; i < NR_GLOBAL_LOCKS; i++)
 
                mutex_init(&sbi->fs_lock[i]);
 
+       spin_lock_init(&sbi->spin_lock);
 
        mutex_init(&sbi->node_write);
 
        sbi->por_doing = 0;
 
        spin_lock_init(&sbi->stat_lock);
Quoted 8 lines
Quoted 7 lines

(END)

------------------------------------------------------------------------------
How ServiceNow helps IT people transform IT departments:
1. Consolidate legacy IT systems to a single system of record for IT
2. Standardize and globalize service processes across IT
3. Implement zero-touch automation to replace manual, redundant tasks
http://pubads.g.doubleclick.net/gampad/clk?id=51271111&iu=/4140/ostg.clktrk
_______________________________________________
Linux-f2fs-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

#3 lore raw
In reply to Jaegeuk Kim

Hi Jaegeuk, Chao,

Quoted 9 lines · Jaegeuk Kim

Hi,

At first, thank you for the report and please follow the email writing rules. :)

Anyway, I agree to the below issue.
One thing that I can think of is that we don't need to use the spin_lock, since we don't care about the exact lock number, but just need to get any not-collided number.

IMHO, just moving sbi->next_lock_num++ before mutex_lock(&sbi->fs_lock[next_lock]) can avoid unbalance issue mostly.
IMO, the case two or more threads increase sbi->next_lock_num in the same time is really very very little. If you think it is not rigorous, change next_lock_num to atomic one can fix it.
What's your opinion?

Regards,
Gu

Quoted 138 lines

So, how about removing the spin_lock?
And how about using a random number?

Thanks,

2013-09-06 (금), 09:48 +0000, Chao Yu:

Quoted 131 lines

Hi Kim:

     I think there is a performance problem: when all sbi->fs_lock is

holded,

then all other threads may get the same next_lock value from sbi->next_lock_num in function mutex_lock_op,

and wait to get the same lock at position fs_lock[next_lock], it unbalance the fs_lock usage.

It may lost performance when we do the multithread test.

Here is the patch to fix this problem:

fs/f2fs/f2fs.h +0 -0

old mode 100644

new mode 100755

index 467d42d..983bb45

--- a/fs/f2fs/f2fs.h

+++ b/fs/f2fs/f2fs.h

fs/f2fs/f2fs.h +0 -0
@@ -371,6 +371,7 @@ struct f2fs_sb_info {
 
        struct mutex fs_lock[NR_GLOBAL_LOCKS];  /* blocking FS

operations */

struct mutex node_write; /* locking node writes
*/

struct mutex writepages; /* mutex for
writepages() */

+ spinlock_t spin_lock; /* lock for
next_lock_num */

unsigned char next_lock_num; /* round-robin global
locks */

int por_doing; /* recovery is doing
or not */

int on_build_free_nids; /* build_free_nids is
doing */

fs/f2fs/f2fs.h +6 -2
@@ -533,15 +534,19 @@ static inline void mutex_unlock_all(struct
 f2fs_sb_info *sbi)
 
 
 
 static inline int mutex_lock_op(struct f2fs_sb_info *sbi)
 
 {
 
-       unsigned char next_lock = sbi->next_lock_num %
 NR_GLOBAL_LOCKS;
 
+       unsigned char next_lock;
 
        int i = 0;
 
 
 
        for (; i < NR_GLOBAL_LOCKS; i++)
 
                if (mutex_trylock(&sbi->fs_lock[i]))
 
                        return i;
 
 
 
-       mutex_lock(&sbi->fs_lock[next_lock]);
 
+       spin_lock(&sbi->spin_lock);
 
+       next_lock = sbi->next_lock_num % NR_GLOBAL_LOCKS;
 
        sbi->next_lock_num++;
 
+       spin_unlock(&sbi->spin_lock);
 
+
 
+       mutex_lock(&sbi->fs_lock[next_lock]);
 
        return next_lock;
 
 }
fs/f2fs/super.c +0 -0

old mode 100644

new mode 100755

index 75c7dc3..4f27596

--- a/fs/f2fs/super.c

+++ b/fs/f2fs/super.c

fs/f2fs/super.c +1 -0
@@ -657,6 +657,7 @@ static int f2fs_fill_super(struct super_block *sb,
 void *data, int silent)
 
        mutex_init(&sbi->cp_mutex);
 
        for (i = 0; i < NR_GLOBAL_LOCKS; i++)
 
                mutex_init(&sbi->fs_lock[i]);
 
+       spin_lock_init(&sbi->spin_lock);
 
        mutex_init(&sbi->node_write);
 
        sbi->por_doing = 0;
 
        spin_lock_init(&sbi->stat_lock);
Quoted 8 lines
Quoted 7 lines

(END)

------------------------------------------------------------------------------
How ServiceNow helps IT people transform IT departments:
1. Consolidate legacy IT systems to a single system of record for IT
2. Standardize and globalize service processes across IT
3. Implement zero-touch automation to replace manual, redundant tasks
http://pubads.g.doubleclick.net/gampad/clk?id=51271111&iu=/4140/ostg.clktrk
_______________________________________________
Linux-f2fs-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

#4 lore raw

Hi Gu,

2013/9/11 Gu Zheng <[email protected]>:

Quoted 14 lines

Hi Jaegeuk, Chao,

Quoted 9 lines · Jaegeuk Kim

Hi,

At first, thank you for the report and please follow the email writing rules. :)

Anyway, I agree to the below issue.
One thing that I can think of is that we don't need to use the spin_lock, since we don't care about the exact lock number, but just need to get any not-collided number.

Quoted 6 lines

IMHO, just moving sbi->next_lock_num++ before mutex_lock(&sbi->fs_lock[next_lock]) can avoid unbalance issue mostly.
IMO, the case two or more threads increase sbi->next_lock_num in the same time is really very very little. If you think it is not rigorous, change next_lock_num to atomic one can fix it.
What's your opinion?

As your opinion, I think it is enough to replace it with simple sbi->next_lock_num++.
Thanks,

Quoted 153 lines

Regards,
Gu

Quoted 138 lines

So, how about removing the spin_lock?
And how about using a random number?

Thanks,

2013-09-06 (금), 09:48 +0000, Chao Yu:

Quoted 131 lines

Hi Kim:

     I think there is a performance problem: when all sbi->fs_lock is

holded,

then all other threads may get the same next_lock value from sbi->next_lock_num in function mutex_lock_op,

and wait to get the same lock at position fs_lock[next_lock], it unbalance the fs_lock usage.

It may lost performance when we do the multithread test.

Here is the patch to fix this problem:

fs/f2fs/f2fs.h +0 -0

old mode 100644

new mode 100755

index 467d42d..983bb45

--- a/fs/f2fs/f2fs.h

+++ b/fs/f2fs/f2fs.h

fs/f2fs/f2fs.h +0 -0
@@ -371,6 +371,7 @@ struct f2fs_sb_info {
 
        struct mutex fs_lock[NR_GLOBAL_LOCKS];  /* blocking FS

operations */

struct mutex node_write; /* locking node writes
*/

struct mutex writepages; /* mutex for
writepages() */

+ spinlock_t spin_lock; /* lock for
next_lock_num */

unsigned char next_lock_num; /* round-robin global
locks */

int por_doing; /* recovery is doing
or not */

int on_build_free_nids; /* build_free_nids is
doing */

fs/f2fs/f2fs.h +6 -2
@@ -533,15 +534,19 @@ static inline void mutex_unlock_all(struct
 f2fs_sb_info *sbi)
 
 
 
 static inline int mutex_lock_op(struct f2fs_sb_info *sbi)
 
 {
 
-       unsigned char next_lock = sbi->next_lock_num %
 NR_GLOBAL_LOCKS;
 
+       unsigned char next_lock;
 
        int i = 0;
 
 
 
        for (; i < NR_GLOBAL_LOCKS; i++)
 
                if (mutex_trylock(&sbi->fs_lock[i]))
 
                        return i;
 
 
 
-       mutex_lock(&sbi->fs_lock[next_lock]);
 
+       spin_lock(&sbi->spin_lock);
 
+       next_lock = sbi->next_lock_num % NR_GLOBAL_LOCKS;
 
        sbi->next_lock_num++;
 
+       spin_unlock(&sbi->spin_lock);
 
+
 
+       mutex_lock(&sbi->fs_lock[next_lock]);
 
        return next_lock;
 
 }
fs/f2fs/super.c +0 -0

old mode 100644

new mode 100755

index 75c7dc3..4f27596

--- a/fs/f2fs/super.c

+++ b/fs/f2fs/super.c

fs/f2fs/super.c +1 -0
@@ -657,6 +657,7 @@ static int f2fs_fill_super(struct super_block *sb,
 void *data, int silent)
 
        mutex_init(&sbi->cp_mutex);
 
        for (i = 0; i < NR_GLOBAL_LOCKS; i++)
 
                mutex_init(&sbi->fs_lock[i]);
 
+       spin_lock_init(&sbi->spin_lock);
 
        mutex_init(&sbi->node_write);
 
        sbi->por_doing = 0;
 
        spin_lock_init(&sbi->stat_lock);
Quoted 8 lines
Quoted 7 lines

(END)

Quoted 10 lines

------------------------------------------------------------------------------
How ServiceNow helps IT people transform IT departments:
1. Consolidate legacy IT systems to a single system of record for IT
2. Standardize and globalize service processes across IT
3. Implement zero-touch automation to replace manual, redundant tasks
http://pubads.g.doubleclick.net/gampad/clk?id=51271111&iu=/4140/ostg.clktrk
_______________________________________________
Linux-f2fs-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#5 lore raw
In reply to Gu Zheng

Hi Gu

Quoted 31 lines

-----Original Message-----
From: Gu Zheng [mailto:[email protected]]
Sent: Wednesday, September 11, 2013 1:38 PM
To: [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [f2fs-dev][PATCH] f2fs: optimize fs_lock for better performance

Hi Jaegeuk, Chao,

Quoted 9 lines · Jaegeuk Kim

Hi,

At first, thank you for the report and please follow the email writing rules. :)

Anyway, I agree to the below issue.
One thing that I can think of is that we don't need to use the spin_lock, since we don't care about the exact lock number, but just need to get any not-collided number.

IMHO, just moving sbi->next_lock_num++ before mutex_lock(&sbi->fs_lock[next_lock])
can avoid unbalance issue mostly.
IMO, the case two or more threads increase sbi->next_lock_num in the same time is really very very little. If you think it is not rigorous, change next_lock_num to atomic one can fix it.
What's your opinion?

Quoted 2 lines

Regards,
Gu

I did the test sbi->next_lock_num++ compare with the atomic one,
And I found performance of them is almost the same under a small number thread racing. So as your and Kim's opinion, it's enough to use "sbi->next_lock_num++" to fix this issue.

Thanks for the advice.

Quoted 151 lines
Quoted 39 lines

So, how about removing the spin_lock?
And how about using a random number?

Thanks,

2013-09-06 (금), 09:48 +0000, Chao Yu:

Quoted 31 lines

Hi Kim:

     I think there is a performance problem: when all sbi->fs_lock is

holded,

then all other threads may get the same next_lock value from sbi->next_lock_num in function mutex_lock_op,

and wait to get the same lock at position fs_lock[next_lock], it unbalance the fs_lock usage.

It may lost performance when we do the multithread test.

Here is the patch to fix this problem:

fs/f2fs/f2fs.h +0 -0

old mode 100644

new mode 100755

index 467d42d..983bb45

--- a/fs/f2fs/f2fs.h

Quoted 8 lines
Quoted 8 lines

+++ b/fs/f2fs/f2fs.h

fs/f2fs/f2fs.h +0 -0
@@ -371,6 +371,7 @@ struct f2fs_sb_info {
 
        struct mutex fs_lock[NR_GLOBAL_LOCKS];  /* blocking FS

operations */

struct mutex node_write; /* locking node

writes

Quoted 9 lines
Quoted 9 lines

*/

struct mutex writepages; /* mutex for
writepages() */

+ spinlock_t spin_lock; /* lock for
next_lock_num */

unsigned char next_lock_num; /* round-robin

global

Quoted 83 lines
Quoted 83 lines

locks */

int por_doing; /* recovery is doing
or not */

int on_build_free_nids; /* build_free_nids is
doing */

fs/f2fs/f2fs.h +6 -2
@@ -533,15 +534,19 @@ static inline void mutex_unlock_all(struct
 f2fs_sb_info *sbi)
 
 
 
 static inline int mutex_lock_op(struct f2fs_sb_info *sbi)
 
 {
 
-       unsigned char next_lock = sbi->next_lock_num %
 NR_GLOBAL_LOCKS;
 
+       unsigned char next_lock;
 
        int i = 0;
 
 
 
        for (; i < NR_GLOBAL_LOCKS; i++)
 
                if (mutex_trylock(&sbi->fs_lock[i]))
 
                        return i;
 
 
 
-       mutex_lock(&sbi->fs_lock[next_lock]);
 
+       spin_lock(&sbi->spin_lock);
 
+       next_lock = sbi->next_lock_num % NR_GLOBAL_LOCKS;
 
        sbi->next_lock_num++;
 
+       spin_unlock(&sbi->spin_lock);
 
+
 
+       mutex_lock(&sbi->fs_lock[next_lock]);
 
        return next_lock;
 
 }
fs/f2fs/super.c +0 -0

old mode 100644

new mode 100755

index 75c7dc3..4f27596

--- a/fs/f2fs/super.c

+++ b/fs/f2fs/super.c

fs/f2fs/super.c +1 -0
@@ -657,6 +657,7 @@ static int f2fs_fill_super(struct super_block
 *sb, void *data, int silent)
 
        mutex_init(&sbi->cp_mutex);
 
        for (i = 0; i < NR_GLOBAL_LOCKS; i++)
 
                mutex_init(&sbi->fs_lock[i]);
 
+       spin_lock_init(&sbi->spin_lock);
 
        mutex_init(&sbi->node_write);
 
        sbi->por_doing = 0;
 
        spin_lock_init(&sbi->stat_lock);
Quoted 8 lines
Quoted 7 lines

(END)

Quoted 1 line

=

------------------------------------------------------------------------------
How ServiceNow helps IT people transform IT departments:
1. Consolidate legacy IT systems to a single system of record for IT
2. Standardize and globalize service processes across IT
3. Implement zero-touch automation to replace manual, redundant tasks
http://pubads.g.doubleclick.net/gampad/clk?id=51271111&iu=/4140/ostg.clktrk
_______________________________________________
Linux-f2fs-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

#6 lore raw

PATCH v2 1/1 f2fs: optimize fs_lock for better performance

From: Yu Chao <[email protected]>

There is a performance problem: when all sbi->fs_lock are holded, then all the following threads may get the same next_lock value from sbi->next_lock_num in function mutex_lock_op, and wait for the same lock(fs_lock[next_lock]), it may cause performance reduce.
So we move the sbi->next_lock_num++ before getting lock, this will average the following threads if all sbi->fs_lock are holded.

v1-->v2:
Drop the needless spin_lock as Jaegeuk suggested.

---
 fs/f2fs/f2fs.h |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)
fs/f2fs/f2fs.h +2 -2
index 608f0df..7fd99d8 100644
@@ -544,15 +544,15 @@ static inline void mutex_unlock_all(struct f2fs_sb_info *sbi)
 
 static inline int mutex_lock_op(struct f2fs_sb_info *sbi)
 {
-	unsigned char next_lock = sbi->next_lock_num % NR_GLOBAL_LOCKS;
+	unsigned char next_lock;
 	int i = 0;
 
 	for (; i < NR_GLOBAL_LOCKS; i++)
 		if (mutex_trylock(&sbi->fs_lock[i]))
 			return i;
 
+	next_lock = sbi->next_lock_num++ % NR_GLOBAL_LOCKS;
 	mutex_lock(&sbi->fs_lock[next_lock]);
-	sbi->next_lock_num++;
 	return next_lock;
 }
 
Signature · 10 lines
1.7.7



------------------------------------------------------------------------------
How ServiceNow helps IT people transform IT departments:
1. Consolidate legacy IT systems to a single system of record for IT
2. Standardize and globalize service processes across IT
3. Implement zero-touch automation to replace manual, redundant tasks
http://pubads.g.doubleclick.net/gampad/clk?id=51271111&iu=/4140/ostg.clktrk
#7 lore raw
In reply to 俞超

Hi Chao,

Quoted 33 lines · 俞超

Hi Gu

Quoted 31 lines

-----Original Message-----
From: Gu Zheng [mailto:[email protected]]
Sent: Wednesday, September 11, 2013 1:38 PM
To: [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [f2fs-dev][PATCH] f2fs: optimize fs_lock for better performance

Hi Jaegeuk, Chao,

Quoted 9 lines · Jaegeuk Kim

Hi,

At first, thank you for the report and please follow the email writing rules. :)

Anyway, I agree to the below issue.
One thing that I can think of is that we don't need to use the spin_lock, since we don't care about the exact lock number, but just need to get any not-collided number.

IMHO, just moving sbi->next_lock_num++ before mutex_lock(&sbi->fs_lock[next_lock])
can avoid unbalance issue mostly.
IMO, the case two or more threads increase sbi->next_lock_num in the same time is really very very little. If you think it is not rigorous, change next_lock_num to atomic one can fix it.
What's your opinion?

Quoted 6 lines
Quoted 2 lines

Regards,
Gu

I did the test sbi->next_lock_num++ compare with the atomic one,
And I found performance of them is almost the same under a small number thread racing. So as your and Kim's opinion, it's enough to use "sbi->next_lock_num++" to fix this issue.

Good, but it seems that your replay patch is out of format, and it's hard for Jaegeuk to merge. I'll format it, see the following thread.

Thanks,
Gu

Quoted 156 lines

Thanks for the advice.

Quoted 152 lines
Quoted 39 lines

So, how about removing the spin_lock?
And how about using a random number?

Thanks,

2013-09-06 (금), 09:48 +0000, Chao Yu:

Quoted 31 lines

Hi Kim:

     I think there is a performance problem: when all sbi->fs_lock is

holded,

then all other threads may get the same next_lock value from sbi->next_lock_num in function mutex_lock_op,

and wait to get the same lock at position fs_lock[next_lock], it unbalance the fs_lock usage.

It may lost performance when we do the multithread test.

Here is the patch to fix this problem:

fs/f2fs/f2fs.h +0 -0

old mode 100644

new mode 100755

index 467d42d..983bb45

--- a/fs/f2fs/f2fs.h

Quoted 8 lines
Quoted 8 lines

+++ b/fs/f2fs/f2fs.h

fs/f2fs/f2fs.h +0 -0
@@ -371,6 +371,7 @@ struct f2fs_sb_info {
 
        struct mutex fs_lock[NR_GLOBAL_LOCKS];  /* blocking FS

operations */

struct mutex node_write; /* locking node

writes

Quoted 9 lines
Quoted 9 lines

*/

struct mutex writepages; /* mutex for
writepages() */

+ spinlock_t spin_lock; /* lock for
next_lock_num */

unsigned char next_lock_num; /* round-robin

global

Quoted 83 lines
Quoted 83 lines

locks */

int por_doing; /* recovery is doing
or not */

int on_build_free_nids; /* build_free_nids is
doing */

fs/f2fs/f2fs.h +6 -2
@@ -533,15 +534,19 @@ static inline void mutex_unlock_all(struct
 f2fs_sb_info *sbi)
 
 
 
 static inline int mutex_lock_op(struct f2fs_sb_info *sbi)
 
 {
 
-       unsigned char next_lock = sbi->next_lock_num %
 NR_GLOBAL_LOCKS;
 
+       unsigned char next_lock;
 
        int i = 0;
 
 
 
        for (; i < NR_GLOBAL_LOCKS; i++)
 
                if (mutex_trylock(&sbi->fs_lock[i]))
 
                        return i;
 
 
 
-       mutex_lock(&sbi->fs_lock[next_lock]);
 
+       spin_lock(&sbi->spin_lock);
 
+       next_lock = sbi->next_lock_num % NR_GLOBAL_LOCKS;
 
        sbi->next_lock_num++;
 
+       spin_unlock(&sbi->spin_lock);
 
+
 
+       mutex_lock(&sbi->fs_lock[next_lock]);
 
        return next_lock;
 
 }
fs/f2fs/super.c +0 -0

old mode 100644

new mode 100755

index 75c7dc3..4f27596

--- a/fs/f2fs/super.c

+++ b/fs/f2fs/super.c

fs/f2fs/super.c +1 -0
@@ -657,6 +657,7 @@ static int f2fs_fill_super(struct super_block
 *sb, void *data, int silent)
 
        mutex_init(&sbi->cp_mutex);
 
        for (i = 0; i < NR_GLOBAL_LOCKS; i++)
 
                mutex_init(&sbi->fs_lock[i]);
 
+       spin_lock_init(&sbi->spin_lock);
 
        mutex_init(&sbi->node_write);
 
        sbi->por_doing = 0;
 
        spin_lock_init(&sbi->stat_lock);
Quoted 8 lines
Quoted 7 lines

(END)

=

------------------------------------------------------------------------------
How ServiceNow helps IT people transform IT departments:
1. Consolidate legacy IT systems to a single system of record for IT
2. Standardize and globalize service processes across IT
3. Implement zero-touch automation to replace manual, redundant tasks
http://pubads.g.doubleclick.net/gampad/clk?id=51271111&iu=/4140/ostg.clktrk
_______________________________________________
Linux-f2fs-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel