Skip to contentSkip to conversation

bit_spinlock: introduce smp_cond_load_relaxed

RFC PATCH In review Gao Xiang 5 participants 19 messages Oct 13, 2018 – Nov 6, 2018

Subsystems (2)
lore
Share & tools
lore

Gao Xiang

author
#0 lore raw

RFC PATCH 1/1 bit_spinlock: introduce smp_cond_load_relaxed

It is better to use smp_cond_load_relaxed instead
of busy waiting for bit_spinlock.

---
 include/linux/bit_spinlock.h | 21 ++++++++-------------
 1 file changed, 8 insertions(+), 13 deletions(-)
include/linux/bit_spinlock.h +8 -13
index bbc4730..713efc4 100644
@@ -15,22 +15,17 @@
  */
 static inline void bit_spin_lock(int bitnum, unsigned long *addr)
 {
-	/*
-	 * Assuming the lock is uncontended, this never enters
-	 * the body of the outer loop. If it is contended, then
-	 * within the inner loop a non-atomic test is used to
-	 * busywait with less bus contention for a good time to
-	 * attempt to acquire the lock bit.
-	 */
-	preempt_disable();
 #if defined(CONFIG_SMP) || defined(CONFIG_DEBUG_SPINLOCK)
-	while (unlikely(test_and_set_bit_lock(bitnum, addr))) {
-		preempt_enable();
-		do {
-			cpu_relax();
-		} while (test_bit(bitnum, addr));
+	while (1) {
+		smp_cond_load_relaxed(&addr[BIT_WORD(bitnum)],
+				      !(VAL >> (bitnum & (BITS_PER_LONG-1))));
 		preempt_disable();
+		if (!test_and_set_bit_lock(bitnum, addr))
+			break;
+		preempt_enable();
 	}
+#else
+	preempt_disable();
 #endif
 	__acquire(bitlock);
 }
Signature · 1 line
2.7.4
#1 lore raw
Quoted 2 lines · Gao Xiang

It is better to use smp_cond_load_relaxed instead
of busy waiting for bit_spinlock.

Why? I think we need some kind of "proof" that this is true before being able to accept a patch like this, don't you agree?

thanks,

greg k-h

author
#2 lore raw

Hi Greg,

Quoted 6 lines · Greg Kroah-Hartman
Quoted 2 lines · Gao Xiang

It is better to use smp_cond_load_relaxed instead
of busy waiting for bit_spinlock.

Why? I think we need some kind of "proof" that this is true before being able to accept a patch like this, don't you agree?

There are some materials which discuss smp_cond_load_* earlier.
https://patchwork.kernel.org/patch/10335991/
https://patchwork.kernel.org/patch/10325057/

In ARM64, they implements a function called "cmpwait", which uses hardware instructions to monitor a value change, I think it is more energy efficient than just do a open-code busy loop...

And it seem smp_cond_load_* is already used in the current kernel, such as:
./kernel/locking/mcs_spinlock.h
./kernel/locking/qspinlock.c
./kernel/sched/core.c
./kernel/smp.c

For other architectures like x86/arm64, I think they could implement smp_cond_load_* later.

Thanks,
Gao Xiang

Quoted 5 lines

thanks,

greg k-h

#3 lore raw
Quoted 18 lines · Gao Xiang

Hi Greg,

Quoted 6 lines · Greg Kroah-Hartman
Quoted 2 lines · Gao Xiang

It is better to use smp_cond_load_relaxed instead
of busy waiting for bit_spinlock.

Why? I think we need some kind of "proof" that this is true before being able to accept a patch like this, don't you agree?

There are some materials which discuss smp_cond_load_* earlier.
https://patchwork.kernel.org/patch/10335991/
https://patchwork.kernel.org/patch/10325057/

In ARM64, they implements a function called "cmpwait", which uses hardware instructions to monitor a value change, I think it is more energy efficient than just do a open-code busy loop...

Quoted 8 lines

And it seem smp_cond_load_* is already used in the current kernel, such as:
./kernel/locking/mcs_spinlock.h
./kernel/locking/qspinlock.c
./kernel/sched/core.c
./kernel/smp.c

For other architectures like x86/arm64, I think they could implement smp_cond_load_* later.

And have you benchmarked this change to show that it provides any benifit?

You need to do that...

thanks,

greg k-h

author
#4 lore raw

Hi Greg,

Quoted 25 lines · Greg Kroah-Hartman
Quoted 24 lines · Gao Xiang

Hi Greg,

Quoted 6 lines · Greg Kroah-Hartman
Quoted 2 lines · Gao Xiang

It is better to use smp_cond_load_relaxed instead
of busy waiting for bit_spinlock.

Why? I think we need some kind of "proof" that this is true before being able to accept a patch like this, don't you agree?

There are some materials which discuss smp_cond_load_* earlier.
https://patchwork.kernel.org/patch/10335991/
https://patchwork.kernel.org/patch/10325057/

In ARM64, they implements a function called "cmpwait", which uses hardware instructions to monitor a value change, I think it is more energy efficient than just do a open-code busy loop...

And it seem smp_cond_load_* is already used in the current kernel, such as:
./kernel/locking/mcs_spinlock.h
./kernel/locking/qspinlock.c
./kernel/sched/core.c
./kernel/smp.c

Quoted 7 lines
Quoted 2 lines

For other architectures like x86/arm64, I think they could implement smp_cond_load_* later.

And have you benchmarked this change to show that it provides any benifit?

You need to do that...

OK, it is my responsibility to test this patch in ARM64 indeed.

I will test it later in ARM64 to see if it has any performance difference after I handled EROFS product landing stuffs (perhaps weeks later, many urgent stuffs for the current product that I need to solve...)

Or if some warm-hearted folks interest in it, I'm very happy to see other implementations or comments about that. :)

Thanks,
Gao Xiang

Quoted 5 lines

thanks,

greg k-h

author
#5 lore raw
In reply to Gao Xiang
Quoted 2 lines · Gao Xiang

For other architectures like x86/arm64, I think they could implement smp_cond_load_* later.

Sorry about that, I mean "amd64".

Actually I don't have performance numbers to proof that now. I think it really depends on the detailed architecture hardware implementation.

In my opinion, I just think it is better to wrap it up rather than do open-coded all around...
do {
cpu_relax()
} while(...);

I was just cleaning up EROFS file system, and saw these piece of code (bit_spinlock) by chance. Therefore I write a patch to get some idea about it....

Thanks,
Gao Xiang

#6 lore raw

PATCH v2 1/1 bit_spinlock: introduce smp_cond_load_relaxed

In reply to Gao Xiang

It is better to use wrapped smp_cond_load_relaxed instead of open-coded busy waiting for bit_spinlock.

---

change log v2:
- fix the incorrect expression !(VAL >> (bitnum & (BITS_PER_LONG-1)))
- the test result is described in the following reply.

Thanks,
Gao Xiang

 include/linux/bit_spinlock.h | 23 ++++++++++-------------
 1 file changed, 10 insertions(+), 13 deletions(-)
include/linux/bit_spinlock.h +10 -13
index bbc4730a6505..d5f922b5ffd9 100644
@@ -15,22 +15,19 @@
  */
 static inline void bit_spin_lock(int bitnum, unsigned long *addr)
 {
-	/*
-	 * Assuming the lock is uncontended, this never enters
-	 * the body of the outer loop. If it is contended, then
-	 * within the inner loop a non-atomic test is used to
-	 * busywait with less bus contention for a good time to
-	 * attempt to acquire the lock bit.
-	 */
-	preempt_disable();
 #if defined(CONFIG_SMP) || defined(CONFIG_DEBUG_SPINLOCK)
-	while (unlikely(test_and_set_bit_lock(bitnum, addr))) {
-		preempt_enable();
-		do {
-			cpu_relax();
-		} while (test_bit(bitnum, addr));
+	const unsigned int bitshift = bitnum & (BITS_PER_LONG - 1);
+
+	while (1) {
+		smp_cond_load_relaxed(&addr[BIT_WORD(bitnum)],
+				      !((VAL >> bitshift) & 1));
 		preempt_disable();
+		if (!test_and_set_bit_lock(bitnum, addr))
+			break;
+		preempt_enable();
 	}
+#else
+	preempt_disable();
 #endif
 	__acquire(bitlock);
 }
Signature · 1 line
2.17.1
#7 lore raw

Hi,

Quoted 4 lines · Gao Xiang

It is better to use wrapped smp_cond_load_relaxed instead of open-coded busy waiting for bit_spinlock.

Quoted 8 lines

---

change log v2:
- fix the incorrect expression !(VAL >> (bitnum & (BITS_PER_LONG-1)))
- the test result is described in the following reply.

Thanks,
Gao Xiang

Simple test script:
#include <linux/kthread.h>
#include <linux/bit_spinlock.h>
#include <linux/module.h>

unsigned long global_lock;

int test_thread(void *data)

{
	unsigned long thread_id = (unsigned long)data;
	int i;
	u64 start = ktime_get_ns();

	for (i = 0; i < 500000; ++i) {
		bit_spin_lock(0, &global_lock);
		__asm__("yield");
		bit_spin_unlock(0, &global_lock);
	}
	pr_err("Thread id: %lu time: %llu\n", thread_id, ktime_get_ns() - start);

	do_exit(0);
}

static int __init bitspinlock_test_module_init(void)

{
	int i;

	for (i = 0; i < 8; ++i) {
		if (IS_ERR(kthread_run(test_thread, (void *)(unsigned long)i, "thread-%d", i)))
			pr_err("fail to create thread %d\n", i);
	}

	return 0;
}

static void __exit bitspinlock_test_module_exit(void) {
}

module_init(bitspinlock_test_module_init);
module_exit(bitspinlock_test_module_exit);
MODULE_LICENSE("GPL");

...and tested in the following ARM server environment:

Processor: HI1616 (https://en.wikichip.org/wiki/hisilicon/hi16xx/hi1616)
Board: HiSilicon D05 Development Board (http://open-estuary.org/d05/)
Memory: 512GB
Host OS: Ubuntu 18.04.1 LTS (Ubuntu 4.15.0-29.31-generic 4.15.18)
QEMU KVM OS: Linux 4.19 + buildroot
QEMU KVM cmdline: qemu-system-aarch64 -enable-kvm -cpu host -smp 4 -m 256M -kernel Image -M virt,kernel_irqchip=on -nographic -hda rootfs.ext2 -append 'root=/dev/vda console=ttyAMA0 earlycon=pl011,0x9000000' -serial mon:stdio -net none

Without this patch:
Thread 0 Thread 1 Thread 2 Thread 3 Thread 4 Thread 5 Thread 6 Thread 7

1 1283709480 1271869280  454742480 1173673820 1145643640 1118846920  774616920 1144146140
2  643580180  625143860  576841700  322982340  649987880  585749000  529178880  373374780
3  672307220  847315000  880801860 1039502040  667086380 1033939940 1035381120 1046898300
4  568635580  440547020  737000380  910040880  804543740  712314280  868896880  867049000
5  749107320  726397720  776134480  611970100  756721040  753449440  711691300  609343300

With this patch:
Thread 0 Thread 1 Thread 2 Thread 3 Thread 4 Thread 5 Thread 6 Thread 7

1  170327620  196322160  169434180   74723860  178145600  178873460  143843260   70998780
2  166415220  129649200  166161240  175241520  155474460  112811860  157003140  150087420
3  511420780  117655640  598641860  596213720  462888760  430838600  554346300  428035120 
4  174520240  156311800  120274280   87465380  172781400  136118620  163728340   63026360
5  153677940  202786860  183626500  140721300  150311360  161266840  168154340  107247460

Thanks,
Gao Xiang

author
#8 lore raw

Hi all,

ping... is there someone interested in this patch?

It seems bit_spinlock performs much better by using smp_cond_load_relaxed than just do busy-loop in arm64...

Thanks,
Gao Xiang

Quoted 90 lines · Gao Xiang

Hi,

Quoted 4 lines · Gao Xiang

It is better to use wrapped smp_cond_load_relaxed instead of open-coded busy waiting for bit_spinlock.

Quoted 8 lines

---

change log v2:
- fix the incorrect expression !(VAL >> (bitnum & (BITS_PER_LONG-1)))
- the test result is described in the following reply.

Thanks,
Gao Xiang

Simple test script:
#include <linux/kthread.h>
#include <linux/bit_spinlock.h>
#include <linux/module.h>

unsigned long global_lock;

int test_thread(void *data)

{
	unsigned long thread_id = (unsigned long)data;
	int i;
	u64 start = ktime_get_ns();

	for (i = 0; i < 500000; ++i) {
		bit_spin_lock(0, &global_lock);
		__asm__("yield");
		bit_spin_unlock(0, &global_lock);
	}
	pr_err("Thread id: %lu time: %llu\n", thread_id, ktime_get_ns() - start);

	do_exit(0);
}

static int __init bitspinlock_test_module_init(void)

{
	int i;

	for (i = 0; i < 8; ++i) {
		if (IS_ERR(kthread_run(test_thread, (void *)(unsigned long)i, "thread-%d", i)))
			pr_err("fail to create thread %d\n", i);
	}

	return 0;
}

static void __exit bitspinlock_test_module_exit(void) {
}

module_init(bitspinlock_test_module_init);
module_exit(bitspinlock_test_module_exit);
MODULE_LICENSE("GPL");

...and tested in the following ARM server environment:

Processor: HI1616 (https://en.wikichip.org/wiki/hisilicon/hi16xx/hi1616)
Board: HiSilicon D05 Development Board (http://open-estuary.org/d05/)
Memory: 512GB
Host OS: Ubuntu 18.04.1 LTS (Ubuntu 4.15.0-29.31-generic 4.15.18)
QEMU KVM OS: Linux 4.19 + buildroot
QEMU KVM cmdline: qemu-system-aarch64 -enable-kvm -cpu host -smp 4 -m 256M -kernel Image -M virt,kernel_irqchip=on -nographic -hda rootfs.ext2 -append 'root=/dev/vda console=ttyAMA0 earlycon=pl011,0x9000000' -serial mon:stdio -net none

Without this patch:
Thread 0 Thread 1 Thread 2 Thread 3 Thread 4 Thread 5 Thread 6 Thread 7

1 1283709480 1271869280  454742480 1173673820 1145643640 1118846920  774616920 1144146140
2  643580180  625143860  576841700  322982340  649987880  585749000  529178880  373374780
3  672307220  847315000  880801860 1039502040  667086380 1033939940 1035381120 1046898300
4  568635580  440547020  737000380  910040880  804543740  712314280  868896880  867049000
5  749107320  726397720  776134480  611970100  756721040  753449440  711691300  609343300

With this patch:
Thread 0 Thread 1 Thread 2 Thread 3 Thread 4 Thread 5 Thread 6 Thread 7

1  170327620  196322160  169434180   74723860  178145600  178873460  143843260   70998780
2  166415220  129649200  166161240  175241520  155474460  112811860  157003140  150087420
3  511420780  117655640  598641860  596213720  462888760  430838600  554346300  428035120 
4  174520240  156311800  120274280   87465380  172781400  136118620  163728340   63026360
5  153677940  202786860  183626500  140721300  150311360  161266840  168154340  107247460

Thanks,
Gao Xiang

#9 lore raw
In reply to Gao Xiang

[+PeterZ -- please include him on stuff like this]

Hi Gao,

Quoted 9 lines · Gao Xiang

It is better to use wrapped smp_cond_load_relaxed instead of open-coded busy waiting for bit_spinlock.

---

change log v2:
- fix the incorrect expression !(VAL >> (bitnum & (BITS_PER_LONG-1)))
- the test result is described in the following reply.

Please include the results in the commit message, so that this change is justified.

Quoted 24 lines
include/linux/bit_spinlock.h +2 -13
index bbc4730a6505..d5f922b5ffd9 100644
@@ -15,22 +15,19 @@
  */
 static inline void bit_spin_lock(int bitnum, unsigned long *addr)
 {
-	/*
-	 * Assuming the lock is uncontended, this never enters
-	 * the body of the outer loop. If it is contended, then
-	 * within the inner loop a non-atomic test is used to
-	 * busywait with less bus contention for a good time to
-	 * attempt to acquire the lock bit.
-	 */
-	preempt_disable();
 #if defined(CONFIG_SMP) || defined(CONFIG_DEBUG_SPINLOCK)
-	while (unlikely(test_and_set_bit_lock(bitnum, addr))) {
-		preempt_enable();
-		do {
-			cpu_relax();
-		} while (test_bit(bitnum, addr));
+	const unsigned int bitshift = bitnum & (BITS_PER_LONG - 1);
+

include/linux/bit_spinlock.h line 30

Quoted 10 lines
include/linux/bit_spinlock.h +8 -0
+	while (1) {
+		smp_cond_load_relaxed(&addr[BIT_WORD(bitnum)],
+				      !((VAL >> bitshift) & 1));
 		preempt_disable();
+		if (!test_and_set_bit_lock(bitnum, addr))
+			break;
+		preempt_enable();
 	}
+#else
+	preempt_disable();

This appears to introduce a bunch of overhead for the uncontended fastpath. How about the much-simpler-but-completely-untested (tm) patch below?

Will

--->8

include/asm-generic/bitops/lock.h +9 -0
index 3ae021368f48..9de8d3544630 100644
@@ -6,6 +6,15 @@
 #include <linux/compiler.h>
 #include <asm/barrier.h>
 
+static inline void spin_until_bit_unlock(unsigned int nr,
+					 volatile unsigned long *p)
+{
+	unsigned long mask = BIT_MASK(bitnum);
+
+	p += BIT_WORD(nr);
+	smp_cond_load_relaxed(p, VAL & mask);
+}
+
 /**
  * test_and_set_bit_lock - Set a bit and return its old value, for lock
  * @nr: Bit to set
include/linux/bit_spinlock.h +1 -3
index bbc4730a6505..d711c62e718c 100644
@@ -26,9 +26,7 @@ static inline void bit_spin_lock(int bitnum, unsigned long *addr)
 #if defined(CONFIG_SMP) || defined(CONFIG_DEBUG_SPINLOCK)
 	while (unlikely(test_and_set_bit_lock(bitnum, addr))) {
 		preempt_enable();
-		do {
-			cpu_relax();
-		} while (test_bit(bitnum, addr));
+		spin_until_bit_unlock(bitnum, addr);
 		preempt_disable();
 	}
 #endif
#10 lore raw

Hi Will,

Quoted 6 lines · Will Deacon

Hi Gao,

Quoted 3 lines · Gao Xiang

It is better to use wrapped smp_cond_load_relaxed instead of open-coded busy waiting for bit_spinlock.

Quoted 8 lines
Quoted 6 lines

---

change log v2:
- fix the incorrect expression !(VAL >> (bitnum & (BITS_PER_LONG-1)))
- the test result is described in the following reply.

Please include the results in the commit message, so that this change is justified.

Will add in the next version...

Quoted 3 lines

This appears to introduce a bunch of overhead for the uncontended fastpath. How about the much-simpler-but-completely-untested (tm) patch below?

Actually I thought to do like the following (much simpler indeed) at first...

But the current implementation of smp_cond_load_relaxed will do a judgement immediately which seems unnecessary (right after the test_and_set_bit_lock rather than after __cmpwait_relaxed...)

        for (;;) {							\
		VAL = READ_ONCE(*__PTR);				\
		if (cond_expr)						\
			break;						\
		__cmpwait_relaxed(__PTR, VAL);				\
	}								\

p.s. I have no idea the original uncontended fastpath really works effectively... some idea about this? Thanks in advance...

Thanks,
Gao Xiang

Quoted 40 lines

Will

--->8

include/asm-generic/bitops/lock.h +9 -0
index 3ae021368f48..9de8d3544630 100644
@@ -6,6 +6,15 @@
 #include <linux/compiler.h>
 #include <asm/barrier.h>
 
+static inline void spin_until_bit_unlock(unsigned int nr,
+					 volatile unsigned long *p)
+{
+	unsigned long mask = BIT_MASK(bitnum);
+
+	p += BIT_WORD(nr);
+	smp_cond_load_relaxed(p, VAL & mask);
+}
+
 /**
  * test_and_set_bit_lock - Set a bit and return its old value, for lock
  * @nr: Bit to set
include/linux/bit_spinlock.h +1 -3
index bbc4730a6505..d711c62e718c 100644
@@ -26,9 +26,7 @@ static inline void bit_spin_lock(int bitnum, unsigned long *addr)
 #if defined(CONFIG_SMP) || defined(CONFIG_DEBUG_SPINLOCK)
 	while (unlikely(test_and_set_bit_lock(bitnum, addr))) {
 		preempt_enable();
-		do {
-			cpu_relax();
-		} while (test_bit(bitnum, addr));
+		spin_until_bit_unlock(bitnum, addr);
 		preempt_disable();
 	}
 #endif
#11 lore raw
In reply to Will Deacon
Quoted 25 lines · Will Deacon
include/asm-generic/bitops/lock.h +9 -0
index 3ae021368f48..9de8d3544630 100644
@@ -6,6 +6,15 @@
 #include <linux/compiler.h>
 #include <asm/barrier.h>
 
+static inline void spin_until_bit_unlock(unsigned int nr,
+					 volatile unsigned long *p)
+{
+	unsigned long mask = BIT_MASK(bitnum);
+
+	p += BIT_WORD(nr);
+	smp_cond_load_relaxed(p, VAL & mask);
+}
+
 /**
  * test_and_set_bit_lock - Set a bit and return its old value, for lock
  * @nr: Bit to set
include/linux/bit_spinlock.h +0 -0
index bbc4730a6505..d711c62e718c 100644

include/linux/bit_spinlock.h static inline void bit_spin_lock(int bitnum, unsigned long *addr) · line 32

Quoted 10 lines
include/linux/bit_spinlock.h +1 -3
@@ -26,9 +26,7 @@ static inline void bit_spin_lock(int bitnum, unsigned long *addr)
 #if defined(CONFIG_SMP) || defined(CONFIG_DEBUG_SPINLOCK)
 	while (unlikely(test_and_set_bit_lock(bitnum, addr))) {
 		preempt_enable();
-		do {
-			cpu_relax();
-		} while (test_bit(bitnum, addr));
+		spin_until_bit_unlock(bitnum, addr);
 		preempt_disable();
 	}
 #endif

Yes, that's much better. Ideally though, we'd get rid of bit spinlocks that have significant enough contention for this to matter.

#12 lore raw

Hi Peter,

Quoted 31 lines · Peter Zijlstra
Quoted 30 lines · Will Deacon
include/asm-generic/bitops/lock.h +9 -0
index 3ae021368f48..9de8d3544630 100644
@@ -6,6 +6,15 @@
 #include <linux/compiler.h>
 #include <asm/barrier.h>
 
+static inline void spin_until_bit_unlock(unsigned int nr,
+					 volatile unsigned long *p)
+{
+	unsigned long mask = BIT_MASK(bitnum);
+
+	p += BIT_WORD(nr);
+	smp_cond_load_relaxed(p, VAL & mask);
+}
+
 /**
  * test_and_set_bit_lock - Set a bit and return its old value, for lock
  * @nr: Bit to set
include/linux/bit_spinlock.h +0 -2
index bbc4730a6505..d711c62e718c 100644
@@ -26,9 +26,7 @@ static inline void bit_spin_lock(int bitnum, unsigned long *addr)
 #if defined(CONFIG_SMP) || defined(CONFIG_DEBUG_SPINLOCK)
 	while (unlikely(test_and_set_bit_lock(bitnum, addr))) {
 		preempt_enable();
-		do {
-			cpu_relax();
Quoted 8 lines

include/linux/bit_spinlock.h static inline void bit_spin_lock(int bitnum, unsigned long *addr) · line 32

Quoted 5 lines
include/linux/bit_spinlock.h +1 -1
-		} while (test_bit(bitnum, addr));
+		spin_until_bit_unlock(bitnum, addr);
 		preempt_disable();
 	}
 #endif

Yes, that's much better. Ideally though, we'd get rid of bit spinlocks that have significant enough contention for this to matter.

OK, I will send v3 to fix like the above.

Thanks,
Gao Xiang

Quoted 2 lines
#13 lore raw
Quoted 43 lines · Gao Xiang

Hi Peter,

Quoted 31 lines · Peter Zijlstra
Quoted 30 lines · Will Deacon
include/asm-generic/bitops/lock.h +9 -0
index 3ae021368f48..9de8d3544630 100644
@@ -6,6 +6,15 @@
 #include <linux/compiler.h>
 #include <asm/barrier.h>
 
+static inline void spin_until_bit_unlock(unsigned int nr,
+					 volatile unsigned long *p)
+{
+	unsigned long mask = BIT_MASK(bitnum);
+
+	p += BIT_WORD(nr);
+	smp_cond_load_relaxed(p, VAL & mask);
+}
+
 /**
  * test_and_set_bit_lock - Set a bit and return its old value, for lock
  * @nr: Bit to set
include/linux/bit_spinlock.h +0 -2
index bbc4730a6505..d711c62e718c 100644
@@ -26,9 +26,7 @@ static inline void bit_spin_lock(int bitnum, unsigned long *addr)
 #if defined(CONFIG_SMP) || defined(CONFIG_DEBUG_SPINLOCK)
 	while (unlikely(test_and_set_bit_lock(bitnum, addr))) {
 		preempt_enable();
-		do {
-			cpu_relax();
Quoted 8 lines

include/linux/bit_spinlock.h static inline void bit_spin_lock(int bitnum, unsigned long *addr) · line 32

Quoted 5 lines
include/linux/bit_spinlock.h +1 -1
-		} while (test_bit(bitnum, addr));
+		spin_until_bit_unlock(bitnum, addr);
 		preempt_disable();
 	}
 #endif

Yes, that's much better. Ideally though, we'd get rid of bit spinlocks that have significant enough contention for this to matter.

Quoted 1 line

OK, I will send v3 to fix like the above.

That's not answering the full question though. What bit spinlocks did you find where this matters? And can't we convert them to proper spinlocks instead?

#14 lore raw

Hi Peter,

Quoted 6 lines · Peter Zijlstra
Quoted 3 lines
Quoted 2 lines

Yes, that's much better. Ideally though, we'd get rid of bit spinlocks that have significant enough contention for this to matter.

OK, I will send v3 to fix like the above.

That's not answering the full question though. What bit spinlocks did you find where this matters? And can't we convert them to proper spinlocks instead?

I just misunderstood your question and I get your point now.

"What bit spinlocks did you find where this matters?" nope..I said the original background to Greg before, that is I tried to use smp_cond_load_relaxed instead of busy spining in the development of erofs file system and I saw this bit_spinlock implementation by chance...This is not a big modification but since I raised the question before and I want to trace to the end...

"And can't we convert them to proper spinlocks instead?" I think bit_spinlock is sometime preferred since it requires little memory and can be integrated into some fields(eg. flags)...It is selectable for spinlocks don't have to many users at the same time, so I think it depends on the detailed real use scenerio... It is just a tool for user code to select case by case... That is my personal idea...

IMO, to use wrapped up function for the detailed scenario could be better than open-coded all the time (eg. do cpu_relax(); while(...)) since it could be optimizated even more for the specific architecture...

Thanks,
Gao Xiang

#15 lore raw
Quoted 3 lines · Gao Xiang

IMO, to use wrapped up function for the detailed scenario could be better than open-coded all the time (eg. do cpu_relax(); while(...)) since it could be optimizated even more for the specific architecture...

That's the whole point though; if this actually matters, you're doing it wrong.

#16 lore raw

Hi Peter,

Quoted 6 lines · Peter Zijlstra
Quoted 3 lines · Gao Xiang

IMO, to use wrapped up function for the detailed scenario could be better than open-coded all the time (eg. do cpu_relax(); while(...)) since it could be optimizated even more for the specific architecture...

That's the whole point though; if this actually matters, you're doing it wrong.

I cannot fully understand your point...Sorry about my English...

To the point, you mean it is much better to fix it as Will suggested before or leave the matter as it is since the performance of bit_spinlock itself doesn't matter?

Thanks in advance.

Thanks,
Gao Xiang

#17 lore raw

Hi Peter,

OK, I think I understand your point. If you think the performance of bit_spinlock doesn't matter, Let's keep the current status.

Thanks,
Gao Xiang

Quoted 20 lines · Gao Xiang

Hi Peter,

Quoted 6 lines · Peter Zijlstra
Quoted 3 lines · Gao Xiang

IMO, to use wrapped up function for the detailed scenario could be better than open-coded all the time (eg. do cpu_relax(); while(...)) since it could be optimizated even more for the specific architecture...

That's the whole point though; if this actually matters, you're doing it wrong.

I cannot fully understand your point...Sorry about my English...

To the point, you mean it is much better to fix it as Will suggested before or leave the matter as it is since the performance of bit_spinlock itself doesn't matter?

Thanks in advance.

Thanks,
Gao Xiang

#18 lore raw
In reply to Gao Xiang
Quoted 7 lines · Gao Xiang

Hi Peter,

Quoted 4 lines · Peter Zijlstra
Quoted 3 lines · Gao Xiang

IMO, to use wrapped up function for the detailed scenario could be better than open-coded all the time (eg. do cpu_relax(); while(...)) since it could be optimizated even more for the specific architecture...

Quoted 7 lines
Quoted 2 lines

That's the whole point though; if this actually matters, you're doing it wrong.

I cannot fully understand your point...Sorry about my English...

To the point, you mean it is much better to fix it as Will suggested before or leave the matter as it is since the performance of bit_spinlock itself doesn't matter?

Right, bit-spinlocks are terrible when contended. If the contended behaviour of bit-spinlocks start to matter, you've lost already.