Skip to contentSkip to conversation

Use WFE for spinlock and ring

PATCH Superseded Gavin Hu 9 participants 60 messages Apr 24, 2020 – Jul 9, 2021 Arm/Amlogic Meson SoC support

lore
Share & tools
lore

Gavin Hu

author
#0 lore raw

cover Use WFE for spinlock and ring

The rte_wait_until_equal_xxx APIs abstract the functionality of 'polling for a memory location to become equal to a given value'[1].

Use the API for the rte spinlock and ring implementations.

[1] http://patches.dpdk.org/cover/62703/

Gavin Hu (2):
spinlock: use wfe to reduce contention on aarch64
ring: use wfe to wait for ring tail update on aarch64

 lib/librte_eal/include/generic/rte_spinlock.h | 5 ++---
 lib/librte_ring/rte_ring_c11_mem.h            | 4 ++--
 lib/librte_ring/rte_ring_generic.h            | 3 +--
 3 files changed, 5 insertions(+), 7 deletions(-)
Signature · 1 line
2.17.1
author
#1 lore raw

PATCH 1/2 spinlock: use wfe to reduce contention on aarch64

In acquiring a spinlock, cores repeatedly poll the lock variable. This is replaced by rte_wait_until_equal API.

Running the micro benchmarking and the testpmd and l3fwd traffic tests on ThunderX2, Ampere eMAG80 and Arm N1SDP, everything went well and no notable performance gain nor degradation was measured.

---
 lib/librte_eal/include/generic/rte_spinlock.h | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)
lib/librte_eal/include/generic/rte_spinlock.h +2 -3
index 87ae7a4f1..5cc123247 100644
@@ -28,7 +28,7 @@
  * The rte_spinlock_t type.
  */
 typedef struct {
-	volatile int locked; /**< lock status 0 = unlocked, 1 = locked */
+	volatile uint32_t locked; /**< lock status 0 = unlocked, 1 = locked */
 } rte_spinlock_t;
 
 /**
@@ -65,8 +65,7 @@ rte_spinlock_lock(rte_spinlock_t *sl)
 
 	while (!__atomic_compare_exchange_n(&sl->locked, &exp, 1, 0,
 				__ATOMIC_ACQUIRE, __ATOMIC_RELAXED)) {
-		while (__atomic_load_n(&sl->locked, __ATOMIC_RELAXED))
-			rte_pause();
+		rte_wait_until_equal_32(&sl->locked, 0, __ATOMIC_RELAXED);
 		exp = 0;
 	}
 }
Signature · 1 line
2.17.1
author
#2 lore raw

PATCH 2/2 ring: use wfe to wait for ring tail update on aarch64

In reply to Gavin Hu

Instead of polling for tail to be updated, use wfe instruction.

---
 lib/librte_ring/rte_ring_c11_mem.h | 4 ++--
 lib/librte_ring/rte_ring_generic.h | 3 +--
 2 files changed, 3 insertions(+), 4 deletions(-)
lib/librte_ring/rte_ring_c11_mem.h +2 -2
index 0fb73a337..764d8f186 100644
@@ -2,6 +2,7 @@
  *
  * Copyright (c) 2017,2018 HXT-semitech Corporation.
  * Copyright (c) 2007-2009 Kip Macy [email protected]
+ * Copyright (c) 2019 Arm Limited
  * All rights reserved.
  * Derived from FreeBSD's bufring.h
  * Used as BSD-3 Licensed with permission from Kip Macy.
@@ -21,8 +22,7 @@ update_tail(struct rte_ring_headtail *ht, uint32_t old_val, uint32_t new_val,
 	 * we need to wait for them to complete
 	 */
 	if (!single)
-		while (unlikely(ht->tail != old_val))
-			rte_pause();
+		rte_wait_until_equal_relaxed_32(&ht->tail, old_val);
 
 	__atomic_store_n(&ht->tail, new_val, __ATOMIC_RELEASE);
 }
lib/librte_ring/rte_ring_generic.h +1 -2
index 953cdbbd5..682852783 100644
@@ -23,8 +23,7 @@ update_tail(struct rte_ring_headtail *ht, uint32_t old_val, uint32_t new_val,
 	 * we need to wait for them to complete
 	 */
 	if (!single)
-		while (unlikely(ht->tail != old_val))
-			rte_pause();
+		rte_wait_until_equal_relaxed_32(&ht->tail, old_val);
 
 	ht->tail = new_val;
 }
Signature · 1 line
2.17.1
#3 lore raw

Re: [dpdk-dev] [PATCH v1 2/2] ring: use wfe to wait for ring tail update on aarch64

Quoted 41 lines

Instead of polling for tail to be updated, use wfe instruction.

---
 lib/librte_ring/rte_ring_c11_mem.h | 4 ++--
 lib/librte_ring/rte_ring_generic.h | 3 +--
 2 files changed, 3 insertions(+), 4 deletions(-)
lib/librte_ring/rte_ring_c11_mem.h +2 -2
index 0fb73a337..764d8f186 100644
@@ -2,6 +2,7 @@
  *
  * Copyright (c) 2017,2018 HXT-semitech Corporation.
  * Copyright (c) 2007-2009 Kip Macy [email protected]
+ * Copyright (c) 2019 Arm Limited
  * All rights reserved.
  * Derived from FreeBSD's bufring.h
  * Used as BSD-3 Licensed with permission from Kip Macy.
@@ -21,8 +22,7 @@ update_tail(struct rte_ring_headtail *ht, uint32_t old_val, uint32_t new_val,
 	 * we need to wait for them to complete
 	 */
 	if (!single)
-		while (unlikely(ht->tail != old_val))
-			rte_pause();
+		rte_wait_until_equal_relaxed_32(&ht->tail, old_val);
 
 	__atomic_store_n(&ht->tail, new_val, __ATOMIC_RELEASE);
 }
lib/librte_ring/rte_ring_generic.h +0 -0
index 953cdbbd5..682852783 100644

lib/librte_ring/rte_ring_generic.h update_tail(struct rte_ring_headtail *ht, uint32_t old_val, uint32_t new_val, · line 26

Quoted 6 lines
lib/librte_ring/rte_ring_generic.h +1 -2
@@ -23,8 +23,7 @@ update_tail(struct rte_ring_headtail *ht, uint32_t old_val, uint32_t new_val,
 	 * we need to wait for them to complete
 	 */
 	if (!single)
-		while (unlikely(ht->tail != old_val))
-			rte_pause();
+		rte_wait_until_equal_relaxed_32(&ht->tail, old_val);

./lib/librte_ring/rte_ring_generic.h:26:3: error: implicit declaration of function 'rte_wait_until_equal_relaxed_32'; did you mean 'rte_wait_until_equal_32'? [-Werror=implicit-function-declaration]
rte_wait_until_equal_relaxed_32(&ht->tail, old_val);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

BTW, after patch #1 compilation fails also:
../lib/librte_eal/x86/include/rte_spinlock.h:125:24: error: pointer targets in passing argument 1 of 'rte_try_tm' differ in signedness [-Werror=pointer-sign]
if (likely(rte_try_tm(&sl->locked)))

                        ^
Quoted 5 lines
lib/librte_ring/rte_ring_generic.h +0 -1
 	ht->tail = new_val;
 }
--

2.17.1

author
#4 lore raw

Re: [dpdk-dev] [PATCH v1 2/2] ring: use wfe to wait for ring tail update on aarch64

Hi Konstantin,

Quoted 67 lines

-----Original Message-----
From: Ananyev, Konstantin <[email protected]>
Sent: Friday, April 24, 2020 9:24 PM
To: Gavin Hu <[email protected]>; [email protected]
Cc: nd <[email protected]>; [email protected]; [email protected]; [email protected]; Honnappa Nagarahalli <[email protected]>; Ruifeng Wang <[email protected]>; Phil Yang <[email protected]>; Joyce Kong <[email protected]>
Subject: RE: [dpdk-dev] [PATCH v1 2/2] ring: use wfe to wait for ring tail update on aarch64

Quoted 8 lines

Instead of polling for tail to be updated, use wfe instruction.

Review

Quoted 6 lines
---
 lib/librte_ring/rte_ring_c11_mem.h | 4 ++--
 lib/librte_ring/rte_ring_generic.h | 3 +--
 2 files changed, 3 insertions(+), 4 deletions(-)
(quoted diff) +0 -0

b/lib/librte_ring/rte_ring_c11_mem.h

lib/librte_ring/rte_ring_c11_mem.h update_tail(struct rte_ring_headtail *ht, uint32_t

Quoted 12 lines

index 0fb73a337..764d8f186 100644

lib/librte_ring/rte_ring_c11_mem.h +1 -0
@@ -2,6 +2,7 @@
  *
  * Copyright (c) 2017,2018 HXT-semitech Corporation.
  * Copyright (c) 2007-2009 Kip Macy [email protected]
+ * Copyright (c) 2019 Arm Limited
  * All rights reserved.
  * Derived from FreeBSD's bufring.h
  * Used as BSD-3 Licensed with permission from Kip Macy.
@@ -21,8 +22,7 @@ update_tail(struct rte_ring_headtail *ht, uint32_t

old_val, uint32_t new_val,

Review

Quoted 10 lines
lib/librte_ring/rte_ring_c11_mem.h +1 -2
 	 * we need to wait for them to complete
 	 */
 	if (!single)
-		while (unlikely(ht->tail != old_val))
-			rte_pause();
+		rte_wait_until_equal_relaxed_32(&ht->tail, old_val);
 
 	__atomic_store_n(&ht->tail, new_val, __ATOMIC_RELEASE);
 }
(quoted diff) +0 -0

b/lib/librte_ring/rte_ring_generic.h

lib/librte_ring/rte_ring_generic.h update_tail(struct rte_ring_headtail *ht, uint32_t

Quoted 4 lines

index 953cdbbd5..682852783 100644

lib/librte_ring/rte_ring_generic.h +0 -0
@@ -23,8 +23,7 @@ update_tail(struct rte_ring_headtail *ht, uint32_t

old_val, uint32_t new_val,

lib/librte_ring/rte_ring_generic.h update_tail(struct rte_ring_headtail *ht, uint32_t · line 26

Quoted 6 lines
lib/librte_ring/rte_ring_generic.h +1 -2
 	 * we need to wait for them to complete
 	 */
 	if (!single)
-		while (unlikely(ht->tail != old_val))
-			rte_pause();
+		rte_wait_until_equal_relaxed_32(&ht->tail, old_val);

./lib/librte_ring/rte_ring_generic.h:26:3: error: implicit declaration of function 'rte_wait_until_equal_relaxed_32'; did you mean 'rte_wait_until_equal_32'? [-Werror=implicit-function-declaration]

Quoted 2 lines
   rte_wait_until_equal_relaxed_32(&ht->tail, old_val);
   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Sorry for the leakage, fixed in v2.

Quoted 6 lines

BTW, after patch #1 compilation fails also:
../lib/librte_eal/x86/include/rte_spinlock.h:125:24: error: pointer targets in passing argument 1 of 'rte_try_tm' differ in signedness [-Werror=pointer- sign]
if (likely(rte_try_tm(&sl->locked)))

Fixed in v2, thanks!

Quoted 9 lines
                        ^
Quoted 5 lines
lib/librte_ring/rte_ring_generic.h +0 -1
 	ht->tail = new_val;
 }
--

2.17.1

#5 lore raw

Re: [dpdk-dev] [PATCH v1 2/2] ring: use wfe to wait for ring tail update on aarch64

In reply to Gavin Hu

<snip>

Hi Gavin,
There are other sync modes added to rte_ring library. Can you please extend this to other rte_pause() calls in the library?

Thanks,
Honnappa

Quoted 54 lines

Subject: [PATCH v1 2/2] ring: use wfe to wait for ring tail update on aarch64

Instead of polling for tail to be updated, use wfe instruction.

---
 lib/librte_ring/rte_ring_c11_mem.h | 4 ++--
lib/librte_ring/rte_ring_generic.h | 3 +--
 2 files changed, 3 insertions(+), 4 deletions(-)
lib/librte_ring/rte_ring_c11_mem.h +2 -2
index 0fb73a337..764d8f186 100644
@@ -2,6 +2,7 @@
  *
  * Copyright (c) 2017,2018 HXT-semitech Corporation.
  * Copyright (c) 2007-2009 Kip Macy [email protected]
+ * Copyright (c) 2019 Arm Limited
  * All rights reserved.
  * Derived from FreeBSD's bufring.h
  * Used as BSD-3 Licensed with permission from Kip Macy.
@@ -21,8 +22,7 @@ update_tail(struct rte_ring_headtail *ht, uint32_t
 old_val, uint32_t new_val,
 	 * we need to wait for them to complete
 	 */
 	if (!single)
-		while (unlikely(ht->tail != old_val))
-			rte_pause();
+		rte_wait_until_equal_relaxed_32(&ht->tail, old_val);
 
 	__atomic_store_n(&ht->tail, new_val, __ATOMIC_RELEASE);  } diff --

git a/lib/librte_ring/rte_ring_generic.h b/lib/librte_ring/rte_ring_generic.h index 953cdbbd5..682852783 100644

lib/librte_ring/rte_ring_generic.h +1 -2
@@ -23,8 +23,7 @@ update_tail(struct rte_ring_headtail *ht, uint32_t
 old_val, uint32_t new_val,
 	 * we need to wait for them to complete
 	 */
 	if (!single)
-		while (unlikely(ht->tail != old_val))
-			rte_pause();
+		rte_wait_until_equal_relaxed_32(&ht->tail, old_val);
 
 	ht->tail = new_val;

}
--
2.17.1

author
#6 lore raw

Re: [dpdk-dev] [PATCH v1 2/2] ring: use wfe to wait for ring tail update on aarch64

Hi Honnappa,

Quoted 12 lines

-----Original Message-----
From: Honnappa Nagarahalli <[email protected]>
Sent: Friday, April 24, 2020 10:46 PM
To: Gavin Hu <[email protected]>; [email protected]
Cc: nd <[email protected]>; [email protected]; [email protected]; [email protected]; Ruifeng Wang <[email protected]>; Phil Yang <[email protected]>; Joyce Kong <[email protected]>; Honnappa Nagarahalli <[email protected]>; nd <[email protected]>
Subject: RE: [PATCH v1 2/2] ring: use wfe to wait for ring tail update on aarch64

Quoted 5 lines

<snip>

Hi Gavin,
There are other sync modes added to rte_ring library. Can you
please extend this to other rte_pause() calls in the library?

I looked into the other two calls to rte_cause, they can't use the rte_wait_until_equal_XX APIs simply. Maybe new APIs are required?
Here is the 32-bit API for your reference:
rte_wait_until_equal_32(volatile uint32_t *addr, uint32_t expected, int memorder)

Quoted 59 lines

Thanks,
Honnappa

Quoted 1 line

Subject: [PATCH v1 2/2] ring: use wfe to wait for ring tail update on

aarch64

Quoted 53 lines

Instead of polling for tail to be updated, use wfe instruction.

---
 lib/librte_ring/rte_ring_c11_mem.h | 4 ++--
lib/librte_ring/rte_ring_generic.h | 3 +--
 2 files changed, 3 insertions(+), 4 deletions(-)
lib/librte_ring/rte_ring_c11_mem.h +2 -2
index 0fb73a337..764d8f186 100644
@@ -2,6 +2,7 @@
  *
  * Copyright (c) 2017,2018 HXT-semitech Corporation.
  * Copyright (c) 2007-2009 Kip Macy [email protected]
+ * Copyright (c) 2019 Arm Limited
  * All rights reserved.
  * Derived from FreeBSD's bufring.h
  * Used as BSD-3 Licensed with permission from Kip Macy.
@@ -21,8 +22,7 @@ update_tail(struct rte_ring_headtail *ht, uint32_t
 old_val, uint32_t new_val,
 	 * we need to wait for them to complete
 	 */
 	if (!single)
-		while (unlikely(ht->tail != old_val))
-			rte_pause();
+		rte_wait_until_equal_relaxed_32(&ht->tail, old_val);
 
 	__atomic_store_n(&ht->tail, new_val, __ATOMIC_RELEASE);  } diff --

git a/lib/librte_ring/rte_ring_generic.h b/lib/librte_ring/rte_ring_generic.h index 953cdbbd5..682852783 100644

lib/librte_ring/rte_ring_generic.h +1 -2
@@ -23,8 +23,7 @@ update_tail(struct rte_ring_headtail *ht, uint32_t
 old_val, uint32_t new_val,
 	 * we need to wait for them to complete
 	 */
 	if (!single)
-		while (unlikely(ht->tail != old_val))
-			rte_pause();
+		rte_wait_until_equal_relaxed_32(&ht->tail, old_val);
 
 	ht->tail = new_val;

}
--
2.17.1

author
#7 lore raw

v2 cover Use WFE for spinlock and ring

In reply to Gavin Hu

The rte_wait_until_equal_xxx APIs abstract the functionality of 'polling for a memory location to become equal to a given value'[1].

Use the API for the rte spinlock and ring implementations.

[1] http://patches.dpdk.org/cover/62703/

Gavin Hu (2):
spinlock: use wfe to reduce contention on aarch64
ring: use wfe to wait for ring tail update on aarch64

 lib/librte_eal/include/generic/rte_spinlock.h | 4 ++--
 lib/librte_ring/rte_ring_c11_mem.h            | 4 ++--
 lib/librte_ring/rte_ring_generic.h            | 3 +--
 3 files changed, 5 insertions(+), 6 deletions(-)
Signature · 1 line
2.17.1
author
#8 lore raw

PATCH v2 1/2 spinlock: use wfe to reduce contention on aarch64

In acquiring a spinlock, cores repeatedly poll the lock variable. This is replaced by rte_wait_until_equal API.

Running the micro benchmarking and the testpmd and l3fwd traffic tests on ThunderX2, Ampere eMAG80 and Arm N1SDP, everything went well and no notable performance gain nor degradation was measured.

---
 lib/librte_eal/include/generic/rte_spinlock.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
lib/librte_eal/include/generic/rte_spinlock.h +2 -2
index 87ae7a4f1..40fe49d5a 100644
@@ -65,8 +65,8 @@ rte_spinlock_lock(rte_spinlock_t *sl)
 
 	while (!__atomic_compare_exchange_n(&sl->locked, &exp, 1, 0,
 				__ATOMIC_ACQUIRE, __ATOMIC_RELAXED)) {
-		while (__atomic_load_n(&sl->locked, __ATOMIC_RELAXED))
-			rte_pause();
+		rte_wait_until_equal_32((volatile uint32_t *)&sl->locked,
+			       0, __ATOMIC_RELAXED);
 		exp = 0;
 	}
 }
Signature · 1 line
2.17.1
Acked-by on v2 1/2
#9 lore raw

Re: [dpdk-dev] [PATCH v2 1/2] spinlock: use wfe to reduce contention on aarch64

Quoted 8 lines · Gavin Hu

In acquiring a spinlock, cores repeatedly poll the lock variable. This is replaced by rte_wait_until_equal API.

Running the micro benchmarking and the testpmd and l3fwd traffic tests on ThunderX2, Ampere eMAG80 and Arm N1SDP, everything went well and no notable performance gain nor degradation was measured.

Quoted 7 lines
Quoted 22 lines
---
 lib/librte_eal/include/generic/rte_spinlock.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
lib/librte_eal/include/generic/rte_spinlock.h +2 -3
index 87ae7a4f1..40fe49d5a 100644
@@ -65,8 +65,8 @@ rte_spinlock_lock(rte_spinlock_t *sl)
 
        while (!__atomic_compare_exchange_n(&sl->locked, &exp, 1, 0,
                                __ATOMIC_ACQUIRE, __ATOMIC_RELAXED)) {
-               while (__atomic_load_n(&sl->locked, __ATOMIC_RELAXED))
-                       rte_pause();
+               rte_wait_until_equal_32((volatile uint32_t *)&sl->locked,
+                              0, __ATOMIC_RELAXED);
                exp = 0;
        }
 }
--

2.17.1

author
#10 lore raw

PATCH v2 2/2 ring: use wfe to wait for ring tail update on aarch64

In reply to Gavin Hu

Instead of polling for tail to be updated, use wfe instruction.

---
 lib/librte_ring/rte_ring_c11_mem.h | 4 ++--
 lib/librte_ring/rte_ring_generic.h | 3 +--
 2 files changed, 3 insertions(+), 4 deletions(-)
lib/librte_ring/rte_ring_c11_mem.h +2 -2
index 0fb73a337..b37ef5995 100644
@@ -2,6 +2,7 @@
  *
  * Copyright (c) 2017,2018 HXT-semitech Corporation.
  * Copyright (c) 2007-2009 Kip Macy [email protected]
+ * Copyright (c) 2019 Arm Limited
  * All rights reserved.
  * Derived from FreeBSD's bufring.h
  * Used as BSD-3 Licensed with permission from Kip Macy.
@@ -21,8 +22,7 @@ update_tail(struct rte_ring_headtail *ht, uint32_t old_val, uint32_t new_val,
 	 * we need to wait for them to complete
 	 */
 	if (!single)
-		while (unlikely(ht->tail != old_val))
-			rte_pause();
+		rte_wait_until_equal_32(&ht->tail, old_val, __ATOMIC_RELAXED);
 
 	__atomic_store_n(&ht->tail, new_val, __ATOMIC_RELEASE);
 }
lib/librte_ring/rte_ring_generic.h +1 -2
index 953cdbbd5..afce89aec 100644
@@ -23,8 +23,7 @@ update_tail(struct rte_ring_headtail *ht, uint32_t old_val, uint32_t new_val,
 	 * we need to wait for them to complete
 	 */
 	if (!single)
-		while (unlikely(ht->tail != old_val))
-			rte_pause();
+		rte_wait_until_equal_32(&ht->tail, old_val, __ATOMIC_RELAXED);
 
 	ht->tail = new_val;
 }
Signature · 1 line
2.17.1
#11 lore raw
In reply to Gavin Hu
Quoted 8 lines

The rte_wait_until_equal_xxx APIs abstract the functionality of 'polling for a memory location to become equal to a given value'[1].

Use the API for the rte spinlock and ring implementations.

[1] http://patches.dpdk.org/cover/62703/

Quoted 9 lines

Gavin Hu (2):
spinlock: use wfe to reduce contention on aarch64
ring: use wfe to wait for ring tail update on aarch64

 lib/librte_eal/include/generic/rte_spinlock.h | 4 ++--
 lib/librte_ring/rte_ring_c11_mem.h            | 4 ++--
 lib/librte_ring/rte_ring_generic.h            | 3 +--
 3 files changed, 5 insertions(+), 6 deletions(-)

Series Acked-by: Konstantin Ananyev <[email protected]>

Quoted 2 lines

--
2.17.1

#12 lore raw
In reply to Gavin Hu
Quoted 11 lines · Gavin Hu

The rte_wait_until_equal_xxx APIs abstract the functionality of 'polling for a memory location to become equal to a given value'[1].

Use the API for the rte spinlock and ring implementations.

[1] http://patches.dpdk.org/cover/62703/

Gavin Hu (2):
spinlock: use wfe to reduce contention on aarch64
ring: use wfe to wait for ring tail update on aarch64

This would result in rte_ring and rte_spinlock APIs becoming experimental and this breaks compilation for external applications using stable ring and spinlock APIs.
IIRC, it was the reason why these patches were dropped with the introduction of the rte_wait_until_equal_* API.

Signature · 1 line
David Marchand
#13 lore raw

<snip>

Quoted 15 lines

Subject: Re: [PATCH v2 0/2] Use WFE for spinlock and ring

Quoted 11 lines · Gavin Hu

The rte_wait_until_equal_xxx APIs abstract the functionality of 'polling for a memory location to become equal to a given value'[1].

Use the API for the rte spinlock and ring implementations.

[1] http://patches.dpdk.org/cover/62703/

Gavin Hu (2):
spinlock: use wfe to reduce contention on aarch64
ring: use wfe to wait for ring tail update on aarch64

Quoted 5 lines

This would result in rte_ring and rte_spinlock APIs becoming experimental and this breaks compilation for external applications using stable ring and spinlock APIs.
IIRC, it was the reason why these patches were dropped with the introduction of the rte_wait_until_equal_* API.

Agreed, the rte_ring new sync modes are resulting in different use cases for these APIs. We need to take a relook at the APIs.

Quoted 4 lines

--
David Marchand

#14 lore raw

Hello Honnappa,

Quoted 18 lines · Honnappa Nagarahalli

<snip>

Quoted 15 lines

Subject: Re: [PATCH v2 0/2] Use WFE for spinlock and ring

Quoted 11 lines · Gavin Hu

The rte_wait_until_equal_xxx APIs abstract the functionality of 'polling for a memory location to become equal to a given value'[1].

Use the API for the rte spinlock and ring implementations.

[1] http://patches.dpdk.org/cover/62703/

Gavin Hu (2):
spinlock: use wfe to reduce contention on aarch64
ring: use wfe to wait for ring tail update on aarch64

Quoted 6 lines
Quoted 5 lines

This would result in rte_ring and rte_spinlock APIs becoming experimental and this breaks compilation for external applications using stable ring and spinlock APIs.
IIRC, it was the reason why these patches were dropped with the introduction of the rte_wait_until_equal_* API.

Agreed, the rte_ring new sync modes are resulting in different use cases for these APIs. We need to take a relook at the APIs.

Will we reconsider this series now that the wait until equal is going stable?

Signature · 1 line
David Marchand
#15 lore raw

Hello,

Quoted 23 lines · David Marchand

Hello Honnappa,

Quoted 18 lines · Honnappa Nagarahalli

<snip>

Quoted 15 lines

Subject: Re: [PATCH v2 0/2] Use WFE for spinlock and ring

Quoted 11 lines · Gavin Hu

The rte_wait_until_equal_xxx APIs abstract the functionality of 'polling for a memory location to become equal to a given value'[1].

Use the API for the rte spinlock and ring implementations.

[1] http://patches.dpdk.org/cover/62703/

Gavin Hu (2):
spinlock: use wfe to reduce contention on aarch64
ring: use wfe to wait for ring tail update on aarch64

Quoted 8 lines
Quoted 6 lines
Quoted 5 lines

This would result in rte_ring and rte_spinlock APIs becoming experimental and this breaks compilation for external applications using stable ring and spinlock APIs.
IIRC, it was the reason why these patches were dropped with the introduction of the rte_wait_until_equal_* API.

Agreed, the rte_ring new sync modes are resulting in different use cases for these APIs. We need to take a relook at the APIs.

Will we reconsider this series now that the wait until equal is going stable?

Any update?
I suppose this would need some rebasing after the ring library changes.

Thanks.

Signature · 1 line
David Marchand
#16 lore raw
Quoted 42 lines

-----Original Message-----
From: David Marchand <[email protected]>
Sent: Thursday, March 25, 2021 10:58 PM
To: Honnappa Nagarahalli <[email protected]>
Cc: dev <[email protected]>; nd <[email protected]>; [email protected]; [email protected]; Ruifeng Wang <[email protected]>; Phil Yang <[email protected]>; Joyce Kong <[email protected]>; Ananyev, Konstantin <[email protected]>
Subject: Re: [dpdk-dev] [PATCH v2 0/2] Use WFE for spinlock and ring

Hello,

Quoted 28 lines · David Marchand

Hello Honnappa,

Quoted 23 lines · Honnappa Nagarahalli

<snip>

Quoted 20 lines

Subject: Re: [PATCH v2 0/2] Use WFE for spinlock and ring

Quoted 11 lines · Gavin Hu

The rte_wait_until_equal_xxx APIs abstract the functionality of 'polling for a memory location to become equal to a given value'[1].

Use the API for the rte spinlock and ring implementations.

[1] http://patches.dpdk.org/cover/62703/

Gavin Hu (2):
spinlock: use wfe to reduce contention on aarch64
ring: use wfe to wait for ring tail update on aarch64

This would result in rte_ring and rte_spinlock APIs becoming experimental and this breaks compilation for external applications using stable ring and spinlock APIs.
IIRC, it was the reason why these patches were dropped with the introduction of the rte_wait_until_equal_* API.

Quoted 7 lines
Quoted 1 line
Quoted 1 line

Agreed, the rte_ring new sync modes are resulting in different use cases

for these APIs. We need to take a relook at the APIs.

Quoted 2 lines

Will we reconsider this series now that the wait until equal is going stable?

Any update?
I suppose this would need some rebasing after the ring library changes.

Sorry for the late response.
This series has been rebased and v3 posted.
I have problem to mark v2 as superseded.

Thank you.

Quoted 6 lines

Thanks.

--
David Marchand

#17 lore raw

Ruifeng Wang:

Quoted 7 lines

From: David Marchand <[email protected]>

Quoted 2 lines

Any update?
I suppose this would need some rebasing after the ring library changes.

Sorry for the late response.
This series has been rebased and v3 posted.
I have problem to mark v2 as superseded.

I did change v2 as superseded.

#18 lore raw
In reply to Gavin Hu

The rte_wait_until_equal_xxx APIs abstract the functionality of 'polling for a memory location to become equal to a given value'[1].

Use the API for the rte spinlock and ring implementations. With the wait until equal APIs being stable, changes will not impact ABI.

[1] http://patches.dpdk.org/cover/62703/

v3:
Series rebased. (David)

Gavin Hu (1):
spinlock: use wfe to reduce contention on aarch64

Ruifeng Wang (1):
ring: use wfe to wait for ring tail update on aarch64

 lib/eal/include/generic/rte_spinlock.h | 4 ++--
 lib/ring/rte_ring_c11_pvt.h            | 4 ++--
 lib/ring/rte_ring_generic_pvt.h        | 3 +--
 3 files changed, 5 insertions(+), 6 deletions(-)
Signature · 1 line
2.25.1
#19 lore raw

PATCH v3 1/2 spinlock: use wfe to reduce contention on aarch64

From: Gavin Hu <[email protected]>

In acquiring a spinlock, cores repeatedly poll the lock variable. This is replaced by rte_wait_until_equal API.

Running the micro benchmarking and the testpmd and l3fwd traffic tests on ThunderX2, Ampere eMAG80 and Arm N1SDP, everything went well and no notable performance gain nor degradation was measured.

---
 lib/eal/include/generic/rte_spinlock.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
lib/eal/include/generic/rte_spinlock.h +2 -2
index 87ae7a4f18..40fe49d5ad 100644
@@ -65,8 +65,8 @@ rte_spinlock_lock(rte_spinlock_t *sl)
 
 	while (!__atomic_compare_exchange_n(&sl->locked, &exp, 1, 0,
 				__ATOMIC_ACQUIRE, __ATOMIC_RELAXED)) {
-		while (__atomic_load_n(&sl->locked, __ATOMIC_RELAXED))
-			rte_pause();
+		rte_wait_until_equal_32((volatile uint32_t *)&sl->locked,
+			       0, __ATOMIC_RELAXED);
 		exp = 0;
 	}
 }
Signature · 1 line
2.25.1
#20 lore raw

PATCH v3 2/2 ring: use wfe to wait for ring tail update on aarch64

In reply to Ruifeng Wang

Instead of polling for tail to be updated, use wfe instruction.

---
 lib/ring/rte_ring_c11_pvt.h     | 4 ++--
 lib/ring/rte_ring_generic_pvt.h | 3 +--
 2 files changed, 3 insertions(+), 4 deletions(-)
lib/ring/rte_ring_c11_pvt.h +2 -2
index 759192f4c4..37e0b2afd6 100644
@@ -2,6 +2,7 @@
  *
  * Copyright (c) 2017,2018 HXT-semitech Corporation.
  * Copyright (c) 2007-2009 Kip Macy [email protected]
+ * Copyright (c) 2021 Arm Limited
  * All rights reserved.
  * Derived from FreeBSD's bufring.h
  * Used as BSD-3 Licensed with permission from Kip Macy.
@@ -21,8 +22,7 @@ __rte_ring_update_tail(struct rte_ring_headtail *ht, uint32_t old_val,
 	 * we need to wait for them to complete
 	 */
 	if (!single)
-		while (unlikely(ht->tail != old_val))
-			rte_pause();
+		rte_wait_until_equal_32(&ht->tail, old_val, __ATOMIC_RELAXED);
 
 	__atomic_store_n(&ht->tail, new_val, __ATOMIC_RELEASE);
 }
lib/ring/rte_ring_generic_pvt.h +1 -2
index 532deb5e7a..c95ad7e12c 100644
@@ -23,8 +23,7 @@ __rte_ring_update_tail(struct rte_ring_headtail *ht, uint32_t old_val,
 	 * we need to wait for them to complete
 	 */
 	if (!single)
-		while (unlikely(ht->tail != old_val))
-			rte_pause();
+		rte_wait_until_equal_32(&ht->tail, old_val, __ATOMIC_RELAXED);
 
 	ht->tail = new_val;
 }
Signature · 1 line
2.25.1
Acked-by on v3 2/2
#21 lore raw

Re: [dpdk-dev] [PATCH v3 2/2] ring: use wfe to wait for ring tail update on aarch64

Quoted 9 lines · Ruifeng Wang

Instead of polling for tail to be updated, use wfe instruction.

Quoted 44 lines
---
 lib/ring/rte_ring_c11_pvt.h     | 4 ++--
 lib/ring/rte_ring_generic_pvt.h | 3 +--
 2 files changed, 3 insertions(+), 4 deletions(-)
lib/ring/rte_ring_c11_pvt.h +2 -2
index 759192f4c4..37e0b2afd6 100644
@@ -2,6 +2,7 @@
  *
  * Copyright (c) 2017,2018 HXT-semitech Corporation.
  * Copyright (c) 2007-2009 Kip Macy [email protected]
+ * Copyright (c) 2021 Arm Limited
  * All rights reserved.
  * Derived from FreeBSD's bufring.h
  * Used as BSD-3 Licensed with permission from Kip Macy.
@@ -21,8 +22,7 @@ __rte_ring_update_tail(struct rte_ring_headtail *ht, uint32_t old_val,
         * we need to wait for them to complete
         */
        if (!single)
-               while (unlikely(ht->tail != old_val))
-                       rte_pause();
+               rte_wait_until_equal_32(&ht->tail, old_val, __ATOMIC_RELAXED);
 
        __atomic_store_n(&ht->tail, new_val, __ATOMIC_RELEASE);
 }
lib/ring/rte_ring_generic_pvt.h +1 -3
index 532deb5e7a..c95ad7e12c 100644
@@ -23,8 +23,7 @@ __rte_ring_update_tail(struct rte_ring_headtail *ht, uint32_t old_val,
         * we need to wait for them to complete
         */
        if (!single)
-               while (unlikely(ht->tail != old_val))
-                       rte_pause();
+               rte_wait_until_equal_32(&ht->tail, old_val, __ATOMIC_RELAXED);
 
        ht->tail = new_val;
 }
--

2.25.1

#22 lore raw

Re: [dpdk-dev] [PATCH v3 2/2] ring: use wfe to wait for ring tail update on aarch64

In reply to Ruifeng Wang
Quoted 8 lines · On Sun, 25 Apr 2021 05:56:53 +0000 Ruifeng Wang <[email protected]> wrote:

Instead of polling for tail to be updated, use wfe instruction.

Looks ok to me, but it does raise an interesting question. Shouldn't the original code have been using atomic load to look at ht->tail.

This another place where "volatile considered harmful" applies.

#23 lore raw

Re: [dpdk-dev] [PATCH v3 2/2] ring: use wfe to wait for ring tail update on aarch64

Quoted 24 lines

-----Original Message-----
From: Stephen Hemminger <[email protected]>
Sent: Thursday, April 29, 2021 1:17 AM
To: Ruifeng Wang <[email protected]>
Cc: Honnappa Nagarahalli <[email protected]>; Konstantin Ananyev <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; nd <[email protected]>; Gavin Hu <[email protected]>; Steve Capper <[email protected]>; Ola Liljedahl <[email protected]>
Subject: Re: [dpdk-dev] [PATCH v3 2/2] ring: use wfe to wait for ring tail update on aarch64

Quoted 8 lines · On Sun, 25 Apr 2021 05:56:53 +0000 Ruifeng Wang <[email protected]> wrote:

Instead of polling for tail to be updated, use wfe instruction.

Quoted 4 lines

Looks ok to me, but it does raise an interesting question. Shouldn't the original code have been using atomic load to look at ht->tail.

This another place where "volatile considered harmful" applies.

Do you mean 'volatile' should be removed from rte_wait_until_equal_xxx parameters?

#24 lore raw

Re: [dpdk-dev] [PATCH v3 2/2] ring: use wfe to wait for ring tail update on aarch64

Quoted 24 lines · On Thu, 29 Apr 2021 14:35:35 +0000 Ruifeng Wang <[email protected]> wrote:
Quoted 24 lines

-----Original Message-----
From: Stephen Hemminger <[email protected]>
Sent: Thursday, April 29, 2021 1:17 AM
To: Ruifeng Wang <[email protected]>
Cc: Honnappa Nagarahalli <[email protected]>; Konstantin Ananyev <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; nd <[email protected]>; Gavin Hu <[email protected]>; Steve Capper <[email protected]>; Ola Liljedahl <[email protected]>
Subject: Re: [dpdk-dev] [PATCH v3 2/2] ring: use wfe to wait for ring tail update on aarch64

Quoted 8 lines · On Sun, 25 Apr 2021 05:56:53 +0000 Ruifeng Wang <[email protected]> wrote:

Instead of polling for tail to be updated, use wfe instruction.

Quoted 7 lines
Quoted 4 lines

Looks ok to me, but it does raise an interesting question. Shouldn't the original code have been using atomic load to look at ht->tail.

This another place where "volatile considered harmful" applies.

Do you mean 'volatile' should be removed from rte_wait_until_equal_xxx parameters?

I meant that all access to tail should be via C11 atomic builtin. At that point, the volatile on the data structure elements does not matter.

#25 lore raw

Re: [dpdk-dev] [PATCH v3 2/2] ring: use wfe to wait for ring tail update on aarch64

Quoted 43 lines

-----Original Message-----
From: Stephen Hemminger <[email protected]>
Sent: Thursday, April 29, 2021 11:06 PM
To: Ruifeng Wang <[email protected]>
Cc: Honnappa Nagarahalli <[email protected]>; Konstantin Ananyev <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; nd <[email protected]>; Gavin Hu <[email protected]>; Steve Capper <[email protected]>; Ola Liljedahl <[email protected]>
Subject: Re: [dpdk-dev] [PATCH v3 2/2] ring: use wfe to wait for ring tail update on aarch64

Quoted 7 lines · On Thu, 29 Apr 2021 14:35:35 +0000 Ruifeng Wang <[email protected]> wrote:
Quoted 7 lines

-----Original Message-----
From: Stephen Hemminger <[email protected]>
Sent: Thursday, April 29, 2021 1:17 AM
To: Ruifeng Wang <[email protected]>
Cc: Honnappa Nagarahalli <[email protected]>; Konstantin Ananyev <[email protected]>; [email protected]; [email protected]; [email protected];

[email protected];

Quoted 20 lines
Quoted 20 lines

nd <[email protected]>; Gavin Hu <[email protected]>; Steve Capper <[email protected]>; Ola Liljedahl <[email protected]>
Subject: Re: [dpdk-dev] [PATCH v3 2/2] ring: use wfe to wait for ring tail update on aarch64

Quoted 8 lines · On Sun, 25 Apr 2021 05:56:53 +0000 Ruifeng Wang <[email protected]> wrote:

Instead of polling for tail to be updated, use wfe instruction.

Looks ok to me, but it does raise an interesting question. Shouldn't the original code have been using atomic load to look at ht->tail.

Quoted 8 lines
Quoted 3 lines
Quoted 1 line

This another place where "volatile considered harmful" applies.

Do you mean 'volatile' should be removed from rte_wait_until_equal_xxx

parameters?

Quoted 1 line

I meant that all access to tail should be via C11 atomic builtin. At that point, the volatile on the data structure elements does not matter.

Agreed. If synchronization is ensured by using C11 atomic builtin, 'volatile' on elements can be removed.

#26 lore raw
In reply to Ruifeng Wang

Hello Ruifeng,

Quoted 6 lines · Ruifeng Wang

The rte_wait_until_equal_xxx APIs abstract the functionality of 'polling for a memory location to become equal to a given value'[1].

Use the API for the rte spinlock and ring implementations. With the wait until equal APIs being stable, changes will not impact ABI.

Afaics, there is no ARM target with WFE enabled and we lost ability to enable WFE support with removal of the make build system.

$ git grep RTE_ARM_USE_WFE
config/arm/meson.build:        ['RTE_ARM_USE_WFE', false],
lib/eal/arm/include/rte_pause_64.h:#ifdef RTE_ARM_USE_WFE

How did you enable WFE to test this series?

Signature · 1 line
David Marchand
#27 lore raw
Quoted 9 lines

-----Original Message-----
From: David Marchand <[email protected]>
Sent: Wednesday, April 28, 2021 3:42 PM
To: Ruifeng Wang <[email protected]>
Cc: dev <[email protected]>; [email protected]; [email protected]; nd <[email protected]>; Honnappa Nagarahalli <[email protected]>
Subject: Re: Use WFE for spinlock and ring

Hello Ruifeng,

Hello David,

Quoted 4 lines
Quoted 1 line · Ruifeng Wang
Quoted 8 lines
Quoted 5 lines

The rte_wait_until_equal_xxx APIs abstract the functionality of 'polling for a memory location to become equal to a given value'[1].

Use the API for the rte spinlock and ring implementations. With the wait until equal APIs being stable, changes will not impact ABI.

Afaics, there is no ARM target with WFE enabled and we lost ability to enable WFE support with removal of the make build system.

WFE can be enabled with direct meson file change.
WFE is not intended to be enabled by default. It can be enabled based on benchmarking result on hardware.

Quoted 6 lines
$ git grep RTE_ARM_USE_WFE
config/arm/meson.build:        ['RTE_ARM_USE_WFE', false],
lib/eal/arm/include/rte_pause_64.h:#ifdef RTE_ARM_USE_WFE

How did you enable WFE to test this series?

I modified meson file to test.
Tests were also done with WFE disabled to make sure no degradation with generic implementation.

Quoted 4 lines

--
David Marchand

#28 lore raw

28/04/2021 11:30, Ruifeng Wang:

Quoted 17 lines

From: David Marchand <[email protected]>

Quoted 11 lines
Quoted 6 lines · Ruifeng Wang

The rte_wait_until_equal_xxx APIs abstract the functionality of 'polling for a memory location to become equal to a given value'[1].

Use the API for the rte spinlock and ring implementations. With the wait until equal APIs being stable, changes will not impact ABI.

Afaics, there is no ARM target with WFE enabled and we lost ability to enable WFE support with removal of the make build system.

WFE can be enabled with direct meson file change.
WFE is not intended to be enabled by default. It can be enabled based on benchmarking result on hardware.

Quoted 1 line
Quoted 8 lines
Quoted 5 lines
$ git grep RTE_ARM_USE_WFE
config/arm/meson.build:        ['RTE_ARM_USE_WFE', false],
lib/eal/arm/include/rte_pause_64.h:#ifdef RTE_ARM_USE_WFE

How did you enable WFE to test this series?

I modified meson file to test.
Tests were also done with WFE disabled to make sure no degradation with generic implementation.

I don't understand the usage.
Which platform should use it?
Should it be a compile-time option?

#29 lore raw
Quoted 31 lines

-----Original Message-----
From: Thomas Monjalon <[email protected]>
Sent: Wednesday, April 28, 2021 7:14 PM
To: Ruifeng Wang <[email protected]>
Cc: David Marchand <[email protected]>; dev <[email protected]>; [email protected]; nd <[email protected]>; Honnappa Nagarahalli <[email protected]>
Subject: Re: Use WFE for spinlock and ring

28/04/2021 11:30, Ruifeng Wang:

Quoted 21 lines

From: David Marchand <[email protected]>

Quoted 11 lines
Quoted 6 lines · Ruifeng Wang

The rte_wait_until_equal_xxx APIs abstract the functionality of 'polling for a memory location to become equal to a given value'[1].

Use the API for the rte spinlock and ring implementations. With the wait until equal APIs being stable, changes will not impact ABI.

Afaics, there is no ARM target with WFE enabled and we lost ability to enable WFE support with removal of the make build system.

WFE can be enabled with direct meson file change.
WFE is not intended to be enabled by default. It can be enabled based on benchmarking result on hardware.

Quoted 5 lines
$ git grep RTE_ARM_USE_WFE
config/arm/meson.build:        ['RTE_ARM_USE_WFE', false],
lib/eal/arm/include/rte_pause_64.h:#ifdef RTE_ARM_USE_WFE
Quoted 8 lines
Quoted 4 lines
Quoted 1 line

How did you enable WFE to test this series?

I modified meson file to test.
Tests were also done with WFE disabled to make sure no degradation with

generic implementation.

I don't understand the usage.
Which platform should use it?

Platforms that implement WFE semantic (e.g. N1) can use.
The user can enable this feature for power efficiency purpose. But there is something to note as described in commit message 1be7855d77 when the API was introduced.

Quoted 1 line

Should it be a compile-time option?

Yes, it should be a compile-time option.
It can be configured via c_args meson option?

Quoted 1 line
#30 lore raw

29/04/2021 16:28, Ruifeng Wang:

Quoted 32 lines

From: Thomas Monjalon <[email protected]>

Quoted 22 lines

28/04/2021 11:30, Ruifeng Wang:

Quoted 21 lines

From: David Marchand <[email protected]>

Quoted 11 lines
Quoted 6 lines · Ruifeng Wang

The rte_wait_until_equal_xxx APIs abstract the functionality of 'polling for a memory location to become equal to a given value'[1].

Use the API for the rte spinlock and ring implementations. With the wait until equal APIs being stable, changes will not impact ABI.

Afaics, there is no ARM target with WFE enabled and we lost ability to enable WFE support with removal of the make build system.

WFE can be enabled with direct meson file change.
WFE is not intended to be enabled by default. It can be enabled based on benchmarking result on hardware.

Quoted 5 lines
$ git grep RTE_ARM_USE_WFE
config/arm/meson.build:        ['RTE_ARM_USE_WFE', false],
lib/eal/arm/include/rte_pause_64.h:#ifdef RTE_ARM_USE_WFE
Quoted 8 lines
Quoted 4 lines
Quoted 1 line

How did you enable WFE to test this series?

I modified meson file to test.
Tests were also done with WFE disabled to make sure no degradation with

generic implementation.

I don't understand the usage.
Which platform should use it?

Quoted 8 lines

Platforms that implement WFE semantic (e.g. N1) can use.
The user can enable this feature for power efficiency purpose. But there is something to note as described in commit message 1be7855d77 when the API was introduced.

Quoted 1 line

Should it be a compile-time option?

Yes, it should be a compile-time option.
It can be configured via c_args meson option?

+Cc Bruce for discussing how to enable such feature.

The problem with c_args is that the application has no way to know.

#31 lore raw
Quoted 37 lines · Thomas Monjalon

29/04/2021 16:28, Ruifeng Wang:

Quoted 36 lines

From: Thomas Monjalon <[email protected]>

Quoted 22 lines

28/04/2021 11:30, Ruifeng Wang:

Quoted 21 lines

From: David Marchand <[email protected]>

Quoted 11 lines
Quoted 6 lines · Ruifeng Wang

The rte_wait_until_equal_xxx APIs abstract the functionality of 'polling for a memory location to become equal to a given value'[1].

Use the API for the rte spinlock and ring implementations. With the wait until equal APIs being stable, changes will not impact ABI.

Afaics, there is no ARM target with WFE enabled and we lost ability to enable WFE support with removal of the make build system.

WFE can be enabled with direct meson file change.
WFE is not intended to be enabled by default. It can be enabled based on benchmarking result on hardware.

Quoted 5 lines
$ git grep RTE_ARM_USE_WFE
config/arm/meson.build:        ['RTE_ARM_USE_WFE', false],
lib/eal/arm/include/rte_pause_64.h:#ifdef RTE_ARM_USE_WFE
Quoted 8 lines
Quoted 4 lines
Quoted 1 line

How did you enable WFE to test this series?

I modified meson file to test.
Tests were also done with WFE disabled to make sure no degradation with

generic implementation.

I don't understand the usage.
Which platform should use it?

Platforms that implement WFE semantic (e.g. N1) can use.
The user can enable this feature for power efficiency purpose. But there is something to note as described in commit message 1be7855d77 when the API was introduced.

Quoted 9 lines
Quoted 4 lines
Quoted 1 line

Should it be a compile-time option?

Yes, it should be a compile-time option.
It can be configured via c_args meson option?

+Cc Bruce for discussing how to enable such feature.

The problem with c_args is that the application has no way to know.

Agree about c_args not being a great choice. Why does this need to be a compile-time option? Can runtime support not be detected in some manner?

#32 lore raw

<snip>

Quoted 41 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

The rte_wait_until_equal_xxx APIs abstract the functionality of 'polling for a memory location to become equal to a given

value'[1].

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Use the API for the rte spinlock and ring implementations. With the wait until equal APIs being stable, changes will not impact

ABI.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Afaics, there is no ARM target with WFE enabled and we lost ability to enable WFE support with removal of the make build

system.

Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines

WFE can be enabled with direct meson file change.
WFE is not intended to be enabled by default. It can be enabled based on benchmarking result on hardware.

Quoted 6 lines
$ git grep RTE_ARM_USE_WFE
config/arm/meson.build:        ['RTE_ARM_USE_WFE', false],
lib/eal/arm/include/rte_pause_64.h:#ifdef RTE_ARM_USE_WFE

How did you enable WFE to test this series?

I modified meson file to test.
Tests were also done with WFE disabled to make sure no degradation with

Quoted 8 lines
Quoted 8 lines
Quoted 4 lines

generic implementation.

I don't understand the usage.
Which platform should use it?

Platforms that implement WFE semantic (e.g. N1) can use. The user can enable this feature for power efficiency purpose. But there is something to note as described in commit message 1be7855d77

when the API was introduced.

Quoted 6 lines
Quoted 5 lines
Quoted 2 lines

Should it be a compile-time option?

Yes, it should be a compile-time option.
It can be configured via c_args meson option?

Quoted 7 lines
Quoted 4 lines

+Cc Bruce for discussing how to enable such feature.

The problem with c_args is that the application has no way to know.

Agree about c_args not being a great choice. Why does this need to be a compile-time option? Can runtime support not be detected in some manner?

The problem is inconsistency in performance on different Arm platforms. We had decided that each platform needs to enable it after some testing.

#33 lore raw
Quoted 43 lines · Honnappa Nagarahalli

<snip>

Quoted 41 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

The rte_wait_until_equal_xxx APIs abstract the functionality of 'polling for a memory location to become equal to a given

value'[1].

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Use the API for the rte spinlock and ring implementations. With the wait until equal APIs being stable, changes will not impact

ABI.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Afaics, there is no ARM target with WFE enabled and we lost ability to enable WFE support with removal of the make build

system.

Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines

WFE can be enabled with direct meson file change.
WFE is not intended to be enabled by default. It can be enabled based on benchmarking result on hardware.

Quoted 6 lines
$ git grep RTE_ARM_USE_WFE
config/arm/meson.build:        ['RTE_ARM_USE_WFE', false],
lib/eal/arm/include/rte_pause_64.h:#ifdef RTE_ARM_USE_WFE

How did you enable WFE to test this series?

I modified meson file to test.
Tests were also done with WFE disabled to make sure no degradation with

Quoted 8 lines
Quoted 8 lines
Quoted 4 lines

generic implementation.

I don't understand the usage.
Which platform should use it?

Platforms that implement WFE semantic (e.g. N1) can use. The user can enable this feature for power efficiency purpose. But there is something to note as described in commit message 1be7855d77

when the API was introduced.

Quoted 6 lines
Quoted 5 lines
Quoted 2 lines

Should it be a compile-time option?

Yes, it should be a compile-time option.
It can be configured via c_args meson option?

Quoted 8 lines
Quoted 7 lines
Quoted 4 lines

+Cc Bruce for discussing how to enable such feature.

The problem with c_args is that the application has no way to know.

Agree about c_args not being a great choice. Why does this need to be a compile-time option? Can runtime support not be detected in some manner?

The problem is inconsistency in performance on different Arm platforms. We had decided that each platform needs to enable it after some testing.

Then it sounds like it does indeed need to be a build option. Does it need to be added to the meson_options.txt, or can it just be specified in cross-files and optionally via c_args?

#34 lore raw
Quoted 64 lines

-----Original Message-----
From: Bruce Richardson <[email protected]>
Sent: Friday, April 30, 2021 10:19 PM
To: Honnappa Nagarahalli <[email protected]>
Cc: [email protected]; Ruifeng Wang <[email protected]>; David Marchand <[email protected]>; dev <[email protected]>; [email protected]; nd <[email protected]>
Subject: Re: Use WFE for spinlock and ring

Quoted 54 lines · Honnappa Nagarahalli

<snip>

Quoted 52 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

The rte_wait_until_equal_xxx APIs abstract the functionality of 'polling for a memory location to become equal to a given

value'[1].

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

Use the API for the rte spinlock and ring implementations. With the wait until equal APIs being stable, changes will not impact

ABI.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

Afaics, there is no ARM target with WFE enabled and we lost ability to enable WFE support with removal of the make build

system.

Quoted 16 lines
Quoted 16 lines
Quoted 16 lines
Quoted 9 lines

WFE can be enabled with direct meson file change. WFE is not intended to be enabled by default. It can be enabled based on benchmarking result on hardware.

Quoted 5 lines
$ git grep RTE_ARM_USE_WFE
config/arm/meson.build:        ['RTE_ARM_USE_WFE', false],
lib/eal/arm/include/rte_pause_64.h:#ifdef RTE_ARM_USE_WFE
Quoted 5 lines
Quoted 1 line

How did you enable WFE to test this series?

I modified meson file to test.
Tests were also done with WFE disabled to make sure no degradation with

generic implementation.

Quoted 7 lines
Quoted 7 lines
Quoted 2 lines

I don't understand the usage.
Which platform should use it?

Platforms that implement WFE semantic (e.g. N1) can use. The user can enable this feature for power efficiency purpose. But there is something to note as described in commit message 1be7855d77

when the API was introduced.

Quoted 10 lines
Quoted 5 lines
Quoted 2 lines

Should it be a compile-time option?

Yes, it should be a compile-time option.
It can be configured via c_args meson option?

+Cc Bruce for discussing how to enable such feature.

The problem with c_args is that the application has no way to know.

Agree about c_args not being a great choice. Why does this need to be a compile-time option? Can runtime support not be detected in some manner?

Quoted 6 lines
Quoted 1 line

The problem is inconsistency in performance on different Arm platforms.

We had decided that each platform needs to enable it after some testing.

Then it sounds like it does indeed need to be a build option. Does it need to be added to the meson_options.txt, or can it just be specified in cross-files and optionally via c_args?

Add it to the meson_options.txt is good as the option will be clearly exposed. My concern is more options (e.g. RTE_ARCH_ARM64_MEMCPY) need to be added in. Will the options bloat meson_options.txt?

#35 lore raw
Quoted 67 lines · Ruifeng Wang
Quoted 67 lines

-----Original Message-----
From: Bruce Richardson <[email protected]>
Sent: Friday, April 30, 2021 10:19 PM
To: Honnappa Nagarahalli <[email protected]>
Cc: [email protected]; Ruifeng Wang <[email protected]>; David Marchand <[email protected]>; dev <[email protected]>; [email protected]; nd <[email protected]>
Subject: Re: Use WFE for spinlock and ring

Quoted 47 lines · Honnappa Nagarahalli

<snip>

Quoted 45 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

The rte_wait_until_equal_xxx APIs abstract the functionality of 'polling for a memory location to become equal to a given

value'[1].

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

Use the API for the rte spinlock and ring implementations. With the wait until equal APIs being stable, changes will not impact

ABI.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

Afaics, there is no ARM target with WFE enabled and we lost ability to enable WFE support with removal of the make build

system.

Quoted 16 lines
Quoted 16 lines
Quoted 16 lines
Quoted 9 lines

WFE can be enabled with direct meson file change. WFE is not intended to be enabled by default. It can be enabled based on benchmarking result on hardware.

Quoted 5 lines
$ git grep RTE_ARM_USE_WFE
config/arm/meson.build:        ['RTE_ARM_USE_WFE', false],
lib/eal/arm/include/rte_pause_64.h:#ifdef RTE_ARM_USE_WFE
Quoted 5 lines
Quoted 1 line

How did you enable WFE to test this series?

I modified meson file to test.
Tests were also done with WFE disabled to make sure no degradation with

generic implementation.

Quoted 7 lines
Quoted 7 lines
Quoted 2 lines

I don't understand the usage.
Which platform should use it?

Platforms that implement WFE semantic (e.g. N1) can use. The user can enable this feature for power efficiency purpose. But there is something to note as described in commit message 1be7855d77

when the API was introduced.

Quoted 6 lines
Quoted 5 lines
Quoted 2 lines

Should it be a compile-time option?

Yes, it should be a compile-time option.
It can be configured via c_args meson option?

Quoted 8 lines
Quoted 7 lines
Quoted 4 lines

+Cc Bruce for discussing how to enable such feature.

The problem with c_args is that the application has no way to know.

Agree about c_args not being a great choice. Why does this need to be a compile-time option? Can runtime support not be detected in some manner?

The problem is inconsistency in performance on different Arm platforms.

We had decided that each platform needs to enable it after some testing.

Quoted 7 lines
Quoted 3 lines

Then it sounds like it does indeed need to be a build option. Does it need to be added to the meson_options.txt, or can it just be specified in cross-files and optionally via c_args?

Add it to the meson_options.txt is good as the option will be clearly exposed. My concern is more options (e.g. RTE_ARCH_ARM64_MEMCPY) need to be added in. Will the options bloat meson_options.txt?

That bloat would indeed become a concern. We may need to look at more use of auto-detection and cross files for such options.

#36 lore raw
Quoted 7 lines · Bruce Richardson
Quoted 7 lines
Quoted 6 lines
Quoted 1 line

The problem is inconsistency in performance on different Arm platforms.

We had decided that each platform needs to enable it after some testing.

Then it sounds like it does indeed need to be a build option. Does it need to be added to the meson_options.txt, or can it just be specified in cross-files and optionally via c_args?

Quoted 6 lines
Quoted 3 lines

Add it to the meson_options.txt is good as the option will be clearly exposed. My concern is more options (e.g. RTE_ARCH_ARM64_MEMCPY) need to be added in. Will the options bloat meson_options.txt?

That bloat would indeed become a concern. We may need to look at more use of auto-detection and cross files for such options.

Ruifeng,

This series is blocked on this last topic.
Do you intend to send a n+1?

Signature · 1 line
David Marchand
#37 lore raw
Quoted 26 lines

-----Original Message-----
From: David Marchand <[email protected]>
Sent: Monday, July 5, 2021 4:52 PM
To: Ruifeng Wang <[email protected]>
Cc: Bruce Richardson <[email protected]>; Honnappa Nagarahalli <[email protected]>; [email protected]; dev <[email protected]>; [email protected]; nd <[email protected]>
Subject: Re: Use WFE for spinlock and ring

Quoted 1 line · Bruce Richardson
Quoted 1 line
Quoted 1 line
Quoted 1 line

The problem is inconsistency in performance on different Arm

platforms.

Quoted 1 line
Quoted 1 line
Quoted 1 line

We had decided that each platform needs to enable it after some

testing.

Quoted 6 lines
Quoted 6 lines
Quoted 4 lines

Then it sounds like it does indeed need to be a build option. Does it need to be added to the meson_options.txt, or can it just be specified in cross-files and optionally via c_args?

Add it to the meson_options.txt is good as the option will be clearly

exposed.

Quoted 1 line
Quoted 1 line

My concern is more options (e.g. RTE_ARCH_ARM64_MEMCPY) need to

be added in.

Quoted 2 lines
Quoted 1 line

Will the options bloat meson_options.txt?

Quoted 7 lines
Quoted 2 lines

That bloat would indeed become a concern. We may need to look at more use of auto-detection and cross files for such options.

Ruifeng,

This series is blocked on this last topic. Do you intend to send a n+1?

I think the best way to expose this config is adding it to meson_options. Will send v4 to update.

Quoted 4 lines

--
David Marchand

#38 lore raw
In reply to Ruifeng Wang
Quoted 14 lines · On Sun, 25 Apr 2021 05:56:51 +0000 Ruifeng Wang <[email protected]> wrote:

The rte_wait_until_equal_xxx APIs abstract the functionality of 'polling for a memory location to become equal to a given value'[1].

Use the API for the rte spinlock and ring implementations. With the wait until equal APIs being stable, changes will not impact ABI.

[1] http://patches.dpdk.org/cover/62703/

v3:
Series rebased. (David)

Gavin Hu (1):
spinlock: use wfe to reduce contention on aarch64

Quoted 8 lines

Ruifeng Wang (1):
ring: use wfe to wait for ring tail update on aarch64

 lib/eal/include/generic/rte_spinlock.h | 4 ++--
 lib/ring/rte_ring_c11_pvt.h            | 4 ++--
 lib/ring/rte_ring_generic_pvt.h        | 3 +--
 3 files changed, 5 insertions(+), 6 deletions(-)

Other places that should use WFE:

rte_mcslock.h:rte_mcslock_lock()
rte_mcslock_unlock:rte_mcslock_unlock()

rte_pflock.h:rte_pflock_lock()
rte_rwlock.h:rte_rwlock_read_lock()
rte_rwlock.h:rte_rwlock_write_lock()

You should also introduce rte_wait_while_XXX variants to handle some of these cases.

#39 lore raw
Quoted 29 lines

-----Original Message-----
From: Stephen Hemminger <[email protected]>
Sent: Wednesday, July 7, 2021 10:48 PM
To: Ruifeng Wang <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; nd <[email protected]>; Honnappa Nagarahalli <[email protected]>
Subject: Re: [dpdk-dev] Use WFE for spinlock and ring

Quoted 17 lines · On Sun, 25 Apr 2021 05:56:51 +0000 Ruifeng Wang <[email protected]> wrote:

The rte_wait_until_equal_xxx APIs abstract the functionality of 'polling for a memory location to become equal to a given value'[1].

Use the API for the rte spinlock and ring implementations. With the wait until equal APIs being stable, changes will not impact ABI.

[1] http://patches.dpdk.org/cover/62703/

v3:
Series rebased. (David)

Gavin Hu (1):
spinlock: use wfe to reduce contention on aarch64

Ruifeng Wang (1):
ring: use wfe to wait for ring tail update on aarch64

Quoted 7 lines
Quoted 5 lines
 lib/eal/include/generic/rte_spinlock.h | 4 ++--
 lib/ring/rte_ring_c11_pvt.h            | 4 ++--
 lib/ring/rte_ring_generic_pvt.h        | 3 +--
 3 files changed, 5 insertions(+), 6 deletions(-)

Other places that should use WFE:

Thank you Stephen for looking into this.

Quoted 2 lines

rte_mcslock.h:rte_mcslock_lock()

Existing API can be used in this one.

Quoted 1 line

rte_mcslock_unlock:rte_mcslock_unlock()

This one needs rte_wait_while_xxx variant.

Quoted 4 lines

rte_pflock.h:rte_pflock_lock()
rte_rwlock.h:rte_rwlock_read_lock()
rte_rwlock.h:rte_rwlock_write_lock()

These occurrences have extra logic (AND, conditional branch, CAS) in the loop. I'm not sure generic API can be abstracted from these use cases.

Quoted 5 lines

You should also introduce rte_wait_while_XXX variants to handle some of these cases.

#40 lore raw

<snip>

Quoted 35 lines
Quoted 21 lines
Quoted 17 lines · On Sun, 25 Apr 2021 05:56:51 +0000 Ruifeng Wang <[email protected]> wrote:

The rte_wait_until_equal_xxx APIs abstract the functionality of 'polling for a memory location to become equal to a given value'[1].

Use the API for the rte spinlock and ring implementations. With the wait until equal APIs being stable, changes will not impact ABI.

[1] http://patches.dpdk.org/cover/62703/

v3:
Series rebased. (David)

Gavin Hu (1):
spinlock: use wfe to reduce contention on aarch64

Ruifeng Wang (1):
ring: use wfe to wait for ring tail update on aarch64

Quoted 7 lines
Quoted 5 lines
 lib/eal/include/generic/rte_spinlock.h | 4 ++--
 lib/ring/rte_ring_c11_pvt.h            | 4 ++--
 lib/ring/rte_ring_generic_pvt.h        | 3 +--
 3 files changed, 5 insertions(+), 6 deletions(-)

Other places that should use WFE:

Thank you Stephen for looking into this.

Quoted 2 lines

rte_mcslock.h:rte_mcslock_lock()

Existing API can be used in this one.

Quoted 1 line

rte_mcslock_unlock:rte_mcslock_unlock()

Quoted 8 lines

This one needs rte_wait_while_xxx variant.

Quoted 4 lines

rte_pflock.h:rte_pflock_lock()
rte_rwlock.h:rte_rwlock_read_lock()
rte_rwlock.h:rte_rwlock_write_lock()

These occurrences have extra logic (AND, conditional branch, CAS) in the loop. I'm not sure generic API can be abstracted from these use cases.

I think it is possible to create additional abstractions to address these cases.

Quoted 8 lines
Quoted 6 lines

You should also introduce rte_wait_while_XXX variants to handle some of these cases.

#41 lore raw

v4 cover Use WFE for spinlock and ring

In reply to Gavin Hu

The rte_wait_until_equal_xxx APIs abstract the functionality of 'polling for a memory location to become equal to a given value'[1].

Use the API for the rte spinlock and ring implementations. With the wait until equal APIs being stable, changes will not impact ABI.

[1] http://patches.dpdk.org/cover/62703/

v4:
Added meson option to expose WFE. (David, Bruce)

v3:
Series rebased. (David)

Gavin Hu (1):
spinlock: use wfe to reduce contention on aarch64

Ruifeng Wang (2):
ring: use wfe to wait for ring tail update on aarch64
build: add option to enable wait until equal

 config/arm/meson.build                 | 2 +-
 lib/eal/include/generic/rte_spinlock.h | 4 ++--
 lib/ring/rte_ring_c11_pvt.h            | 4 ++--
 lib/ring/rte_ring_generic_pvt.h        | 3 +--
 meson_options.txt                      | 2 ++
 5 files changed, 8 insertions(+), 7 deletions(-)
Signature · 1 line
2.25.1
#42 lore raw

PATCH v4 1/3 spinlock: use wfe to reduce contention on aarch64

From: Gavin Hu <[email protected]>

In acquiring a spinlock, cores repeatedly poll the lock variable. This is replaced by rte_wait_until_equal API.

Running the micro benchmarking and the testpmd and l3fwd traffic tests on ThunderX2, Ampere eMAG80 and Arm N1SDP, everything went well and no notable performance gain nor degradation was measured.

---
 lib/eal/include/generic/rte_spinlock.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
lib/eal/include/generic/rte_spinlock.h +2 -2
index 87ae7a4f18..40fe49d5ad 100644
@@ -65,8 +65,8 @@ rte_spinlock_lock(rte_spinlock_t *sl)
 
 	while (!__atomic_compare_exchange_n(&sl->locked, &exp, 1, 0,
 				__ATOMIC_ACQUIRE, __ATOMIC_RELAXED)) {
-		while (__atomic_load_n(&sl->locked, __ATOMIC_RELAXED))
-			rte_pause();
+		rte_wait_until_equal_32((volatile uint32_t *)&sl->locked,
+			       0, __ATOMIC_RELAXED);
 		exp = 0;
 	}
 }
Signature · 1 line
2.25.1
#43 lore raw

PATCH v4 2/3 ring: use wfe to wait for ring tail update on aarch64

In reply to Ruifeng Wang

Instead of polling for tail to be updated, use wfe instruction.

---
 lib/ring/rte_ring_c11_pvt.h     | 4 ++--
 lib/ring/rte_ring_generic_pvt.h | 3 +--
 2 files changed, 3 insertions(+), 4 deletions(-)
lib/ring/rte_ring_c11_pvt.h +2 -2
index 759192f4c4..37e0b2afd6 100644
@@ -2,6 +2,7 @@
  *
  * Copyright (c) 2017,2018 HXT-semitech Corporation.
  * Copyright (c) 2007-2009 Kip Macy [email protected]
+ * Copyright (c) 2021 Arm Limited
  * All rights reserved.
  * Derived from FreeBSD's bufring.h
  * Used as BSD-3 Licensed with permission from Kip Macy.
@@ -21,8 +22,7 @@ __rte_ring_update_tail(struct rte_ring_headtail *ht, uint32_t old_val,
 	 * we need to wait for them to complete
 	 */
 	if (!single)
-		while (unlikely(ht->tail != old_val))
-			rte_pause();
+		rte_wait_until_equal_32(&ht->tail, old_val, __ATOMIC_RELAXED);
 
 	__atomic_store_n(&ht->tail, new_val, __ATOMIC_RELEASE);
 }
lib/ring/rte_ring_generic_pvt.h +1 -2
index 532deb5e7a..c95ad7e12c 100644
@@ -23,8 +23,7 @@ __rte_ring_update_tail(struct rte_ring_headtail *ht, uint32_t old_val,
 	 * we need to wait for them to complete
 	 */
 	if (!single)
-		while (unlikely(ht->tail != old_val))
-			rte_pause();
+		rte_wait_until_equal_32(&ht->tail, old_val, __ATOMIC_RELAXED);
 
 	ht->tail = new_val;
 }
Signature · 1 line
2.25.1
#44 lore raw

PATCH v4 3/3 build: add option to enable wait until equal

In reply to Ruifeng Wang

Introduce a meson option 'use_wfe' to select wait until equal method. The default is disable. Traditional polling loop is used. When enabled, architecture specific mechanism is relied on to do the wait.

---
 config/arm/meson.build | 2 +-
 meson_options.txt      | 2 ++
 2 files changed, 3 insertions(+), 1 deletion(-)
config/arm/meson.build +1 -1
index 9b147c0b93..9ea478fb77 100644
@@ -18,7 +18,6 @@ flags_common = [
         #    ['RTE_ARM64_MEMCPY_STRICT_ALIGN', false],
 
         ['RTE_SCHED_VECTOR', false],
-        ['RTE_ARM_USE_WFE', false],
         ['RTE_ARCH_ARM64', true],
         ['RTE_CACHE_LINE_SIZE', 128]
 ]
@@ -371,6 +370,7 @@ socs = {
 
 dpdk_conf.set('RTE_ARCH_ARM', 1)
 dpdk_conf.set('RTE_FORCE_INTRINSICS', 1)
+dpdk_conf.set('RTE_ARM_USE_WFE', get_option('use_wfe'))
 
 if dpdk_conf.get('RTE_ARCH_32')
     # armv7 build
meson_options.txt +2 -0
index 56bdfd0f0a..5012803c77 100644
@@ -44,3 +44,5 @@ option('tests', type: 'boolean', value: true, description:
        'build unit tests')
 option('use_hpet', type: 'boolean', value: false, description:
        'use HPET timer in EAL')
+option('use_wfe', type: 'boolean', value: false, description:
+       'use Wait Until Equal for polling loops')
Signature · 1 line
2.25.1
#45 lore raw

Re: [dpdk-dev] [PATCH v4 3/3] build: add option to enable wait until equal

meson_options.txt option('tests', type: 'boolean', value: true, description: · line 48

Quoted 10 lines · Ruifeng Wang
meson_options.txt +2 -0
index 56bdfd0f0a..5012803c77 100644
@@ -44,3 +44,5 @@ option('tests', type: 'boolean', value: true, description:
        'build unit tests')
 option('use_hpet', type: 'boolean', value: false, description:
        'use HPET timer in EAL')
+option('use_wfe', type: 'boolean', value: false, description:
+       'use Wait Until Equal for polling loops')

This is badly described.
We have a wait until equal API.

What you want to enable with this option is the hw assisted implementation.

Signature · 1 line
David Marchand
#46 lore raw

v4 cover Use WFE for spinlock and ring

In reply to Gavin Hu

The rte_wait_until_equal_xxx APIs abstract the functionality of 'polling for a memory location to become equal to a given value'[1].

Use the API for the rte spinlock and ring implementations. With the wait until equal APIs being stable, changes will not impact ABI.

[1] http://patches.dpdk.org/cover/62703/

v4:
Added meson option to expose WFE. (David, Bruce)

v3:
Series rebased. (David)

Gavin Hu (1):
spinlock: use wfe to reduce contention on aarch64

Ruifeng Wang (2):
ring: use wfe to wait for ring tail update on aarch64
build: add option to enable wait until equal

 config/arm/meson.build                 | 2 +-
 lib/eal/include/generic/rte_spinlock.h | 4 ++--
 lib/ring/rte_ring_c11_pvt.h            | 4 ++--
 lib/ring/rte_ring_generic_pvt.h        | 3 +--
 meson_options.txt                      | 2 ++
 5 files changed, 8 insertions(+), 7 deletions(-)
Signature · 1 line
2.25.1
#47 lore raw

PATCH v4 1/3 spinlock: use wfe to reduce contention on aarch64

From: Gavin Hu <[email protected]>

In acquiring a spinlock, cores repeatedly poll the lock variable. This is replaced by rte_wait_until_equal API.

Running the micro benchmarking and the testpmd and l3fwd traffic tests on ThunderX2, Ampere eMAG80 and Arm N1SDP, everything went well and no notable performance gain nor degradation was measured.

---
 lib/eal/include/generic/rte_spinlock.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
lib/eal/include/generic/rte_spinlock.h +2 -2
index 87ae7a4f18..40fe49d5ad 100644
@@ -65,8 +65,8 @@ rte_spinlock_lock(rte_spinlock_t *sl)
 
 	while (!__atomic_compare_exchange_n(&sl->locked, &exp, 1, 0,
 				__ATOMIC_ACQUIRE, __ATOMIC_RELAXED)) {
-		while (__atomic_load_n(&sl->locked, __ATOMIC_RELAXED))
-			rte_pause();
+		rte_wait_until_equal_32((volatile uint32_t *)&sl->locked,
+			       0, __ATOMIC_RELAXED);
 		exp = 0;
 	}
 }
Signature · 1 line
2.25.1
#48 lore raw

PATCH v4 2/3 ring: use wfe to wait for ring tail update on aarch64

In reply to Ruifeng Wang

Instead of polling for tail to be updated, use wfe instruction.

---
 lib/ring/rte_ring_c11_pvt.h     | 4 ++--
 lib/ring/rte_ring_generic_pvt.h | 3 +--
 2 files changed, 3 insertions(+), 4 deletions(-)
lib/ring/rte_ring_c11_pvt.h +2 -2
index 759192f4c4..37e0b2afd6 100644
@@ -2,6 +2,7 @@
  *
  * Copyright (c) 2017,2018 HXT-semitech Corporation.
  * Copyright (c) 2007-2009 Kip Macy [email protected]
+ * Copyright (c) 2021 Arm Limited
  * All rights reserved.
  * Derived from FreeBSD's bufring.h
  * Used as BSD-3 Licensed with permission from Kip Macy.
@@ -21,8 +22,7 @@ __rte_ring_update_tail(struct rte_ring_headtail *ht, uint32_t old_val,
 	 * we need to wait for them to complete
 	 */
 	if (!single)
-		while (unlikely(ht->tail != old_val))
-			rte_pause();
+		rte_wait_until_equal_32(&ht->tail, old_val, __ATOMIC_RELAXED);
 
 	__atomic_store_n(&ht->tail, new_val, __ATOMIC_RELEASE);
 }
lib/ring/rte_ring_generic_pvt.h +1 -2
index 532deb5e7a..c95ad7e12c 100644
@@ -23,8 +23,7 @@ __rte_ring_update_tail(struct rte_ring_headtail *ht, uint32_t old_val,
 	 * we need to wait for them to complete
 	 */
 	if (!single)
-		while (unlikely(ht->tail != old_val))
-			rte_pause();
+		rte_wait_until_equal_32(&ht->tail, old_val, __ATOMIC_RELAXED);
 
 	ht->tail = new_val;
 }
Signature · 1 line
2.25.1
#49 lore raw

PATCH v4 3/3 build: add option to enable wait until equal

In reply to Ruifeng Wang

Introduce a meson option 'use_wfe' to select wait until equal method. The default is disable. Traditional polling loop is used. When enabled, architecture specific mechanism is relied on to do the wait.

---
 config/arm/meson.build | 2 +-
 meson_options.txt      | 2 ++
 2 files changed, 3 insertions(+), 1 deletion(-)
config/arm/meson.build +1 -1
index 9b147c0b93..9ea478fb77 100644
@@ -18,7 +18,6 @@ flags_common = [
         #    ['RTE_ARM64_MEMCPY_STRICT_ALIGN', false],
 
         ['RTE_SCHED_VECTOR', false],
-        ['RTE_ARM_USE_WFE', false],
         ['RTE_ARCH_ARM64', true],
         ['RTE_CACHE_LINE_SIZE', 128]
 ]
@@ -371,6 +370,7 @@ socs = {
 
 dpdk_conf.set('RTE_ARCH_ARM', 1)
 dpdk_conf.set('RTE_FORCE_INTRINSICS', 1)
+dpdk_conf.set('RTE_ARM_USE_WFE', get_option('use_wfe'))
 
 if dpdk_conf.get('RTE_ARCH_32')
     # armv7 build
meson_options.txt +2 -0
index 56bdfd0f0a..5012803c77 100644
@@ -44,3 +44,5 @@ option('tests', type: 'boolean', value: true, description:
        'build unit tests')
 option('use_hpet', type: 'boolean', value: false, description:
        'use HPET timer in EAL')
+option('use_wfe', type: 'boolean', value: false, description:
+       'use Wait Until Equal for polling loops')
Signature · 1 line
2.25.1
#50 lore raw

Re: [dpdk-dev] [PATCH v4 3/3] build: add option to enable wait until equal

07/07/2021 07:48, Ruifeng Wang:

Quoted 4 lines

Introduce a meson option 'use_wfe' to select wait until equal method. The default is disable. Traditional polling loop is used. When enabled, architecture specific mechanism is relied on to do the wait.

Why do we need an option?
Can it be automatic to enable it when supported?

#51 lore raw

Re: [dpdk-dev] [PATCH v4 3/3] build: add option to enable wait until equal

Quoted 10 lines

-----Original Message-----
From: Thomas Monjalon <[email protected]>
Sent: Wednesday, July 7, 2021 2:32 PM
To: Ruifeng Wang <[email protected]>
Cc: Jan Viktorin <[email protected]>; [email protected]; Bruce Richardson <[email protected]>; [email protected]; [email protected]; nd <[email protected]>; Honnappa Nagarahalli <[email protected]>
Subject: Re: [PATCH v4 3/3] build: add option to enable wait until equal

Quoted 9 lines

07/07/2021 07:48, Ruifeng Wang:

Quoted 4 lines

Introduce a meson option 'use_wfe' to select wait until equal method. The default is disable. Traditional polling loop is used. When enabled, architecture specific mechanism is relied on to do the wait.

Why do we need an option?
Can it be automatic to enable it when supported?

The problem is inconsistency in performance on different Arm platforms. We had decided that each platform needs to enable it after some testing.

#52 lore raw

Re: [dpdk-dev] [PATCH v4 3/3] build: add option to enable wait until equal

Quoted 16 lines · Ruifeng Wang
Quoted 16 lines

-----Original Message-----
From: Thomas Monjalon <[email protected]>
Sent: Wednesday, July 7, 2021 2:32 PM
To: Ruifeng Wang <[email protected]>
Cc: Jan Viktorin <[email protected]>; [email protected]; Bruce Richardson <[email protected]>; [email protected]; [email protected]; nd <[email protected]>; Honnappa Nagarahalli <[email protected]>
Subject: Re: [PATCH v4 3/3] build: add option to enable wait until equal

07/07/2021 07:48, Ruifeng Wang:

Quoted 4 lines

Introduce a meson option 'use_wfe' to select wait until equal method. The default is disable. Traditional polling loop is used. When enabled, architecture specific mechanism is relied on to do the wait.

Quoted 5 lines
Quoted 3 lines

Why do we need an option?
Can it be automatic to enable it when supported?

The problem is inconsistency in performance on different Arm platforms. We had decided that each platform needs to enable it after some testing.

Can that not be done via variables in the cross-file for the builds, or via automatic detection if it's a native build? Is it likely that individual users of DPDK will be knowledgable enough to use this option correctly?

#53 lore raw

Re: [dpdk-dev] [PATCH v4 3/3] build: add option to enable wait until equal

Quoted 18 lines · Bruce Richardson
Quoted 16 lines · Ruifeng Wang
Quoted 16 lines

-----Original Message-----
From: Thomas Monjalon <[email protected]>
Sent: Wednesday, July 7, 2021 2:32 PM
To: Ruifeng Wang <[email protected]>
Cc: Jan Viktorin <[email protected]>; [email protected]; Bruce Richardson <[email protected]>; [email protected]; [email protected]; nd <[email protected]>; Honnappa Nagarahalli <[email protected]>
Subject: Re: [PATCH v4 3/3] build: add option to enable wait until equal

07/07/2021 07:48, Ruifeng Wang:

Quoted 4 lines

Introduce a meson option 'use_wfe' to select wait until equal method. The default is disable. Traditional polling loop is used. When enabled, architecture specific mechanism is relied on to do the wait.

Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

Why do we need an option?
Can it be automatic to enable it when supported?

The problem is inconsistency in performance on different Arm platforms. We had decided that each platform needs to enable it after some testing.

Can that not be done via variables in the cross-file for the builds, or via automatic detection if it's a native build? Is it likely that individual users of DPDK will be knowledgable enough to use this option correctly?

+1 to add this in cross-file instead of the top of config option as scope if is only for arm64 builds.

#54 lore raw

Re: [dpdk-dev] [PATCH v4 3/3] build: add option to enable wait until equal

Quoted 39 lines

-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Wednesday, July 7, 2021 8:37 PM
To: Bruce Richardson <[email protected]>
Cc: Ruifeng Wang <[email protected]>; [email protected]; Jan Viktorin <[email protected]>; [email protected]; [email protected]; [email protected]; nd <[email protected]>; Honnappa Nagarahalli <[email protected]>
Subject: Re: [dpdk-dev] [PATCH v4 3/3] build: add option to enable wait until equal

Quoted 9 lines · Bruce Richardson
Quoted 7 lines · Ruifeng Wang
Quoted 7 lines

-----Original Message-----
From: Thomas Monjalon <[email protected]>
Sent: Wednesday, July 7, 2021 2:32 PM
To: Ruifeng Wang <[email protected]>
Cc: Jan Viktorin <[email protected]>; [email protected]; Bruce Richardson <[email protected]>; [email protected]; [email protected]; nd <[email protected]>; Honnappa

Nagarahalli

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines

<[email protected]>
Subject: Re: [PATCH v4 3/3] build: add option to enable wait until equal

07/07/2021 07:48, Ruifeng Wang:

Quoted 4 lines

Introduce a meson option 'use_wfe' to select wait until equal method. The default is disable. Traditional polling loop is used. When enabled, architecture specific mechanism is relied on to do the wait.

Quoted 4 lines
Quoted 4 lines
Quoted 3 lines

Why do we need an option?
Can it be automatic to enable it when supported?

The problem is inconsistency in performance on different Arm platforms.

We had decided that each platform needs to enable it after some testing.

Quoted 1 line
Quoted 1 line
Quoted 7 lines
Quoted 3 lines

Can that not be done via variables in the cross-file for the builds, or via automatic detection if it's a native build? Is it likely that individual users of DPDK will be knowledgable enough to use this option

correctly?

+1 to add this in cross-file instead of the top of config option as scope if is only for arm64 builds.

Currently this option is in config/arm/meson.build (flags_common). SoCs can build with this option enabled/disabled. And the ability is available for both native build and cross build as cross build also goes through meson.build.
If a SoC needs to enable the option by default, an entry and be added to the SoC flags.

The key difference here is whether this option need to be exposed to the top level config.

#55 lore raw

Re: [dpdk-dev] [PATCH v4 3/3] build: add option to enable wait until equal

Quoted 48 lines · Ruifeng Wang
Quoted 40 lines

-----Original Message-----
From: Jerin Jacob <[email protected]>
Sent: Wednesday, July 7, 2021 8:37 PM
To: Bruce Richardson <[email protected]>
Cc: Ruifeng Wang <[email protected]>; [email protected]; Jan Viktorin <[email protected]>; [email protected]; [email protected]; [email protected]; nd <[email protected]>; Honnappa Nagarahalli <[email protected]>
Subject: Re: [dpdk-dev] [PATCH v4 3/3] build: add option to enable wait until equal

Quoted 9 lines · Bruce Richardson
Quoted 7 lines · Ruifeng Wang
Quoted 7 lines

-----Original Message-----
From: Thomas Monjalon <[email protected]>
Sent: Wednesday, July 7, 2021 2:32 PM
To: Ruifeng Wang <[email protected]>
Cc: Jan Viktorin <[email protected]>; [email protected]; Bruce Richardson <[email protected]>; [email protected]; [email protected]; nd <[email protected]>; Honnappa

Nagarahalli

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines

<[email protected]>
Subject: Re: [PATCH v4 3/3] build: add option to enable wait until equal

07/07/2021 07:48, Ruifeng Wang:

Quoted 4 lines

Introduce a meson option 'use_wfe' to select wait until equal method. The default is disable. Traditional polling loop is used. When enabled, architecture specific mechanism is relied on to do the wait.

Quoted 4 lines
Quoted 4 lines
Quoted 3 lines

Why do we need an option?
Can it be automatic to enable it when supported?

The problem is inconsistency in performance on different Arm platforms.

We had decided that each platform needs to enable it after some testing.

Quoted 1 line
Quoted 1 line
Quoted 7 lines
Quoted 3 lines

Can that not be done via variables in the cross-file for the builds, or via automatic detection if it's a native build? Is it likely that individual users of DPDK will be knowledgable enough to use this option

correctly?

+1 to add this in cross-file instead of the top of config option as scope if is only for arm64 builds.

Quoted 6 lines

Currently this option is in config/arm/meson.build (flags_common). SoCs can build with this option enabled/disabled. And the ability is available for both native build and cross build as cross build also goes through meson.build.
If a SoC needs to enable the option by default, an entry and be added to the SoC flags.

The key difference here is whether this option need to be exposed to the top level config.

In the view of limiting top-level config options and it is specific to Arm, I think, it better to
be a cross file only option.

#56 lore raw

Re: [dpdk-dev] [PATCH v4 3/3] build: add option to enable wait until equal

08/07/2021 08:32, Jerin Jacob:

Quoted 31 lines · Ruifeng Wang
Quoted 30 lines

From: Jerin Jacob <[email protected]>

Quoted 16 lines
Quoted 8 lines · Bruce Richardson
Quoted 7 lines · Ruifeng Wang

From: Thomas Monjalon <[email protected]>

Quoted 6 lines

07/07/2021 07:48, Ruifeng Wang:

Quoted 4 lines

Introduce a meson option 'use_wfe' to select wait until equal method. The default is disable. Traditional polling loop is used. When enabled, architecture specific mechanism is relied on to do the wait.

Quoted 4 lines
Quoted 4 lines
Quoted 3 lines

Why do we need an option?
Can it be automatic to enable it when supported?

The problem is inconsistency in performance on different Arm platforms.

We had decided that each platform needs to enable it after some testing.

Quoted 1 line
Quoted 1 line
Quoted 7 lines
Quoted 3 lines

Can that not be done via variables in the cross-file for the builds, or via automatic detection if it's a native build? Is it likely that individual users of DPDK will be knowledgable enough to use this option

correctly?

+1 to add this in cross-file instead of the top of config option as scope if is only for arm64 builds.

Currently this option is in config/arm/meson.build (flags_common). SoCs can build with this option enabled/disabled. And the ability is available for both native build and cross build as cross build also goes through meson.build.
If a SoC needs to enable the option by default, an entry and be added to the SoC flags.

Quoted 5 lines
Quoted 1 line

The key difference here is whether this option need to be exposed to the top level config.

In the view of limiting top-level config options and it is specific to Arm, I think, it better to
be a cross file only option.

+1, sorry for the late notice.
I would advocate to take this patch in 21.08-rc2.

#57 lore raw

Re: [dpdk-dev] [PATCH v4 3/3] build: add option to enable wait until equal

Quoted 52 lines

-----Original Message-----
From: Thomas Monjalon <[email protected]>
Sent: Thursday, July 8, 2021 3:33 PM
To: Ruifeng Wang <[email protected]>; Jerin Jacob <[email protected]>
Cc: Bruce Richardson <[email protected]>; Jan Viktorin <[email protected]>; [email protected]; [email protected]; [email protected]; nd <[email protected]>; Honnappa Nagarahalli <[email protected]>
Subject: Re: [dpdk-dev] [PATCH v4 3/3] build: add option to enable wait until equal

08/07/2021 08:32, Jerin Jacob:

Quoted 1 line

On Thu, Jul 8, 2021 at 11:55 AM Ruifeng Wang <[email protected]>

Quoted 7 lines · wrote:
Quoted 7 lines

From: Jerin Jacob <[email protected]>

Quoted 6 lines
Quoted 4 lines · Bruce Richardson
Quoted 3 lines · Ruifeng Wang

From: Thomas Monjalon <[email protected]>

Quoted 2 lines

07/07/2021 07:48, Ruifeng Wang:

Quoted 1 line

Introduce a meson option 'use_wfe' to select wait until equal

method.

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
Quoted 3 lines

The default is disable. Traditional polling loop is used. When enabled, architecture specific mechanism is relied on to do the wait.

Why do we need an option?
Can it be automatic to enable it when supported?

The problem is inconsistency in performance on different Arm

platforms.

Quoted 1 line
Quoted 1 line
Quoted 1 line

We had decided that each platform needs to enable it after some

testing.

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
Quoted 1 line

Can that not be done via variables in the cross-file for the builds, or via automatic detection if it's a native build? Is it likely that individual users of DPDK will be knowledgable enough to use this option

correctly?

Quoted 7 lines
Quoted 7 lines
Quoted 3 lines

+1 to add this in cross-file instead of the top of config option +as
scope if is only for arm64 builds.

Currently this option is in config/arm/meson.build (flags_common). SoCs can build with this option enabled/disabled. And the ability is available for both native build and cross build as cross build also goes

through meson.build.

Quoted 1 line
Quoted 1 line

If a SoC needs to enable the option by default, an entry and be added to

the SoC flags.

Quoted 1 line
Quoted 1 line
Quoted 9 lines
Quoted 1 line
Quoted 1 line

The key difference here is whether this option need to be exposed to

the top level config.

Quoted 3 lines

In the view of limiting top-level config options and it is specific to Arm, I think, it better to be a cross file only option.

+1, sorry for the late notice.
I would advocate to take this patch in 21.08-rc2.

If the decision is not to expose this option in top-level config, we can just drop 3/3 patch. Do I need to send a new version?

#58 lore raw

Re: [dpdk-dev] [PATCH v4 3/3] build: add option to enable wait until equal

08/07/2021 11:21, Ruifeng Wang:

Quoted 44 lines

From: Thomas Monjalon <[email protected]>

Quoted 43 lines

08/07/2021 08:32, Jerin Jacob:

Quoted 1 line

On Thu, Jul 8, 2021 at 11:55 AM Ruifeng Wang <[email protected]>

Quoted 7 lines · wrote:
Quoted 7 lines

From: Jerin Jacob <[email protected]>

Quoted 6 lines
Quoted 4 lines · Bruce Richardson
Quoted 3 lines · Ruifeng Wang

From: Thomas Monjalon <[email protected]>

Quoted 2 lines

07/07/2021 07:48, Ruifeng Wang:

Quoted 1 line

Introduce a meson option 'use_wfe' to select wait until equal

method.

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
Quoted 3 lines

The default is disable. Traditional polling loop is used. When enabled, architecture specific mechanism is relied on to do the wait.

Why do we need an option?
Can it be automatic to enable it when supported?

The problem is inconsistency in performance on different Arm

platforms.

Quoted 1 line
Quoted 1 line
Quoted 1 line

We had decided that each platform needs to enable it after some

testing.

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
Quoted 1 line

Can that not be done via variables in the cross-file for the builds, or via automatic detection if it's a native build? Is it likely that individual users of DPDK will be knowledgable enough to use this option

correctly?

Quoted 7 lines
Quoted 7 lines
Quoted 3 lines

+1 to add this in cross-file instead of the top of config option +as
scope if is only for arm64 builds.

Currently this option is in config/arm/meson.build (flags_common). SoCs can build with this option enabled/disabled. And the ability is available for both native build and cross build as cross build also goes

through meson.build.

Quoted 1 line
Quoted 1 line

If a SoC needs to enable the option by default, an entry and be added to

the SoC flags.

Quoted 2 lines
Quoted 2 lines

The key difference here is whether this option need to be exposed to

the top level config.

Quoted 1 line
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

In the view of limiting top-level config options and it is specific to Arm, I think, it better to be a cross file only option.

+1, sorry for the late notice.
I would advocate to take this patch in 21.08-rc2.

If the decision is not to expose this option in top-level config, we can just drop 3/3 patch. Do I need to send a new version?

No it looks not needed to send a new version, I can take patches 1 & 2.

#59 lore raw
In reply to Ruifeng Wang

07/07/2021 07:48, Ruifeng Wang:

Quoted 6 lines

The rte_wait_until_equal_xxx APIs abstract the functionality of 'polling for a memory location to become equal to a given value'[1].

Use the API for the rte spinlock and ring implementations. With the wait until equal APIs being stable, changes will not impact ABI.

Quoted 6 lines

Gavin Hu (1):
spinlock: use wfe to reduce contention on aarch64

Ruifeng Wang (2):
ring: use wfe to wait for ring tail update on aarch64
build: add option to enable wait until equal

As discussed in the thread, patches 1 & 2 are applied. The patch 3 (meson option) is rejected.