Skip to contentSkip to conversation

ring: fix unaligned memory access on aarch32

PATCH Applied Phil Yang 6 participants 13 messages Mar 9, 2020 – Nov 13, 2023

lore
Share & tools
lore

Phil Yang

author
#0 lore raw

PATCH 1/1 ring: fix unaligned memory access on aarch32

The 32-bit arm machine doesn't support unaligned memory access. It will cause a bus error on aarch32 with the custom element size ring.

Thread 1 "test" received signal SIGBUS, Bus error. __rte_ring_enqueue_elems_64 (n=1, obj_table=0xf5edfe41, prod_head=0, \
r=0xf5edfb80) at /build/dpdk/build/include/rte_ring_elem.h:177
177 ring[idx++] = obj[i++];

Fixes cc4b218790f6 ("ring: support configurable element size")
---
 lib/librte_ring/rte_ring_elem.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
lib/librte_ring/rte_ring_elem.h +2 -2
index 3976757..663addc 100644
@@ -160,7 +160,7 @@ __rte_ring_enqueue_elems_64(struct rte_ring *r, uint32_t prod_head,
 	const uint32_t size = r->size;
 	uint32_t idx = prod_head & r->mask;
 	uint64_t *ring = (uint64_t *)&r[1];
-	const uint64_t *obj = (const uint64_t *)obj_table;
+	const unaligned_uint64_t *obj = (const unaligned_uint64_t *)obj_table;
 	if (likely(idx + n < size)) {
 		for (i = 0; i < (n & ~0x3); i += 4, idx += 4) {
 			ring[idx] = obj[i];
@@ -294,7 +294,7 @@ __rte_ring_dequeue_elems_64(struct rte_ring *r, uint32_t prod_head,
 	const uint32_t size = r->size;
 	uint32_t idx = prod_head & r->mask;
 	uint64_t *ring = (uint64_t *)&r[1];
-	uint64_t *obj = (uint64_t *)obj_table;
+	unaligned_uint64_t *obj = (unaligned_uint64_t *)obj_table;
 	if (likely(idx + n < size)) {
 		for (i = 0; i < (n & ~0x3); i += 4, idx += 4) {
 			obj[i] = ring[idx];
Signature · 1 line
2.7.4
#1 lore raw
Quoted 9 lines · Phil Yang

The 32-bit arm machine doesn't support unaligned memory access. It will cause a bus error on aarch32 with the custom element size ring.

Thread 1 "test" received signal SIGBUS, Bus error. __rte_ring_enqueue_elems_64 (n=1, obj_table=0xf5edfe41, prod_head=0, \
r=0xf5edfb80) at /build/dpdk/build/include/rte_ring_elem.h:177
177 ring[idx++] = obj[i++];

Quoted 5 lines
Fixes cc4b218790f6 ("ring: support configurable element size")

Applied, thanks.

Signature · 1 line
David Marchand
#2 lore raw
In reply to Phil Yang

I have for a long time now wondered why the ring functions for enqueue/dequeue of 64-bit objects supports unaligned addresses, and now I finally found the patch introducing it.

Quoted 10 lines

From: dev [mailto:[email protected]] On Behalf Of Phil Yang
Sent: Monday, 9 March 2020 18.20

The 32-bit arm machine doesn't support unaligned memory access. It will cause a bus error on aarch32 with the custom element size ring.

Thread 1 "test" received signal SIGBUS, Bus error. __rte_ring_enqueue_elems_64 (n=1, obj_table=0xf5edfe41, prod_head=0, \
r=0xf5edfb80) at /build/dpdk/build/include/rte_ring_elem.h:177
177 ring[idx++] = obj[i++];

Which test is this? Why is it using an unaligned array of 64-bit objects? (Notice that obj_table=0xf5edfe41.)

Nobody in their right mind would use an unaligned array of 64-bit objects. You can only create such an array if you force the compiler to prevent automatic alignment! And all the functions in your application using this array would also need to support unaligned addressing of these objects.

This seems extremely exotic, and not something any real application would do!

I would like to revert this patch for performance reasons.

Quoted 35 lines
Fixes cc4b218790f6 ("ring: support configurable element size")
---
 lib/librte_ring/rte_ring_elem.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
lib/librte_ring/rte_ring_elem.h +1 -1
index 3976757..663addc 100644
@@ -160,7 +160,7 @@ __rte_ring_enqueue_elems_64(struct rte_ring *r,
 uint32_t prod_head,
 	const uint32_t size = r->size;
 	uint32_t idx = prod_head & r->mask;
 	uint64_t *ring = (uint64_t *)&r[1];
-	const uint64_t *obj = (const uint64_t *)obj_table;
+	const unaligned_uint64_t *obj = (const unaligned_uint64_t
 *)obj_table;
 	if (likely(idx + n < size)) {
lib/librte_ring/rte_ring_elem.h +1 -1
 		for (i = 0; i < (n & ~0x3); i += 4, idx += 4) {
 			ring[idx] = obj[i];
@@ -294,7 +294,7 @@ __rte_ring_dequeue_elems_64(struct rte_ring *r,
 uint32_t prod_head,
 	const uint32_t size = r->size;
 	uint32_t idx = prod_head & r->mask;
 	uint64_t *ring = (uint64_t *)&r[1];
-	uint64_t *obj = (uint64_t *)obj_table;
+	unaligned_uint64_t *obj = (unaligned_uint64_t *)obj_table;
 	if (likely(idx + n < size)) {
 		for (i = 0; i < (n & ~0x3); i += 4, idx += 4) {
Quoted 4 lines

obj[i] = ring[idx];
--
2.7.4

References:
https://git.dpdk.org/dpdk/commit/lib/librte_ring/rte_ring_elem.h?id=3ba51478a3ab3132c33effc8b132641233275b36
https://patchwork.dpdk.org/project/dpdk/patch/[email protected]/

#3 lore raw
Quoted 23 lines

-----Original Message-----
From: Morten Brørup <[email protected]>
Sent: Friday, November 3, 2023 7:04 PM
To: Phil Yang <[email protected]>; Honnappa Nagarahalli <[email protected]>; Ruifeng Wang <[email protected]>; [email protected]
Cc: [email protected]; [email protected]; Dharmik Jayesh Thakkar <[email protected]>; Gavin Hu <[email protected]>; nd <[email protected]>; [email protected]
Subject: RE: [dpdk-dev] [PATCH] ring: fix unaligned memory access on aarch32

I have for a long time now wondered why the ring functions for enqueue/dequeue of 64-bit objects supports unaligned addresses, and now I finally found the patch introducing it.

Quoted 6 lines

From: dev [mailto:[email protected]] On Behalf Of Phil Yang
Sent: Monday, 9 March 2020 18.20

The 32-bit arm machine doesn't support unaligned memory access. It will cause a bus error on aarch32 with the custom element size ring.

Quoted 7 lines
Quoted 4 lines

Thread 1 "test" received signal SIGBUS, Bus error. __rte_ring_enqueue_elems_64 (n=1, obj_table=0xf5edfe41, prod_head=0, \
r=0xf5edfb80) at /build/dpdk/build/include/rte_ring_elem.h:177
177 ring[idx++] = obj[i++];

Which test is this? Why is it using an unaligned array of 64-bit objects? (Notice that obj_table=0xf5edfe41.)

Can't recollect which test it is. I am guessing one of the unit test cases. We might have to reinvestigate, not sure why the obj_table is unaligned.

Quoted 9 lines

Nobody in their right mind would use an unaligned array of 64-bit objects. You can only create such an array if you force the compiler to prevent automatic alignment! And all the functions in your application using this array would also need to support unaligned addressing of these objects.

This seems extremely exotic, and not something any real application would do!

I would like to revert this patch for performance reasons.

Can you provide more details? Platform, test, how much is the regression?

Quoted 45 lines
Quoted 35 lines
Fixes cc4b218790f6 ("ring: support configurable element size")
---
 lib/librte_ring/rte_ring_elem.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
(quoted diff) +0 -0

b/lib/librte_ring/rte_ring_elem.h index 3976757..663addc 100644

lib/librte_ring/rte_ring_elem.h +1 -1
@@ -160,7 +160,7 @@ __rte_ring_enqueue_elems_64(struct rte_ring *r,
 uint32_t prod_head,
 	const uint32_t size = r->size;
 	uint32_t idx = prod_head & r->mask;
 	uint64_t *ring = (uint64_t *)&r[1];
-	const uint64_t *obj = (const uint64_t *)obj_table;
+	const unaligned_uint64_t *obj = (const unaligned_uint64_t
 *)obj_table;
 	if (likely(idx + n < size)) {
lib/librte_ring/rte_ring_elem.h +1 -1
 		for (i = 0; i < (n & ~0x3); i += 4, idx += 4) {
 			ring[idx] = obj[i];
@@ -294,7 +294,7 @@ __rte_ring_dequeue_elems_64(struct rte_ring *r,
 uint32_t prod_head,
 	const uint32_t size = r->size;
 	uint32_t idx = prod_head & r->mask;
 	uint64_t *ring = (uint64_t *)&r[1];
-	uint64_t *obj = (uint64_t *)obj_table;
+	unaligned_uint64_t *obj = (unaligned_uint64_t *)obj_table;
 	if (likely(idx + n < size)) {
 		for (i = 0; i < (n & ~0x3); i += 4, idx += 4) {
Quoted 4 lines

obj[i] = ring[idx];
--
2.7.4

References:
https://git.dpdk.org/dpdk/commit/lib/librte_ring/rte_ring_elem.h?id=3ba514
78a3ab3132c33effc8b132641233275b36
https://patchwork.dpdk.org/project/dpdk/patch/1583774395-10233-1-git-
[email protected]/

#4 lore raw
Quoted 24 lines

From: Honnappa Nagarahalli [mailto:[email protected]]
Sent: Saturday, 4 November 2023 17.32

Quoted 5 lines

From: Morten Brørup <[email protected]>
Sent: Friday, November 3, 2023 7:04 PM

I have for a long time now wondered why the ring functions for enqueue/dequeue of 64-bit objects supports unaligned addresses, and

now I

Quoted 7 lines

finally found the patch introducing it.

Quoted 5 lines

From: dev [mailto:[email protected]] On Behalf Of Phil Yang
Sent: Monday, 9 March 2020 18.20

The 32-bit arm machine doesn't support unaligned memory access. It will cause a bus error on aarch32 with the custom element size

ring.

Quoted 3 lines
Quoted 3 lines

Thread 1 "test" received signal SIGBUS, Bus error. __rte_ring_enqueue_elems_64 (n=1, obj_table=0xf5edfe41,

prod_head=0, \

Quoted 3 lines
Quoted 2 lines

r=0xf5edfb80) at /build/dpdk/build/include/rte_ring_elem.h:177
177 ring[idx++] = obj[i++];

Quoted 6 lines
Quoted 1 line

Which test is this? Why is it using an unaligned array of 64-bit

objects? (Notice

Quoted 1 line

that obj_table=0xf5edfe41.)

Can't recollect which test it is. I am guessing one of the unit test cases. We might have to reinvestigate, not sure why the obj_table is unaligned.

Thank you for picking this up, Honnappa.

Quoted 8 lines
Quoted 3 lines

Nobody in their right mind would use an unaligned array of 64-bit

objects. You

Quoted 1 line

can only create such an array if you force the compiler to prevent

automatic

Quoted 1 line

alignment! And all the functions in your application using this array

would also

Quoted 8 lines
Quoted 3 lines

need to support unaligned addressing of these objects.

This seems extremely exotic, and not something any real application

would do!

Quoted 2 lines

I would like to revert this patch for performance reasons.

Can you provide more details? Platform, test, how much is the regression?

I haven't seen a regression, but I speculate some performance cost on low-end CPUs. Maybe it is purely academic.

Maybe not purely academic... I just tested on Godbolt, which shows different code generated:

uint64_t fa(void *p)

{
    return *(uint64_t *)p;
}

uint64_t fu(void *p)

{
    return *(unaligned_uint64_t *)p;
}

Generates different output:

fa:

        ldrd    r0, [r0]
        bx      lr

fu:

        mov     r3, r0
        ldr     r0, [r0]  @ unaligned
        ldr     r1, [r3, #4]      @ unaligned
        bx      lr
Quoted 48 lines
Quoted 34 lines
Quoted 34 lines
Fixes cc4b218790f6 ("ring: support configurable element size")
---
 lib/librte_ring/rte_ring_elem.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
(quoted diff) +0 -0

b/lib/librte_ring/rte_ring_elem.h index 3976757..663addc 100644

lib/librte_ring/rte_ring_elem.h +1 -1
@@ -160,7 +160,7 @@ __rte_ring_enqueue_elems_64(struct rte_ring *r,
 uint32_t prod_head,
 	const uint32_t size = r->size;
 	uint32_t idx = prod_head & r->mask;
 	uint64_t *ring = (uint64_t *)&r[1];
-	const uint64_t *obj = (const uint64_t *)obj_table;
+	const unaligned_uint64_t *obj = (const unaligned_uint64_t
 *)obj_table;
 	if (likely(idx + n < size)) {
lib/librte_ring/rte_ring_elem.h +1 -1
 		for (i = 0; i < (n & ~0x3); i += 4, idx += 4) {
 			ring[idx] = obj[i];
@@ -294,7 +294,7 @@ __rte_ring_dequeue_elems_64(struct rte_ring *r,
 uint32_t prod_head,
 	const uint32_t size = r->size;
 	uint32_t idx = prod_head & r->mask;
 	uint64_t *ring = (uint64_t *)&r[1];
-	uint64_t *obj = (uint64_t *)obj_table;
+	unaligned_uint64_t *obj = (unaligned_uint64_t *)obj_table;
Quoted 9 lines
Quoted 6 lines
lib/librte_ring/rte_ring_elem.h +0 -0
 	if (likely(idx + n < size)) {
 		for (i = 0; i < (n & ~0x3); i += 4, idx += 4) {

obj[i] = ring[idx];
--
2.7.4

References:

https://git.dpdk.org/dpdk/commit/lib/librte_ring/rte_ring_elem.h?id=3ba
514

Quoted 3 lines

78a3ab3132c33effc8b132641233275b36
https://patchwork.dpdk.org/project/dpdk/patch/1583774395-10233-1-git-
[email protected]/

#5 lore raw
In reply to Morten Brørup
Quoted 8 lines · Morten Brørup

I have for a long time now wondered why the ring functions for enqueue/dequeue of 64-bit objects supports unaligned addresses, and now I finally found the patch introducing it.

Quoted 6 lines

From: dev [mailto:[email protected]] On Behalf Of Phil Yang
Sent: Monday, 9 March 2020 18.20

The 32-bit arm machine doesn't support unaligned memory access. It will cause a bus error on aarch32 with the custom element size ring.

Quoted 6 lines
Quoted 4 lines

Thread 1 "test" received signal SIGBUS, Bus error. __rte_ring_enqueue_elems_64 (n=1, obj_table=0xf5edfe41, prod_head=0, \
r=0xf5edfb80) at /build/dpdk/build/include/rte_ring_elem.h:177
177 ring[idx++] = obj[i++];

Which test is this? Why is it using an unaligned array of 64-bit objects? (Notice that obj_table=0xf5edfe41.)

The test case is:
https://elixir.bootlin.com/dpdk/latest/source/app/test/test_ring.c#L1121
This case deliberately use unaligned objects.

Quoted 51 lines

Nobody in their right mind would use an unaligned array of 64-bit objects. You can only create such an array if you force the compiler to prevent automatic alignment! And all the functions in your application using this array would also need to support unaligned addressing of these objects.

This seems extremely exotic, and not something any real application would do!

I would like to revert this patch for performance reasons.

Quoted 35 lines
Fixes cc4b218790f6 ("ring: support configurable element size")
---
  lib/librte_ring/rte_ring_elem.h | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)
lib/librte_ring/rte_ring_elem.h +1 -1
index 3976757..663addc 100644
@@ -160,7 +160,7 @@ __rte_ring_enqueue_elems_64(struct rte_ring *r,
 uint32_t prod_head,
  	const uint32_t size = r->size;
  	uint32_t idx = prod_head & r->mask;
  	uint64_t *ring = (uint64_t *)&r[1];
-	const uint64_t *obj = (const uint64_t *)obj_table;
+	const unaligned_uint64_t *obj = (const unaligned_uint64_t
 *)obj_table;
  	if (likely(idx + n < size)) {
  		for (i = 0; i < (n & ~0x3); i += 4, idx += 4) {
  			ring[idx] = obj[i];
lib/librte_ring/rte_ring_elem.h +1 -1
@@ -294,7 +294,7 @@ __rte_ring_dequeue_elems_64(struct rte_ring *r,
 uint32_t prod_head,
  	const uint32_t size = r->size;
  	uint32_t idx = prod_head & r->mask;
  	uint64_t *ring = (uint64_t *)&r[1];
-	uint64_t *obj = (uint64_t *)obj_table;
+	unaligned_uint64_t *obj = (unaligned_uint64_t *)obj_table;
  	if (likely(idx + n < size)) {
  		for (i = 0; i < (n & ~0x3); i += 4, idx += 4) {
Quoted 4 lines

obj[i] = ring[idx];
--
2.7.4

References:
https://git.dpdk.org/dpdk/commit/lib/librte_ring/rte_ring_elem.h?id=3ba51478a3ab3132c33effc8b132641233275b36
https://patchwork.dpdk.org/project/dpdk/patch/[email protected]/

#6 lore raw

+CC Gavin, reviewed the test case

Quoted 20 lines

From: Ruifeng Wang [mailto:[email protected]]
Sent: Friday, 10 November 2023 09.40

Quoted 1 line · Morten Brørup

I have for a long time now wondered why the ring functions for

enqueue/dequeue of 64-bit objects supports unaligned addresses, and now I finally found the patch introducing it.

Quoted 9 lines
Quoted 9 lines

From: dev [mailto:[email protected]] On Behalf Of Phil Yang
Sent: Monday, 9 March 2020 18.20

The 32-bit arm machine doesn't support unaligned memory access. It will cause a bus error on aarch32 with the custom element size ring.

Thread 1 "test" received signal SIGBUS, Bus error. __rte_ring_enqueue_elems_64 (n=1, obj_table=0xf5edfe41, prod_head=0,

\

Quoted 3 lines
Quoted 2 lines

r=0xf5edfb80) at /build/dpdk/build/include/rte_ring_elem.h:177
177 ring[idx++] = obj[i++];

Quoted 7 lines
Quoted 1 line

Which test is this? Why is it using an unaligned array of 64-bit

objects? (Notice that obj_table=0xf5edfe41.)

The test case is:
https://elixir.bootlin.com/dpdk/latest/source/app/test/test_ring.c#L112
1
This case deliberately use unaligned objects.

Thank you, Ruifeng.

Honnappa, I see you signed off on the patch introducing the test for unaligned objects:
http://git.dpdk.org/dpdk/commit/app/test/test_ring.c?id=a9fe152363e283d4c590ab8e8d51396f2ffab9ff

What was the rationale behind testing support for unaligned object pointers? Did any applications/customers use unaligned object pointers, or is it a purely academic test case?

Quoted 12 lines
Quoted 3 lines

Nobody in their right mind would use an unaligned array of 64-bit

objects. You can only create such an array if you force the compiler to prevent automatic alignment! And all the functions in your application using this array would also need to support unaligned addressing of these objects.

Quoted 2 lines

This seems extremely exotic, and not something any real application

would do!

Quoted 2 lines

I would like to revert this patch for performance reasons.

I could add an RTE_ASSERT() to verify that the pointer is aligned, for debugging purposes.

Quoted 48 lines
Quoted 34 lines
Quoted 34 lines
Fixes cc4b218790f6 ("ring: support configurable element size")
---
  lib/librte_ring/rte_ring_elem.h | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)
lib/librte_ring/rte_ring_elem.h +1 -1
index 3976757..663addc 100644
@@ -160,7 +160,7 @@ __rte_ring_enqueue_elems_64(struct rte_ring *r,
 uint32_t prod_head,
  	const uint32_t size = r->size;
  	uint32_t idx = prod_head & r->mask;
  	uint64_t *ring = (uint64_t *)&r[1];
-	const uint64_t *obj = (const uint64_t *)obj_table;
+	const unaligned_uint64_t *obj = (const unaligned_uint64_t
 *)obj_table;
  	if (likely(idx + n < size)) {
  		for (i = 0; i < (n & ~0x3); i += 4, idx += 4) {
  			ring[idx] = obj[i];
lib/librte_ring/rte_ring_elem.h +1 -1
@@ -294,7 +294,7 @@ __rte_ring_dequeue_elems_64(struct rte_ring *r,
 uint32_t prod_head,
  	const uint32_t size = r->size;
  	uint32_t idx = prod_head & r->mask;
  	uint64_t *ring = (uint64_t *)&r[1];
-	uint64_t *obj = (uint64_t *)obj_table;
+	unaligned_uint64_t *obj = (unaligned_uint64_t *)obj_table;
Quoted 9 lines
Quoted 6 lines
lib/librte_ring/rte_ring_elem.h +0 -0
  	if (likely(idx + n < size)) {
  		for (i = 0; i < (n & ~0x3); i += 4, idx += 4) {

obj[i] = ring[idx];
--
2.7.4

References:

https://git.dpdk.org/dpdk/commit/lib/librte_ring/rte_ring_elem.h?id=3ba
51478a3ab3132c33effc8b132641233275b36

Quoted 1 line

https://patchwork.dpdk.org/project/dpdk/patch/1583774395-10233-1-git-

[email protected]/

Quoted 1 line
#7 lore raw
Quoted 40 lines

-----Original Message-----
From: Morten Brørup <[email protected]>
Sent: Friday, November 10, 2023 9:34 AM
To: Ruifeng Wang <[email protected]>; [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Gavin Hu <[email protected]>
Subject: RE: [dpdk-dev] [PATCH] ring: fix unaligned memory access on aarch32

+CC Gavin, reviewed the test case

Quoted 20 lines

From: Ruifeng Wang [mailto:[email protected]]
Sent: Friday, 10 November 2023 09.40

Quoted 1 line · Morten Brørup

I have for a long time now wondered why the ring functions for

enqueue/dequeue of 64-bit objects supports unaligned addresses, and now I finally found the patch introducing it.

Quoted 9 lines
Quoted 9 lines

From: dev [mailto:[email protected]] On Behalf Of Phil Yang
Sent: Monday, 9 March 2020 18.20

The 32-bit arm machine doesn't support unaligned memory access. It will cause a bus error on aarch32 with the custom element size ring.

Thread 1 "test" received signal SIGBUS, Bus error. __rte_ring_enqueue_elems_64 (n=1, obj_table=0xf5edfe41, prod_head=0,

\

Quoted 3 lines
Quoted 2 lines

r=0xf5edfb80) at /build/dpdk/build/include/rte_ring_elem.h:177
177 ring[idx++] = obj[i++];

Quoted 7 lines
Quoted 1 line

Which test is this? Why is it using an unaligned array of 64-bit

objects? (Notice that obj_table=0xf5edfe41.)

The test case is:
https://elixir.bootlin.com/dpdk/latest/source/app/test/test_ring.c#L112
1
This case deliberately use unaligned objects.

Quoted 15 lines

Thank you, Ruifeng.

Honnappa, I see you signed off on the patch introducing the test for unaligned objects:
http://git.dpdk.org/dpdk/commit/app/test/test_ring.c?id=a9fe152363e283d4c590ab8e8d51396f2ffab9ff

What was the rationale behind testing support for unaligned object pointers? Did any applications/customers use unaligned object pointers, or is it a purely academic test case?

Quoted 7 lines
Quoted 3 lines

Nobody in their right mind would use an unaligned array of 64-bit

objects. You can only create such an array if you force the compiler to prevent automatic alignment! And all the functions in your application using this array would also need to support unaligned addressing of these objects.

It could be just one elem, not an array.
And the user can use 'packed' struct or so... Agree, not a common case, but probably still possible.

Quoted 5 lines
Quoted 5 lines
Quoted 2 lines

This seems extremely exotic, and not something any real application

would do!

Quoted 2 lines

I would like to revert this patch for performance reasons.

Morten, could you probably explain first the problems you encountered with this patch? You mention about 'performance reasons', so did you notice any real slowdown?

Quoted 51 lines

I could add an RTE_ASSERT() to verify that the pointer is aligned, for debugging purposes.

Quoted 48 lines
Quoted 34 lines
Quoted 34 lines
Fixes cc4b218790f6 ("ring: support configurable element size")
---
  lib/librte_ring/rte_ring_elem.h | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)
lib/librte_ring/rte_ring_elem.h +1 -1
index 3976757..663addc 100644
@@ -160,7 +160,7 @@ __rte_ring_enqueue_elems_64(struct rte_ring *r,
 uint32_t prod_head,
  	const uint32_t size = r->size;
  	uint32_t idx = prod_head & r->mask;
  	uint64_t *ring = (uint64_t *)&r[1];
-	const uint64_t *obj = (const uint64_t *)obj_table;
+	const unaligned_uint64_t *obj = (const unaligned_uint64_t
 *)obj_table;
  	if (likely(idx + n < size)) {
  		for (i = 0; i < (n & ~0x3); i += 4, idx += 4) {
  			ring[idx] = obj[i];
lib/librte_ring/rte_ring_elem.h +1 -1
@@ -294,7 +294,7 @@ __rte_ring_dequeue_elems_64(struct rte_ring *r,
 uint32_t prod_head,
  	const uint32_t size = r->size;
  	uint32_t idx = prod_head & r->mask;
  	uint64_t *ring = (uint64_t *)&r[1];
-	uint64_t *obj = (uint64_t *)obj_table;
+	unaligned_uint64_t *obj = (unaligned_uint64_t *)obj_table;
Quoted 9 lines
Quoted 6 lines
lib/librte_ring/rte_ring_elem.h +0 -0
  	if (likely(idx + n < size)) {
  		for (i = 0; i < (n & ~0x3); i += 4, idx += 4) {

obj[i] = ring[idx];
--
2.7.4

References:

https://git.dpdk.org/dpdk/commit/lib/librte_ring/rte_ring_elem.h?id=3ba
51478a3ab3132c33effc8b132641233275b36

Quoted 1 line

https://patchwork.dpdk.org/project/dpdk/patch/1583774395-10233-1-git-

[email protected]/

Quoted 1 line
#8 lore raw
Quoted 58 lines

From: Konstantin Ananyev [mailto:[email protected]]
Sent: Friday, 10 November 2023 10.45

Quoted 11 lines

From: Morten Brørup <[email protected]>
Sent: Friday, November 10, 2023 9:34 AM

+CC Gavin, reviewed the test case

Quoted 6 lines

From: Ruifeng Wang [mailto:[email protected]]
Sent: Friday, 10 November 2023 09.40

Quoted 1 line · Morten Brørup

I have for a long time now wondered why the ring functions for

enqueue/dequeue of 64-bit objects supports unaligned addresses, and

now

Quoted 6 lines
Quoted 6 lines

I finally found the patch introducing it.

Quoted 5 lines
Quoted 5 lines

From: dev [mailto:[email protected]] On Behalf Of Phil Yang
Sent: Monday, 9 March 2020 18.20

The 32-bit arm machine doesn't support unaligned memory access.

It

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

will cause a bus error on aarch32 with the custom element size

ring.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Thread 1 "test" received signal SIGBUS, Bus error. __rte_ring_enqueue_elems_64 (n=1, obj_table=0xf5edfe41,

prod_head=0,

Quoted 9 lines
Quoted 9 lines

\

Quoted 4 lines
Quoted 2 lines

r=0xf5edfb80) at /build/dpdk/build/include/rte_ring_elem.h:177
177 ring[idx++] = obj[i++];

Which test is this? Why is it using an unaligned array of 64-bit

objects? (Notice that obj_table=0xf5edfe41.)

The test case is:

https://elixir.bootlin.com/dpdk/latest/source/app/test/test_ring.c#L112

Quoted 6 lines
Quoted 2 lines

1
This case deliberately use unaligned objects.

Thank you, Ruifeng.

Honnappa, I see you signed off on the patch introducing the test for

unaligned objects:

Quoted 1 line

http://git.dpdk.org/dpdk/commit/app/test/test_ring.c?id=a9fe152363e283d
4c590ab8e8d51396f2ffab9ff

Quoted 2 lines

What was the rationale behind testing support for unaligned object

pointers? Did any applications/customers use unaligned object

Quoted 6 lines

pointers, or is it a purely academic test case?

Quoted 4 lines
Quoted 3 lines

Nobody in their right mind would use an unaligned array of 64-bit

objects. You can only create such an array if you force the

compiler to

Quoted 8 lines
Quoted 1 line
Quoted 1 line

prevent automatic alignment! And all the functions in your

application

Quoted 2 lines
Quoted 2 lines

using this array would also need to support unaligned addressing of these objects.

It could be just one elem, not an array.
And the user can use 'packed' struct or so... Agree, not a common case, but probably still possible.

Very good point, Konstantin. A single unaligned object in a packed structure is not as exotic as an unaligned array of objects (which I consider completely unrealistic).

If anyone is using an architecture which doesn't support unaligned accesses, I would expect them to completely avoid using unaligned objects. But perhaps you are right; however unlikely, it might happen.

If we think this is a real use case, should we add support for unaligned 32 bit objects?
(128 bit objects already support unaligned access; they are type casted to void pointer and accessed using memcpy().)

And how about the stack library, should it support unaligned objects too?

Quoted 4 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

This seems extremely exotic, and not something any real

application

Quoted 8 lines
Quoted 3 lines
Quoted 3 lines

would do!

Quoted 2 lines

I would like to revert this patch for performance reasons.

Morten, could you probably explain first the problems you encountered with this patch?
You mention about 'performance reasons', so did you notice any real slowdown?

Please check my reply to the same question here:
http://inbox.dpdk.org/dev/[email protected]/

Quoted 57 lines
Quoted 3 lines

I could add an RTE_ASSERT() to verify that the pointer is aligned,

for debugging purposes.

Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Fixes cc4b218790f6 ("ring: support configurable element size")
---
  lib/librte_ring/rte_ring_elem.h | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

lib/librte_ring/rte_ring_elem.h __rte_ring_enqueue_elems_64(struct rte_ring

Quoted 6 lines
lib/librte_ring/rte_ring_elem.h +0 -0
index 3976757..663addc 100644
@@ -160,7 +160,7 @@ __rte_ring_enqueue_elems_64(struct rte_ring

*r,

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

lib/librte_ring/rte_ring_elem.h __rte_ring_dequeue_elems_64(struct rte_ring

Quoted 11 lines
lib/librte_ring/rte_ring_elem.h +1 -1
 uint32_t prod_head,
  	const uint32_t size = r->size;
  	uint32_t idx = prod_head & r->mask;
  	uint64_t *ring = (uint64_t *)&r[1];
-	const uint64_t *obj = (const uint64_t *)obj_table;
+	const unaligned_uint64_t *obj = (const unaligned_uint64_t
 *)obj_table;
  	if (likely(idx + n < size)) {
  		for (i = 0; i < (n & ~0x3); i += 4, idx += 4) {
  			ring[idx] = obj[i];
lib/librte_ring/rte_ring_elem.h +0 -0
@@ -294,7 +294,7 @@ __rte_ring_dequeue_elems_64(struct rte_ring

*r,

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
lib/librte_ring/rte_ring_elem.h +1 -1
 uint32_t prod_head,
  	const uint32_t size = r->size;
  	uint32_t idx = prod_head & r->mask;
  	uint64_t *ring = (uint64_t *)&r[1];
-	uint64_t *obj = (uint64_t *)obj_table;
+	unaligned_uint64_t *obj = (unaligned_uint64_t *)obj_table;
  	if (likely(idx + n < size)) {
  		for (i = 0; i < (n & ~0x3); i += 4, idx += 4) {
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
Quoted 4 lines

obj[i] = ring[idx];
--
2.7.4

References:

https://git.dpdk.org/dpdk/commit/lib/librte_ring/rte_ring_elem.h?id=3ba

Quoted 2 lines
Quoted 2 lines

51478a3ab3132c33effc8b132641233275b36

Quoted 1 line

https://patchwork.dpdk.org/project/dpdk/patch/1583774395-10233-1-

git-

Quoted 2 lines
Quoted 2 lines

[email protected]/

Quoted 1 line
#9 lore raw

Dear Ruifeng,
+CC: all CPU architecture maintainers,

I'm trying to figure out the requirements for supporting unaligned memory access in DPDK (specifically the ring library), and need your expert feedback.

The #define RTE_ARCH_STRICT_ALIGN - which is undocumented, but probably means that CPU memory access must be aligned - is only set by "generic_aarch32".

So we will assume that all other CPU architectures supported by DPDK can access unaligned memory.

As discussed in this thread, "generic_aarch32" has special requirements for performing 64-bit load/store at unaligned addresses.

Now comes the big question: Can "generic_aarch32" perform 32-bit load/store at unaligned addresses without similar special requirements? Then the ring library already supports unaligned 32-bit objects, and doesn't need to be fixed in this regard.

Med venlig hilsen / Kind regards,
-Morten Brørup

Quoted 174 lines

From: Morten Brørup [mailto:[email protected]]
Sent: Friday, 10 November 2023 11.44

Quoted 14 lines

From: Konstantin Ananyev [mailto:[email protected]]
Sent: Friday, 10 November 2023 10.45

Quoted 11 lines

From: Morten Brørup <[email protected]>
Sent: Friday, November 10, 2023 9:34 AM

+CC Gavin, reviewed the test case

Quoted 6 lines

From: Ruifeng Wang [mailto:[email protected]]
Sent: Friday, 10 November 2023 09.40

Quoted 1 line · Morten Brørup

I have for a long time now wondered why the ring functions for

enqueue/dequeue of 64-bit objects supports unaligned addresses,

and

Quoted 7 lines

now

Quoted 6 lines
Quoted 6 lines

I finally found the patch introducing it.

Quoted 5 lines
Quoted 5 lines

From: dev [mailto:[email protected]] On Behalf Of Phil Yang
Sent: Monday, 9 March 2020 18.20

The 32-bit arm machine doesn't support unaligned memory

access.

Quoted 12 lines

It

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

will cause a bus error on aarch32 with the custom element size

ring.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Thread 1 "test" received signal SIGBUS, Bus error. __rte_ring_enqueue_elems_64 (n=1, obj_table=0xf5edfe41,

prod_head=0,

Quoted 5 lines
Quoted 5 lines

\

Quoted 4 lines
Quoted 2 lines

r=0xf5edfb80) at /build/dpdk/build/include/rte_ring_elem.h:177
177 ring[idx++] = obj[i++];

Which test is this? Why is it using an unaligned array of 64-

bit

Quoted 5 lines
Quoted 4 lines
Quoted 4 lines

objects? (Notice that obj_table=0xf5edfe41.)

The test case is:

https://elixir.bootlin.com/dpdk/latest/source/app/test/test_ring.c#L112

Quoted 6 lines
Quoted 6 lines
Quoted 2 lines

1
This case deliberately use unaligned objects.

Thank you, Ruifeng.

Honnappa, I see you signed off on the patch introducing the test

for

Quoted 3 lines

unaligned objects:

Quoted 1 line

http://git.dpdk.org/dpdk/commit/app/test/test_ring.c?id=a9fe152363e283d

Quoted 9 lines

4c590ab8e8d51396f2ffab9ff

Quoted 2 lines

What was the rationale behind testing support for unaligned object

pointers? Did any applications/customers use unaligned object

Quoted 5 lines

pointers, or is it a purely academic test case?

Quoted 3 lines
Quoted 3 lines

Nobody in their right mind would use an unaligned array of 64-

bit

Quoted 5 lines
Quoted 1 line
Quoted 1 line

objects. You can only create such an array if you force the

compiler to

Quoted 1 line
Quoted 1 line

prevent automatic alignment! And all the functions in your

application

Quoted 1 line
Quoted 1 line

using this array would also need to support unaligned addressing

of

Quoted 5 lines
Quoted 1 line
Quoted 1 line

these objects.

It could be just one elem, not an array.
And the user can use 'packed' struct or so... Agree, not a common case, but probably still possible.

Very good point, Konstantin. A single unaligned object in a packed structure is not as exotic as an unaligned array of objects (which I consider completely unrealistic).

If anyone is using an architecture which doesn't support unaligned accesses, I would expect them to completely avoid using unaligned objects. But perhaps you are right; however unlikely, it might happen.

If we think this is a real use case, should we add support for unaligned 32 bit objects?
(128 bit objects already support unaligned access; they are type casted to void pointer and accessed using memcpy().)

And how about the stack library, should it support unaligned objects too?

Quoted 4 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

This seems extremely exotic, and not something any real

application

Quoted 8 lines
Quoted 3 lines
Quoted 3 lines

would do!

Quoted 2 lines

I would like to revert this patch for performance reasons.

Morten, could you probably explain first the problems you encountered with this patch?
You mention about 'performance reasons', so did you notice any real slowdown?

Please check my reply to the same question here:
http://inbox.dpdk.org/dev/98CBD80474FA8B44BF855DF32C47DC35E9EFD3@smarts
erver.smartshare.dk/

Quoted 8 lines
Quoted 3 lines

I could add an RTE_ASSERT() to verify that the pointer is aligned,

for debugging purposes.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Fixes cc4b218790f6 ("ring: support configurable element

size")

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Reviewed-by Honnappa Nagarahalli

<[email protected]>

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines

lib/librte_ring/rte_ring_elem.h __rte_ring_enqueue_elems_64(struct

Quoted 10 lines
---
  lib/librte_ring/rte_ring_elem.h | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)
lib/librte_ring/rte_ring_elem.h +0 -0
index 3976757..663addc 100644
@@ -160,7 +160,7 @@ __rte_ring_enqueue_elems_64(struct

rte_ring

Quoted 12 lines

*r,

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

lib/librte_ring/rte_ring_elem.h __rte_ring_dequeue_elems_64(struct

Quoted 11 lines
lib/librte_ring/rte_ring_elem.h +1 -1
 uint32_t prod_head,
  	const uint32_t size = r->size;
  	uint32_t idx = prod_head & r->mask;
  	uint64_t *ring = (uint64_t *)&r[1];
-	const uint64_t *obj = (const uint64_t *)obj_table;
+	const unaligned_uint64_t *obj = (const unaligned_uint64_t
 *)obj_table;
  	if (likely(idx + n < size)) {
  		for (i = 0; i < (n & ~0x3); i += 4, idx += 4) {
  			ring[idx] = obj[i];
lib/librte_ring/rte_ring_elem.h +0 -0
@@ -294,7 +294,7 @@ __rte_ring_dequeue_elems_64(struct

rte_ring

Quoted 18 lines

*r,

Quoted 16 lines
Quoted 16 lines
Quoted 15 lines
Quoted 8 lines
lib/librte_ring/rte_ring_elem.h +1 -1
 uint32_t prod_head,
  	const uint32_t size = r->size;
  	uint32_t idx = prod_head & r->mask;
  	uint64_t *ring = (uint64_t *)&r[1];
-	uint64_t *obj = (uint64_t *)obj_table;
+	unaligned_uint64_t *obj = (unaligned_uint64_t *)obj_table;
  	if (likely(idx + n < size)) {
  		for (i = 0; i < (n & ~0x3); i += 4, idx += 4) {
Quoted 4 lines

obj[i] = ring[idx];
--
2.7.4

References:

https://git.dpdk.org/dpdk/commit/lib/librte_ring/rte_ring_elem.h?id=3ba

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

51478a3ab3132c33effc8b132641233275b36

Quoted 1 line

https://patchwork.dpdk.org/project/dpdk/patch/1583774395-10233-

1-

Quoted 3 lines

git-

Quoted 2 lines
Quoted 2 lines

[email protected]/

Quoted 1 line
#10 lore raw
Quoted 1 line · Morten Brørup

Dear Ruifeng,

Hi Morten,

Quoted 9 lines

+CC: all CPU architecture maintainers,

I'm trying to figure out the requirements for supporting unaligned memory access in DPDK (specifically the ring library), and need your expert feedback.

The #define RTE_ARCH_STRICT_ALIGN - which is undocumented, but probably means that CPU memory access must be aligned - is only set by "generic_aarch32".

So we will assume that all other CPU architectures supported by DPDK can access unaligned memory.

As discussed in this thread, "generic_aarch32" has special requirements for performing 64-bit load/store at unaligned addresses.

Yes, 64-bit load/store at unaligned addresses causes alignment fault.

Quoted 2 lines

Now comes the big question: Can "generic_aarch32" perform 32-bit load/store at unaligned addresses without similar special requirements? Then the ring library already supports unaligned 32-bit objects, and doesn't need to be fixed in this regard.

Yes, 32-bit load/store support unaligned data accesses to normal memory (not device memory). This is documented in Arm Architecture Reference Manual.

Thanks,
Ruifeng

Quoted 180 lines

Med venlig hilsen / Kind regards,
-Morten Brørup

Quoted 174 lines

From: Morten Brørup [mailto:[email protected]]
Sent: Friday, 10 November 2023 11.44

Quoted 14 lines

From: Konstantin Ananyev [mailto:[email protected]]
Sent: Friday, 10 November 2023 10.45

Quoted 11 lines

From: Morten Brørup <[email protected]>
Sent: Friday, November 10, 2023 9:34 AM

+CC Gavin, reviewed the test case

Quoted 6 lines

From: Ruifeng Wang [mailto:[email protected]]
Sent: Friday, 10 November 2023 09.40

Quoted 1 line · Morten Brørup

I have for a long time now wondered why the ring functions for

enqueue/dequeue of 64-bit objects supports unaligned addresses,

and

Quoted 7 lines

now

Quoted 6 lines
Quoted 6 lines

I finally found the patch introducing it.

Quoted 5 lines
Quoted 5 lines

From: dev [mailto:[email protected]] On Behalf Of Phil Yang
Sent: Monday, 9 March 2020 18.20

The 32-bit arm machine doesn't support unaligned memory

access.

Quoted 12 lines

It

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

will cause a bus error on aarch32 with the custom element size

ring.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Thread 1 "test" received signal SIGBUS, Bus error. __rte_ring_enqueue_elems_64 (n=1, obj_table=0xf5edfe41,

prod_head=0,

Quoted 5 lines
Quoted 5 lines

\

Quoted 4 lines
Quoted 2 lines

r=0xf5edfb80) at /build/dpdk/build/include/rte_ring_elem.h:177
177 ring[idx++] = obj[i++];

Which test is this? Why is it using an unaligned array of 64-

bit

Quoted 5 lines
Quoted 4 lines
Quoted 4 lines

objects? (Notice that obj_table=0xf5edfe41.)

The test case is:

https://elixir.bootlin.com/dpdk/latest/source/app/test/test_ring.c#L112

Quoted 6 lines
Quoted 6 lines
Quoted 2 lines

1
This case deliberately use unaligned objects.

Thank you, Ruifeng.

Honnappa, I see you signed off on the patch introducing the test

for

Quoted 3 lines

unaligned objects:

Quoted 1 line

http://git.dpdk.org/dpdk/commit/app/test/test_ring.c?id=a9fe152363e283d

Quoted 9 lines

4c590ab8e8d51396f2ffab9ff

Quoted 2 lines

What was the rationale behind testing support for unaligned object

pointers? Did any applications/customers use unaligned object

Quoted 5 lines

pointers, or is it a purely academic test case?

Quoted 3 lines
Quoted 3 lines

Nobody in their right mind would use an unaligned array of 64-

bit

Quoted 5 lines
Quoted 1 line
Quoted 1 line

objects. You can only create such an array if you force the

compiler to

Quoted 1 line
Quoted 1 line

prevent automatic alignment! And all the functions in your

application

Quoted 1 line
Quoted 1 line

using this array would also need to support unaligned addressing

of

Quoted 5 lines
Quoted 1 line
Quoted 1 line

these objects.

It could be just one elem, not an array.
And the user can use 'packed' struct or so... Agree, not a common case, but probably still possible.

Very good point, Konstantin. A single unaligned object in a packed structure is not as exotic as an unaligned array of objects (which I consider completely unrealistic).

If anyone is using an architecture which doesn't support unaligned accesses, I would expect them to completely avoid using unaligned objects. But perhaps you are right; however unlikely, it might happen.

If we think this is a real use case, should we add support for unaligned 32 bit objects?
(128 bit objects already support unaligned access; they are type casted to void pointer and accessed using memcpy().)

And how about the stack library, should it support unaligned objects too?

Quoted 4 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

This seems extremely exotic, and not something any real

application

Quoted 8 lines
Quoted 3 lines
Quoted 3 lines

would do!

Quoted 2 lines

I would like to revert this patch for performance reasons.

Morten, could you probably explain first the problems you encountered with this patch?
You mention about 'performance reasons', so did you notice any real slowdown?

Please check my reply to the same question here:
http://inbox.dpdk.org/dev/98CBD80474FA8B44BF855DF32C47DC35E9EFD3@smarts
erver.smartshare.dk/

Quoted 8 lines
Quoted 3 lines

I could add an RTE_ASSERT() to verify that the pointer is aligned,

for debugging purposes.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Fixes cc4b218790f6 ("ring: support configurable element

size")

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Reviewed-by Honnappa Nagarahalli

<[email protected]>

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines

lib/librte_ring/rte_ring_elem.h __rte_ring_enqueue_elems_64(struct

Quoted 10 lines
---
   lib/librte_ring/rte_ring_elem.h | 4 ++--
   1 file changed, 2 insertions(+), 2 deletions(-)
lib/librte_ring/rte_ring_elem.h +0 -0
index 3976757..663addc 100644
@@ -160,7 +160,7 @@ __rte_ring_enqueue_elems_64(struct

rte_ring

Quoted 12 lines

*r,

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

lib/librte_ring/rte_ring_elem.h __rte_ring_dequeue_elems_64(struct

Quoted 11 lines
lib/librte_ring/rte_ring_elem.h +1 -1
 uint32_t prod_head,
   	const uint32_t size = r->size;
   	uint32_t idx = prod_head & r->mask;
   	uint64_t *ring = (uint64_t *)&r[1];
-	const uint64_t *obj = (const uint64_t *)obj_table;
+	const unaligned_uint64_t *obj = (const unaligned_uint64_t
 *)obj_table;
   	if (likely(idx + n < size)) {
   		for (i = 0; i < (n & ~0x3); i += 4, idx += 4) {
   			ring[idx] = obj[i];
lib/librte_ring/rte_ring_elem.h +0 -0
@@ -294,7 +294,7 @@ __rte_ring_dequeue_elems_64(struct

rte_ring

Quoted 18 lines

*r,

Quoted 16 lines
Quoted 16 lines
Quoted 15 lines
Quoted 9 lines
lib/librte_ring/rte_ring_elem.h +1 -1
 uint32_t prod_head,
   	const uint32_t size = r->size;
   	uint32_t idx = prod_head & r->mask;
   	uint64_t *ring = (uint64_t *)&r[1];
-	uint64_t *obj = (uint64_t *)obj_table;
+	unaligned_uint64_t *obj = (unaligned_uint64_t *)obj_table;
   	if (likely(idx + n < size)) {
   		for (i = 0; i < (n & ~0x3); i += 4, idx += 4) {
   			obj[i] = ring[idx];
Quoted 3 lines

--
2.7.4

References:

https://git.dpdk.org/dpdk/commit/lib/librte_ring/rte_ring_elem.h?id=3ba

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

51478a3ab3132c33effc8b132641233275b36

Quoted 1 line

https://patchwork.dpdk.org/project/dpdk/patch/1583774395-10233-

1-

Quoted 3 lines

git-

Quoted 2 lines
Quoted 2 lines

[email protected]/

Quoted 1 line
#11 lore raw
In reply to Morten Brørup
Quoted 71 lines
Quoted 59 lines

From: Konstantin Ananyev [mailto:[email protected]]
Sent: Friday, 10 November 2023 10.45

Quoted 11 lines

From: Morten Brørup <[email protected]>
Sent: Friday, November 10, 2023 9:34 AM

+CC Gavin, reviewed the test case

Quoted 6 lines

From: Ruifeng Wang [mailto:[email protected]]
Sent: Friday, 10 November 2023 09.40

Quoted 1 line · Morten Brørup

I have for a long time now wondered why the ring functions for

enqueue/dequeue of 64-bit objects supports unaligned addresses, and

now

Quoted 6 lines
Quoted 6 lines

I finally found the patch introducing it.

Quoted 5 lines
Quoted 5 lines

From: dev [mailto:[email protected]] On Behalf Of Phil Yang
Sent: Monday, 9 March 2020 18.20

The 32-bit arm machine doesn't support unaligned memory access.

It

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

will cause a bus error on aarch32 with the custom element size

ring.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Thread 1 "test" received signal SIGBUS, Bus error. __rte_ring_enqueue_elems_64 (n=1, obj_table=0xf5edfe41,

prod_head=0,

Quoted 9 lines
Quoted 9 lines

\

Quoted 4 lines
Quoted 2 lines

r=0xf5edfb80) at /build/dpdk/build/include/rte_ring_elem.h:177
177 ring[idx++] = obj[i++];

Which test is this? Why is it using an unaligned array of 64-bit

objects? (Notice that obj_table=0xf5edfe41.)

The test case is:

https://elixir.bootlin.com/dpdk/latest/source/app/test/test_ring.c#L112

Quoted 6 lines
Quoted 2 lines

1
This case deliberately use unaligned objects.

Thank you, Ruifeng.

Honnappa, I see you signed off on the patch introducing the test for

unaligned objects:

Quoted 1 line

http://git.dpdk.org/dpdk/commit/app/test/test_ring.c?id=a9fe152363e283d
4c590ab8e8d51396f2ffab9ff

Quoted 2 lines

What was the rationale behind testing support for unaligned object

pointers? Did any applications/customers use unaligned object

Quoted 6 lines

pointers, or is it a purely academic test case?

Quoted 4 lines
Quoted 3 lines

Nobody in their right mind would use an unaligned array of 64-bit

objects. You can only create such an array if you force the

compiler to

Quoted 8 lines
Quoted 1 line
Quoted 1 line

prevent automatic alignment! And all the functions in your

application

Quoted 2 lines
Quoted 2 lines

using this array would also need to support unaligned addressing of these objects.

It could be just one elem, not an array.
And the user can use 'packed' struct or so... Agree, not a common case, but probably still possible.

Very good point, Konstantin. A single unaligned object in a packed structure is not as exotic as an unaligned array of objects (which I consider completely unrealistic).

Quoted 7 lines

If anyone is using an architecture which doesn't support unaligned accesses, I would expect them to completely avoid using unaligned objects. But perhaps you are right; however unlikely, it might happen.

If we think this is a real use case, should we add support for unaligned 32 bit objects?
(128 bit objects already support unaligned access; they are type casted to void pointer and accessed using memcpy().)

And how about the stack library, should it support unaligned objects too?

Hmm... good point, from my understanding - yes.
Probably we just been lucky so far here - no-one hit un-aligned case yet.

Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

This seems extremely exotic, and not something any real

application

Quoted 3 lines
Quoted 3 lines

would do!

Quoted 2 lines

I would like to revert this patch for performance reasons.

Quoted 7 lines
Quoted 4 lines

Morten, could you probably explain first the problems you encountered with this patch?
You mention about 'performance reasons', so did you notice any real slowdown?

Please check my reply to the same question here:
http://inbox.dpdk.org/dev/[email protected]/

As I remember, right now unaligned_uint(64,32)_t are really unaligned only for one specific architecture - aarch32. All others should be unaffected.

Quoted 58 lines
Quoted 58 lines
Quoted 4 lines

I could add an RTE_ASSERT() to verify that the pointer is aligned,

for debugging purposes.

Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Fixes cc4b218790f6 ("ring: support configurable element size")
---
  lib/librte_ring/rte_ring_elem.h | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

lib/librte_ring/rte_ring_elem.h __rte_ring_enqueue_elems_64(struct rte_ring

Quoted 6 lines
lib/librte_ring/rte_ring_elem.h +0 -0
index 3976757..663addc 100644
@@ -160,7 +160,7 @@ __rte_ring_enqueue_elems_64(struct rte_ring

*r,

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

lib/librte_ring/rte_ring_elem.h __rte_ring_dequeue_elems_64(struct rte_ring

Quoted 11 lines
lib/librte_ring/rte_ring_elem.h +1 -1
 uint32_t prod_head,
  	const uint32_t size = r->size;
  	uint32_t idx = prod_head & r->mask;
  	uint64_t *ring = (uint64_t *)&r[1];
-	const uint64_t *obj = (const uint64_t *)obj_table;
+	const unaligned_uint64_t *obj = (const unaligned_uint64_t
 *)obj_table;
  	if (likely(idx + n < size)) {
  		for (i = 0; i < (n & ~0x3); i += 4, idx += 4) {
  			ring[idx] = obj[i];
lib/librte_ring/rte_ring_elem.h +0 -0
@@ -294,7 +294,7 @@ __rte_ring_dequeue_elems_64(struct rte_ring

*r,

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
lib/librte_ring/rte_ring_elem.h +1 -1
 uint32_t prod_head,
  	const uint32_t size = r->size;
  	uint32_t idx = prod_head & r->mask;
  	uint64_t *ring = (uint64_t *)&r[1];
-	uint64_t *obj = (uint64_t *)obj_table;
+	unaligned_uint64_t *obj = (unaligned_uint64_t *)obj_table;
  	if (likely(idx + n < size)) {
  		for (i = 0; i < (n & ~0x3); i += 4, idx += 4) {
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
Quoted 4 lines

obj[i] = ring[idx];
--
2.7.4

References:

https://git.dpdk.org/dpdk/commit/lib/librte_ring/rte_ring_elem.h?id=3ba

Quoted 2 lines
Quoted 2 lines

51478a3ab3132c33effc8b132641233275b36

Quoted 1 line

https://patchwork.dpdk.org/project/dpdk/patch/1583774395-10233-1-

git-

Quoted 2 lines
Quoted 2 lines

[email protected]/

Quoted 1 line
#12 lore raw
In reply to Morten Brørup
Quoted 81 lines

-----Original Message-----
From: Morten Brørup <[email protected]>
Sent: Friday, November 10, 2023 4:44 AM
To: Konstantin Ananyev <[email protected]>; Ruifeng Wang <[email protected]>; Honnappa Nagarahalli <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; Dharmik Jayesh Thakkar <[email protected]>; nd <[email protected]>; [email protected]; Gavin Hu <[email protected]>
Subject: RE: [dpdk-dev] [PATCH] ring: fix unaligned memory access on aarch32

Quoted 67 lines

From: Konstantin Ananyev [mailto:[email protected]]
Sent: Friday, 10 November 2023 10.45

Quoted 12 lines

From: Morten Brørup <[email protected]>
Sent: Friday, November 10, 2023 9:34 AM

+CC Gavin, reviewed the test case

Quoted 7 lines

From: Ruifeng Wang [mailto:[email protected]]
Sent: Friday, 10 November 2023 09.40

Quoted 1 line · Morten Brørup

I have for a long time now wondered why the ring functions for

enqueue/dequeue of 64-bit objects supports unaligned addresses, and

now

Quoted 6 lines
Quoted 6 lines

I finally found the patch introducing it.

Quoted 5 lines
Quoted 5 lines

From: dev [mailto:[email protected]] On Behalf Of Phil Yang
Sent: Monday, 9 March 2020 18.20

The 32-bit arm machine doesn't support unaligned memory access.

It

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

will cause a bus error on aarch32 with the custom element size

ring.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Thread 1 "test" received signal SIGBUS, Bus error. __rte_ring_enqueue_elems_64 (n=1, obj_table=0xf5edfe41,

prod_head=0,

Quoted 9 lines
Quoted 9 lines

\

Quoted 4 lines
Quoted 2 lines

r=0xf5edfb80) at /build/dpdk/build/include/rte_ring_elem.h:177
177 ring[idx++] = obj[i++];

Which test is this? Why is it using an unaligned array of 64-bit

objects? (Notice that obj_table=0xf5edfe41.)

The test case is:

https://elixir.bootlin.com/dpdk/latest/source/app/test/test_ring.c#L11
2

Quoted 6 lines
Quoted 2 lines

1
This case deliberately use unaligned objects.

Thank you, Ruifeng.

Honnappa, I see you signed off on the patch introducing the test for

unaligned objects:

Quoted 1 line

http://git.dpdk.org/dpdk/commit/app/test/test_ring.c?id=a9fe152363e283
d
4c590ab8e8d51396f2ffab9ff

Quoted 2 lines

What was the rationale behind testing support for unaligned object

pointers? Did any applications/customers use unaligned object

Quoted 7 lines

pointers, or is it a purely academic test case?

Quoted 5 lines
Quoted 4 lines

Nobody in their right mind would use an unaligned array of 64-bit

objects. You can only create such an array if you force the

compiler to

Quoted 1 line
Quoted 1 line

prevent automatic alignment! And all the functions in your

application

Quoted 2 lines
Quoted 2 lines

using this array would also need to support unaligned addressing of these objects.

Quoted 7 lines
Quoted 3 lines

It could be just one elem, not an array.
And the user can use 'packed' struct or so... Agree, not a common case, but probably still possible.

Very good point, Konstantin. A single unaligned object in a packed structure is not as exotic as an unaligned array of objects (which I consider completely unrealistic).

The requirement is coming from RCU library, it has a requirement to dequeue 1 element.

Quoted 20 lines

If anyone is using an architecture which doesn't support unaligned accesses, I would expect them to completely avoid using unaligned objects. But perhaps you are right; however unlikely, it might happen.

If we think this is a real use case, should we add support for unaligned 32 bit objects?
(128 bit objects already support unaligned access; they are type casted to void pointer and accessed using memcpy().)

And how about the stack library, should it support unaligned objects too?

Quoted 8 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

This seems extremely exotic, and not something any real

application

Quoted 3 lines
Quoted 3 lines

would do!

Quoted 2 lines

I would like to revert this patch for performance reasons.

Quoted 8 lines
Quoted 4 lines

Morten, could you probably explain first the problems you encountered with this patch?
You mention about 'performance reasons', so did you notice any real slowdown?

Please check my reply to the same question here:
http://inbox.dpdk.org/dev/98CBD80474FA8B44BF855DF32C47DC35E9EFD3@
smartserver.smartshare.dk/

Typically, the performance tests are run and made sure to capture any perf differences. In this case, I am sure, there was not any or much difference. Otherwise, the change would have been highly contested.

Quoted 60 lines
Quoted 60 lines
Quoted 4 lines

I could add an RTE_ASSERT() to verify that the pointer is aligned,

for debugging purposes.

Quoted 13 lines
Quoted 13 lines
Quoted 13 lines
Quoted 13 lines
Fixes cc4b218790f6 ("ring: support configurable element size")
Reviewed-by Honnappa Nagarahalli

<[email protected]>

---
  lib/librte_ring/rte_ring_elem.h | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

lib/librte_ring/rte_ring_elem.h __rte_ring_enqueue_elems_64(struct rte_ring

Quoted 5 lines
(quoted diff) +0 -0

b/lib/librte_ring/rte_ring_elem.h index 3976757..663addc 100644

lib/librte_ring/rte_ring_elem.h +0 -0
@@ -160,7 +160,7 @@ __rte_ring_enqueue_elems_64(struct rte_ring

*r,

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

lib/librte_ring/rte_ring_elem.h __rte_ring_dequeue_elems_64(struct rte_ring

Quoted 11 lines
lib/librte_ring/rte_ring_elem.h +1 -1
 uint32_t prod_head,
  	const uint32_t size = r->size;
  	uint32_t idx = prod_head & r->mask;
  	uint64_t *ring = (uint64_t *)&r[1];
-	const uint64_t *obj = (const uint64_t *)obj_table;
+	const unaligned_uint64_t *obj = (const unaligned_uint64_t
 *)obj_table;
  	if (likely(idx + n < size)) {
  		for (i = 0; i < (n & ~0x3); i += 4, idx += 4) {
  			ring[idx] = obj[i];
lib/librte_ring/rte_ring_elem.h +0 -0
@@ -294,7 +294,7 @@ __rte_ring_dequeue_elems_64(struct rte_ring

*r,

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
lib/librte_ring/rte_ring_elem.h +1 -1
 uint32_t prod_head,
  	const uint32_t size = r->size;
  	uint32_t idx = prod_head & r->mask;
  	uint64_t *ring = (uint64_t *)&r[1];
-	uint64_t *obj = (uint64_t *)obj_table;
+	unaligned_uint64_t *obj = (unaligned_uint64_t *)obj_table;
  	if (likely(idx + n < size)) {
  		for (i = 0; i < (n & ~0x3); i += 4, idx += 4) {
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
Quoted 4 lines

obj[i] = ring[idx];
--
2.7.4

References:

https://git.dpdk.org/dpdk/commit/lib/librte_ring/rte_ring_elem.h?id=3b
a

Quoted 3 lines
Quoted 3 lines

51478a3ab3132c33effc8b132641233275b36

Quoted 2 lines

https://patchwork.dpdk.org/project/dpdk/patch/1583774395-10233-1
-

git-

Quoted 2 lines
Quoted 2 lines

[email protected]/

Quoted 1 line