__rte_raw_cksum() crash with misaligned pointer

+CC Olivier Matz <[email protected]>, Network Headers maintainer

Quoted 95 lines

From: Morten Brørup <[email protected]>
Sent: den 15 juni 2022 16:41
Quoted 4 lines
From: [email protected] [mailto:[email protected]]
Sent: Wednesday, 15 June 2022 09.16
https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-
45444
Quoted 1 line
5555731-2e92ae6bf759c0c5&q=1&e=b3fc70af-5d37-4ffb-b34d-
9a51927f5f6d&u=
Quoted 14 lines
https%3A%2F%2Fbugs.dpdk.org%2Fshow_bug.cgi%3Fid%3D1035
            Bug ID: 1035
           Summary: __rte_raw_cksum() crash with misaligned pointer
           Product: DPDK
           Version: 21.11
          Hardware: All
                OS: All
            Status: UNCONFIRMED
          Severity: normal
          Priority: Normal
         Component: ethdev
          Assignee: [email protected]
          Reporter: [email protected]
Quoted 7 lines
Target Milestone: ---
See rte_raw_cksum() in rte_ip.h, which is part of the public API. See also the subfunction __rte_raw_cksum().
_rte_raw_cksum assumes that the buffer over which the checksum is calculated is an even address (divisible by two). See for example
this
Quoted 9 lines
stack overflow
post:
https://stackoverflow.com/questions/46790550/c-undefined-behavior-
strict-aliasing-rule-or-incorrect-alignment
The post explains that there is undefined behavior in C11 when "conversion between two pointer types produces a result that is incorrectly aligned". When the buf argument starts on an odd address we thus have undefined behavior, since a pointer is cast from void*
to
Quoted 8 lines
uint16_t*.
In most cases (at least on x86) that isn't a problem, but with higher optimization levels it may break due to vector instructions. This new function seems to be easier to optimize by the compiler, resulting in a crash when the buf argument is odd. Please note that the undefined behavior is present in earlier versions of dpdk as well.
Quoted 6 lines
Now you're probably thinking: "Just align your buffers". The problem is that we have a packet buffer which is aligned. The checksum is calculated on a subset of that aligned packet buffer, and that sometimes lies on odd addresses.
The question remains if this is an issue with dpdk or not.
I can imagine other systems doing what you describe too. So it needs to be addressed.
Off the top of my head, an easy fix would be updating __rte_raw_cksum() like this:
static inline uint32_t
__rte_raw_cksum(const void *buf, size_t len, uint32_t sum) {
	if (likely((buf & 1) == 0)) {
		/* The buffer is 16 bit aligned. */
		Keep the existing, optimized implementation here.
	} else {
		/* The buffer is not 16 bit aligned. */
		Add a new odd-buf tolerant implementation here.
	}
}
However, I'm not sure that it covers your scenario!
The checksum is 16 bit wide, so if you calculate the checksum of e.g. 4 bytes of memory starting at offset 1 in a 6 byte packet buffer, the memory block can be treated as either 4 or 6 bytes relative to the data covered by the checksum, i.e.:
A: XX [01 02] [03 04] XX --> cksum = [04 06]
B: [XX 01] [02 03] [04 XX] --> cksum = [06 04]
Which one do you need?
Perhaps an additional function is required to support your use case, and the documentation for rte_raw_cksum() and __rte_raw_cksum() needs to reflect that the buffer must be 16 bit aligned.
Or the rte_raw_cksum() function can be modified to support an odd buffer pointer as outlined above, with documentation added about alignment of the running checksum.

Quoted 8 lines

-----Original Message-----
From: Emil Berg [mailto:[email protected]]
Sent: Thursday, 16 June 2022 07.45
Hi!
We want the B option, i.e. the 6 bytes option. Perhaps adding alignment detection to __rte_raw_cksum() is a good idea.

With option B, the invariant that the running checksum is being calculated on a 16 bit aligned packet buffer remains unchanged. So I think that support for option B is appropriate to add to __rte_raw_cksum(), rather than adding a separate function.

Quoted 7 lines

A minor comment but I think buf & 1 won't work since buf isn't an integral type, but something along that way.
I'm starting to think about an efficient way to do this.
Thank you!

Sounds good. Please CC me on your patch, when ready for review. :-)

Morten Brørup

2022-06-16 06:44 UTC #5 lore raw

+CC Olivier Matz <[email protected]>, Network Headers maintainer

Quoted 132 lines

-----Original Message-----
From: Morten Brørup <[email protected]>
Sent: den 15 juni 2022 16:41
To: Emil Berg <[email protected]>; [email protected]
Cc: [email protected]
Subject: RE: [Bug 1035] __rte_raw_cksum() crash with misaligned pointer
Quoted 4 lines
From: [email protected] [mailto:[email protected]]
Sent: Wednesday, 15 June 2022 09.16
https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-
45444
Quoted 1 line
5555731-2e92ae6bf759c0c5&q=1&e=b3fc70af-5d37-4ffb-b34d-
9a51927f5f6d&u=
Quoted 14 lines
https%3A%2F%2Fbugs.dpdk.org%2Fshow_bug.cgi%3Fid%3D1035
            Bug ID: 1035
           Summary: __rte_raw_cksum() crash with misaligned pointer
           Product: DPDK
           Version: 21.11
          Hardware: All
                OS: All
            Status: UNCONFIRMED
          Severity: normal
          Priority: Normal
         Component: ethdev
          Assignee: [email protected]
          Reporter: [email protected]
Quoted 7 lines
Target Milestone: ---
See rte_raw_cksum() in rte_ip.h, which is part of the public API. See also the subfunction __rte_raw_cksum().
_rte_raw_cksum assumes that the buffer over which the checksum is calculated is an even address (divisible by two). See for example
this
Quoted 9 lines
stack overflow
post:
https://stackoverflow.com/questions/46790550/c-undefined-behavior-
strict-aliasing-rule-or-incorrect-alignment
The post explains that there is undefined behavior in C11 when "conversion between two pointer types produces a result that is incorrectly aligned". When the buf argument starts on an odd address we thus have undefined behavior, since a pointer is cast from void*
to
Quoted 8 lines
uint16_t*.
In most cases (at least on x86) that isn't a problem, but with higher optimization levels it may break due to vector instructions. This new function seems to be easier to optimize by the compiler, resulting in a crash when the buf argument is odd. Please note that the undefined behavior is present in earlier versions of dpdk as well.
Quoted 6 lines
Now you're probably thinking: "Just align your buffers". The problem is that we have a packet buffer which is aligned. The checksum is calculated on a subset of that aligned packet buffer, and that sometimes lies on odd addresses.
The question remains if this is an issue with dpdk or not.
I can imagine other systems doing what you describe too. So it needs to be addressed.
Off the top of my head, an easy fix would be updating __rte_raw_cksum() like this:
static inline uint32_t
__rte_raw_cksum(const void *buf, size_t len, uint32_t sum) {
	if (likely((buf & 1) == 0)) {
		/* The buffer is 16 bit aligned. */
		Keep the existing, optimized implementation here.
	} else {
		/* The buffer is not 16 bit aligned. */
		Add a new odd-buf tolerant implementation here.
	}
}
However, I'm not sure that it covers your scenario!
The checksum is 16 bit wide, so if you calculate the checksum of e.g. 4 bytes of memory starting at offset 1 in a 6 byte packet buffer, the memory block can be treated as either 4 or 6 bytes relative to the data covered by the checksum, i.e.:
A: XX [01 02] [03 04] XX --> cksum = [04 06]
B: [XX 01] [02 03] [04 XX] --> cksum = [06 04]
Which one do you need?
Perhaps an additional function is required to support your use case, and the documentation for rte_raw_cksum() and __rte_raw_cksum() needs to reflect that the buffer must be 16 bit aligned.
Or the rte_raw_cksum() function can be modified to support an odd buffer pointer as outlined above, with documentation added about alignment of the running checksum.
-----Original Message-----
From: Emil Berg
Sent: den 16 juni 2022 07:45
To: Morten Brørup <[email protected]>; [email protected]
Cc: [email protected]
Subject: RE: [Bug 1035] __rte_raw_cksum() crash with misaligned pointer
Hi!
We want the B option, i.e. the 6 bytes option. Perhaps adding alignment detection to __rte_raw_cksum() is a good idea.
A minor comment but I think buf & 1 won't work since buf isn't an integral type, but something along that way.
I'm starting to think about an efficient way to do this.
Thank you!
-----Original Message-----
From: Emil Berg [mailto:[email protected]]
Sent: Thursday, 16 June 2022 08.32
To: Morten Brørup; [email protected]
Cc: [email protected]
Subject: RE: [Bug 1035] __rte_raw_cksum() crash with misaligned pointer
I've been sketching on an efficient solution to this. What about something along the way below? I've run it with the combinations of:
even buf, even len
even buf, odd len
odd buf, even len
odd buf, odd len

Quoted 4 lines

and it seems to give the same results as the older version of __rte_raw_cksum, before 21.03. I ran it without optimizations and such to ensure the compiler didn't insert vector instructions and such so the results were comparable.

The performance, when using an aligned buffer, needs to be comparable with full compiler optimization, or the patch will not be accepted.

Quoted 12 lines

static inline uint32_t
__rte_raw_cksum_newest(const void *buf, size_t len, uint32_t sum)
{
	const uint8_t *end = buf + len;

	uint32_t sum_even = 0;
	for (const uint8_t *p = buf + 1; p < end; p += 2) {
		sum_even += *p;
	}
	sum += sum_even << 8;

Quoted 8 lines

	uint32_t sum_odd = 0;
	for (const uint8_t *p = buf; p < end; p += 2) {
		sum_odd += *p;
	}
	sum += sum_odd;

	return sum;
}

This function does not work on both little and big endian, when mixed with other checksum functions.

The checksum functions read the buffer in CPU native endian and stores the checksum in CPU native endian; and magically it becomes correct in network endian. Please refer to RFC 1071 for the details. Perhaps you can also find some inspiration in RFC 1141.

PS: Please don't top post.

Emil Berg

2022-06-16 10:48 UTC #2 lore raw

Hi!

We want the B option, i.e. the 6 bytes option. Perhaps adding alignment detection to __rte_raw_cksum() is a good idea.

A minor comment but I think buf & 1 won't work since buf isn't an integral type, but something along that way.

I'm starting to think about an efficient way to do this.

Thank you!

-----Original Message-----
From: Morten Brørup <[email protected]>
Sent: den 15 juni 2022 16:41
To: Emil Berg <[email protected]>; [email protected]
Cc: [email protected]
Subject: RE: [Bug 1035] __rte_raw_cksum() crash with misaligned pointer

Quoted 43 lines

From: [email protected] [mailto:[email protected]]
Sent: Wednesday, 15 June 2022 09.16
https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-45444
5555731-2e92ae6bf759c0c5&q=1&e=b3fc70af-5d37-4ffb-b34d-9a51927f5f6d&u= https%3A%2F%2Fbugs.dpdk.org%2Fshow_bug.cgi%3Fid%3D1035
            Bug ID: 1035
           Summary: __rte_raw_cksum() crash with misaligned pointer
           Product: DPDK
           Version: 21.11
          Hardware: All
                OS: All
            Status: UNCONFIRMED
          Severity: normal
          Priority: Normal
         Component: ethdev
          Assignee: [email protected]
          Reporter: [email protected]
Target Milestone: ---
See rte_raw_cksum() in rte_ip.h, which is part of the public API. See also the subfunction __rte_raw_cksum().
_rte_raw_cksum assumes that the buffer over which the checksum is calculated is an even address (divisible by two). See for example this stack overflow
post:
https://stackoverflow.com/questions/46790550/c-undefined-behavior-
strict-aliasing-rule-or-incorrect-alignment
The post explains that there is undefined behavior in C11 when "conversion between two pointer types produces a result that is incorrectly aligned". When the buf argument starts on an odd address we thus have undefined behavior, since a pointer is cast from void* to uint16_t*.
In most cases (at least on x86) that isn't a problem, but with higher optimization levels it may break due to vector instructions. This new function seems to be easier to optimize by the compiler, resulting in a crash when the buf argument is odd. Please note that the undefined behavior is present in earlier versions of dpdk as well.

Quoted 6 lines

Now you're probably thinking: "Just align your buffers". The problem is that we have a packet buffer which is aligned. The checksum is calculated on a subset of that aligned packet buffer, and that sometimes lies on odd addresses.
The question remains if this is an issue with dpdk or not.

I can imagine other systems doing what you describe too. So it needs to be addressed.

Off the top of my head, an easy fix would be updating __rte_raw_cksum() like this:

static inline uint32_t

__rte_raw_cksum(const void *buf, size_t len, uint32_t sum) {
	if (likely((buf & 1) == 0)) {
		/* The buffer is 16 bit aligned. */
		Keep the existing, optimized implementation here.
	} else {
		/* The buffer is not 16 bit aligned. */
		Add a new odd-buf tolerant implementation here.
	}
}

However, I'm not sure that it covers your scenario!

A: XX [01 02] [03 04] XX --> cksum = [04 06]

B: [XX 01] [02 03] [04 XX] --> cksum = [06 04]

Which one do you need?

Perhaps an additional function is required to support your use case, and the documentation for rte_raw_cksum() and __rte_raw_cksum() needs to reflect that the buffer must be 16 bit aligned.

Or the rte_raw_cksum() function can be modified to support an odd buffer pointer as outlined above, with documentation added about alignment of the running checksum.

Emil Berg

2022-06-16 10:48 UTC #4 lore raw

I've been sketching on an efficient solution to this. What about something along the way below? I've run it with the combinations of:
even buf, even len
even buf, odd len
odd buf, even len
odd buf, odd len

and it seems to give the same results as the older version of __rte_raw_cksum, before 21.03. I ran it without optimizations and such to ensure the compiler didn't insert vector instructions and such so the results were comparable.

static inline uint32_t
__rte_raw_cksum_newest(const void *buf, size_t len, uint32_t sum)

{
	const uint8_t *end = buf + len;

	uint32_t sum_even = 0;
	for (const uint8_t *p = buf + 1; p < end; p += 2) {
		sum_even += *p;
	}
	sum += sum_even << 8;

	uint32_t sum_odd = 0;
	for (const uint8_t *p = buf; p < end; p += 2) {
		sum_odd += *p;
	}
	sum += sum_odd;

	return sum;
}

/Emil

-----Original Message-----
From: Emil Berg
Sent: den 16 juni 2022 07:45
To: Morten Brørup <[email protected]>; [email protected]
Cc: [email protected]
Subject: RE: [Bug 1035] __rte_raw_cksum() crash with misaligned pointer

Hi!

We want the B option, i.e. the 6 bytes option. Perhaps adding alignment detection to __rte_raw_cksum() is a good idea.

A minor comment but I think buf & 1 won't work since buf isn't an integral type, but something along that way.

I'm starting to think about an efficient way to do this.

Thank you!

Quoted 43 lines

From: [email protected] [mailto:[email protected]]
Sent: Wednesday, 15 June 2022 09.16
https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-45444
5555731-2e92ae6bf759c0c5&q=1&e=b3fc70af-5d37-4ffb-b34d-9a51927f5f6d&u= https%3A%2F%2Fbugs.dpdk.org%2Fshow_bug.cgi%3Fid%3D1035
            Bug ID: 1035
           Summary: __rte_raw_cksum() crash with misaligned pointer
           Product: DPDK
           Version: 21.11
          Hardware: All
                OS: All
            Status: UNCONFIRMED
          Severity: normal
          Priority: Normal
         Component: ethdev
          Assignee: [email protected]
          Reporter: [email protected]
Target Milestone: ---
See rte_raw_cksum() in rte_ip.h, which is part of the public API. See also the subfunction __rte_raw_cksum().
_rte_raw_cksum assumes that the buffer over which the checksum is calculated is an even address (divisible by two). See for example this stack overflow
post:
https://stackoverflow.com/questions/46790550/c-undefined-behavior-
strict-aliasing-rule-or-incorrect-alignment
The post explains that there is undefined behavior in C11 when "conversion between two pointer types produces a result that is incorrectly aligned". When the buf argument starts on an odd address we thus have undefined behavior, since a pointer is cast from void* to uint16_t*.
In most cases (at least on x86) that isn't a problem, but with higher optimization levels it may break due to vector instructions. This new function seems to be easier to optimize by the compiler, resulting in a crash when the buf argument is odd. Please note that the undefined behavior is present in earlier versions of dpdk as well.

Quoted 6 lines

Now you're probably thinking: "Just align your buffers". The problem is that we have a packet buffer which is aligned. The checksum is calculated on a subset of that aligned packet buffer, and that sometimes lies on odd addresses.
The question remains if this is an issue with dpdk or not.

I can imagine other systems doing what you describe too. So it needs to be addressed.

Off the top of my head, an easy fix would be updating __rte_raw_cksum() like this:

static inline uint32_t

__rte_raw_cksum(const void *buf, size_t len, uint32_t sum) {
	if (likely((buf & 1) == 0)) {
		/* The buffer is 16 bit aligned. */
		Keep the existing, optimized implementation here.
	} else {
		/* The buffer is not 16 bit aligned. */
		Add a new odd-buf tolerant implementation here.
	}
}

However, I'm not sure that it covers your scenario!

A: XX [01 02] [03 04] XX --> cksum = [04 06]

B: [XX 01] [02 03] [04 XX] --> cksum = [06 04]

Which one do you need?

Perhaps an additional function is required to support your use case, and the documentation for rte_raw_cksum() and __rte_raw_cksum() needs to reflect that the buffer must be 16 bit aligned.

Or the rte_raw_cksum() function can be modified to support an odd buffer pointer as outlined above, with documentation added about alignment of the running checksum.

Mattias Rönnblom

2022-06-16 13:58 UTC #6 lore raw

Quoted 134 lines · Morten Brørup

+CC Olivier Matz <[email protected]>, Network Headers maintainer
Quoted 132 lines
-----Original Message-----
From: Morten Brørup <[email protected]>
Sent: den 15 juni 2022 16:41
To: Emil Berg <[email protected]>; [email protected]
Cc: [email protected]
Subject: RE: [Bug 1035] __rte_raw_cksum() crash with misaligned pointer
Quoted 4 lines
From: [email protected] [mailto:[email protected]]
Sent: Wednesday, 15 June 2022 09.16
https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-
45444
Quoted 1 line
5555731-2e92ae6bf759c0c5&q=1&e=b3fc70af-5d37-4ffb-b34d-
9a51927f5f6d&u=
Quoted 14 lines
https%3A%2F%2Fbugs.dpdk.org%2Fshow_bug.cgi%3Fid%3D1035
             Bug ID: 1035
            Summary: __rte_raw_cksum() crash with misaligned pointer
            Product: DPDK
            Version: 21.11
           Hardware: All
                 OS: All
             Status: UNCONFIRMED
           Severity: normal
           Priority: Normal
          Component: ethdev
           Assignee: [email protected]
           Reporter: [email protected]
Quoted 7 lines
Target Milestone: ---
See rte_raw_cksum() in rte_ip.h, which is part of the public API. See also the subfunction __rte_raw_cksum().
_rte_raw_cksum assumes that the buffer over which the checksum is calculated is an even address (divisible by two). See for example
this
Quoted 9 lines
stack overflow
post:
https://stackoverflow.com/questions/46790550/c-undefined-behavior-
strict-aliasing-rule-or-incorrect-alignment
The post explains that there is undefined behavior in C11 when "conversion between two pointer types produces a result that is incorrectly aligned". When the buf argument starts on an odd address we thus have undefined behavior, since a pointer is cast from void*
to
Quoted 8 lines
uint16_t*.
In most cases (at least on x86) that isn't a problem, but with higher optimization levels it may break due to vector instructions. This new function seems to be easier to optimize by the compiler, resulting in a crash when the buf argument is odd. Please note that the undefined behavior is present in earlier versions of dpdk as well.
Quoted 6 lines
Now you're probably thinking: "Just align your buffers". The problem is that we have a packet buffer which is aligned. The checksum is calculated on a subset of that aligned packet buffer, and that sometimes lies on odd addresses.
The question remains if this is an issue with dpdk or not.
I can imagine other systems doing what you describe too. So it needs to be addressed.
Off the top of my head, an easy fix would be updating __rte_raw_cksum() like this:
static inline uint32_t
__rte_raw_cksum(const void *buf, size_t len, uint32_t sum) {
	if (likely((buf & 1) == 0)) {
		/* The buffer is 16 bit aligned. */
		Keep the existing, optimized implementation here.
	} else {
		/* The buffer is not 16 bit aligned. */
		Add a new odd-buf tolerant implementation here.
	}
}
However, I'm not sure that it covers your scenario!
The checksum is 16 bit wide, so if you calculate the checksum of e.g. 4 bytes of memory starting at offset 1 in a 6 byte packet buffer, the memory block can be treated as either 4 or 6 bytes relative to the data covered by the checksum, i.e.:
A: XX [01 02] [03 04] XX --> cksum = [04 06]
B: [XX 01] [02 03] [04 XX] --> cksum = [06 04]
Which one do you need?
Perhaps an additional function is required to support your use case, and the documentation for rte_raw_cksum() and __rte_raw_cksum() needs to reflect that the buffer must be 16 bit aligned.
Or the rte_raw_cksum() function can be modified to support an odd buffer pointer as outlined above, with documentation added about alignment of the running checksum.
-----Original Message-----
From: Emil Berg
Sent: den 16 juni 2022 07:45
To: Morten Brørup <[email protected]>; [email protected]
Cc: [email protected]
Subject: RE: [Bug 1035] __rte_raw_cksum() crash with misaligned pointer
Hi!
We want the B option, i.e. the 6 bytes option. Perhaps adding alignment detection to __rte_raw_cksum() is a good idea.
A minor comment but I think buf & 1 won't work since buf isn't an integral type, but something along that way.
I'm starting to think about an efficient way to do this.
Thank you!
-----Original Message-----
From: Emil Berg [mailto:[email protected]]
Sent: Thursday, 16 June 2022 08.32
To: Morten Brørup; [email protected]
Cc: [email protected]
Subject: RE: [Bug 1035] __rte_raw_cksum() crash with misaligned pointer
I've been sketching on an efficient solution to this. What about something along the way below? I've run it with the combinations of:
even buf, even len
even buf, odd len
odd buf, even len
odd buf, odd len

Quoted 7 lines

Quoted 4 lines
and it seems to give the same results as the older version of __rte_raw_cksum, before 21.03. I ran it without optimizations and such to ensure the compiler didn't insert vector instructions and such so the results were comparable.
The performance, when using an aligned buffer, needs to be comparable with full compiler optimization, or the patch will not be accepted.

I think the question is: does rte_raw_cksum() have any alignment requirements, from an API contract point of view? The documentation says nothing about any such. In that case, it seems reasonable to me to assume that there are none.

Quoted 27 lines

Quoted 12 lines
static inline uint32_t
__rte_raw_cksum_newest(const void *buf, size_t len, uint32_t sum)
{
	const uint8_t *end = buf + len;

	uint32_t sum_even = 0;
	for (const uint8_t *p = buf + 1; p < end; p += 2) {
		sum_even += *p;
	}
	sum += sum_even << 8;
Quoted 8 lines
	uint32_t sum_odd = 0;
	for (const uint8_t *p = buf; p < end; p += 2) {
		sum_odd += *p;
	}
	sum += sum_odd;

	return sum;
}
This function does not work on both little and big endian, when mixed with other checksum functions.
The checksum functions read the buffer in CPU native endian and stores the checksum in CPU native endian; and magically it becomes correct in network endian. Please refer to RFC 1071 for the details. Perhaps you can also find some inspiration in RFC 1141.
PS: Please don't top post.

Mattias Rönnblom

2022-06-16 14:09 UTC #72 lore raw

Quoted 8 lines · Emil Berg

I've been sketching on an efficient solution to this. What about something along the way below? I've run it with the combinations of:
even buf, even len
even buf, odd len
odd buf, even len
odd buf, odd len
and it seems to give the same results as the older version of __rte_raw_cksum, before 21.03. I ran it without optimizations and such to ensure the compiler didn't insert vector instructions and such so the results were comparable.

...but you *want* the compiler to vectorize this code. There's much to gain, and it can likely be done also in the non-aligned case. What you don't want is for the compiler to assume the data is 16-bit aligned (and output SIMD load/store instructions which require alignment).

I don't see why you just can't take the current implementation, and replace the direct assignment ("*u16_buf") with a temporary variable, and a memcpy(). This also eliminates the need for the may_alias attribute (at least on the u16_buf pointer).

Quoted 125 lines

static inline uint32_t
__rte_raw_cksum_newest(const void *buf, size_t len, uint32_t sum)
{
	const uint8_t *end = buf + len;

	uint32_t sum_even = 0;
	for (const uint8_t *p = buf + 1; p < end; p += 2) {
		sum_even += *p;
	}
	sum += sum_even << 8;

	uint32_t sum_odd = 0;
	for (const uint8_t *p = buf; p < end; p += 2) {
		sum_odd += *p;
	}
	sum += sum_odd;

	return sum;
}
/Emil
-----Original Message-----
From: Emil Berg
Sent: den 16 juni 2022 07:45
To: Morten Brørup <[email protected]>; [email protected]
Cc: [email protected]
Subject: RE: [Bug 1035] __rte_raw_cksum() crash with misaligned pointer
Hi!
We want the B option, i.e. the 6 bytes option. Perhaps adding alignment detection to __rte_raw_cksum() is a good idea.
A minor comment but I think buf & 1 won't work since buf isn't an integral type, but something along that way.
I'm starting to think about an efficient way to do this.
Thank you!
-----Original Message-----
From: Morten Brørup <[email protected]>
Sent: den 15 juni 2022 16:41
To: Emil Berg <[email protected]>; [email protected]
Cc: [email protected]
Subject: RE: [Bug 1035] __rte_raw_cksum() crash with misaligned pointer
Quoted 43 lines
From: [email protected] [mailto:[email protected]]
Sent: Wednesday, 15 June 2022 09.16
https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-45444
5555731-2e92ae6bf759c0c5&q=1&e=b3fc70af-5d37-4ffb-b34d-9a51927f5f6d&u= https%3A%2F%2Fbugs.dpdk.org%2Fshow_bug.cgi%3Fid%3D1035
             Bug ID: 1035
            Summary: __rte_raw_cksum() crash with misaligned pointer
            Product: DPDK
            Version: 21.11
           Hardware: All
                 OS: All
             Status: UNCONFIRMED
           Severity: normal
           Priority: Normal
          Component: ethdev
           Assignee: [email protected]
           Reporter: [email protected]
Target Milestone: ---
See rte_raw_cksum() in rte_ip.h, which is part of the public API. See also the subfunction __rte_raw_cksum().
_rte_raw_cksum assumes that the buffer over which the checksum is calculated is an even address (divisible by two). See for example this stack overflow
post:
https://stackoverflow.com/questions/46790550/c-undefined-behavior-
strict-aliasing-rule-or-incorrect-alignment
The post explains that there is undefined behavior in C11 when "conversion between two pointer types produces a result that is incorrectly aligned". When the buf argument starts on an odd address we thus have undefined behavior, since a pointer is cast from void* to uint16_t*.
In most cases (at least on x86) that isn't a problem, but with higher optimization levels it may break due to vector instructions. This new function seems to be easier to optimize by the compiler, resulting in a crash when the buf argument is odd. Please note that the undefined behavior is present in earlier versions of dpdk as well.
Quoted 6 lines
Now you're probably thinking: "Just align your buffers". The problem is that we have a packet buffer which is aligned. The checksum is calculated on a subset of that aligned packet buffer, and that sometimes lies on odd addresses.
The question remains if this is an issue with dpdk or not.
I can imagine other systems doing what you describe too. So it needs to be addressed.
Off the top of my head, an easy fix would be updating __rte_raw_cksum() like this:
static inline uint32_t
__rte_raw_cksum(const void *buf, size_t len, uint32_t sum) {
	if (likely((buf & 1) == 0)) {
		/* The buffer is 16 bit aligned. */
		Keep the existing, optimized implementation here.
	} else {
		/* The buffer is not 16 bit aligned. */
		Add a new odd-buf tolerant implementation here.
	}
}
However, I'm not sure that it covers your scenario!
The checksum is 16 bit wide, so if you calculate the checksum of e.g. 4 bytes of memory starting at offset 1 in a 6 byte packet buffer, the memory block can be treated as either 4 or 6 bytes relative to the data covered by the checksum, i.e.:
A: XX [01 02] [03 04] XX --> cksum = [04 06]
B: [XX 01] [02 03] [04 XX] --> cksum = [06 04]
Which one do you need?
Perhaps an additional function is required to support your use case, and the documentation for rte_raw_cksum() and __rte_raw_cksum() needs to reflect that the buffer must be 16 bit aligned.
Or the rte_raw_cksum() function can be modified to support an odd buffer pointer as outlined above, with documentation added about alignment of the running checksum.

Morten Brørup

2022-06-16 14:36 UTC #7 lore raw

[1] https://elixir.bootlin.com/dpdk/latest/source/lib/net/rte_ether.h#L273

Quoted 161 lines

From: Mattias Rönnblom [mailto:[email protected]]
Sent: Thursday, 16 June 2022 15.58
Quoted 8 lines · Morten Brørup
+CC Olivier Matz <[email protected]>, Network Headers maintainer
Quoted 6 lines
-----Original Message-----
From: Morten Brørup <[email protected]>
Sent: den 15 juni 2022 16:41
To: Emil Berg <[email protected]>; [email protected]
Cc: [email protected]
Subject: RE: [Bug 1035] __rte_raw_cksum() crash with misaligned
pointer
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
From: [email protected] [mailto:[email protected]]
Sent: Wednesday, 15 June 2022 09.16
Quoted 8 lines
Quoted 8 lines
Quoted 1 line
https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-
45444
Quoted 1 line
5555731-2e92ae6bf759c0c5&q=1&e=b3fc70af-5d37-4ffb-b34d-
9a51927f5f6d&u=
Quoted 4 lines
https%3A%2F%2Fbugs.dpdk.org%2Fshow_bug.cgi%3Fid%3D1035
Bug ID: 1035
Summary: __rte_raw_cksum() crash with misaligned
pointer
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
            Product: DPDK
            Version: 21.11
           Hardware: All
                 OS: All
             Status: UNCONFIRMED
           Severity: normal
           Priority: Normal
          Component: ethdev
           Assignee: [email protected]
           Reporter: [email protected]
Target Milestone: ---
Quoted 1 line
Quoted 1 line
Quoted 1 line
See rte_raw_cksum() in rte_ip.h, which is part of the public API.
See
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
also the subfunction __rte_raw_cksum().
_rte_raw_cksum assumes that the buffer over which the checksum is calculated is an even address (divisible by two). See for example
this
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
stack overflow
post:
https://stackoverflow.com/questions/46790550/c-undefined-behavior-
strict-aliasing-rule-or-incorrect-alignment
The post explains that there is undefined behavior in C11 when "conversion between two pointer types produces a result that is incorrectly aligned". When the buf argument starts on an odd
address
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
we thus have undefined behavior, since a pointer is cast from void*
to
Quoted 3 lines
uint16_t*.
In most cases (at least on x86) that isn't a problem, but with
higher
Quoted 1 line
Quoted 1 line
Quoted 1 line
optimization levels it may break due to vector instructions. This
new
Quoted 1 line
Quoted 1 line
Quoted 1 line
function seems to be easier to optimize by the compiler, resulting
in
Quoted 1 line
Quoted 1 line
Quoted 1 line
a crash when the buf argument is odd. Please note that the
undefined
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
behavior is present in earlier versions of dpdk as well.
Now you're probably thinking: "Just align your buffers". The
problem
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
is that we have a packet buffer which is aligned. The checksum is calculated on a subset of that aligned packet buffer, and that sometimes lies on odd addresses.
The question remains if this is an issue with dpdk or not.
I can imagine other systems doing what you describe too. So it needs
to
Quoted 3 lines
Quoted 3 lines
be addressed.
Off the top of my head, an easy fix would be updating
__rte_raw_cksum()
Quoted 13 lines
Quoted 13 lines
like this:
static inline uint32_t
__rte_raw_cksum(const void *buf, size_t len, uint32_t sum) {
	if (likely((buf & 1) == 0)) {
		/* The buffer is 16 bit aligned. */
		Keep the existing, optimized implementation here.
	} else {
		/* The buffer is not 16 bit aligned. */
		Add a new odd-buf tolerant implementation here.
	}
}
Quoted 3 lines
Quoted 3 lines
However, I'm not sure that it covers your scenario!
The checksum is 16 bit wide, so if you calculate the checksum of
e.g. 4
Quoted 2 lines
Quoted 2 lines
bytes of memory starting at offset 1 in a 6 byte packet buffer, the memory block can be treated as either 4 or 6 bytes relative to the
data
Quoted 10 lines
Quoted 10 lines
covered by the checksum, i.e.:
A: XX [01 02] [03 04] XX --> cksum = [04 06]
B: [XX 01] [02 03] [04 XX] --> cksum = [06 04]
Which one do you need?
Perhaps an additional function is required to support your use case, and the documentation for rte_raw_cksum() and __rte_raw_cksum()
needs
Quoted 6 lines
Quoted 5 lines
to reflect that the buffer must be 16 bit aligned.
Or the rte_raw_cksum() function can be modified to support an odd buffer pointer as outlined above, with documentation added about alignment of the running checksum.
Quoted 6 lines
Quoted 6 lines
-----Original Message-----
From: Emil Berg
Sent: den 16 juni 2022 07:45
To: Morten Brørup <[email protected]>; [email protected]
Cc: [email protected]
Subject: RE: [Bug 1035] __rte_raw_cksum() crash with misaligned
pointer
Quoted 4 lines
Quoted 4 lines
Hi!
We want the B option, i.e. the 6 bytes option. Perhaps adding
alignment
Quoted 7 lines
Quoted 7 lines
detection to __rte_raw_cksum() is a good idea.
A minor comment but I think buf & 1 won't work since buf isn't an integral type, but something along that way.
I'm starting to think about an efficient way to do this.
Quoted 8 lines
Quoted 8 lines
Thank you!
-----Original Message-----
From: Emil Berg [mailto:[email protected]]
Sent: Thursday, 16 June 2022 08.32
To: Morten Brørup; [email protected]
Cc: [email protected]
Subject: RE: [Bug 1035] __rte_raw_cksum() crash with misaligned
pointer
Quoted 10 lines
Quoted 10 lines
I've been sketching on an efficient solution to this. What about something along the way below? I've run it with the combinations of:
even buf, even len
even buf, odd len
odd buf, even len
odd buf, odd len
and it seems to give the same results as the older version of __rte_raw_cksum, before 21.03. I ran it without optimizations and
such
Quoted 3 lines
Quoted 2 lines
to ensure the compiler didn't insert vector instructions and such so the results were comparable.

Quoted 8 lines

Quoted 1 line
The performance, when using an aligned buffer, needs to be comparable
with full compiler optimization, or the patch will not be accepted.
Quoted 1 line
I think the question is: does rte_raw_cksum() have any alignment requirements, from an API contract point of view? The documentation says
nothing about any such. In that case, it seems reasonable to me to assume that there are none.

The packet buffer must be 16 bit aligned. Many structures in DPDK are designed with this invariant, e.g. the Ethernet header (struct rte_ether_hdr) [1].

I agree that the documentation could mention this invariant in more places.

When calculating the checksum of a part of packet buffer, it should allow that part of the buffer to start at a non-16 bit aligned address, which is what Emil needs, and this is what I suggest adding support for in this function.

Regarding any implementation suggestions, please refer to my examples A and B above: The running checksum for a partial buffer of 4 bytes differs, depending on how you calculate it. The checksum calculation must see the buffer as part of the packet buffer, i.e. aligned with the (16 bit aligned) packet buffer and the (16 bit aligned) running checksum, as described by example B.

Morten Brørup

2022-06-17 07:32 UTC #8 lore raw

Quoted 202 lines

From: Morten Brørup
Sent: Thursday, 16 June 2022 16.36
Quoted 5 lines
From: Mattias Rönnblom [mailto:[email protected]]
Sent: Thursday, 16 June 2022 15.58
Quoted 1 line · Morten Brørup
+CC Olivier Matz <[email protected]>, Network Headers
maintainer
Quoted 33 lines
Quoted 7 lines
Quoted 7 lines
-----Original Message-----
From: Morten Brørup <[email protected]>
Sent: den 15 juni 2022 16:41
To: Emil Berg <[email protected]>; [email protected]
Cc: [email protected]
Subject: RE: [Bug 1035] __rte_raw_cksum() crash with misaligned
pointer
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
From: [email protected] [mailto:[email protected]]
Sent: Wednesday, 15 June 2022 09.16
Quoted 8 lines
Quoted 8 lines
Quoted 1 line
https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-
45444
Quoted 1 line
5555731-2e92ae6bf759c0c5&q=1&e=b3fc70af-5d37-4ffb-b34d-
9a51927f5f6d&u=
Quoted 4 lines
https%3A%2F%2Fbugs.dpdk.org%2Fshow_bug.cgi%3Fid%3D1035
Bug ID: 1035
Summary: __rte_raw_cksum() crash with misaligned
pointer
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
            Product: DPDK
            Version: 21.11
           Hardware: All
                 OS: All
             Status: UNCONFIRMED
           Severity: normal
           Priority: Normal
          Component: ethdev
           Assignee: [email protected]
           Reporter: [email protected]
Target Milestone: ---
Quoted 10 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
See rte_raw_cksum() in rte_ip.h, which is part of the public API.
See
Quoted 8 lines
Quoted 8 lines
Quoted 4 lines
also the subfunction __rte_raw_cksum().
_rte_raw_cksum assumes that the buffer over which the checksum is calculated is an even address (divisible by two). See for example
this
Quoted 3 lines
stack overflow
post:
https://stackoverflow.com/questions/46790550/c-undefined-
behavior-
Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
strict-aliasing-rule-or-incorrect-alignment
The post explains that there is undefined behavior in C11 when "conversion between two pointer types produces a result that is incorrectly aligned". When the buf argument starts on an odd
address
Quoted 1 line
Quoted 1 line
Quoted 1 line
we thus have undefined behavior, since a pointer is cast from
void*
Quoted 8 lines
Quoted 4 lines
Quoted 4 lines
to
Quoted 3 lines
uint16_t*.
In most cases (at least on x86) that isn't a problem, but with
higher
Quoted 1 line
Quoted 1 line
Quoted 1 line
optimization levels it may break due to vector instructions. This
new
Quoted 1 line
Quoted 1 line
Quoted 1 line
function seems to be easier to optimize by the compiler,
resulting
Quoted 14 lines
in
Quoted 1 line
Quoted 1 line
Quoted 1 line
a crash when the buf argument is odd. Please note that the
undefined
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
behavior is present in earlier versions of dpdk as well.
Now you're probably thinking: "Just align your buffers". The
problem
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
is that we have a packet buffer which is aligned. The checksum is calculated on a subset of that aligned packet buffer, and that sometimes lies on odd addresses.
The question remains if this is an issue with dpdk or not.
I can imagine other systems doing what you describe too. So it
needs
Quoted 18 lines
to
Quoted 3 lines
Quoted 3 lines
be addressed.
Off the top of my head, an easy fix would be updating
__rte_raw_cksum()
Quoted 13 lines
Quoted 13 lines
like this:
static inline uint32_t
__rte_raw_cksum(const void *buf, size_t len, uint32_t sum) {
	if (likely((buf & 1) == 0)) {
		/* The buffer is 16 bit aligned. */
		Keep the existing, optimized implementation here.
	} else {
		/* The buffer is not 16 bit aligned. */
		Add a new odd-buf tolerant implementation here.
	}
}
Quoted 5 lines
Quoted 3 lines
Quoted 3 lines
However, I'm not sure that it covers your scenario!
The checksum is 16 bit wide, so if you calculate the checksum of
e.g. 4
Quoted 1 line
Quoted 1 line
bytes of memory starting at offset 1 in a 6 byte packet buffer,
the
Quoted 11 lines
Quoted 1 line
Quoted 1 line
memory block can be treated as either 4 or 6 bytes relative to the
data
Quoted 9 lines
Quoted 9 lines
covered by the checksum, i.e.:
A: XX [01 02] [03 04] XX --> cksum = [04 06]
B: [XX 01] [02 03] [04 XX] --> cksum = [06 04]
Which one do you need?
Perhaps an additional function is required to support your use
case,
Quoted 29 lines
Quoted 1 line
Quoted 1 line
and the documentation for rte_raw_cksum() and __rte_raw_cksum()
needs
Quoted 6 lines
Quoted 5 lines
to reflect that the buffer must be 16 bit aligned.
Or the rte_raw_cksum() function can be modified to support an odd buffer pointer as outlined above, with documentation added about alignment of the running checksum.
Quoted 6 lines
Quoted 6 lines
-----Original Message-----
From: Emil Berg
Sent: den 16 juni 2022 07:45
To: Morten Brørup <[email protected]>; [email protected]
Cc: [email protected]
Subject: RE: [Bug 1035] __rte_raw_cksum() crash with misaligned
pointer
Quoted 4 lines
Quoted 4 lines
Hi!
We want the B option, i.e. the 6 bytes option. Perhaps adding
alignment
Quoted 9 lines
Quoted 8 lines
detection to __rte_raw_cksum() is a good idea.
A minor comment but I think buf & 1 won't work since buf isn't an integral type, but something along that way.
I'm starting to think about an efficient way to do this.
Thank you!
Quoted 10 lines
Quoted 6 lines
Quoted 6 lines
-----Original Message-----
From: Emil Berg [mailto:[email protected]]
Sent: Thursday, 16 June 2022 08.32
To: Morten Brørup; [email protected]
Cc: [email protected]
Subject: RE: [Bug 1035] __rte_raw_cksum() crash with misaligned
pointer
Quoted 3 lines
Quoted 3 lines
I've been sketching on an efficient solution to this. What about something along the way below? I've run it with the combinations
of:
Quoted 9 lines
Quoted 7 lines
Quoted 7 lines
even buf, even len
even buf, odd len
odd buf, even len
odd buf, odd len
and it seems to give the same results as the older version of __rte_raw_cksum, before 21.03. I ran it without optimizations and
such
Quoted 1 line
Quoted 1 line
to ensure the compiler didn't insert vector instructions and such
so
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
the results were comparable.
The performance, when using an aligned buffer, needs to be
comparable
Quoted 7 lines
with full compiler optimization, or the patch will not be accepted.
Quoted 1 line
I think the question is: does rte_raw_cksum() have any alignment requirements, from an API contract point of view? The documentation says
nothing about any such. In that case, it seems reasonable to me to assume that there are none.
The packet buffer must be 16 bit aligned. Many structures in DPDK are designed with this invariant, e.g. the Ethernet header (struct rte_ether_hdr) [1].
I agree that the documentation could mention this invariant in more places.
When calculating the checksum of a part of packet buffer, it should allow that part of the buffer to start at a non-16 bit aligned address, which is what Emil needs, and this is what I suggest adding support for in this function.
Regarding any implementation suggestions, please refer to my examples A and B above: The running checksum for a partial buffer of 4 bytes differs, depending on how you calculate it. The checksum calculation must see the buffer as part of the packet buffer, i.e. aligned with the (16 bit aligned) packet buffer and the (16 bit aligned) running checksum, as described by example B.

Quoted 2 lines

[1]
https://elixir.bootlin.com/dpdk/latest/source/lib/net/rte_ether.h#L273

I tried a few things in Godbolt, and will submit a patch shortly.

Please hang on...

Morten Brørup

2022-06-17 08:45 UTC #9 lore raw

PATCH 1/1 net: fix checksum with unaligned buffer

With this patch, the checksum can be calculated on an unligned part of a packet buffer.
I.e. the buf parameter is no longer required to be 16 bit aligned.

The DPDK invariant that packet buffers must be 16 bit aligned remains unchanged.
This invariant also defines how to calculate the 16 bit checksum on an unaligned part of a packet buffer.

Bugzilla ID: 1035
Cc: [email protected]

---
 lib/net/rte_ip.h | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)

lib/net/rte_ip.h +15 -2

index b502481670..8e301d9c26 100644
@@ -162,9 +162,22 @@ __rte_raw_cksum(const void *buf, size_t len, uint32_t sum)
 {
 	/* extend strict-aliasing rules */
 	typedef uint16_t __attribute__((__may_alias__)) u16_p;
-	const u16_p *u16_buf = (const u16_p *)buf;
-	const u16_p *end = u16_buf + len / sizeof(*u16_buf);
+	const u16_p *u16_buf;
+	const u16_p *end;
+
+	/* if buffer is unaligned, keeping it byte order independent */
+	if (unlikely((uintptr_t)buf & 1)) {
+		uint16_t first = 0;
+		if (unlikely(len == 0))
+			return 0;
+		((unsigned char *)&first)[1] = *(const unsigned char *)buf;
+		sum += first;
+		buf = (const void *)((uintptr_t)buf + 1);
+		len--;
+	}
 
+	u16_buf = (const u16_p *)buf;
+	end = u16_buf + len / sizeof(*u16_buf);
 	for (; u16_buf != end; ++u16_buf)
 		sum += *u16_buf;

Signature · 1 line

2.17.1

Morten Brørup

2022-06-17 09:07 UTC #10 lore raw

RE: [PATCH] net: fix checksum with unaligned buffer

Quoted 51 lines

From: Morten Brørup [mailto:[email protected]]
Sent: Friday, 17 June 2022 10.45
With this patch, the checksum can be calculated on an unligned part of a packet buffer.
I.e. the buf parameter is no longer required to be 16 bit aligned.
The DPDK invariant that packet buffers must be 16 bit aligned remains unchanged.
This invariant also defines how to calculate the 16 bit checksum on an unaligned part of a packet buffer.
Bugzilla ID: 1035
Cc: [email protected]
Signed-off-by Morten Brørup <[email protected]>
---
 lib/net/rte_ip.h | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)
lib/net/rte_ip.h +15 -3
index b502481670..8e301d9c26 100644
@@ -162,9 +162,22 @@ __rte_raw_cksum(const void *buf, size_t len,
 uint32_t sum)
 {
 	/* extend strict-aliasing rules */
 	typedef uint16_t __attribute__((__may_alias__)) u16_p;
-	const u16_p *u16_buf = (const u16_p *)buf;
-	const u16_p *end = u16_buf + len / sizeof(*u16_buf);
+	const u16_p *u16_buf;
+	const u16_p *end;
+
+	/* if buffer is unaligned, keeping it byte order independent */
+	if (unlikely((uintptr_t)buf & 1)) {
+		uint16_t first = 0;
+		if (unlikely(len == 0))
+			return 0;
+		((unsigned char *)&first)[1] = *(const unsigned char *)buf;
+		sum += first;
+		buf = (const void *)((uintptr_t)buf + 1);
+		len--;
+	}
 
+	u16_buf = (const u16_p *)buf;
+	end = u16_buf + len / sizeof(*u16_buf);
 	for (; u16_buf != end; ++u16_buf)
 		sum += *u16_buf;
 
--

Quoted 1 line

2.17.1

@Emil, can you please test this patch with an unaligned buffer on your application to confirm that it produces the expected result.

Emil Berg

2022-06-17 13:57 UTC #11 lore raw

RE: [PATCH] net: fix checksum with unaligned buffer

Quoted 58 lines

-----Original Message-----
From: Morten Brørup <[email protected]>
Sent: den 17 juni 2022 11:07
To: Emil Berg <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: RE: [PATCH] net: fix checksum with unaligned buffer
Quoted 29 lines
From: Morten Brørup [mailto:[email protected]]
Sent: Friday, 17 June 2022 10.45
With this patch, the checksum can be calculated on an unligned part of a packet buffer.
I.e. the buf parameter is no longer required to be 16 bit aligned.
The DPDK invariant that packet buffers must be 16 bit aligned remains unchanged.
This invariant also defines how to calculate the 16 bit checksum on an unaligned part of a packet buffer.
Bugzilla ID: 1035
Cc: [email protected]
Signed-off-by Morten Brørup <[email protected]>
---
 lib/net/rte_ip.h | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)
lib/net/rte_ip.h index renamed from lib/net/rte_ip.h +0 -0
b502481670..8e301d9c26 100644
lib/net/rte_ip.h +0 -1
@@ -162,9 +162,22 @@ __rte_raw_cksum(const void *buf, size_t len,
 uint32_t sum)  {
 	/* extend strict-aliasing rules */
 	typedef uint16_t __attribute__((__may_alias__)) u16_p;
-	const u16_p *u16_buf = (const u16_p *)buf;
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, size_t len, · line 173
Quoted 10 lines
lib/net/rte_ip.h +9 -1
-	const u16_p *end = u16_buf + len / sizeof(*u16_buf);
+	const u16_p *u16_buf;
+	const u16_p *end;
+
+	/* if buffer is unaligned, keeping it byte order independent */
+	if (unlikely((uintptr_t)buf & 1)) {
+		uint16_t first = 0;
+		if (unlikely(len == 0))
+			return 0;
+		((unsigned char *)&first)[1] = *(const unsigned
char *)buf;
Quoted 10 lines
lib/net/rte_ip.h +6 -0
+		sum += first;
+		buf = (const void *)((uintptr_t)buf + 1);
+		len--;
+	}
 
+	u16_buf = (const u16_p *)buf;
+	end = u16_buf + len / sizeof(*u16_buf);
 	for (; u16_buf != end; ++u16_buf)
 		sum += *u16_buf;
 

Quoted 5 lines

Quoted 2 lines
--
2.17.1
@Emil, can you please test this patch with an unaligned buffer on your application to confirm that it produces the expected result.

All right. I'll test, perhaps on Monday.

Morten Brørup

2022-06-20 10:57 UTC #13 lore raw

RE: [PATCH] net: fix checksum with unaligned buffer

Quoted 66 lines

From: Emil Berg [mailto:[email protected]]
Sent: Monday, 20 June 2022 12.38
Quoted 7 lines
From: Morten Brørup <[email protected]>
Sent: den 17 juni 2022 11:07
Quoted 4 lines
From: Morten Brørup [mailto:[email protected]]
Sent: Friday, 17 June 2022 10.45
With this patch, the checksum can be calculated on an unligned part
of
Quoted 4 lines
Quoted 4 lines
a packet buffer.
I.e. the buf parameter is no longer required to be 16 bit aligned.
The DPDK invariant that packet buffers must be 16 bit aligned
remains
Quoted 2 lines
Quoted 2 lines
unchanged.
This invariant also defines how to calculate the 16 bit checksum on
an
Quoted 40 lines
Quoted 19 lines
unaligned part of a packet buffer.
Bugzilla ID: 1035
Cc: [email protected]
Signed-off-by Morten Brørup <[email protected]>
---
 lib/net/rte_ip.h | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)
lib/net/rte_ip.h index renamed from lib/net/rte_ip.h +0 -0
b502481670..8e301d9c26 100644
lib/net/rte_ip.h +0 -1
@@ -162,9 +162,22 @@ __rte_raw_cksum(const void *buf, size_t len,
 uint32_t sum)  {
 	/* extend strict-aliasing rules */
 	typedef uint16_t __attribute__((__may_alias__)) u16_p;
-	const u16_p *u16_buf = (const u16_p *)buf;
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, size_t len, · line 173
Quoted 10 lines
lib/net/rte_ip.h +9 -1
-	const u16_p *end = u16_buf + len / sizeof(*u16_buf);
+	const u16_p *u16_buf;
+	const u16_p *end;
+
+	/* if buffer is unaligned, keeping it byte order independent */
+	if (unlikely((uintptr_t)buf & 1)) {
+		uint16_t first = 0;
+		if (unlikely(len == 0))
+			return 0;
+		((unsigned char *)&first)[1] = *(const unsigned
char *)buf;
Quoted 10 lines
lib/net/rte_ip.h +6 -0
+		sum += first;
+		buf = (const void *)((uintptr_t)buf + 1);
+		len--;
+	}
 
+	u16_buf = (const u16_p *)buf;
+	end = u16_buf + len / sizeof(*u16_buf);
 	for (; u16_buf != end; ++u16_buf)
 		sum += *u16_buf;
 
Quoted 4 lines
Quoted 2 lines
--
2.17.1
@Emil, can you please test this patch with an unaligned buffer on
your
Quoted 1 line
application to confirm that it produces the expected result.

Quoted 7 lines

Hi!
I tested the patch. It doesn't seem to produce the same results. I think the problem is that it always starts summing from an even address, the sum should always start from the first byte according to the checksum specification. Can I instead propose something Mattias Rönnblom sent me?

I assume that it produces the same result when the "buf" parameter is aligned?

And when the "buf" parameter is unaligned, I don't expect it to produce the same results as the simple algorithm!

This was the whole point of the patch: I expect the overall packet buffer to be 16 bit aligned, and the checksum to be a partial checksum of such a 16 bit aligned packet buffer. When calling this function, I assume that the "buf" and "len" parameters point to a part of such a packet buffer. If these expectations are correct, the simple algorithm will produce incorrect results when "buf" is unaligned.

I was asking you to test if the checksum on the packet is correct when your application modifies an unaligned part of the packet and uses this function to update the checksum.

Quoted 54 lines

-----------------------------------------------------------------------
---------------------------------------
const void *end = RTE_PTR_ADD(buf, (len / sizeof(uint16_t)) * sizeof(uint16_t));
for (; buf != end; buf = RTE_PTR_ADD(buf, sizeof(uint16_t))) {
    uint16_t v;
    memcpy(&v, buf, sizeof(uint16_t));
    sum += v;
}
/* if length is odd, keeping it byte order independent */
if (unlikely(len % 2)) {
    uint16_t left = 0;
    *(unsigned char *)&left = *(const unsigned char *)end;
    sum += left;
}
-----------------------------------------------------------------------
---------------------------------------
Note that the last block is the same as before. Amazingly I see no measurable performance hit from this compared to the previous one (-O3, march=native). Looking at the previous the loop body may compile to (x86):
-----------------------------------------------------------------------
---------------------------------------
vmovdqa (%rdx),%xmm1
vpmovzxwd %xmm1,%xmm0
vpsrldq $0x8,%xmm1,%xmm1
vpmovzxwd %xmm1,%xmm1
vpaddd %xmm1,%xmm0,%xmm0
cmp $0xf,%rax
jbe 0x7ff7a0dfb1a9
-----------------------------------------------------------------------
---------------------------------------
while Mattias' memcpy solution:
-----------------------------------------------------------------------
---------------------------------------
vmovdqu (%rcx),%ymm0
add $0x20,%rcx
vpmovzxwd %xmm0,%ymm1
vextracti128 $0x1,%ymm0,%xmm0
vpmovzxwd %xmm0,%ymm0
vpaddd %ymm0,%ymm1,%ymm0
vpaddd %ymm0,%ymm2,%ymm2
cmp %r9,%rcx
jne 0x555555556380
-----------------------------------------------------------------------
---------------------------------------
Thus two extra instructions in the loop, but I suspect it may be memory bound, leading to no measurable performance difference.
Any comments?
/Emil

Morten Brørup

2022-06-21 08:05 UTC #15 lore raw

RE: [PATCH] net: fix checksum with unaligned buffer

[1] http://git.dpdk.org/dpdk/commit/lib/librte_net/rte_ether.h?id=da5350ef29afd35c1adabe76f60832f3092269ad

+TO: @Bruce and @Stephen: You signed off on the 16 bit alignment requirement. We need background info on this.

Quoted 119 lines

From: Emil Berg [mailto:[email protected]]
Sent: Tuesday, 21 June 2022 09.17
Quoted 17 lines
From: Morten Brørup <[email protected]>
Sent: den 20 juni 2022 12:58
Quoted 14 lines
From: Emil Berg [mailto:[email protected]]
Sent: Monday, 20 June 2022 12.38
Quoted 8 lines
From: Morten Brørup <[email protected]>
Sent: den 17 juni 2022 11:07
Quoted 5 lines
From: Morten Brørup [mailto:[email protected]]
Sent: Friday, 17 June 2022 10.45
With this patch, the checksum can be calculated on an unligned part
of
Quoted 2 lines
Quoted 2 lines
a packet buffer.
I.e. the buf parameter is no longer required to be 16 bit
aligned.
Quoted 5 lines
Quoted 5 lines
Quoted 2 lines
Quoted 2 lines
The DPDK invariant that packet buffers must be 16 bit aligned
remains
Quoted 2 lines
Quoted 2 lines
unchanged.
This invariant also defines how to calculate the 16 bit
checksum
Quoted 12 lines
Quoted 12 lines
Quoted 1 line
Quoted 1 line
on
an
Quoted 10 lines
Quoted 10 lines
unaligned part of a packet buffer.
Bugzilla ID: 1035
Cc: [email protected]
Signed-off-by Morten Brørup <[email protected]>
---
 lib/net/rte_ip.h | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, size_t
Quoted 5 lines
lib/net/rte_ip.h index renamed from lib/net/rte_ip.h +0 -0
b502481670..8e301d9c26 100644
lib/net/rte_ip.h +0 -0
@@ -162,9 +162,22 @@ __rte_raw_cksum(const void *buf, size_t
len,
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, size_t len, uint32_t sum)
Quoted 9 lines
lib/net/rte_ip.h +4 -2
 uint32_t sum)  {
 	/* extend strict-aliasing rules */
 	typedef uint16_t __attribute__((__may_alias__)) u16_p;
-	const u16_p *u16_buf = (const u16_p *)buf;
-	const u16_p *end = u16_buf + len / sizeof(*u16_buf);
+	const u16_p *u16_buf;
+	const u16_p *end;
+
+	/* if buffer is unaligned, keeping it byte order
independent */
Quoted 20 lines
Quoted 20 lines
Quoted 20 lines
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, size_t len, uint32_t sum)
Quoted 5 lines
lib/net/rte_ip.h +5 -0
+	if (unlikely((uintptr_t)buf & 1)) {
+		uint16_t first = 0;
+		if (unlikely(len == 0))
+			return 0;
+		((unsigned char *)&first)[1] = *(const unsigned
char *)buf;
Quoted 11 lines
lib/net/rte_ip.h +6 -1
+		sum += first;
+		buf = (const void *)((uintptr_t)buf + 1);
+		len--;
+	}
 
+	u16_buf = (const u16_p *)buf;
+	end = u16_buf + len / sizeof(*u16_buf);
 	for (; u16_buf != end; ++u16_buf)
 		sum += *u16_buf;
 
--
Quoted 1 line
2.17.1
@Emil, can you please test this patch with an unaligned buffer on
Quoted 8 lines
Quoted 8 lines
your
Quoted 1 line
application to confirm that it produces the expected result.
Hi!
I tested the patch. It doesn't seem to produce the same results. I think the problem is that it always starts summing from an even address, the sum should always start from the first byte according
to
Quoted 7 lines
Quoted 2 lines
the checksum specification. Can I instead propose something Mattias Rönnblom sent me?
I assume that it produces the same result when the "buf" parameter is aligned?
And when the "buf" parameter is unaligned, I don't expect it to
produce the
Quoted 3 lines
same results as the simple algorithm!
This was the whole point of the patch: I expect the overall packet
buffer to
Quoted 1 line
be 16 bit aligned, and the checksum to be a partial checksum of such
a 16 bit
Quoted 1 line
aligned packet buffer. When calling this function, I assume that the
"buf" and
Quoted 2 lines
"len" parameters point to a part of such a packet buffer. If these expectations are correct, the simple algorithm will produce incorrect
results
Quoted 3 lines
when "buf" is unaligned.
I was asking you to test if the checksum on the packet is correct
when your
Quoted 1 line
application modifies an unaligned part of the packet and uses this
function to
Quoted 2 lines
update the checksum.
Now I understand your use case. Your use case seems to be about partial checksums, of which some partial checksums may start on unaligned addresses in an otherwise aligned packet.
Our use case is about calculating the full checksum on a nested packet. That nested packet may start on unaligned addresses.

Quoted 6 lines

The difference is basically if we want to sum over aligned addresses or not, handling the heading and trailing bytes appropriately.
Your method does not work in our case since we want to treat the first two bytes as the first word in our case. But I do understand that both methods are useful.

Yes, that certainly are two different use cases, requiring two different ways of calculating the 16 bit checksum.

Quoted 6 lines

Note that your method breaks the API. Previously (assuming no crashing due to low optimization levels, more accepting hardware, or a different compiler (version)) the current method would calculate the checksum assuming the first two bytes is the first word.

Depending on the point of view, my patch either fixes a bug (where the checksum was calculated incorrectly when the buf pointer was unaligned) or breaks the API (by calculating the differently when the buffer is unaligned).

I cannot say with certainty which one is correct, but perhaps some of the people with a deeper DPDK track record can...

@Bruce and @Stephen, in 2019 you signed off on a patch [1] introducing a 16 bit alignment requirement to the Ethernet address structure.

It is my understanding that DPDK has an invariant requiring packets to be 16 bit aligned, which that patch supports. Is this invariant documented anywhere, or am I completely wrong? If I'm wrong, then the alignment requirement introduced in that patch needs to be removed, as well as any similar alignment requirements elsewhere in DPDK.

@Emil, we should wait for a conclusion about the alignment invariant before we proceed.

If there is no such invariant, my patch is wrong, and we need to provide a v2 of the patch, which will then fit your use case. If there is such an invariant, my patch is correct, and another function must be added for your use case.

Bruce Richardson

2022-06-21 08:23 UTC #16 lore raw

Re: [PATCH] net: fix checksum with unaligned buffer

Quoted 137 lines · Morten Brørup

+TO: @Bruce and @Stephen: You signed off on the 16 bit alignment requirement. We need background info on this.
Quoted 119 lines
From: Emil Berg [mailto:[email protected]]
Sent: Tuesday, 21 June 2022 09.17
Quoted 17 lines
From: Morten Brørup <[email protected]>
Sent: den 20 juni 2022 12:58
Quoted 14 lines
From: Emil Berg [mailto:[email protected]]
Sent: Monday, 20 June 2022 12.38
Quoted 8 lines
From: Morten Brørup <[email protected]>
Sent: den 17 juni 2022 11:07
Quoted 5 lines
From: Morten Brørup [mailto:[email protected]]
Sent: Friday, 17 June 2022 10.45
With this patch, the checksum can be calculated on an unligned part
of
Quoted 2 lines
Quoted 2 lines
a packet buffer.
I.e. the buf parameter is no longer required to be 16 bit
aligned.
Quoted 5 lines
Quoted 5 lines
Quoted 2 lines
Quoted 2 lines
The DPDK invariant that packet buffers must be 16 bit aligned
remains
Quoted 2 lines
Quoted 2 lines
unchanged.
This invariant also defines how to calculate the 16 bit
checksum
Quoted 12 lines
Quoted 12 lines
Quoted 1 line
Quoted 1 line
on
an
Quoted 10 lines
Quoted 10 lines
unaligned part of a packet buffer.
Bugzilla ID: 1035
Cc: [email protected]
Signed-off-by Morten Brørup <[email protected]>
---
 lib/net/rte_ip.h | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, size_t
Quoted 5 lines
lib/net/rte_ip.h index renamed from lib/net/rte_ip.h +0 -0
b502481670..8e301d9c26 100644
lib/net/rte_ip.h +0 -0
@@ -162,9 +162,22 @@ __rte_raw_cksum(const void *buf, size_t
len,
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, size_t len, uint32_t sum)
Quoted 9 lines
lib/net/rte_ip.h +4 -2
 uint32_t sum)  {
 	/* extend strict-aliasing rules */
 	typedef uint16_t __attribute__((__may_alias__)) u16_p;
-	const u16_p *u16_buf = (const u16_p *)buf;
-	const u16_p *end = u16_buf + len / sizeof(*u16_buf);
+	const u16_p *u16_buf;
+	const u16_p *end;
+
+	/* if buffer is unaligned, keeping it byte order
independent */
Quoted 20 lines
Quoted 20 lines
Quoted 20 lines
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, size_t len, uint32_t sum)
Quoted 5 lines
lib/net/rte_ip.h +5 -0
+	if (unlikely((uintptr_t)buf & 1)) {
+		uint16_t first = 0;
+		if (unlikely(len == 0))
+			return 0;
+		((unsigned char *)&first)[1] = *(const unsigned
char *)buf;
Quoted 11 lines
lib/net/rte_ip.h +6 -1
+		sum += first;
+		buf = (const void *)((uintptr_t)buf + 1);
+		len--;
+	}
 
+	u16_buf = (const u16_p *)buf;
+	end = u16_buf + len / sizeof(*u16_buf);
 	for (; u16_buf != end; ++u16_buf)
 		sum += *u16_buf;
 
--
Quoted 1 line
2.17.1
@Emil, can you please test this patch with an unaligned buffer on
Quoted 8 lines
Quoted 8 lines
your
Quoted 1 line
application to confirm that it produces the expected result.
Hi!
I tested the patch. It doesn't seem to produce the same results. I think the problem is that it always starts summing from an even address, the sum should always start from the first byte according
to
Quoted 7 lines
Quoted 2 lines
the checksum specification. Can I instead propose something Mattias Rönnblom sent me?
I assume that it produces the same result when the "buf" parameter is aligned?
And when the "buf" parameter is unaligned, I don't expect it to
produce the
Quoted 3 lines
same results as the simple algorithm!
This was the whole point of the patch: I expect the overall packet
buffer to
Quoted 1 line
be 16 bit aligned, and the checksum to be a partial checksum of such
a 16 bit
Quoted 1 line
aligned packet buffer. When calling this function, I assume that the
"buf" and
Quoted 2 lines
"len" parameters point to a part of such a packet buffer. If these expectations are correct, the simple algorithm will produce incorrect
results
Quoted 3 lines
when "buf" is unaligned.
I was asking you to test if the checksum on the packet is correct
when your
Quoted 1 line
application modifies an unaligned part of the packet and uses this
function to
Quoted 2 lines
update the checksum.
Now I understand your use case. Your use case seems to be about partial checksums, of which some partial checksums may start on unaligned addresses in an otherwise aligned packet.
Our use case is about calculating the full checksum on a nested packet. That nested packet may start on unaligned addresses.
Quoted 6 lines
The difference is basically if we want to sum over aligned addresses or not, handling the heading and trailing bytes appropriately.
Your method does not work in our case since we want to treat the first two bytes as the first word in our case. But I do understand that both methods are useful.
Yes, that certainly are two different use cases, requiring two different ways of calculating the 16 bit checksum.
Quoted 6 lines
Note that your method breaks the API. Previously (assuming no crashing due to low optimization levels, more accepting hardware, or a different compiler (version)) the current method would calculate the checksum assuming the first two bytes is the first word.

Quoted 7 lines

Depending on the point of view, my patch either fixes a bug (where the checksum was calculated incorrectly when the buf pointer was unaligned) or breaks the API (by calculating the differently when the buffer is unaligned).
I cannot say with certainty which one is correct, but perhaps some of the people with a deeper DPDK track record can...
@Bruce and @Stephen, in 2019 you signed off on a patch [1] introducing a 16 bit alignment requirement to the Ethernet address structure.
It is my understanding that DPDK has an invariant requiring packets to be 16 bit aligned, which that patch supports. Is this invariant documented anywhere, or am I completely wrong? If I'm wrong, then the alignment requirement introduced in that patch needs to be removed, as well as any similar alignment requirements elsewhere in DPDK.

I don't believe it is explicitly documented as a global invariant, but I think it should be unless there is a definite case where we need to allow packets to be completely unaligned. Across all packet headers we looked at, there was no tunneling protocol where the resulting packet was left unaligned.

That said, if there are real use cases where we need to allow packets to start at an unaligned address, then I agree with you that we need to roll back the patch and work to ensure everything works with unaligned addresses.

/Bruce

Quoted 8 lines

[1] http://git.dpdk.org/dpdk/commit/lib/librte_net/rte_ether.h?id=da5350ef29afd35c1adabe76f60832f3092269ad
@Emil, we should wait for a conclusion about the alignment invariant before we proceed.
If there is no such invariant, my patch is wrong, and we need to provide a v2 of the patch, which will then fit your use case. If there is such an invariant, my patch is correct, and another function must be added for your use case.

Morten Brørup

2022-06-21 09:35 UTC #17 lore raw

RE: [PATCH] net: fix checksum with unaligned buffer

Quoted 192 lines

From: Bruce Richardson [mailto:[email protected]]
Sent: Tuesday, 21 June 2022 10.23
Quoted 1 line · Morten Brørup
+TO: @Bruce and @Stephen: You signed off on the 16 bit alignment
requirement. We need background info on this.
Quoted 10 lines
Quoted 10 lines
From: Emil Berg [mailto:[email protected]]
Sent: Tuesday, 21 June 2022 09.17
Quoted 6 lines
From: Morten Brørup <[email protected]>
Sent: den 20 juni 2022 12:58
Quoted 3 lines
From: Emil Berg [mailto:[email protected]]
Sent: Monday, 20 June 2022 12.38
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
From: Morten Brørup <[email protected]>
Sent: den 17 juni 2022 11:07
Quoted 4 lines
From: Morten Brørup [mailto:[email protected]]
Sent: Friday, 17 June 2022 10.45
With this patch, the checksum can be calculated on an
unligned
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
Quoted 1 line
part
of
Quoted 2 lines
Quoted 2 lines
a packet buffer.
I.e. the buf parameter is no longer required to be 16 bit
aligned.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
The DPDK invariant that packet buffers must be 16 bit
aligned
Quoted 16 lines
Quoted 16 lines
Quoted 3 lines
Quoted 3 lines
remains
Quoted 2 lines
Quoted 2 lines
unchanged.
This invariant also defines how to calculate the 16 bit
checksum
Quoted 12 lines
Quoted 12 lines
Quoted 1 line
Quoted 1 line
on
an
Quoted 10 lines
Quoted 10 lines
unaligned part of a packet buffer.
Bugzilla ID: 1035
Cc: [email protected]
Signed-off-by Morten Brørup <[email protected]>
---
 lib/net/rte_ip.h | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
lib/net/rte_ip.h __rte_raw_cksum(const void *buf,
Quoted 5 lines
lib/net/rte_ip.h index renamed from lib/net/rte_ip.h +0 -0
b502481670..8e301d9c26 100644
lib/net/rte_ip.h +0 -0
@@ -162,9 +162,22 @@ __rte_raw_cksum(const void *buf,
size_t
Quoted 4 lines
Quoted 4 lines
len,
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
lib/net/rte_ip.h +0 -0
 uint32_t sum)  {
 	/* extend strict-aliasing rules */
 	typedef uint16_t __attribute__((__may_alias__))
u16_p;
Quoted 23 lines
Quoted 23 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, · line 168
Quoted 6 lines
lib/net/rte_ip.h +4 -2
-	const u16_p *u16_buf = (const u16_p *)buf;
-	const u16_p *end = u16_buf + len / sizeof(*u16_buf);
+	const u16_p *u16_buf;
+	const u16_p *end;
+
+	/* if buffer is unaligned, keeping it byte order
independent */
Quoted 16 lines
Quoted 16 lines
Quoted 16 lines
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, · line 173
Quoted 5 lines
lib/net/rte_ip.h +5 -0
+	if (unlikely((uintptr_t)buf & 1)) {
+		uint16_t first = 0;
+		if (unlikely(len == 0))
+			return 0;
+		((unsigned char *)&first)[1] = *(const unsigned
char *)buf;
Quoted 10 lines
lib/net/rte_ip.h +6 -0
+		sum += first;
+		buf = (const void *)((uintptr_t)buf + 1);
+		len--;
+	}
 
+	u16_buf = (const u16_p *)buf;
+	end = u16_buf + len / sizeof(*u16_buf);
 	for (; u16_buf != end; ++u16_buf)
 		sum += *u16_buf;
 
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
--
2.17.1
@Emil, can you please test this patch with an unaligned
buffer on
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
your
Quoted 1 line
application to confirm that it produces the expected result.
Hi!
I tested the patch. It doesn't seem to produce the same
results. I
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
think the problem is that it always starts summing from an even address, the sum should always start from the first byte
according
Quoted 2 lines
Quoted 2 lines
to
Quoted 1 line
Quoted 1 line
the checksum specification. Can I instead propose something
Mattias
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Rönnblom sent me?
I assume that it produces the same result when the "buf"
parameter is
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines
aligned?
And when the "buf" parameter is unaligned, I don't expect it to
produce the
Quoted 3 lines
same results as the simple algorithm!
This was the whole point of the patch: I expect the overall
packet
Quoted 2 lines
Quoted 2 lines
buffer to
Quoted 1 line
be 16 bit aligned, and the checksum to be a partial checksum of
such
Quoted 2 lines
Quoted 2 lines
a 16 bit
Quoted 1 line
aligned packet buffer. When calling this function, I assume that
the
Quoted 2 lines
Quoted 2 lines
"buf" and
Quoted 1 line
"len" parameters point to a part of such a packet buffer. If
these
Quoted 1 line
Quoted 1 line
Quoted 1 line
expectations are correct, the simple algorithm will produce
incorrect
Quoted 6 lines
Quoted 6 lines
results
Quoted 3 lines
when "buf" is unaligned.
I was asking you to test if the checksum on the packet is correct
when your
Quoted 1 line
application modifies an unaligned part of the packet and uses
this
Quoted 5 lines
Quoted 5 lines
function to
Quoted 2 lines
update the checksum.
Now I understand your use case. Your use case seems to be about
partial
Quoted 4 lines
Quoted 4 lines
checksums, of which some partial checksums may start on unaligned addresses in an otherwise aligned packet.
Our use case is about calculating the full checksum on a nested
packet.
Quoted 3 lines
Quoted 3 lines
That nested packet may start on unaligned addresses.
The difference is basically if we want to sum over aligned
addresses or
Quoted 3 lines
Quoted 3 lines
not, handling the heading and trailing bytes appropriately.
Your method does not work in our case since we want to treat the
first
Quoted 1 line
Quoted 1 line
two bytes as the first word in our case. But I do understand that
both
Quoted 3 lines
Quoted 1 line
methods are useful.
Yes, that certainly are two different use cases, requiring two
different ways of calculating the 16 bit checksum.
Quoted 3 lines
Quoted 3 lines
Note that your method breaks the API. Previously (assuming no
crashing
Quoted 1 line
Quoted 1 line
due to low optimization levels, more accepting hardware, or a
different
Quoted 5 lines
Quoted 3 lines
compiler (version)) the current method would calculate the checksum assuming the first two bytes is the first word.
Depending on the point of view, my patch either fixes a bug (where
the checksum was calculated incorrectly when the buf pointer was unaligned) or breaks the API (by calculating the differently when the buffer is unaligned).
Quoted 2 lines
I cannot say with certainty which one is correct, but perhaps some of
the people with a deeper DPDK track record can...
Quoted 2 lines
@Bruce and @Stephen, in 2019 you signed off on a patch [1]
introducing a 16 bit alignment requirement to the Ethernet address structure.
Quoted 2 lines
It is my understanding that DPDK has an invariant requiring packets
to be 16 bit aligned, which that patch supports. Is this invariant documented anywhere, or am I completely wrong? If I'm wrong, then the alignment requirement introduced in that patch needs to be removed, as well as any similar alignment requirements elsewhere in DPDK.
I don't believe it is explicitly documented as a global invariant, but I
think it should be unless there is a definite case where we need to allow
packets to be completely unaligned. Across all packet headers we looked at,
there was no tunneling protocol where the resulting packet was left unaligned.

Quoted 9 lines

That said, if there are real use cases where we need to allow packets to
start at an unaligned address, then I agree with you that we need to roll
back the patch and work to ensure everything works with unaligned addresses.
/Bruce

@Emil, can you please describe or refer to which tunneling protocol you are using, where the nested packet can be unaligned?

I am asking to determine if your use case is exotic (maybe some Ericsson proprietary protocol), or more generic (rooted in some standard protocol). This information affects the DPDK community's opinion about how it should be supported by DPDK.

If possible, please provide more details about the tunneling protocol and nested packets... E.g. do the nested packets also contain Layer 2 (Ethernet, VLAN, etc.) headers, or only Layer 3 (IP) or Layer 4 (TCP, UDP, etc.)? And how about ARP packets and Layer 2 control protocol packets (STP, LACP, etc.)?

Quoted 13 lines

Quoted 2 lines
[1]
http://git.dpdk.org/dpdk/commit/lib/librte_net/rte_ether.h?id=da5350ef2
9afd35c1adabe76f60832f3092269ad
Quoted 2 lines
@Emil, we should wait for a conclusion about the alignment invariant
before we proceed.
Quoted 2 lines
If there is no such invariant, my patch is wrong, and we need to
provide a v2 of the patch, which will then fit your use case.
Quoted 1 line
If there is such an invariant, my patch is correct, and another
function must be added for your use case.
Quoted 1 line

Emil Berg

2022-06-22 06:01 UTC #12 lore raw

RE: [PATCH] net: fix checksum with unaligned buffer

Quoted 58 lines

-----Original Message-----
From: Morten Brørup <[email protected]>
Sent: den 17 juni 2022 11:07
To: Emil Berg <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: RE: [PATCH] net: fix checksum with unaligned buffer
Quoted 29 lines
From: Morten Brørup [mailto:[email protected]]
Sent: Friday, 17 June 2022 10.45
With this patch, the checksum can be calculated on an unligned part of a packet buffer.
I.e. the buf parameter is no longer required to be 16 bit aligned.
The DPDK invariant that packet buffers must be 16 bit aligned remains unchanged.
This invariant also defines how to calculate the 16 bit checksum on an unaligned part of a packet buffer.
Bugzilla ID: 1035
Cc: [email protected]
Signed-off-by Morten Brørup <[email protected]>
---
 lib/net/rte_ip.h | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)
lib/net/rte_ip.h index renamed from lib/net/rte_ip.h +0 -0
b502481670..8e301d9c26 100644
lib/net/rte_ip.h +0 -1
@@ -162,9 +162,22 @@ __rte_raw_cksum(const void *buf, size_t len,
 uint32_t sum)  {
 	/* extend strict-aliasing rules */
 	typedef uint16_t __attribute__((__may_alias__)) u16_p;
-	const u16_p *u16_buf = (const u16_p *)buf;
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, size_t len, · line 173
Quoted 10 lines
lib/net/rte_ip.h +9 -1
-	const u16_p *end = u16_buf + len / sizeof(*u16_buf);
+	const u16_p *u16_buf;
+	const u16_p *end;
+
+	/* if buffer is unaligned, keeping it byte order independent */
+	if (unlikely((uintptr_t)buf & 1)) {
+		uint16_t first = 0;
+		if (unlikely(len == 0))
+			return 0;
+		((unsigned char *)&first)[1] = *(const unsigned
char *)buf;
Quoted 10 lines
lib/net/rte_ip.h +6 -0
+		sum += first;
+		buf = (const void *)((uintptr_t)buf + 1);
+		len--;
+	}
 
+	u16_buf = (const u16_p *)buf;
+	end = u16_buf + len / sizeof(*u16_buf);
 	for (; u16_buf != end; ++u16_buf)
 		sum += *u16_buf;
 

Quoted 5 lines

Quoted 2 lines
--
2.17.1
@Emil, can you please test this patch with an unaligned buffer on your application to confirm that it produces the expected result.

Hi!

I tested the patch. It doesn't seem to produce the same results. I think the problem is that it always starts summing from an even address, the sum should always start from the first byte according to the checksum specification. Can I instead propose something Mattias Rönnblom sent me?
--------------------------------------------------------------------------------------------------------------
const void *end = RTE_PTR_ADD(buf, (len / sizeof(uint16_t)) * sizeof(uint16_t));

for (; buf != end; buf = RTE_PTR_ADD(buf, sizeof(uint16_t))) {
    uint16_t v;
    memcpy(&v, buf, sizeof(uint16_t));
    sum += v;
}

/* if length is odd, keeping it byte order independent */

if (unlikely(len % 2)) {
    uint16_t left = 0;
    *(unsigned char *)&left = *(const unsigned char *)end;
    sum += left;
}

--------------------------------------------------------------------------------------------------------------
Note that the last block is the same as before. Amazingly I see no measurable performance hit from this compared to the previous one (-O3, march=native). Looking at the previous the loop body may compile to (x86):
--------------------------------------------------------------------------------------------------------------
vmovdqa (%rdx),%xmm1
vpmovzxwd %xmm1,%xmm0
vpsrldq $0x8,%xmm1,%xmm1
vpmovzxwd %xmm1,%xmm1
vpaddd %xmm1,%xmm0,%xmm0
cmp $0xf,%rax
jbe 0x7ff7a0dfb1a9
--------------------------------------------------------------------------------------------------------------
while Mattias' memcpy solution:
--------------------------------------------------------------------------------------------------------------
vmovdqu (%rcx),%ymm0
add $0x20,%rcx
vpmovzxwd %xmm0,%ymm1
vextracti128 $0x1,%ymm0,%xmm0
vpmovzxwd %xmm0,%ymm0
vpaddd %ymm0,%ymm1,%ymm0
vpaddd %ymm0,%ymm2,%ymm2
cmp %r9,%rcx
jne 0x555555556380
--------------------------------------------------------------------------------------------------------------
Thus two extra instructions in the loop, but I suspect it may be memory bound, leading to no measurable performance difference.

Any comments?

/Emil

Emil Berg

2022-06-22 06:01 UTC #14 lore raw

RE: [PATCH] net: fix checksum with unaligned buffer

Quoted 99 lines

-----Original Message-----
From: Morten Brørup <[email protected]>
Sent: den 20 juni 2022 12:58
To: Emil Berg <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: RE: [PATCH] net: fix checksum with unaligned buffer
Quoted 68 lines
From: Emil Berg [mailto:[email protected]]
Sent: Monday, 20 June 2022 12.38
Quoted 8 lines
From: Morten Brørup <[email protected]>
Sent: den 17 juni 2022 11:07
Quoted 5 lines
From: Morten Brørup [mailto:[email protected]]
Sent: Friday, 17 June 2022 10.45
With this patch, the checksum can be calculated on an unligned part
of
Quoted 4 lines
Quoted 4 lines
a packet buffer.
I.e. the buf parameter is no longer required to be 16 bit aligned.
The DPDK invariant that packet buffers must be 16 bit aligned
remains
Quoted 3 lines
Quoted 3 lines
unchanged.
This invariant also defines how to calculate the 16 bit checksum on
an
Quoted 40 lines
Quoted 19 lines
unaligned part of a packet buffer.
Bugzilla ID: 1035
Cc: [email protected]
Signed-off-by Morten Brørup <[email protected]>
---
 lib/net/rte_ip.h | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)
lib/net/rte_ip.h index renamed from lib/net/rte_ip.h +0 -0
b502481670..8e301d9c26 100644
lib/net/rte_ip.h +0 -1
@@ -162,9 +162,22 @@ __rte_raw_cksum(const void *buf, size_t len,
 uint32_t sum)  {
 	/* extend strict-aliasing rules */
 	typedef uint16_t __attribute__((__may_alias__)) u16_p;
-	const u16_p *u16_buf = (const u16_p *)buf;
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, size_t len, · line 173
Quoted 10 lines
lib/net/rte_ip.h +9 -1
-	const u16_p *end = u16_buf + len / sizeof(*u16_buf);
+	const u16_p *u16_buf;
+	const u16_p *end;
+
+	/* if buffer is unaligned, keeping it byte order independent */
+	if (unlikely((uintptr_t)buf & 1)) {
+		uint16_t first = 0;
+		if (unlikely(len == 0))
+			return 0;
+		((unsigned char *)&first)[1] = *(const unsigned
char *)buf;
Quoted 10 lines
lib/net/rte_ip.h +6 -0
+		sum += first;
+		buf = (const void *)((uintptr_t)buf + 1);
+		len--;
+	}
 
+	u16_buf = (const u16_p *)buf;
+	end = u16_buf + len / sizeof(*u16_buf);
 	for (; u16_buf != end; ++u16_buf)
 		sum += *u16_buf;
 
Quoted 4 lines
Quoted 2 lines
--
2.17.1
@Emil, can you please test this patch with an unaligned buffer on
your
Quoted 1 line
application to confirm that it produces the expected result.
Quoted 7 lines
Hi!
I tested the patch. It doesn't seem to produce the same results. I think the problem is that it always starts summing from an even address, the sum should always start from the first byte according to the checksum specification. Can I instead propose something Mattias Rönnblom sent me?
I assume that it produces the same result when the "buf" parameter is aligned?
And when the "buf" parameter is unaligned, I don't expect it to produce the same results as the simple algorithm!
This was the whole point of the patch: I expect the overall packet buffer to be 16 bit aligned, and the checksum to be a partial checksum of such a 16 bit aligned packet buffer. When calling this function, I assume that the "buf" and "len" parameters point to a part of such a packet buffer. If these expectations are correct, the simple algorithm will produce incorrect results when "buf" is unaligned.

Quoted 4 lines

I was asking you to test if the checksum on the packet is correct when your application modifies an unaligned part of the packet and uses this function to update the checksum.

Now I understand your use case. Your use case seems to be about partial checksums, of which some partial checksums may start on unaligned addresses in an otherwise aligned packet.

Our use case is about calculating the full checksum on a nested packet. That nested packet may start on unaligned addresses.

The difference is basically if we want to sum over aligned addresses or not, handling the heading and trailing bytes appropriately.

Your method does not work in our case since we want to treat the first two bytes as the first word in our case. But I do understand that both methods are useful.

Note that your method breaks the API. Previously (assuming no crashing due to low optimization levels, more accepting hardware, or a different compiler (version)) the current method would calculate the checksum assuming the first two bytes is the first word.

Quoted 62 lines

Quoted 62 lines
----------------------------------------------------------------------
-
---------------------------------------
const void *end = RTE_PTR_ADD(buf, (len / sizeof(uint16_t)) * sizeof(uint16_t));
for (; buf != end; buf = RTE_PTR_ADD(buf, sizeof(uint16_t))) {
    uint16_t v;
    memcpy(&v, buf, sizeof(uint16_t));
    sum += v;
}
/* if length is odd, keeping it byte order independent */ if
(unlikely(len % 2)) {
    uint16_t left = 0;
    *(unsigned char *)&left = *(const unsigned char *)end;
    sum += left;
}
----------------------------------------------------------------------
-
---------------------------------------
Note that the last block is the same as before. Amazingly I see no measurable performance hit from this compared to the previous one (-O3, march=native). Looking at the previous the loop body may compile to
(x86):
----------------------------------------------------------------------
-
---------------------------------------
vmovdqa (%rdx),%xmm1
vpmovzxwd %xmm1,%xmm0
vpsrldq $0x8,%xmm1,%xmm1
vpmovzxwd %xmm1,%xmm1
vpaddd %xmm1,%xmm0,%xmm0
cmp $0xf,%rax
jbe 0x7ff7a0dfb1a9
----------------------------------------------------------------------
-
---------------------------------------
while Mattias' memcpy solution:
----------------------------------------------------------------------
-
---------------------------------------
vmovdqu (%rcx),%ymm0
add $0x20,%rcx
vpmovzxwd %xmm0,%ymm1
vextracti128 $0x1,%ymm0,%xmm0
vpmovzxwd %xmm0,%ymm0
vpaddd %ymm0,%ymm1,%ymm0
vpaddd %ymm0,%ymm2,%ymm2
cmp %r9,%rcx
jne 0x555555556380
----------------------------------------------------------------------
-
---------------------------------------
Thus two extra instructions in the loop, but I suspect it may be memory bound, leading to no measurable performance difference.
Any comments?
/Emil

Bruce Richardson

2022-06-22 09:18 UTC #19 lore raw

Re: [PATCH] net: fix checksum with unaligned buffer

[2] https://bugs.dpdk.org/show_bug.cgi?id=1035

Quoted 218 lines · Emil Berg

Quoted 218 lines
-----Original Message-----
From: Morten Brørup <[email protected]>
Sent: den 21 juni 2022 11:35
To: Emil Berg <[email protected]>
Cc: Bruce Richardson <[email protected]>; Stephen Hemminger <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: RE: [PATCH] net: fix checksum with unaligned buffer
Quoted 191 lines
From: Bruce Richardson [mailto:[email protected]]
Sent: Tuesday, 21 June 2022 10.23
Quoted 1 line · Morten Brørup
+TO: @Bruce and @Stephen: You signed off on the 16 bit alignment
requirement. We need background info on this.
Quoted 10 lines
Quoted 10 lines
From: Emil Berg [mailto:[email protected]]
Sent: Tuesday, 21 June 2022 09.17
Quoted 6 lines
From: Morten Brørup <[email protected]>
Sent: den 20 juni 2022 12:58
Quoted 3 lines
From: Emil Berg [mailto:[email protected]]
Sent: Monday, 20 June 2022 12.38
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
From: Morten Brørup <[email protected]>
Sent: den 17 juni 2022 11:07
Quoted 4 lines
From: Morten Brørup [mailto:[email protected]]
Sent: Friday, 17 June 2022 10.45
With this patch, the checksum can be calculated on an
unligned
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
Quoted 1 line
part
of
Quoted 2 lines
Quoted 2 lines
a packet buffer.
I.e. the buf parameter is no longer required to be 16 bit
aligned.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
The DPDK invariant that packet buffers must be 16 bit
aligned
Quoted 16 lines
Quoted 16 lines
Quoted 3 lines
Quoted 3 lines
remains
Quoted 2 lines
Quoted 2 lines
unchanged.
This invariant also defines how to calculate the 16 bit
checksum
Quoted 12 lines
Quoted 12 lines
Quoted 1 line
Quoted 1 line
on
an
Quoted 10 lines
Quoted 10 lines
unaligned part of a packet buffer.
Bugzilla ID: 1035
Cc: [email protected]
Signed-off-by Morten Brørup <[email protected]>
---
 lib/net/rte_ip.h | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
lib/net/rte_ip.h __rte_raw_cksum(const void *buf,
Quoted 5 lines
lib/net/rte_ip.h index renamed from lib/net/rte_ip.h +0 -0
b502481670..8e301d9c26 100644
lib/net/rte_ip.h +0 -0
@@ -162,9 +162,22 @@ __rte_raw_cksum(const void *buf,
size_t
Quoted 4 lines
Quoted 4 lines
len,
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
lib/net/rte_ip.h +0 -0
 uint32_t sum)  {
 	/* extend strict-aliasing rules */
 	typedef uint16_t __attribute__((__may_alias__))
u16_p;
Quoted 23 lines
Quoted 23 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, · line 168
Quoted 6 lines
lib/net/rte_ip.h +4 -2
-	const u16_p *u16_buf = (const u16_p *)buf;
-	const u16_p *end = u16_buf + len / sizeof(*u16_buf);
+	const u16_p *u16_buf;
+	const u16_p *end;
+
+	/* if buffer is unaligned, keeping it byte order
independent */
Quoted 16 lines
Quoted 16 lines
Quoted 16 lines
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, · line 173
Quoted 5 lines
lib/net/rte_ip.h +5 -0
+	if (unlikely((uintptr_t)buf & 1)) {
+		uint16_t first = 0;
+		if (unlikely(len == 0))
+			return 0;
+		((unsigned char *)&first)[1] = *(const unsigned
char *)buf;
Quoted 10 lines
lib/net/rte_ip.h +6 -0
+		sum += first;
+		buf = (const void *)((uintptr_t)buf + 1);
+		len--;
+	}
 
+	u16_buf = (const u16_p *)buf;
+	end = u16_buf + len / sizeof(*u16_buf);
 	for (; u16_buf != end; ++u16_buf)
 		sum += *u16_buf;
 
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
--
2.17.1
@Emil, can you please test this patch with an unaligned
buffer on
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
your
Quoted 1 line
application to confirm that it produces the expected result.
Hi!
I tested the patch. It doesn't seem to produce the same
results. I
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
think the problem is that it always starts summing from an even address, the sum should always start from the first byte
according
Quoted 2 lines
Quoted 2 lines
to
Quoted 1 line
Quoted 1 line
the checksum specification. Can I instead propose something
Mattias
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Rönnblom sent me?
I assume that it produces the same result when the "buf"
parameter is
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines
aligned?
And when the "buf" parameter is unaligned, I don't expect it to
produce the
Quoted 3 lines
same results as the simple algorithm!
This was the whole point of the patch: I expect the overall
packet
Quoted 2 lines
Quoted 2 lines
buffer to
Quoted 1 line
be 16 bit aligned, and the checksum to be a partial checksum of
such
Quoted 2 lines
Quoted 2 lines
a 16 bit
Quoted 1 line
aligned packet buffer. When calling this function, I assume that
the
Quoted 2 lines
Quoted 2 lines
"buf" and
Quoted 1 line
"len" parameters point to a part of such a packet buffer. If
these
Quoted 1 line
Quoted 1 line
Quoted 1 line
expectations are correct, the simple algorithm will produce
incorrect
Quoted 7 lines
Quoted 7 lines
results
Quoted 4 lines
when "buf" is unaligned.
I was asking you to test if the checksum on the packet is correct
when your
Quoted 1 line
application modifies an unaligned part of the packet and uses
this
Quoted 5 lines
Quoted 5 lines
function to
Quoted 2 lines
update the checksum.
Now I understand your use case. Your use case seems to be about
partial
Quoted 4 lines
Quoted 4 lines
checksums, of which some partial checksums may start on unaligned addresses in an otherwise aligned packet.
Our use case is about calculating the full checksum on a nested
packet.
Quoted 3 lines
Quoted 3 lines
That nested packet may start on unaligned addresses.
The difference is basically if we want to sum over aligned
addresses or
Quoted 3 lines
Quoted 3 lines
not, handling the heading and trailing bytes appropriately.
Your method does not work in our case since we want to treat the
first
Quoted 1 line
Quoted 1 line
two bytes as the first word in our case. But I do understand that
both
Quoted 3 lines
Quoted 1 line
methods are useful.
Yes, that certainly are two different use cases, requiring two
different ways of calculating the 16 bit checksum.
Quoted 3 lines
Quoted 3 lines
Note that your method breaks the API. Previously (assuming no
crashing
Quoted 1 line
Quoted 1 line
due to low optimization levels, more accepting hardware, or a
different
Quoted 5 lines
Quoted 3 lines
compiler (version)) the current method would calculate the checksum assuming the first two bytes is the first word.
Depending on the point of view, my patch either fixes a bug (where
the checksum was calculated incorrectly when the buf pointer was unaligned) or breaks the API (by calculating the differently when the buffer is unaligned).
Quoted 3 lines
I cannot say with certainty which one is correct, but perhaps some of
the people with a deeper DPDK track record can...
Quoted 2 lines
@Bruce and @Stephen, in 2019 you signed off on a patch [1]
introducing a 16 bit alignment requirement to the Ethernet address structure.
Quoted 2 lines
It is my understanding that DPDK has an invariant requiring packets
to be 16 bit aligned, which that patch supports. Is this invariant documented anywhere, or am I completely wrong? If I'm wrong, then the alignment requirement introduced in that patch needs to be removed, as well as any similar alignment requirements elsewhere in DPDK.
I don't believe it is explicitly documented as a global invariant, but I think it should be unless there is a definite case where we need to allow packets to be completely unaligned. Across all packet headers we looked at, there was no tunneling protocol where the resulting packet was left unaligned.
Quoted 7 lines
That said, if there are real use cases where we need to allow packets to start at an unaligned address, then I agree with you that we need to roll back the patch and work to ensure everything works with unaligned addresses.
/Bruce
@Emil, can you please describe or refer to which tunneling protocol you are using, where the nested packet can be unaligned?
I am asking to determine if your use case is exotic (maybe some Ericsson proprietary protocol), or more generic (rooted in some standard protocol). This information affects the DPDK community's opinion about how it should be supported by DPDK.

Quoted 8 lines

Quoted 5 lines
If possible, please provide more details about the tunneling protocol and nested packets... E.g. do the nested packets also contain Layer 2 (Ethernet, VLAN, etc.) headers, or only Layer 3 (IP) or Layer 4 (TCP, UDP, etc.)? And how about ARP packets and Layer 2 control protocol packets (STP, LACP, etc.)?
Well, if you append or adjust an odd number of bytes (e.g. a PDCP header) from a previously aligned payload the entire packet will then be unaligned.

If PDCP headers can leave the rest of the packet field unaligned, then we had better remove the alignment restrictions through all of DPDK.

/Bruce

Morten Brørup

2022-06-22 11:26 UTC #20 lore raw

RE: [PATCH] net: fix checksum with unaligned buffer

Quoted 270 lines

From: Bruce Richardson [mailto:[email protected]]
Sent: Wednesday, 22 June 2022 11.18
Quoted 9 lines · Emil Berg
Quoted 9 lines
From: Morten Brørup <[email protected]>
Sent: den 21 juni 2022 11:35
Quoted 5 lines
From: Bruce Richardson [mailto:[email protected]]
Sent: Tuesday, 21 June 2022 10.23
Quoted 1 line · Morten Brørup
+TO: @Bruce and @Stephen: You signed off on the 16 bit
alignment
Quoted 17 lines
Quoted 17 lines
Quoted 17 lines
requirement. We need background info on this.
Quoted 16 lines
Quoted 16 lines
From: Emil Berg [mailto:[email protected]]
Sent: Tuesday, 21 June 2022 09.17
Quoted 12 lines
From: Morten Brørup <[email protected]>
Sent: den 20 juni 2022 12:58
Quoted 9 lines
From: Emil Berg [mailto:[email protected]]
Sent: Monday, 20 June 2022 12.38
Quoted 6 lines
From: Morten Brørup <[email protected]>
Sent: den 17 juni 2022 11:07
Quoted 3 lines
From: Morten Brørup [mailto:[email protected]]
Sent: Friday, 17 June 2022 10.45
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
With this patch, the checksum can be calculated on an
unligned
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
Quoted 1 line
part
of
Quoted 2 lines
Quoted 2 lines
a packet buffer.
I.e. the buf parameter is no longer required to be 16
bit
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
aligned.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
The DPDK invariant that packet buffers must be 16 bit
aligned
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
remains
Quoted 2 lines
Quoted 2 lines
unchanged.
This invariant also defines how to calculate the 16
bit
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
checksum
Quoted 8 lines
Quoted 8 lines
Quoted 1 line
Quoted 1 line
on
an
Quoted 6 lines
Quoted 6 lines
unaligned part of a packet buffer.
Bugzilla ID: 1035
Cc: [email protected]
Signed-off-by Morten Brørup
<[email protected]>
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
lib/net/rte_ip.h
Quoted 5 lines
---
 lib/net/rte_ip.h | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)
lib/net/rte_ip.h +0 -0
index
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
lib/net/rte_ip.h __rte_raw_cksum(const void *buf,
Quoted 4 lines
b502481670..8e301d9c26 100644
lib/net/rte_ip.h +0 -0
@@ -162,9 +162,22 @@ __rte_raw_cksum(const void *buf,
size_t
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
Quoted 4 lines
len,
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
lib/net/rte_ip.h +0 -0
 uint32_t sum)  {
 	/* extend strict-aliasing rules */
 	typedef uint16_t __attribute__((__may_alias__))
u16_p;
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, · line 166
Quoted 2 lines
lib/net/rte_ip.h +0 -2
-	const u16_p *u16_buf = (const u16_p *)buf;
-	const u16_p *end = u16_buf + len /
sizeof(*u16_buf);
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, · line 168
Quoted 4 lines
lib/net/rte_ip.h +4 -0
+	const u16_p *u16_buf;
+	const u16_p *end;
+
+	/* if buffer is unaligned, keeping it byte
order
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
independent */
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, · line 173
Quoted 5 lines
lib/net/rte_ip.h +5 -0
+	if (unlikely((uintptr_t)buf & 1)) {
+		uint16_t first = 0;
+		if (unlikely(len == 0))
+			return 0;
+		((unsigned char *)&first)[1] = *(const
unsigned
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
char *)buf;
Quoted 10 lines
lib/net/rte_ip.h +6 -0
+		sum += first;
+		buf = (const void *)((uintptr_t)buf + 1);
+		len--;
+	}
 
+	u16_buf = (const u16_p *)buf;
+	end = u16_buf + len / sizeof(*u16_buf);
 	for (; u16_buf != end; ++u16_buf)
 		sum += *u16_buf;
 
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
--
2.17.1
@Emil, can you please test this patch with an unaligned
buffer on
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
your
Quoted 1 line
application to confirm that it produces the expected
result.
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Hi!
I tested the patch. It doesn't seem to produce the same
results. I
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
think the problem is that it always starts summing from
an
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
even address, the sum should always start from the first
byte
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
according
Quoted 2 lines
Quoted 2 lines
to
Quoted 1 line
Quoted 1 line
the checksum specification. Can I instead propose
something
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Mattias
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Rönnblom sent me?
I assume that it produces the same result when the "buf"
parameter is
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
aligned?
And when the "buf" parameter is unaligned, I don't expect
it to
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
Quoted 4 lines
produce the
Quoted 3 lines
same results as the simple algorithm!
This was the whole point of the patch: I expect the overall
packet
Quoted 2 lines
Quoted 2 lines
buffer to
Quoted 1 line
be 16 bit aligned, and the checksum to be a partial
checksum of
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
such
Quoted 2 lines
Quoted 2 lines
a 16 bit
Quoted 1 line
aligned packet buffer. When calling this function, I assume
that
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
the
Quoted 2 lines
Quoted 2 lines
"buf" and
Quoted 1 line
"len" parameters point to a part of such a packet buffer.
If
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
these
Quoted 1 line
Quoted 1 line
Quoted 1 line
expectations are correct, the simple algorithm will produce
incorrect
Quoted 7 lines
Quoted 7 lines
results
Quoted 4 lines
when "buf" is unaligned.
I was asking you to test if the checksum on the packet is correct
when your
Quoted 1 line
application modifies an unaligned part of the packet and
uses
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
this
Quoted 5 lines
Quoted 5 lines
function to
Quoted 2 lines
update the checksum.
Now I understand your use case. Your use case seems to be
about
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
partial
Quoted 1 line
Quoted 1 line
checksums, of which some partial checksums may start on
unaligned
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
addresses in an otherwise aligned packet.
Our use case is about calculating the full checksum on a
nested
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
packet.
Quoted 3 lines
Quoted 3 lines
That nested packet may start on unaligned addresses.
The difference is basically if we want to sum over aligned
addresses or
Quoted 3 lines
Quoted 3 lines
not, handling the heading and trailing bytes appropriately.
Your method does not work in our case since we want to treat
the
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
first
Quoted 1 line
Quoted 1 line
two bytes as the first word in our case. But I do understand
that
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
both
Quoted 3 lines
Quoted 1 line
methods are useful.
Yes, that certainly are two different use cases, requiring two
different ways of calculating the 16 bit checksum.
Quoted 3 lines
Quoted 3 lines
Note that your method breaks the API. Previously (assuming no
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
crashing
Quoted 1 line
Quoted 1 line
due to low optimization levels, more accepting hardware, or a
different
Quoted 5 lines
Quoted 3 lines
compiler (version)) the current method would calculate the checksum assuming the first two bytes is the first word.
Depending on the point of view, my patch either fixes a bug
(where
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
the checksum was calculated incorrectly when the buf pointer was unaligned) or breaks the API (by calculating the differently when
the
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
buffer is unaligned).
Quoted 2 lines
I cannot say with certainty which one is correct, but perhaps
some
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
of
the people with a deeper DPDK track record can...
Quoted 2 lines
@Bruce and @Stephen, in 2019 you signed off on a patch [1]
introducing a 16 bit alignment requirement to the Ethernet
address
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
structure.
Quoted 2 lines
It is my understanding that DPDK has an invariant requiring
packets
Quoted 1 line
Quoted 1 line
Quoted 1 line
to be 16 bit aligned, which that patch supports. Is this
invariant
Quoted 1 line
Quoted 1 line
Quoted 1 line
documented anywhere, or am I completely wrong? If I'm wrong, then
the
Quoted 1 line
Quoted 1 line
Quoted 1 line
alignment requirement introduced in that patch needs to be
removed, as
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
well as any similar alignment requirements elsewhere in DPDK.
I don't believe it is explicitly documented as a global
invariant, but
Quoted 1 line
Quoted 1 line
Quoted 1 line
I think it should be unless there is a definite case where we
need to
Quoted 1 line
Quoted 1 line
Quoted 1 line
allow packets to be completely unaligned. Across all packet
headers we
Quoted 1 line
Quoted 1 line
Quoted 1 line
looked at, there was no tunneling protocol where the resulting
packet
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
was left unaligned.
That said, if there are real use cases where we need to allow
packets
Quoted 1 line
Quoted 1 line
Quoted 1 line
to start at an unaligned address, then I agree with you that we
need
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
to roll back the patch and work to ensure everything works with unaligned addresses.
/Bruce
@Emil, can you please describe or refer to which tunneling protocol
you are
Quoted 3 lines
Quoted 3 lines
using, where the nested packet can be unaligned?
I am asking to determine if your use case is exotic (maybe some
Ericsson
Quoted 1 line
Quoted 1 line
proprietary protocol), or more generic (rooted in some standard
protocol).
Quoted 1 line
Quoted 1 line
This information affects the DPDK community's opinion about how it
should
Quoted 3 lines
Quoted 3 lines
be supported by DPDK.
If possible, please provide more details about the tunneling
protocol and
Quoted 1 line
Quoted 1 line
nested packets... E.g. do the nested packets also contain Layer 2
(Ethernet,
Quoted 1 line
Quoted 1 line
VLAN, etc.) headers, or only Layer 3 (IP) or Layer 4 (TCP, UDP,
etc.)? And how
Quoted 1 line
Quoted 1 line
about ARP packets and Layer 2 control protocol packets (STP, LACP,
etc.)?
Quoted 3 lines
Quoted 1 line
Well, if you append or adjust an odd number of bytes (e.g. a PDCP
header) from a previously aligned payload the entire packet will then be unaligned.
Quoted 1 line

Quoted 5 lines

If PDCP headers can leave the rest of the packet field unaligned, then we
had better remove the alignment restrictions through all of DPDK.
/Bruce

Re-reading the details regarding unaligned pointers in C11, as posted by Emil in Bugzilla [2], I interpret it as follows: Any 16 bit or wider pointer type a must point to data aligned with that type, i.e. a pointer of the type "uint16_t *" must point to 16 bit aligned data, and a pointer of the type "uint64_t *" must point to 64 bit aligned data. Please, someone tell me I got this wrong, and wake me up from my nightmare!

Updating DPDK's packet structures to fully support this C11 limitation with unaligned access would be a nightmare, as we would need to use byte arrays for all structure fields. Functions would also be unable to use other pointer types than "void *" and "char *", which seems to be the actual problem in the __rte_raw_cksum() function. I guess that it also would prevent the compiler from auto-vectorizing the functions.

I am usually a big proponent of academically correct solutions, but such a change would be too wide ranging, so I would like to narrow it down to the actual use case, and perhaps extrapolate a bit from there.

@Emil: Do you only need to calculate the checksum of the (potentially unaligned) embedded packet? Or do you also need to use other DPDK functions with the embedded packet, potentially accessing it at an unaligned address?

I'm trying to determine the scope of this C11 pointer alignment limitation for your use case, i.e. whether or not other DPDK functions need to be updated to support unaligned packet access too.

Morten Brørup

2022-06-22 13:45 UTC #28 lore raw

PATCH v2 1/1 net: fix checksum with unaligned buffer

With this patch, the checksum can be calculated on an unligned buffer. I.e. the buf parameter is no longer required to be 16 bit aligned.

The checksum is still calculated using a 16 bit aligned pointer, so the compiler can auto-vectorize the function's inner loop.

When the buffer is unaligned, the first byte of the buffer is handled separately. Futhermore, the calculated checksum of the buffer is byte shifted before being added to the initial checksum, to compensate for the checksum having been calculated on the buffer shifted by one byte.

v2:
* Do not assume that the buffer is part of an aligned packet buffer.

Bugzilla ID: 1035
Cc: [email protected]

---
 lib/net/rte_ip.h | 31 ++++++++++++++++++++++++++-----
 1 file changed, 26 insertions(+), 5 deletions(-)

lib/net/rte_ip.h +26 -5

index b502481670..ad46bdb443 100644
@@ -162,20 +162,41 @@ __rte_raw_cksum(const void *buf, size_t len, uint32_t sum)
 {
 	/* extend strict-aliasing rules */
 	typedef uint16_t __attribute__((__may_alias__)) u16_p;
-	const u16_p *u16_buf = (const u16_p *)buf;
-	const u16_p *end = u16_buf + len / sizeof(*u16_buf);
+	const u16_p *u16_buf;
+	const u16_p *end;
+	uint32_t bsum = 0;
+	const bool unaligned = (uintptr_t)buf & 1;
+
+	/* if buffer is unaligned, keeping it byte order independent */
+	if (unlikely(unaligned)) {
+		uint16_t first = 0;
+		if (unlikely(len == 0))
+			return 0;
+		((unsigned char *)&first)[1] = *(const unsigned char *)buf;
+		bsum += first;
+		buf = (const void *)((uintptr_t)buf + 1);
+		len--;
+	}
 
+	/* aligned access for compiler auto-vectorization */
+	u16_buf = (const u16_p *)buf;
+	end = u16_buf + len / sizeof(*u16_buf);
 	for (; u16_buf != end; ++u16_buf)
-		sum += *u16_buf;
+		bsum += *u16_buf;
 
 	/* if length is odd, keeping it byte order independent */
 	if (unlikely(len % 2)) {
 		uint16_t left = 0;
 		*(unsigned char *)&left = *(const unsigned char *)end;
-		sum += left;
+		bsum += left;
+	}
+
+	/* if buffer is unaligned, swap the checksum bytes */
+	if (unlikely(unaligned)) {
+		bsum = (bsum & 0xFF00FF00) >> 8 | (bsum & 0x00FF00FF) << 8;
 	}
 
-	return sum;
+	return sum + bsum;
 }
 
 /**

Signature · 1 line

2.17.1

Morten Brørup

2022-06-22 13:54 UTC #29 lore raw

PATCH v3 1/1 net: fix checksum with unaligned buffer

With this patch, the checksum can be calculated on an unligned buffer. I.e. the buf parameter is no longer required to be 16 bit aligned.

The checksum is still calculated using a 16 bit aligned pointer, so the compiler can auto-vectorize the function's inner loop.

When the buffer is unaligned, the first byte of the buffer is handled separately. Furthermore, the calculated checksum of the buffer is byte shifted before being added to the initial checksum, to compensate for the checksum having been calculated on the buffer shifted by one byte.

v3:
* Remove braces from single statement block.
* Fix typo in commit message.
v2:
* Do not assume that the buffer is part of an aligned packet buffer.

Bugzilla ID: 1035
Cc: [email protected]

---
 lib/net/rte_ip.h | 30 +++++++++++++++++++++++++-----
 1 file changed, 25 insertions(+), 5 deletions(-)

lib/net/rte_ip.h +25 -5

index b502481670..3fad448085 100644
@@ -162,20 +162,40 @@ __rte_raw_cksum(const void *buf, size_t len, uint32_t sum)
 {
 	/* extend strict-aliasing rules */
 	typedef uint16_t __attribute__((__may_alias__)) u16_p;
-	const u16_p *u16_buf = (const u16_p *)buf;
-	const u16_p *end = u16_buf + len / sizeof(*u16_buf);
+	const u16_p *u16_buf;
+	const u16_p *end;
+	uint32_t bsum = 0;
+	const bool unaligned = (uintptr_t)buf & 1;
+
+	/* if buffer is unaligned, keeping it byte order independent */
+	if (unlikely(unaligned)) {
+		uint16_t first = 0;
+		if (unlikely(len == 0))
+			return 0;
+		((unsigned char *)&first)[1] = *(const unsigned char *)buf;
+		bsum += first;
+		buf = (const void *)((uintptr_t)buf + 1);
+		len--;
+	}
 
+	/* aligned access for compiler auto-vectorization */
+	u16_buf = (const u16_p *)buf;
+	end = u16_buf + len / sizeof(*u16_buf);
 	for (; u16_buf != end; ++u16_buf)
-		sum += *u16_buf;
+		bsum += *u16_buf;
 
 	/* if length is odd, keeping it byte order independent */
 	if (unlikely(len % 2)) {
 		uint16_t left = 0;
 		*(unsigned char *)&left = *(const unsigned char *)end;
-		sum += left;
+		bsum += left;
 	}
 
-	return sum;
+	/* if buffer is unaligned, swap the checksum bytes */
+	if (unlikely(unaligned))
+		bsum = (bsum & 0xFF00FF00) >> 8 | (bsum & 0x00FF00FF) << 8;
+
+	return sum + bsum;
 }
 
 /**

Signature · 1 line

2.17.1

Morten Brørup

2022-06-22 14:01 UTC #22 lore raw

RE: [PATCH] net: fix checksum with unaligned buffer

Quoted 381 lines

From: Emil Berg [mailto:[email protected]]
Sent: Wednesday, 22 June 2022 14.25
Quoted 15 lines
From: Morten Brørup <[email protected]>
Sent: den 22 juni 2022 13:26
Quoted 12 lines
From: Bruce Richardson [mailto:[email protected]]
Sent: Wednesday, 22 June 2022 11.18
Quoted 8 lines · Emil Berg
Quoted 8 lines
From: Morten Brørup <[email protected]>
Sent: den 21 juni 2022 11:35
Quoted 4 lines
From: Bruce Richardson [mailto:[email protected]]
Sent: Tuesday, 21 June 2022 10.23
On Tue, Jun 21, 2022 at 10:05:15AM +0200, Morten Brørup
Quoted 13 lines · wrote:
Quoted 13 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
+TO: @Bruce and @Stephen: You signed off on the 16 bit
alignment
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
requirement. We need background info on this.
Quoted 10 lines
Quoted 10 lines
From: Emil Berg [mailto:[email protected]]
Sent: Tuesday, 21 June 2022 09.17
Quoted 6 lines
From: Morten Brørup <[email protected]>
Sent: den 20 juni 2022 12:58
Quoted 3 lines
From: Emil Berg [mailto:[email protected]]
Sent: Monday, 20 June 2022 12.38
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
From: Morten Brørup <[email protected]>
Sent: den 17 juni 2022 11:07
Quoted 5 lines
From: Morten Brørup
[mailto:[email protected]]
Sent: Friday, 17 June 2022 10.45
With this patch, the checksum can be calculated
on
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
an
unligned
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
Quoted 1 line
part
of
Quoted 2 lines
Quoted 2 lines
a packet buffer.
I.e. the buf parameter is no longer required to
be
Quoted 10 lines
Quoted 10 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
16
bit
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 4 lines
Quoted 4 lines
aligned.
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
The DPDK invariant that packet buffers must be 16
bit
aligned
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
remains
Quoted 2 lines
Quoted 2 lines
unchanged.
This invariant also defines how to calculate the
16
Quoted 44 lines
Quoted 9 lines
bit
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
checksum
Quoted 7 lines
Quoted 7 lines
Quoted 1 line
Quoted 1 line
on
an
Quoted 5 lines
Quoted 5 lines
unaligned part of a packet buffer.
Bugzilla ID: 1035
Cc: [email protected]
Quoted 18 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Signed-off-by Morten Brørup
<[email protected]>
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
lib/net/rte_ip.h
Quoted 5 lines
---
 lib/net/rte_ip.h | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)
lib/net/rte_ip.h +0 -0
index
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
lib/net/rte_ip.h __rte_raw_cksum(const void · line 162
Quoted 5 lines
b502481670..8e301d9c26 100644
lib/net/rte_ip.h +0 -0
@@ -162,9 +162,22 @@ __rte_raw_cksum(const void
 *buf,
size_t
Quoted 4 lines
Quoted 4 lines
len,
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
lib/net/rte_ip.h +0 -0
 uint32_t sum)  {
 	/* extend strict-aliasing rules */
 	typedef uint16_t
__attribute__((__may_alias__))
Quoted 9 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
u16_p;
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
lib/net/rte_ip.h __rte_raw_cksum(const void · line 167
Quoted 2 lines
lib/net/rte_ip.h +0 -2
-	const u16_p *u16_buf = (const u16_p *)buf;
-	const u16_p *end = u16_buf + len /
sizeof(*u16_buf);
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
lib/net/rte_ip.h __rte_raw_cksum(const void · line 169
Quoted 4 lines
lib/net/rte_ip.h +4 -0
+	const u16_p *u16_buf;
+	const u16_p *end;
+
+	/* if buffer is unaligned, keeping it byte
order
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
independent */
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
lib/net/rte_ip.h __rte_raw_cksum(const void · line 174
Quoted 5 lines
lib/net/rte_ip.h +5 -0
+	if (unlikely((uintptr_t)buf & 1)) {
+		uint16_t first = 0;
+		if (unlikely(len == 0))
+			return 0;
+		((unsigned char *)&first)[1] =
*(const
Quoted 21 lines
Quoted 4 lines
unsigned
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
char *)buf;
lib/net/rte_ip.h __rte_raw_cksum(const void · line 176
Quoted 2 lines
lib/net/rte_ip.h +2 -0
+		sum += first;
+		buf = (const void *)((uintptr_t)buf
+ 1);
Quoted 16 lines
Quoted 16 lines
Quoted 16 lines
Quoted 16 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
lib/net/rte_ip.h +4 -0
+		len--;
+	}
 
+	u16_buf = (const u16_p *)buf;
+	end = u16_buf + len / sizeof(*u16_buf);
 	for (; u16_buf != end; ++u16_buf)
 		sum += *u16_buf;
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 2 lines
--
2.17.1
@Emil, can you please test this patch with an
unaligned
buffer on
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
your
Quoted 1 line
application to confirm that it produces the
expected
Quoted 5 lines
Quoted 5 lines
result.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Hi!
I tested the patch. It doesn't seem to produce the
same
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
results. I
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
think the problem is that it always starts summing
from
Quoted 2 lines
Quoted 2 lines
an
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
even address, the sum should always start from the
first
Quoted 9 lines
Quoted 9 lines
byte
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
according
Quoted 2 lines
Quoted 2 lines
to
Quoted 1 line
Quoted 1 line
the checksum specification. Can I instead propose
something
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Mattias
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Rönnblom sent me?
I assume that it produces the same result when the
"buf"
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
parameter is
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
aligned?
And when the "buf" parameter is unaligned, I don't
expect
Quoted 10 lines
Quoted 10 lines
it to
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Quoted 5 lines
produce the
Quoted 4 lines
same results as the simple algorithm!
This was the whole point of the patch: I expect the overall
packet
Quoted 2 lines
Quoted 2 lines
buffer to
Quoted 1 line
be 16 bit aligned, and the checksum to be a partial
checksum of
Quoted 8 lines
Quoted 8 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
such
Quoted 3 lines
Quoted 3 lines
a 16 bit
Quoted 2 lines
aligned packet buffer. When calling this function, I assume
that
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
the
Quoted 2 lines
Quoted 2 lines
"buf" and
Quoted 1 line
"len" parameters point to a part of such a packet
buffer.
Quoted 9 lines
Quoted 9 lines
If
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
these
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
expectations are correct, the simple algorithm will produce
incorrect
Quoted 4 lines
Quoted 4 lines
results
Quoted 3 lines
when "buf" is unaligned.
I was asking you to test if the checksum on the packet
is
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
correct
when your
Quoted 1 line
application modifies an unaligned part of the packet
and
Quoted 11 lines
Quoted 11 lines
uses
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
this
Quoted 5 lines
Quoted 5 lines
function to
Quoted 2 lines
update the checksum.
Now I understand your use case. Your use case seems to be
about
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
partial
Quoted 1 line
Quoted 1 line
checksums, of which some partial checksums may start on
unaligned
Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
addresses in an otherwise aligned packet.
Our use case is about calculating the full checksum on a
nested
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
packet.
Quoted 3 lines
Quoted 3 lines
That nested packet may start on unaligned addresses.
The difference is basically if we want to sum over
aligned
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
addresses or
Quoted 1 line
Quoted 1 line
not, handling the heading and trailing bytes
appropriately.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Your method does not work in our case since we want to
treat
Quoted 3 lines
Quoted 3 lines
the
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
first
Quoted 1 line
Quoted 1 line
two bytes as the first word in our case. But I do
understand
Quoted 5 lines
Quoted 5 lines
that
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
both
Quoted 3 lines
Quoted 1 line
methods are useful.
Yes, that certainly are two different use cases, requiring
two
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
different ways of calculating the 16 bit checksum.
Quoted 3 lines
Quoted 3 lines
Note that your method breaks the API. Previously
(assuming
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
no
crashing
Quoted 1 line
Quoted 1 line
due to low optimization levels, more accepting hardware,
or
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
a
different
Quoted 1 line
Quoted 1 line
compiler (version)) the current method would calculate
the
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
checksum assuming the first two bytes is the first word.
Depending on the point of view, my patch either fixes a bug
(where
Quoted 1 line
Quoted 1 line
Quoted 1 line
the checksum was calculated incorrectly when the buf pointer
was
Quoted 6 lines
Quoted 6 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
unaligned) or breaks the API (by calculating the differently when
the
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
buffer is unaligned).
Quoted 2 lines
I cannot say with certainty which one is correct, but
perhaps
Quoted 20 lines
Quoted 20 lines
some
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
of
the people with a deeper DPDK track record can...
Quoted 2 lines
@Bruce and @Stephen, in 2019 you signed off on a patch [1]
introducing a 16 bit alignment requirement to the Ethernet
address
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
structure.
Quoted 2 lines
It is my understanding that DPDK has an invariant requiring
packets
Quoted 1 line
Quoted 1 line
Quoted 1 line
to be 16 bit aligned, which that patch supports. Is this
invariant
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
documented anywhere, or am I completely wrong? If I'm wrong, then
the
Quoted 1 line
Quoted 1 line
Quoted 1 line
alignment requirement introduced in that patch needs to be
removed, as
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
well as any similar alignment requirements elsewhere in DPDK.
Quoted 7 lines
Quoted 7 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
I don't believe it is explicitly documented as a global
invariant, but
Quoted 1 line
Quoted 1 line
Quoted 1 line
I think it should be unless there is a definite case where we
need to
Quoted 1 line
Quoted 1 line
Quoted 1 line
allow packets to be completely unaligned. Across all packet
headers we
Quoted 1 line
Quoted 1 line
Quoted 1 line
looked at, there was no tunneling protocol where the
resulting
Quoted 6 lines
Quoted 6 lines
packet
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
was left unaligned.
That said, if there are real use cases where we need to allow
packets
Quoted 1 line
Quoted 1 line
Quoted 1 line
to start at an unaligned address, then I agree with you that
we
Quoted 2 lines
Quoted 2 lines
need
Quoted 1 line
Quoted 1 line
Quoted 1 line
to roll back the patch and work to ensure everything works
with
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
unaligned addresses.
/Bruce
@Emil, can you please describe or refer to which tunneling protocol
Quoted 8 lines
Quoted 8 lines
you are
Quoted 3 lines
Quoted 3 lines
using, where the nested packet can be unaligned?
I am asking to determine if your use case is exotic (maybe some
Ericsson
Quoted 1 line
Quoted 1 line
proprietary protocol), or more generic (rooted in some standard
protocol).
Quoted 1 line
Quoted 1 line
This information affects the DPDK community's opinion about how
it
Quoted 6 lines
Quoted 6 lines
should
Quoted 3 lines
Quoted 3 lines
be supported by DPDK.
If possible, please provide more details about the tunneling
protocol and
Quoted 1 line
Quoted 1 line
nested packets... E.g. do the nested packets also contain Layer
2
Quoted 4 lines
Quoted 4 lines
(Ethernet,
Quoted 1 line
Quoted 1 line
VLAN, etc.) headers, or only Layer 3 (IP) or Layer 4 (TCP, UDP,
etc.)? And how
Quoted 1 line
Quoted 1 line
about ARP packets and Layer 2 control protocol packets (STP,
LACP,
Quoted 5 lines
Quoted 5 lines
etc.)?
Quoted 3 lines
Quoted 1 line
Well, if you append or adjust an odd number of bytes (e.g. a PDCP
header) from a previously aligned payload the entire packet will
then
Quoted 4 lines
Quoted 4 lines
be unaligned.
Quoted 1 line
If PDCP headers can leave the rest of the packet field unaligned,
then
Quoted 1 line
Quoted 1 line
we had better remove the alignment restrictions through all of
DPDK.
Quoted 4 lines
Quoted 2 lines
/Bruce
Re-reading the details regarding unaligned pointers in C11, as posted
by Emil
Quoted 1 line
in Bugzilla [2], I interpret it as follows: Any 16 bit or wider
pointer type a must
Quoted 1 line
point to data aligned with that type, i.e. a pointer of the type
"uint16_t *"
Quoted 1 line
must point to 16 bit aligned data, and a pointer of the type
"uint64_t *" must
Quoted 1 line
point to 64 bit aligned data. Please, someone tell me I got this
wrong, and
Quoted 3 lines
wake me up from my nightmare!
Updating DPDK's packet structures to fully support this C11
limitation with
Quoted 1 line
unaligned access would be a nightmare, as we would need to use byte
arrays
Quoted 1 line
for all structure fields. Functions would also be unable to use other
pointer
Quoted 1 line
types than "void *" and "char *", which seems to be the actual
problem in
Quoted 1 line
the __rte_raw_cksum() function. I guess that it also would prevent
the
Quoted 3 lines
compiler from auto-vectorizing the functions.
I am usually a big proponent of academically correct solutions, but
such a
Quoted 1 line
change would be too wide ranging, so I would like to narrow it down
to the
Quoted 5 lines
actual use case, and perhaps extrapolate a bit from there.
@Emil: Do you only need to calculate the checksum of the (potentially unaligned) embedded packet? Or do you also need to use other DPDK functions with the embedded packet, potentially accessing it at an
unaligned
Quoted 3 lines
address?
I'm trying to determine the scope of this C11 pointer alignment
limitation for
Quoted 1 line
your use case, i.e. whether or not other DPDK functions need to be
updated
Quoted 6 lines
to support unaligned packet access too.
[2] https://protect2.fireeye.com/v1/url?k=31323334-501cfaf3-313273af-
454445554331-2ffe58e5caaeb74e&q=1&e=3f0544d3-8a71-4676-b4f9- 27e0952f7de0&u=https%3A%2F%2Fbugs.dpdk.org%2Fshow_bug.cgi%3Fid% 3D1035
That's my interpretation of the standard as well; For example an uint16_t* must be on even addresses. If not it is undefined behavior. I think this is a bigger problem on ARM for example.

Quoted 5 lines

Without being that invested in dpdk, adding unaligned support for everything seems like a steep step, but I'm not sure what it entails in practice.
We are actually only interested in the checksumming.

Great! Then we can cancel the panic about rewriting DPDK Core completely. Although it might still need some review for similar alignment bugs, where we have been forcing the compiler shut up when trying to warn us. :-)

I have provided v3 of the patch, which should do as requested - and still allow the compiler to auto-vectorize.

@Emil, will you please test v3 of the patch?

Morten Brørup

2022-06-23 07:01 UTC #24 lore raw

RE: [PATCH] net: fix checksum with unaligned buffer

Quoted 466 lines

From: Emil Berg [mailto:[email protected]]
Sent: Thursday, 23 June 2022 07.22
Quoted 18 lines
From: Morten Brørup <[email protected]>
Sent: den 22 juni 2022 16:02
Quoted 15 lines
From: Emil Berg [mailto:[email protected]]
Sent: Wednesday, 22 June 2022 14.25
Quoted 12 lines
From: Morten Brørup <[email protected]>
Sent: den 22 juni 2022 13:26
Quoted 9 lines
From: Bruce Richardson [mailto:[email protected]]
Sent: Wednesday, 22 June 2022 11.18
Quoted 5 lines · Emil Berg
Quoted 5 lines
From: Morten Brørup <[email protected]>
Sent: den 21 juni 2022 11:35
Quoted 1 line
From: Bruce Richardson
[mailto:[email protected]]
Quoted 17 lines
Quoted 17 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Sent: Tuesday, 21 June 2022 10.23
On Tue, Jun 21, 2022 at 10:05:15AM +0200, Morten Brørup
Quoted 13 lines · wrote:
Quoted 13 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
+TO: @Bruce and @Stephen: You signed off on the 16 bit
alignment
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
requirement. We need background info on this.
Quoted 10 lines
Quoted 10 lines
From: Emil Berg [mailto:[email protected]]
Sent: Tuesday, 21 June 2022 09.17
Quoted 6 lines
From: Morten Brørup <[email protected]>
Sent: den 20 juni 2022 12:58
Quoted 3 lines
From: Emil Berg [mailto:[email protected]]
Sent: Monday, 20 June 2022 12.38
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
From: Morten Brørup <[email protected]>
Sent: den 17 juni 2022 11:07
From: Morten Brørup
[mailto:[email protected]]
Sent: Friday, 17 June 2022 10.45
With this patch, the checksum can be
calculated
Quoted 7 lines
Quoted 7 lines
on
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
an
unligned
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
part
of
Quoted 2 lines
a packet buffer.
I.e. the buf parameter is no longer required
to
Quoted 6 lines
Quoted 6 lines
be
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
16
bit
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
aligned.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
The DPDK invariant that packet buffers must
be
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
16 bit
aligned
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
remains
Quoted 2 lines
unchanged.
This invariant also defines how to calculate
the
Quoted 10 lines
Quoted 10 lines
16
Quoted 9 lines
Quoted 9 lines
bit
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
checksum
Quoted 7 lines
Quoted 7 lines
Quoted 1 line
on
an
Quoted 5 lines
unaligned part of a packet buffer.
Bugzilla ID: 1035
Cc: [email protected]
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Signed-off-by: Morten Brørup
<[email protected]>
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
---
lib/net/rte_ip.h | 17 +++++++++++++++--
1 file changed, 15 insertions(+), 2
deletions(-)
diff --git a/lib/net/rte_ip.h
b/lib/net/rte_ip.h
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
index
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
b502481670..8e301d9c26 100644
--- a/lib/net/rte_ip.h
+++ b/lib/net/rte_ip.h
@@ -162,9 +162,22 @@ __rte_raw_cksum(const
void
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
*buf,
size_t
Quoted 4 lines
Quoted 4 lines
len,
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
uint32_t sum) {
/* extend strict-aliasing rules */
typedef uint16_t
__attribute__((__may_alias__))
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
u16_p;
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
- const u16_p *u16_buf = (const u16_p
*)buf;
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
- const u16_p *end = u16_buf + len /
sizeof(*u16_buf);
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
+ const u16_p *u16_buf;
+ const u16_p *end;
+
+ /* if buffer is unaligned, keeping it
byte
Quoted 27 lines
Quoted 27 lines
Quoted 27 lines
Quoted 7 lines
order
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
independent */
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
+ if (unlikely((uintptr_t)buf & 1)) {
+ uint16_t first = 0;
+ if (unlikely(len == 0))
+ return 0;
+ ((unsigned char *)&first)[1] =
*(const
Quoted 4 lines
unsigned
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
char *)buf;
+ sum += first;
+ buf = (const void *)((uintptr_t)buf
+ 1);
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
+ len--;
+ }
+ u16_buf = (const u16_p *)buf;
+ end = u16_buf + len / sizeof(*u16_buf);
for (; u16_buf != end; ++u16_buf)
sum += *u16_buf;
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
--
2.17.1
@Emil, can you please test this patch with an
unaligned
buffer on
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
your
Quoted 1 line
application to confirm that it produces the
expected
Quoted 5 lines
Quoted 5 lines
result.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Hi!
I tested the patch. It doesn't seem to produce
the
Quoted 3 lines
Quoted 3 lines
same
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
results. I
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
think the problem is that it always starts
summing
Quoted 3 lines
Quoted 3 lines
from
Quoted 2 lines
Quoted 2 lines
an
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
even address, the sum should always start from
the
Quoted 22 lines
Quoted 22 lines
first
Quoted 9 lines
Quoted 9 lines
byte
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
according
Quoted 2 lines
Quoted 2 lines
to
Quoted 1 line
Quoted 1 line
the checksum specification. Can I instead propose
something
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Mattias
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Rönnblom sent me?
I assume that it produces the same result when the
"buf"
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
parameter is
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
aligned?
And when the "buf" parameter is unaligned, I don't
expect
Quoted 6 lines
Quoted 6 lines
it to
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
produce the
Quoted 4 lines
same results as the simple algorithm!
This was the whole point of the patch: I expect the overall
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
packet
Quoted 2 lines
Quoted 2 lines
buffer to
Quoted 1 line
be 16 bit aligned, and the checksum to be a partial
checksum of
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
such
Quoted 2 lines
Quoted 2 lines
a 16 bit
Quoted 1 line
aligned packet buffer. When calling this function,
I
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
assume
that
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
the
Quoted 2 lines
Quoted 2 lines
"buf" and
Quoted 1 line
"len" parameters point to a part of such a packet
buffer.
Quoted 1 line
Quoted 1 line
If
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
these
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
expectations are correct, the simple algorithm will produce
incorrect
Quoted 4 lines
Quoted 4 lines
results
Quoted 3 lines
when "buf" is unaligned.
I was asking you to test if the checksum on the
packet
Quoted 4 lines
Quoted 4 lines
is
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
correct
when your
Quoted 1 line
application modifies an unaligned part of the
packet
Quoted 8 lines
Quoted 8 lines
and
Quoted 7 lines
Quoted 7 lines
uses
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
this
Quoted 5 lines
Quoted 5 lines
function to
Quoted 2 lines
update the checksum.
Now I understand your use case. Your use case seems
to
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
be
about
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
partial
Quoted 1 line
Quoted 1 line
checksums, of which some partial checksums may start
on
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
unaligned
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
addresses in an otherwise aligned packet.
Our use case is about calculating the full checksum
on a
Quoted 11 lines
Quoted 11 lines
Quoted 5 lines
Quoted 5 lines
nested
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
packet.
Quoted 3 lines
Quoted 3 lines
That nested packet may start on unaligned addresses.
The difference is basically if we want to sum over
aligned
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
addresses or
Quoted 1 line
Quoted 1 line
not, handling the heading and trailing bytes
appropriately.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Your method does not work in our case since we want
to
Quoted 10 lines
Quoted 10 lines
treat
Quoted 3 lines
Quoted 3 lines
the
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
first
Quoted 1 line
Quoted 1 line
two bytes as the first word in our case. But I do
understand
Quoted 5 lines
Quoted 5 lines
that
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
both
Quoted 3 lines
Quoted 1 line
methods are useful.
Yes, that certainly are two different use cases,
requiring
Quoted 9 lines
Quoted 9 lines
two
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
different ways of calculating the 16 bit checksum.
Quoted 3 lines
Quoted 3 lines
Note that your method breaks the API. Previously
(assuming
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
no
crashing
Quoted 1 line
Quoted 1 line
due to low optimization levels, more accepting
hardware,
Quoted 4 lines
Quoted 4 lines
or
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
a
different
Quoted 1 line
Quoted 1 line
compiler (version)) the current method would
calculate
Quoted 2 lines
Quoted 2 lines
the
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
checksum assuming the first two bytes is the first
word.
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
Depending on the point of view, my patch either fixes a bug
(where
Quoted 1 line
Quoted 1 line
Quoted 1 line
the checksum was calculated incorrectly when the buf
pointer
Quoted 2 lines
Quoted 2 lines
was
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
unaligned) or breaks the API (by calculating the
differently
Quoted 11 lines
Quoted 11 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
when
the
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
buffer is unaligned).
Quoted 2 lines
I cannot say with certainty which one is correct, but
perhaps
Quoted 5 lines
Quoted 5 lines
some
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
of
the people with a deeper DPDK track record can...
Quoted 2 lines
@Bruce and @Stephen, in 2019 you signed off on a patch
[1]
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
introducing a 16 bit alignment requirement to the
Ethernet
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
address
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
structure.
Quoted 3 lines
It is my understanding that DPDK has an invariant
requiring
packets
Quoted 1 line
Quoted 1 line
Quoted 1 line
to be 16 bit aligned, which that patch supports. Is this
invariant
Quoted 1 line
Quoted 1 line
Quoted 1 line
documented anywhere, or am I completely wrong? If I'm
wrong,
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
then
the
Quoted 1 line
Quoted 1 line
Quoted 1 line
alignment requirement introduced in that patch needs to
be
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
removed, as
Quoted 1 line
Quoted 1 line
Quoted 1 line
well as any similar alignment requirements elsewhere in
DPDK.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
I don't believe it is explicitly documented as a global
invariant, but
Quoted 1 line
Quoted 1 line
Quoted 1 line
I think it should be unless there is a definite case
where
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
we
need to
Quoted 1 line
Quoted 1 line
Quoted 1 line
allow packets to be completely unaligned. Across all
packet
Quoted 10 lines
Quoted 10 lines
Quoted 2 lines
Quoted 2 lines
headers we
Quoted 1 line
Quoted 1 line
Quoted 1 line
looked at, there was no tunneling protocol where the
resulting
Quoted 7 lines
Quoted 7 lines
packet
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
was left unaligned.
That said, if there are real use cases where we need to allow
packets
Quoted 1 line
Quoted 1 line
Quoted 1 line
to start at an unaligned address, then I agree with you
that
Quoted 3 lines
Quoted 3 lines
we
Quoted 2 lines
Quoted 2 lines
need
Quoted 1 line
Quoted 1 line
Quoted 1 line
to roll back the patch and work to ensure everything
works
Quoted 16 lines
Quoted 16 lines
with
Quoted 15 lines
Quoted 15 lines
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
unaligned addresses.
/Bruce
@Emil, can you please describe or refer to which tunneling protocol
you are
Quoted 4 lines
Quoted 4 lines
using, where the nested packet can be unaligned?
I am asking to determine if your use case is exotic (maybe some
Ericsson
Quoted 2 lines
Quoted 2 lines
proprietary protocol), or more generic (rooted in some standard
Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
Quoted 3 lines
protocol).
Quoted 2 lines
Quoted 2 lines
This information affects the DPDK community's opinion about how
it
Quoted 4 lines
Quoted 4 lines
should
Quoted 3 lines
Quoted 3 lines
be supported by DPDK.
If possible, please provide more details about the
tunneling
Quoted 9 lines
Quoted 9 lines
Quoted 3 lines
Quoted 3 lines
protocol and
Quoted 2 lines
Quoted 2 lines
nested packets... E.g. do the nested packets also contain Layer
2
Quoted 5 lines
Quoted 5 lines
(Ethernet,
Quoted 2 lines
Quoted 2 lines
VLAN, etc.) headers, or only Layer 3 (IP) or Layer 4 (TCP, UDP,
etc.)? And how
Quoted 1 line
Quoted 1 line
about ARP packets and Layer 2 control protocol packets
(STP,
Quoted 7 lines
Quoted 7 lines
LACP,
Quoted 6 lines
Quoted 6 lines
etc.)?
Quoted 4 lines
Quoted 1 line
Well, if you append or adjust an odd number of bytes (e.g. a PDCP
header) from a previously aligned payload the entire packet
will
Quoted 5 lines
Quoted 5 lines
then
Quoted 4 lines
Quoted 4 lines
be unaligned.
Quoted 1 line
If PDCP headers can leave the rest of the packet field
unaligned,
Quoted 14 lines
Quoted 14 lines
then
Quoted 1 line
Quoted 1 line
we had better remove the alignment restrictions through all of
DPDK.
Quoted 5 lines
Quoted 2 lines
/Bruce
Re-reading the details regarding unaligned pointers in C11, as posted
by Emil
Quoted 1 line
in Bugzilla [2], I interpret it as follows: Any 16 bit or wider
pointer type a must
Quoted 1 line
point to data aligned with that type, i.e. a pointer of the type
"uint16_t *"
Quoted 1 line
must point to 16 bit aligned data, and a pointer of the type
Quoted 8 lines
Quoted 8 lines
"uint64_t *" must
Quoted 1 line
point to 64 bit aligned data. Please, someone tell me I got this
wrong, and
Quoted 3 lines
wake me up from my nightmare!
Updating DPDK's packet structures to fully support this C11
limitation with
Quoted 1 line
unaligned access would be a nightmare, as we would need to use
byte
Quoted 7 lines
Quoted 7 lines
arrays
Quoted 2 lines
for all structure fields. Functions would also be unable to use other
pointer
Quoted 1 line
types than "void *" and "char *", which seems to be the actual
problem in
Quoted 1 line
the __rte_raw_cksum() function. I guess that it also would
prevent
Quoted 4 lines
Quoted 4 lines
the
Quoted 3 lines
compiler from auto-vectorizing the functions.
I am usually a big proponent of academically correct solutions,
but
Quoted 2 lines
Quoted 2 lines
such a
Quoted 1 line
change would be too wide ranging, so I would like to narrow it
down
Quoted 7 lines
Quoted 7 lines
to the
Quoted 6 lines
actual use case, and perhaps extrapolate a bit from there.
@Emil: Do you only need to calculate the checksum of the (potentially
unaligned) embedded packet? Or do you also need to use other DPDK functions with the embedded packet, potentially accessing it at
an
Quoted 6 lines
Quoted 6 lines
unaligned
Quoted 3 lines
address?
I'm trying to determine the scope of this C11 pointer alignment
limitation for
Quoted 1 line
your use case, i.e. whether or not other DPDK functions need to
be
Quoted 12 lines
Quoted 7 lines
updated
Quoted 6 lines
to support unaligned packet access too.
[2]
https://protect2.fireeye.com/v1/url?k=31323334-501cfaf3-313273af-
454445554331-2ffe58e5caaeb74e&q=1&e=3f0544d3-8a71-4676-b4f9-
27e0952f7de0&u=https%3A%2F%2Fbugs.dpdk.org%2Fshow_bug.cgi%3Fid%
Quoted 4 lines
Quoted 1 line
3D1035
That's my interpretation of the standard as well; For example an uint16_t* must be on even addresses. If not it is undefined
behavior.
Quoted 4 lines
Quoted 4 lines
I think this is a bigger problem on ARM for example.
Without being that invested in dpdk, adding unaligned support for everything seems like a steep step, but I'm not sure what it
entails
Quoted 5 lines
Quoted 3 lines
in practice.
We are actually only interested in the checksumming.
Great! Then we can cancel the panic about rewriting DPDK Core
completely.
Quoted 1 line
Although it might still need some review for similar alignment bugs,
where
Quoted 3 lines
we have been forcing the compiler shut up when trying to warn us. :-)
I have provided v3 of the patch, which should do as requested - and
still allow
Quoted 2 lines
the compiler to auto-vectorize.

Quoted 8 lines

Quoted 1 line
@Emil, will you please test v3 of the patch?
It seems to work in these two cases:
* Even address, even length
* Even address, odd length
But it breaks in these two cases:
* Odd address, even length (although it works for small buffers, probably when the sum fits inside a uint16_t integer or something)

Interesting observation, good analysis.

Quoted 1 line

* Odd address, odd length

Does this also work for small buffers?

Quoted 2 lines

I get (and like) the main idea of the algorithm but haven't yet figured out what the problem is with odd addresses.

I wonder if I messed up the algorithm for swapping back the bytes in bsum after the calculation... Is the checksum also wrong when compiling without optimization?

And just to be sure: The algorithm requires that __rte_raw_cksum_reduce() is also applied to the sum. Please confirm that you call rte_raw_cksum() (or __rte_raw_cksum() followed by __rte_raw_cksum_reduce())?

Quoted 2 lines

/Emil

Emil Berg

2022-06-23 07:55 UTC #18 lore raw

RE: [PATCH] net: fix checksum with unaligned buffer

Quoted 218 lines

-----Original Message-----
From: Morten Brørup <[email protected]>
Sent: den 21 juni 2022 11:35
To: Emil Berg <[email protected]>
Cc: Bruce Richardson <[email protected]>; Stephen Hemminger <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: RE: [PATCH] net: fix checksum with unaligned buffer
Quoted 191 lines
From: Bruce Richardson [mailto:[email protected]]
Sent: Tuesday, 21 June 2022 10.23
Quoted 1 line · Morten Brørup
+TO: @Bruce and @Stephen: You signed off on the 16 bit alignment
requirement. We need background info on this.
Quoted 10 lines
Quoted 10 lines
From: Emil Berg [mailto:[email protected]]
Sent: Tuesday, 21 June 2022 09.17
Quoted 6 lines
From: Morten Brørup <[email protected]>
Sent: den 20 juni 2022 12:58
Quoted 3 lines
From: Emil Berg [mailto:[email protected]]
Sent: Monday, 20 June 2022 12.38
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
From: Morten Brørup <[email protected]>
Sent: den 17 juni 2022 11:07
Quoted 4 lines
From: Morten Brørup [mailto:[email protected]]
Sent: Friday, 17 June 2022 10.45
With this patch, the checksum can be calculated on an
unligned
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
Quoted 1 line
part
of
Quoted 2 lines
Quoted 2 lines
a packet buffer.
I.e. the buf parameter is no longer required to be 16 bit
aligned.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
The DPDK invariant that packet buffers must be 16 bit
aligned
Quoted 16 lines
Quoted 16 lines
Quoted 3 lines
Quoted 3 lines
remains
Quoted 2 lines
Quoted 2 lines
unchanged.
This invariant also defines how to calculate the 16 bit
checksum
Quoted 12 lines
Quoted 12 lines
Quoted 1 line
Quoted 1 line
on
an
Quoted 10 lines
Quoted 10 lines
unaligned part of a packet buffer.
Bugzilla ID: 1035
Cc: [email protected]
Signed-off-by Morten Brørup <[email protected]>
---
 lib/net/rte_ip.h | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
lib/net/rte_ip.h __rte_raw_cksum(const void *buf,
Quoted 5 lines
lib/net/rte_ip.h index renamed from lib/net/rte_ip.h +0 -0
b502481670..8e301d9c26 100644
lib/net/rte_ip.h +0 -0
@@ -162,9 +162,22 @@ __rte_raw_cksum(const void *buf,
size_t
Quoted 4 lines
Quoted 4 lines
len,
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
lib/net/rte_ip.h +0 -0
 uint32_t sum)  {
 	/* extend strict-aliasing rules */
 	typedef uint16_t __attribute__((__may_alias__))
u16_p;
Quoted 23 lines
Quoted 23 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, · line 168
Quoted 6 lines
lib/net/rte_ip.h +4 -2
-	const u16_p *u16_buf = (const u16_p *)buf;
-	const u16_p *end = u16_buf + len / sizeof(*u16_buf);
+	const u16_p *u16_buf;
+	const u16_p *end;
+
+	/* if buffer is unaligned, keeping it byte order
independent */
Quoted 16 lines
Quoted 16 lines
Quoted 16 lines
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, · line 173
Quoted 5 lines
lib/net/rte_ip.h +5 -0
+	if (unlikely((uintptr_t)buf & 1)) {
+		uint16_t first = 0;
+		if (unlikely(len == 0))
+			return 0;
+		((unsigned char *)&first)[1] = *(const unsigned
char *)buf;
Quoted 10 lines
lib/net/rte_ip.h +6 -0
+		sum += first;
+		buf = (const void *)((uintptr_t)buf + 1);
+		len--;
+	}
 
+	u16_buf = (const u16_p *)buf;
+	end = u16_buf + len / sizeof(*u16_buf);
 	for (; u16_buf != end; ++u16_buf)
 		sum += *u16_buf;
 
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
--
2.17.1
@Emil, can you please test this patch with an unaligned
buffer on
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
your
Quoted 1 line
application to confirm that it produces the expected result.
Hi!
I tested the patch. It doesn't seem to produce the same
results. I
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
think the problem is that it always starts summing from an even address, the sum should always start from the first byte
according
Quoted 2 lines
Quoted 2 lines
to
Quoted 1 line
Quoted 1 line
the checksum specification. Can I instead propose something
Mattias
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Rönnblom sent me?
I assume that it produces the same result when the "buf"
parameter is
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines
aligned?
And when the "buf" parameter is unaligned, I don't expect it to
produce the
Quoted 3 lines
same results as the simple algorithm!
This was the whole point of the patch: I expect the overall
packet
Quoted 2 lines
Quoted 2 lines
buffer to
Quoted 1 line
be 16 bit aligned, and the checksum to be a partial checksum of
such
Quoted 2 lines
Quoted 2 lines
a 16 bit
Quoted 1 line
aligned packet buffer. When calling this function, I assume that
the
Quoted 2 lines
Quoted 2 lines
"buf" and
Quoted 1 line
"len" parameters point to a part of such a packet buffer. If
these
Quoted 1 line
Quoted 1 line
Quoted 1 line
expectations are correct, the simple algorithm will produce
incorrect
Quoted 7 lines
Quoted 7 lines
results
Quoted 4 lines
when "buf" is unaligned.
I was asking you to test if the checksum on the packet is correct
when your
Quoted 1 line
application modifies an unaligned part of the packet and uses
this
Quoted 5 lines
Quoted 5 lines
function to
Quoted 2 lines
update the checksum.
Now I understand your use case. Your use case seems to be about
partial
Quoted 4 lines
Quoted 4 lines
checksums, of which some partial checksums may start on unaligned addresses in an otherwise aligned packet.
Our use case is about calculating the full checksum on a nested
packet.
Quoted 3 lines
Quoted 3 lines
That nested packet may start on unaligned addresses.
The difference is basically if we want to sum over aligned
addresses or
Quoted 3 lines
Quoted 3 lines
not, handling the heading and trailing bytes appropriately.
Your method does not work in our case since we want to treat the
first
Quoted 1 line
Quoted 1 line
two bytes as the first word in our case. But I do understand that
both
Quoted 3 lines
Quoted 1 line
methods are useful.
Yes, that certainly are two different use cases, requiring two
different ways of calculating the 16 bit checksum.
Quoted 3 lines
Quoted 3 lines
Note that your method breaks the API. Previously (assuming no
crashing
Quoted 1 line
Quoted 1 line
due to low optimization levels, more accepting hardware, or a
different
Quoted 5 lines
Quoted 3 lines
compiler (version)) the current method would calculate the checksum assuming the first two bytes is the first word.
Depending on the point of view, my patch either fixes a bug (where
the checksum was calculated incorrectly when the buf pointer was unaligned) or breaks the API (by calculating the differently when the buffer is unaligned).
Quoted 3 lines
I cannot say with certainty which one is correct, but perhaps some of
the people with a deeper DPDK track record can...
Quoted 2 lines
@Bruce and @Stephen, in 2019 you signed off on a patch [1]
introducing a 16 bit alignment requirement to the Ethernet address structure.
Quoted 2 lines
It is my understanding that DPDK has an invariant requiring packets
to be 16 bit aligned, which that patch supports. Is this invariant documented anywhere, or am I completely wrong? If I'm wrong, then the alignment requirement introduced in that patch needs to be removed, as well as any similar alignment requirements elsewhere in DPDK.
I don't believe it is explicitly documented as a global invariant, but I think it should be unless there is a definite case where we need to allow packets to be completely unaligned. Across all packet headers we looked at, there was no tunneling protocol where the resulting packet was left unaligned.
Quoted 7 lines
That said, if there are real use cases where we need to allow packets to start at an unaligned address, then I agree with you that we need to roll back the patch and work to ensure everything works with unaligned addresses.
/Bruce
@Emil, can you please describe or refer to which tunneling protocol you are using, where the nested packet can be unaligned?
I am asking to determine if your use case is exotic (maybe some Ericsson proprietary protocol), or more generic (rooted in some standard protocol). This information affects the DPDK community's opinion about how it should be supported by DPDK.

Quoted 5 lines

If possible, please provide more details about the tunneling protocol and nested packets... E.g. do the nested packets also contain Layer 2 (Ethernet, VLAN, etc.) headers, or only Layer 3 (IP) or Layer 4 (TCP, UDP, etc.)? And how about ARP packets and Layer 2 control protocol packets (STP, LACP, etc.)?

Well, if you append or adjust an odd number of bytes (e.g. a PDCP header) from a previously aligned payload the entire packet will then be unaligned.

Quoted 19 lines

Quoted 4 lines
Quoted 2 lines
[1]
https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-45444
5555731-713e91ae28ea4a95&q=1&e=91f8f355-4366-43bd-ac93-
4c2f375f8d25&u=
Quoted 1 line
http%3A%2F%2Fgit.dpdk.org%2Fdpdk%2Fcommit%2Flib%2Flibrte_net%2Frt e_eth
Quoted 11 lines
er.h%3Fid%3Dda5350ef2
9afd35c1adabe76f60832f3092269ad
Quoted 2 lines
@Emil, we should wait for a conclusion about the alignment invariant
before we proceed.
Quoted 2 lines
If there is no such invariant, my patch is wrong, and we need to
provide a v2 of the patch, which will then fit your use case.
Quoted 1 line
If there is such an invariant, my patch is correct, and another
function must be added for your use case.
Quoted 1 line

Emil Berg

2022-06-23 07:55 UTC #21 lore raw

RE: [PATCH] net: fix checksum with unaligned buffer

Quoted 327 lines

-----Original Message-----
From: Morten Brørup <[email protected]>
Sent: den 22 juni 2022 13:26
To: Bruce Richardson <[email protected]>; Emil Berg <[email protected]>
Cc: Stephen Hemminger <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: RE: [PATCH] net: fix checksum with unaligned buffer
Quoted 58 lines
From: Bruce Richardson [mailto:[email protected]]
Sent: Wednesday, 22 June 2022 11.18
Quoted 9 lines · Emil Berg
Quoted 9 lines
From: Morten Brørup <[email protected]>
Sent: den 21 juni 2022 11:35
Quoted 5 lines
From: Bruce Richardson [mailto:[email protected]]
Sent: Tuesday, 21 June 2022 10.23
Quoted 1 line · Morten Brørup
+TO: @Bruce and @Stephen: You signed off on the 16 bit
alignment
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
requirement. We need background info on this.
Quoted 17 lines
Quoted 17 lines
From: Emil Berg [mailto:[email protected]]
Sent: Tuesday, 21 June 2022 09.17
Quoted 13 lines
From: Morten Brørup <[email protected]>
Sent: den 20 juni 2022 12:58
Quoted 10 lines
From: Emil Berg [mailto:[email protected]]
Sent: Monday, 20 June 2022 12.38
Quoted 7 lines
From: Morten Brørup <[email protected]>
Sent: den 17 juni 2022 11:07
Quoted 4 lines
From: Morten Brørup
[mailto:[email protected]]
Sent: Friday, 17 June 2022 10.45
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
With this patch, the checksum can be calculated on an
unligned
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
Quoted 1 line
part
of
Quoted 3 lines
Quoted 3 lines
a packet buffer.
I.e. the buf parameter is no longer required to be 16
bit
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 4 lines
Quoted 4 lines
aligned.
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
The DPDK invariant that packet buffers must be 16
bit
aligned
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
remains
Quoted 2 lines
Quoted 2 lines
unchanged.
This invariant also defines how to calculate the 16
bit
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
checksum
Quoted 7 lines
Quoted 7 lines
Quoted 1 line
Quoted 1 line
on
an
Quoted 5 lines
Quoted 5 lines
unaligned part of a packet buffer.
Bugzilla ID: 1035
Cc: [email protected]
Quoted 18 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Signed-off-by Morten Brørup
<[email protected]>
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
lib/net/rte_ip.h
Quoted 5 lines
---
 lib/net/rte_ip.h | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)
lib/net/rte_ip.h +0 -0
index
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
lib/net/rte_ip.h __rte_raw_cksum(const void · line 162
Quoted 5 lines
b502481670..8e301d9c26 100644
lib/net/rte_ip.h +0 -0
@@ -162,9 +162,22 @@ __rte_raw_cksum(const void
 *buf,
size_t
Quoted 4 lines
Quoted 4 lines
len,
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
lib/net/rte_ip.h +0 -0
 uint32_t sum)  {
 	/* extend strict-aliasing rules */
 	typedef uint16_t
__attribute__((__may_alias__))
Quoted 9 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
u16_p;
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
lib/net/rte_ip.h __rte_raw_cksum(const void · line 167
Quoted 2 lines
lib/net/rte_ip.h +0 -2
-	const u16_p *u16_buf = (const u16_p *)buf;
-	const u16_p *end = u16_buf + len /
sizeof(*u16_buf);
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
lib/net/rte_ip.h __rte_raw_cksum(const void · line 169
Quoted 4 lines
lib/net/rte_ip.h +4 -0
+	const u16_p *u16_buf;
+	const u16_p *end;
+
+	/* if buffer is unaligned, keeping it byte
order
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
independent */
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
lib/net/rte_ip.h __rte_raw_cksum(const void · line 174
Quoted 5 lines
lib/net/rte_ip.h +5 -0
+	if (unlikely((uintptr_t)buf & 1)) {
+		uint16_t first = 0;
+		if (unlikely(len == 0))
+			return 0;
+		((unsigned char *)&first)[1] =
*(const
Quoted 4 lines
unsigned
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
char *)buf;
lib/net/rte_ip.h __rte_raw_cksum(const void · line 176
Quoted 2 lines
lib/net/rte_ip.h +2 -0
+		sum += first;
+		buf = (const void *)((uintptr_t)buf
+ 1);
Quoted 185 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
lib/net/rte_ip.h +4 -0
+		len--;
+	}
 
+	u16_buf = (const u16_p *)buf;
+	end = u16_buf + len / sizeof(*u16_buf);
 	for (; u16_buf != end; ++u16_buf)
 		sum += *u16_buf;
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 2 lines
--
2.17.1
@Emil, can you please test this patch with an
unaligned
buffer on
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
your
Quoted 1 line
application to confirm that it produces the expected
result.
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Hi!
I tested the patch. It doesn't seem to produce the same
results. I
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
think the problem is that it always starts summing from
an
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
even address, the sum should always start from the first
byte
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
according
Quoted 2 lines
Quoted 2 lines
to
Quoted 1 line
Quoted 1 line
the checksum specification. Can I instead propose
something
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Mattias
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Rönnblom sent me?
I assume that it produces the same result when the "buf"
parameter is
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
aligned?
And when the "buf" parameter is unaligned, I don't expect
it to
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Quoted 5 lines
produce the
Quoted 4 lines
same results as the simple algorithm!
This was the whole point of the patch: I expect the overall
packet
Quoted 2 lines
Quoted 2 lines
buffer to
Quoted 1 line
be 16 bit aligned, and the checksum to be a partial
checksum of
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
such
Quoted 3 lines
Quoted 3 lines
a 16 bit
Quoted 2 lines
aligned packet buffer. When calling this function, I assume
that
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
the
Quoted 2 lines
Quoted 2 lines
"buf" and
Quoted 1 line
"len" parameters point to a part of such a packet buffer.
If
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
these
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
expectations are correct, the simple algorithm will produce
incorrect
Quoted 7 lines
Quoted 7 lines
results
Quoted 4 lines
when "buf" is unaligned.
I was asking you to test if the checksum on the packet is correct
when your
Quoted 1 line
application modifies an unaligned part of the packet and
uses
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
this
Quoted 5 lines
Quoted 5 lines
function to
Quoted 2 lines
update the checksum.
Now I understand your use case. Your use case seems to be
about
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
partial
Quoted 1 line
Quoted 1 line
checksums, of which some partial checksums may start on
unaligned
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
addresses in an otherwise aligned packet.
Our use case is about calculating the full checksum on a
nested
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
packet.
Quoted 3 lines
Quoted 3 lines
That nested packet may start on unaligned addresses.
The difference is basically if we want to sum over aligned
addresses or
Quoted 3 lines
Quoted 3 lines
not, handling the heading and trailing bytes appropriately.
Your method does not work in our case since we want to treat
the
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
first
Quoted 1 line
Quoted 1 line
two bytes as the first word in our case. But I do understand
that
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
both
Quoted 3 lines
Quoted 1 line
methods are useful.
Yes, that certainly are two different use cases, requiring two
different ways of calculating the 16 bit checksum.
Quoted 4 lines
Quoted 4 lines
Note that your method breaks the API. Previously (assuming no
crashing
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
due to low optimization levels, more accepting hardware, or a
different
Quoted 5 lines
Quoted 3 lines
compiler (version)) the current method would calculate the checksum assuming the first two bytes is the first word.
Depending on the point of view, my patch either fixes a bug
(where
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
the checksum was calculated incorrectly when the buf pointer was unaligned) or breaks the API (by calculating the differently when
the
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
buffer is unaligned).
Quoted 2 lines
I cannot say with certainty which one is correct, but perhaps
some
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
of
the people with a deeper DPDK track record can...
Quoted 2 lines
@Bruce and @Stephen, in 2019 you signed off on a patch [1]
introducing a 16 bit alignment requirement to the Ethernet
address
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
structure.
Quoted 2 lines
It is my understanding that DPDK has an invariant requiring
packets
Quoted 1 line
Quoted 1 line
Quoted 1 line
to be 16 bit aligned, which that patch supports. Is this
invariant
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
documented anywhere, or am I completely wrong? If I'm wrong, then
the
Quoted 1 line
Quoted 1 line
Quoted 1 line
alignment requirement introduced in that patch needs to be
removed, as
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
well as any similar alignment requirements elsewhere in DPDK.
I don't believe it is explicitly documented as a global
invariant, but
Quoted 1 line
Quoted 1 line
Quoted 1 line
I think it should be unless there is a definite case where we
need to
Quoted 1 line
Quoted 1 line
Quoted 1 line
allow packets to be completely unaligned. Across all packet
headers we
Quoted 1 line
Quoted 1 line
Quoted 1 line
looked at, there was no tunneling protocol where the resulting
packet
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
was left unaligned.
That said, if there are real use cases where we need to allow
packets
Quoted 1 line
Quoted 1 line
Quoted 1 line
to start at an unaligned address, then I agree with you that we
need
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
to roll back the patch and work to ensure everything works with unaligned addresses.
/Bruce
@Emil, can you please describe or refer to which tunneling protocol
you are
Quoted 3 lines
Quoted 3 lines
using, where the nested packet can be unaligned?
I am asking to determine if your use case is exotic (maybe some
Ericsson
Quoted 1 line
Quoted 1 line
proprietary protocol), or more generic (rooted in some standard
protocol).
Quoted 1 line
Quoted 1 line
This information affects the DPDK community's opinion about how it
should
Quoted 3 lines
Quoted 3 lines
be supported by DPDK.
If possible, please provide more details about the tunneling
protocol and
Quoted 1 line
Quoted 1 line
nested packets... E.g. do the nested packets also contain Layer 2
(Ethernet,
Quoted 1 line
Quoted 1 line
VLAN, etc.) headers, or only Layer 3 (IP) or Layer 4 (TCP, UDP,
etc.)? And how
Quoted 1 line
Quoted 1 line
about ARP packets and Layer 2 control protocol packets (STP, LACP,
etc.)?
Quoted 3 lines
Quoted 1 line
Well, if you append or adjust an odd number of bytes (e.g. a PDCP
Quoted 8 lines
header) from a previously aligned payload the entire packet will then be unaligned.
Quoted 1 line
If PDCP headers can leave the rest of the packet field unaligned, then we had better remove the alignment restrictions through all of DPDK.
/Bruce
Re-reading the details regarding unaligned pointers in C11, as posted by Emil in Bugzilla [2], I interpret it as follows: Any 16 bit or wider pointer type a must point to data aligned with that type, i.e. a pointer of the type "uint16_t *" must point to 16 bit aligned data, and a pointer of the type "uint64_t *" must point to 64 bit aligned data. Please, someone tell me I got this wrong, and wake me up from my nightmare!
Updating DPDK's packet structures to fully support this C11 limitation with unaligned access would be a nightmare, as we would need to use byte arrays for all structure fields. Functions would also be unable to use other pointer types than "void *" and "char *", which seems to be the actual problem in the __rte_raw_cksum() function. I guess that it also would prevent the compiler from auto-vectorizing the functions.
I am usually a big proponent of academically correct solutions, but such a change would be too wide ranging, so I would like to narrow it down to the actual use case, and perhaps extrapolate a bit from there.
@Emil: Do you only need to calculate the checksum of the (potentially unaligned) embedded packet? Or do you also need to use other DPDK functions with the embedded packet, potentially accessing it at an unaligned address?

Quoted 8 lines

I'm trying to determine the scope of this C11 pointer alignment limitation for your use case, i.e. whether or not other DPDK functions need to be updated to support unaligned packet access too.
[2] https://protect2.fireeye.com/v1/url?k=31323334-501cfaf3-313273af-
454445554331-2ffe58e5caaeb74e&q=1&e=3f0544d3-8a71-4676-b4f9- 27e0952f7de0&u=https%3A%2F%2Fbugs.dpdk.org%2Fshow_bug.cgi%3Fid% 3D1035

That's my interpretation of the standard as well; For example an uint16_t* must be on even addresses. If not it is undefined behavior. I think this is a bigger problem on ARM for example.

Without being that invested in dpdk, adding unaligned support for everything seems like a steep step, but I'm not sure what it entails in practice.

We are actually only interested in the checksumming.

Emil Berg

2022-06-23 07:55 UTC #27 lore raw

RE: [PATCH] net: fix checksum with unaligned buffer

Quoted 414 lines

-----Original Message-----
From: Morten Brørup <[email protected]>
Sent: den 22 juni 2022 16:02
To: Emil Berg <[email protected]>; Bruce Richardson <[email protected]>
Cc: Stephen Hemminger <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: RE: [PATCH] net: fix checksum with unaligned buffer
Quoted 378 lines
From: Emil Berg [mailto:[email protected]]
Sent: Wednesday, 22 June 2022 14.25
Quoted 15 lines
From: Morten Brørup <[email protected]>
Sent: den 22 juni 2022 13:26
Quoted 12 lines
From: Bruce Richardson [mailto:[email protected]]
Sent: Wednesday, 22 June 2022 11.18
Quoted 8 lines · Emil Berg
Quoted 8 lines
From: Morten Brørup <[email protected]>
Sent: den 21 juni 2022 11:35
Quoted 4 lines
From: Bruce Richardson [mailto:[email protected]]
Sent: Tuesday, 21 June 2022 10.23
On Tue, Jun 21, 2022 at 10:05:15AM +0200, Morten Brørup
Quoted 13 lines · wrote:
Quoted 13 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
+TO: @Bruce and @Stephen: You signed off on the 16 bit
alignment
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
requirement. We need background info on this.
Quoted 10 lines
Quoted 10 lines
From: Emil Berg [mailto:[email protected]]
Sent: Tuesday, 21 June 2022 09.17
Quoted 6 lines
From: Morten Brørup <[email protected]>
Sent: den 20 juni 2022 12:58
Quoted 3 lines
From: Emil Berg [mailto:[email protected]]
Sent: Monday, 20 June 2022 12.38
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
From: Morten Brørup <[email protected]>
Sent: den 17 juni 2022 11:07
Quoted 5 lines
From: Morten Brørup
[mailto:[email protected]]
Sent: Friday, 17 June 2022 10.45
With this patch, the checksum can be calculated
on
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
an
unligned
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
Quoted 1 line
part
of
Quoted 2 lines
Quoted 2 lines
a packet buffer.
I.e. the buf parameter is no longer required to
be
Quoted 10 lines
Quoted 10 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
16
bit
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 4 lines
Quoted 4 lines
aligned.
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
The DPDK invariant that packet buffers must be
16 bit
aligned
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
remains
Quoted 2 lines
Quoted 2 lines
unchanged.
This invariant also defines how to calculate the
16
Quoted 45 lines
Quoted 10 lines
bit
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
checksum
Quoted 8 lines
Quoted 8 lines
Quoted 1 line
Quoted 1 line
on
an
Quoted 6 lines
Quoted 6 lines
unaligned part of a packet buffer.
Bugzilla ID: 1035
Cc: [email protected]
Signed-off-by: Morten Brørup
Quoted 18 lines
<[email protected]>
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
---
lib/net/rte_ip.h | 17 +++++++++++++++--
1 file changed, 15 insertions(+), 2
deletions(-)
diff --git a/lib/net/rte_ip.h b/lib/net/rte_ip.h
index
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
b502481670..8e301d9c26 100644
--- a/lib/net/rte_ip.h
+++ b/lib/net/rte_ip.h
@@ -162,9 +162,22 @@ __rte_raw_cksum(const void
*buf,
size_t
Quoted 4 lines
Quoted 4 lines
len,
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
uint32_t sum) {
/* extend strict-aliasing rules */
typedef uint16_t
__attribute__((__may_alias__))
Quoted 9 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
u16_p;
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
- const u16_p *u16_buf = (const u16_p *)buf;
- const u16_p *end = u16_buf + len /
sizeof(*u16_buf);
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
+ const u16_p *u16_buf;
+ const u16_p *end;
+
+ /* if buffer is unaligned, keeping it byte
order
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
independent */
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
+ if (unlikely((uintptr_t)buf & 1)) {
+ uint16_t first = 0;
+ if (unlikely(len == 0))
+ return 0;
+ ((unsigned char *)&first)[1] =
*(const
Quoted 21 lines
Quoted 4 lines
unsigned
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
char *)buf;
Quoted 2 lines
+ sum += first;
+ buf = (const void *)((uintptr_t)buf
+ 1);
Quoted 16 lines
Quoted 16 lines
Quoted 16 lines
Quoted 16 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
+ len--;
+ }
+ u16_buf = (const u16_p *)buf;
+ end = u16_buf + len / sizeof(*u16_buf);
for (; u16_buf != end; ++u16_buf)
sum += *u16_buf;
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 2 lines
--
2.17.1
@Emil, can you please test this patch with an
unaligned
buffer on
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
your
Quoted 1 line
application to confirm that it produces the
expected
Quoted 5 lines
Quoted 5 lines
result.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Hi!
I tested the patch. It doesn't seem to produce the
same
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
results. I
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
think the problem is that it always starts summing
from
Quoted 2 lines
Quoted 2 lines
an
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
even address, the sum should always start from the
first
Quoted 9 lines
Quoted 9 lines
byte
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
according
Quoted 2 lines
Quoted 2 lines
to
Quoted 1 line
Quoted 1 line
the checksum specification. Can I instead propose
something
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Mattias
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Rönnblom sent me?
I assume that it produces the same result when the
"buf"
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
parameter is
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
aligned?
And when the "buf" parameter is unaligned, I don't
expect
Quoted 10 lines
Quoted 10 lines
it to
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Quoted 5 lines
produce the
Quoted 4 lines
same results as the simple algorithm!
This was the whole point of the patch: I expect the overall
packet
Quoted 2 lines
Quoted 2 lines
buffer to
Quoted 1 line
be 16 bit aligned, and the checksum to be a partial
checksum of
Quoted 8 lines
Quoted 8 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
such
Quoted 3 lines
Quoted 3 lines
a 16 bit
Quoted 2 lines
aligned packet buffer. When calling this function, I assume
that
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
the
Quoted 2 lines
Quoted 2 lines
"buf" and
Quoted 1 line
"len" parameters point to a part of such a packet
buffer.
Quoted 9 lines
Quoted 9 lines
If
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
these
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
expectations are correct, the simple algorithm will produce
incorrect
Quoted 4 lines
Quoted 4 lines
results
Quoted 3 lines
when "buf" is unaligned.
I was asking you to test if the checksum on the packet
is
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
correct
when your
Quoted 1 line
application modifies an unaligned part of the packet
and
Quoted 12 lines
Quoted 12 lines
uses
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
this
Quoted 6 lines
Quoted 6 lines
function to
Quoted 2 lines
update the checksum.
Now I understand your use case. Your use case seems to be
about
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
partial
Quoted 1 line
Quoted 1 line
checksums, of which some partial checksums may start on
unaligned
Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
addresses in an otherwise aligned packet.
Our use case is about calculating the full checksum on a
nested
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
packet.
Quoted 3 lines
Quoted 3 lines
That nested packet may start on unaligned addresses.
The difference is basically if we want to sum over
aligned
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
addresses or
Quoted 1 line
Quoted 1 line
not, handling the heading and trailing bytes
appropriately.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Your method does not work in our case since we want to
treat
Quoted 3 lines
Quoted 3 lines
the
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
first
Quoted 1 line
Quoted 1 line
two bytes as the first word in our case. But I do
understand
Quoted 5 lines
Quoted 5 lines
that
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
both
Quoted 3 lines
Quoted 1 line
methods are useful.
Yes, that certainly are two different use cases, requiring
two
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
different ways of calculating the 16 bit checksum.
Quoted 3 lines
Quoted 3 lines
Note that your method breaks the API. Previously
(assuming
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
no
crashing
Quoted 1 line
Quoted 1 line
due to low optimization levels, more accepting hardware,
or
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
a
different
Quoted 1 line
Quoted 1 line
compiler (version)) the current method would calculate
the
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 2 lines
checksum assuming the first two bytes is the first word.
Depending on the point of view, my patch either fixes a bug
(where
Quoted 1 line
Quoted 1 line
Quoted 1 line
the checksum was calculated incorrectly when the buf pointer
was
Quoted 6 lines
Quoted 6 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
unaligned) or breaks the API (by calculating the differently when
the
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
buffer is unaligned).
Quoted 2 lines
I cannot say with certainty which one is correct, but
perhaps
Quoted 21 lines
Quoted 21 lines
some
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
of
the people with a deeper DPDK track record can...
Quoted 2 lines
@Bruce and @Stephen, in 2019 you signed off on a patch [1]
introducing a 16 bit alignment requirement to the Ethernet
address
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
structure.
Quoted 3 lines
It is my understanding that DPDK has an invariant
requiring
packets
Quoted 1 line
Quoted 1 line
Quoted 1 line
to be 16 bit aligned, which that patch supports. Is this
invariant
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
documented anywhere, or am I completely wrong? If I'm wrong, then
the
Quoted 1 line
Quoted 1 line
Quoted 1 line
alignment requirement introduced in that patch needs to be
removed, as
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
well as any similar alignment requirements elsewhere in DPDK.
Quoted 8 lines
Quoted 8 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
I don't believe it is explicitly documented as a global
invariant, but
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
I think it should be unless there is a definite case where we
need to
Quoted 1 line
Quoted 1 line
Quoted 1 line
allow packets to be completely unaligned. Across all packet
headers we
Quoted 1 line
Quoted 1 line
Quoted 1 line
looked at, there was no tunneling protocol where the
resulting
Quoted 7 lines
Quoted 7 lines
packet
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
was left unaligned.
That said, if there are real use cases where we need to allow
packets
Quoted 1 line
Quoted 1 line
Quoted 1 line
to start at an unaligned address, then I agree with you that
we
Quoted 2 lines
Quoted 2 lines
need
Quoted 1 line
Quoted 1 line
Quoted 1 line
to roll back the patch and work to ensure everything works
with
Quoted 10 lines
Quoted 10 lines
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
unaligned addresses.
/Bruce
@Emil, can you please describe or refer to which tunneling protocol
you are
Quoted 2 lines
Quoted 2 lines
using, where the nested packet can be unaligned?
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
I am asking to determine if your use case is exotic (maybe some
Ericsson
Quoted 2 lines
Quoted 2 lines
proprietary protocol), or more generic (rooted in some standard
protocol).
Quoted 2 lines
Quoted 2 lines
This information affects the DPDK community's opinion about how
it
Quoted 7 lines
Quoted 7 lines
should
Quoted 3 lines
Quoted 3 lines
be supported by DPDK.
If possible, please provide more details about the tunneling
protocol and
Quoted 2 lines
Quoted 2 lines
nested packets... E.g. do the nested packets also contain Layer
2
Quoted 5 lines
Quoted 5 lines
(Ethernet,
Quoted 2 lines
Quoted 2 lines
VLAN, etc.) headers, or only Layer 3 (IP) or Layer 4 (TCP, UDP,
etc.)? And how
Quoted 1 line
Quoted 1 line
about ARP packets and Layer 2 control protocol packets (STP,
LACP,
Quoted 6 lines
Quoted 6 lines
etc.)?
Quoted 4 lines
Quoted 1 line
Well, if you append or adjust an odd number of bytes (e.g. a PDCP
header) from a previously aligned payload the entire packet will
then
Quoted 4 lines
Quoted 4 lines
be unaligned.
Quoted 1 line
If PDCP headers can leave the rest of the packet field unaligned,
then
Quoted 1 line
Quoted 1 line
we had better remove the alignment restrictions through all of
DPDK.
Quoted 5 lines
Quoted 2 lines
/Bruce
Re-reading the details regarding unaligned pointers in C11, as posted
by Emil
Quoted 1 line
in Bugzilla [2], I interpret it as follows: Any 16 bit or wider
pointer type a must
Quoted 1 line
point to data aligned with that type, i.e. a pointer of the type
"uint16_t *"
Quoted 1 line
must point to 16 bit aligned data, and a pointer of the type
"uint64_t *" must
Quoted 1 line
point to 64 bit aligned data. Please, someone tell me I got this
wrong, and
Quoted 3 lines
wake me up from my nightmare!
Updating DPDK's packet structures to fully support this C11
limitation with
Quoted 1 line
unaligned access would be a nightmare, as we would need to use byte
arrays
Quoted 2 lines
for all structure fields. Functions would also be unable to use other
pointer
Quoted 1 line
types than "void *" and "char *", which seems to be the actual
problem in
Quoted 1 line
the __rte_raw_cksum() function. I guess that it also would prevent
the
Quoted 3 lines
compiler from auto-vectorizing the functions.
I am usually a big proponent of academically correct solutions, but
such a
Quoted 1 line
change would be too wide ranging, so I would like to narrow it down
to the
Quoted 6 lines
actual use case, and perhaps extrapolate a bit from there.
@Emil: Do you only need to calculate the checksum of the (potentially
unaligned) embedded packet? Or do you also need to use other DPDK functions with the embedded packet, potentially accessing it at an
Quoted 13 lines
unaligned
Quoted 3 lines
address?
I'm trying to determine the scope of this C11 pointer alignment
limitation for
Quoted 1 line
your use case, i.e. whether or not other DPDK functions need to be
updated
Quoted 6 lines
to support unaligned packet access too.
[2]
https://protect2.fireeye.com/v1/url?k=31323334-501cfaf3-313273af-
454445554331-2ffe58e5caaeb74e&q=1&e=3f0544d3-8a71-4676-b4f9-
27e0952f7de0&u=https%3A%2F%2Fbugs.dpdk.org%2Fshow_bug.cgi%3Fid%
Quoted 11 lines
Quoted 1 line
3D1035
That's my interpretation of the standard as well; For example an uint16_t* must be on even addresses. If not it is undefined behavior. I think this is a bigger problem on ARM for example.
Without being that invested in dpdk, adding unaligned support for everything seems like a steep step, but I'm not sure what it entails in practice.
We are actually only interested in the checksumming.

Quoted 8 lines

Great! Then we can cancel the panic about rewriting DPDK Core completely. Although it might still need some review for similar alignment bugs, where we have been forcing the compiler shut up when trying to warn us. :-)
I have provided v3 of the patch, which should do as requested - and still allow the compiler to auto-vectorize.
@Emil, will you please test v3 of the patch?

I will test the patch tomorrow.

Emil Berg

2022-06-23 07:55 UTC #23 lore raw

RE: [PATCH] net: fix checksum with unaligned buffer

Quoted 416 lines

-----Original Message-----
From: Morten Brørup <[email protected]>
Sent: den 22 juni 2022 16:02
To: Emil Berg <[email protected]>; Bruce Richardson <[email protected]>
Cc: Stephen Hemminger <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: RE: [PATCH] net: fix checksum with unaligned buffer
Quoted 378 lines
From: Emil Berg [mailto:[email protected]]
Sent: Wednesday, 22 June 2022 14.25
Quoted 15 lines
From: Morten Brørup <[email protected]>
Sent: den 22 juni 2022 13:26
Quoted 12 lines
From: Bruce Richardson [mailto:[email protected]]
Sent: Wednesday, 22 June 2022 11.18
Quoted 8 lines · Emil Berg
Quoted 8 lines
From: Morten Brørup <[email protected]>
Sent: den 21 juni 2022 11:35
Quoted 4 lines
From: Bruce Richardson [mailto:[email protected]]
Sent: Tuesday, 21 June 2022 10.23
On Tue, Jun 21, 2022 at 10:05:15AM +0200, Morten Brørup
Quoted 13 lines · wrote:
Quoted 13 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
+TO: @Bruce and @Stephen: You signed off on the 16 bit
alignment
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
requirement. We need background info on this.
Quoted 10 lines
Quoted 10 lines
From: Emil Berg [mailto:[email protected]]
Sent: Tuesday, 21 June 2022 09.17
Quoted 6 lines
From: Morten Brørup <[email protected]>
Sent: den 20 juni 2022 12:58
Quoted 3 lines
From: Emil Berg [mailto:[email protected]]
Sent: Monday, 20 June 2022 12.38
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
From: Morten Brørup <[email protected]>
Sent: den 17 juni 2022 11:07
Quoted 5 lines
From: Morten Brørup
[mailto:[email protected]]
Sent: Friday, 17 June 2022 10.45
With this patch, the checksum can be calculated
on
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
an
unligned
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
Quoted 1 line
part
of
Quoted 2 lines
Quoted 2 lines
a packet buffer.
I.e. the buf parameter is no longer required to
be
Quoted 10 lines
Quoted 10 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
16
bit
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 4 lines
Quoted 4 lines
aligned.
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
The DPDK invariant that packet buffers must be
16 bit
aligned
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
remains
Quoted 2 lines
Quoted 2 lines
unchanged.
This invariant also defines how to calculate the
16
Quoted 45 lines
Quoted 10 lines
bit
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
checksum
Quoted 8 lines
Quoted 8 lines
Quoted 1 line
Quoted 1 line
on
an
Quoted 6 lines
Quoted 6 lines
unaligned part of a packet buffer.
Bugzilla ID: 1035
Cc: [email protected]
Signed-off-by: Morten Brørup
Quoted 18 lines
<[email protected]>
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
---
lib/net/rte_ip.h | 17 +++++++++++++++--
1 file changed, 15 insertions(+), 2
deletions(-)
diff --git a/lib/net/rte_ip.h b/lib/net/rte_ip.h
index
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
b502481670..8e301d9c26 100644
--- a/lib/net/rte_ip.h
+++ b/lib/net/rte_ip.h
@@ -162,9 +162,22 @@ __rte_raw_cksum(const void
*buf,
size_t
Quoted 4 lines
Quoted 4 lines
len,
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
uint32_t sum) {
/* extend strict-aliasing rules */
typedef uint16_t
__attribute__((__may_alias__))
Quoted 9 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
u16_p;
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
- const u16_p *u16_buf = (const u16_p *)buf;
- const u16_p *end = u16_buf + len /
sizeof(*u16_buf);
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
+ const u16_p *u16_buf;
+ const u16_p *end;
+
+ /* if buffer is unaligned, keeping it byte
order
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
independent */
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
+ if (unlikely((uintptr_t)buf & 1)) {
+ uint16_t first = 0;
+ if (unlikely(len == 0))
+ return 0;
+ ((unsigned char *)&first)[1] =
*(const
Quoted 21 lines
Quoted 4 lines
unsigned
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
char *)buf;
Quoted 2 lines
+ sum += first;
+ buf = (const void *)((uintptr_t)buf
+ 1);
Quoted 16 lines
Quoted 16 lines
Quoted 16 lines
Quoted 16 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
+ len--;
+ }
+ u16_buf = (const u16_p *)buf;
+ end = u16_buf + len / sizeof(*u16_buf);
for (; u16_buf != end; ++u16_buf)
sum += *u16_buf;
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 2 lines
--
2.17.1
@Emil, can you please test this patch with an
unaligned
buffer on
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
your
Quoted 1 line
application to confirm that it produces the
expected
Quoted 5 lines
Quoted 5 lines
result.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Hi!
I tested the patch. It doesn't seem to produce the
same
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
results. I
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
think the problem is that it always starts summing
from
Quoted 2 lines
Quoted 2 lines
an
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
even address, the sum should always start from the
first
Quoted 9 lines
Quoted 9 lines
byte
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
according
Quoted 2 lines
Quoted 2 lines
to
Quoted 1 line
Quoted 1 line
the checksum specification. Can I instead propose
something
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Mattias
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Rönnblom sent me?
I assume that it produces the same result when the
"buf"
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
parameter is
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
aligned?
And when the "buf" parameter is unaligned, I don't
expect
Quoted 10 lines
Quoted 10 lines
it to
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Quoted 5 lines
produce the
Quoted 4 lines
same results as the simple algorithm!
This was the whole point of the patch: I expect the overall
packet
Quoted 2 lines
Quoted 2 lines
buffer to
Quoted 1 line
be 16 bit aligned, and the checksum to be a partial
checksum of
Quoted 8 lines
Quoted 8 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
such
Quoted 3 lines
Quoted 3 lines
a 16 bit
Quoted 2 lines
aligned packet buffer. When calling this function, I assume
that
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
the
Quoted 2 lines
Quoted 2 lines
"buf" and
Quoted 1 line
"len" parameters point to a part of such a packet
buffer.
Quoted 9 lines
Quoted 9 lines
If
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
these
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
expectations are correct, the simple algorithm will produce
incorrect
Quoted 4 lines
Quoted 4 lines
results
Quoted 3 lines
when "buf" is unaligned.
I was asking you to test if the checksum on the packet
is
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
correct
when your
Quoted 1 line
application modifies an unaligned part of the packet
and
Quoted 12 lines
Quoted 12 lines
uses
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
this
Quoted 6 lines
Quoted 6 lines
function to
Quoted 2 lines
update the checksum.
Now I understand your use case. Your use case seems to be
about
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
partial
Quoted 1 line
Quoted 1 line
checksums, of which some partial checksums may start on
unaligned
Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
addresses in an otherwise aligned packet.
Our use case is about calculating the full checksum on a
nested
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
packet.
Quoted 3 lines
Quoted 3 lines
That nested packet may start on unaligned addresses.
The difference is basically if we want to sum over
aligned
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
addresses or
Quoted 1 line
Quoted 1 line
not, handling the heading and trailing bytes
appropriately.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Your method does not work in our case since we want to
treat
Quoted 3 lines
Quoted 3 lines
the
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
first
Quoted 1 line
Quoted 1 line
two bytes as the first word in our case. But I do
understand
Quoted 5 lines
Quoted 5 lines
that
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
both
Quoted 3 lines
Quoted 1 line
methods are useful.
Yes, that certainly are two different use cases, requiring
two
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
different ways of calculating the 16 bit checksum.
Quoted 3 lines
Quoted 3 lines
Note that your method breaks the API. Previously
(assuming
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
no
crashing
Quoted 1 line
Quoted 1 line
due to low optimization levels, more accepting hardware,
or
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
a
different
Quoted 1 line
Quoted 1 line
compiler (version)) the current method would calculate
the
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 2 lines
checksum assuming the first two bytes is the first word.
Depending on the point of view, my patch either fixes a bug
(where
Quoted 1 line
Quoted 1 line
Quoted 1 line
the checksum was calculated incorrectly when the buf pointer
was
Quoted 6 lines
Quoted 6 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
unaligned) or breaks the API (by calculating the differently when
the
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
buffer is unaligned).
Quoted 2 lines
I cannot say with certainty which one is correct, but
perhaps
Quoted 21 lines
Quoted 21 lines
some
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
of
the people with a deeper DPDK track record can...
Quoted 2 lines
@Bruce and @Stephen, in 2019 you signed off on a patch [1]
introducing a 16 bit alignment requirement to the Ethernet
address
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
structure.
Quoted 3 lines
It is my understanding that DPDK has an invariant
requiring
packets
Quoted 1 line
Quoted 1 line
Quoted 1 line
to be 16 bit aligned, which that patch supports. Is this
invariant
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
documented anywhere, or am I completely wrong? If I'm wrong, then
the
Quoted 1 line
Quoted 1 line
Quoted 1 line
alignment requirement introduced in that patch needs to be
removed, as
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
well as any similar alignment requirements elsewhere in DPDK.
Quoted 8 lines
Quoted 8 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
I don't believe it is explicitly documented as a global
invariant, but
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
I think it should be unless there is a definite case where we
need to
Quoted 1 line
Quoted 1 line
Quoted 1 line
allow packets to be completely unaligned. Across all packet
headers we
Quoted 1 line
Quoted 1 line
Quoted 1 line
looked at, there was no tunneling protocol where the
resulting
Quoted 7 lines
Quoted 7 lines
packet
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
was left unaligned.
That said, if there are real use cases where we need to allow
packets
Quoted 1 line
Quoted 1 line
Quoted 1 line
to start at an unaligned address, then I agree with you that
we
Quoted 2 lines
Quoted 2 lines
need
Quoted 1 line
Quoted 1 line
Quoted 1 line
to roll back the patch and work to ensure everything works
with
Quoted 10 lines
Quoted 10 lines
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
unaligned addresses.
/Bruce
@Emil, can you please describe or refer to which tunneling protocol
you are
Quoted 2 lines
Quoted 2 lines
using, where the nested packet can be unaligned?
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
I am asking to determine if your use case is exotic (maybe some
Ericsson
Quoted 2 lines
Quoted 2 lines
proprietary protocol), or more generic (rooted in some standard
protocol).
Quoted 2 lines
Quoted 2 lines
This information affects the DPDK community's opinion about how
it
Quoted 7 lines
Quoted 7 lines
should
Quoted 3 lines
Quoted 3 lines
be supported by DPDK.
If possible, please provide more details about the tunneling
protocol and
Quoted 2 lines
Quoted 2 lines
nested packets... E.g. do the nested packets also contain Layer
2
Quoted 5 lines
Quoted 5 lines
(Ethernet,
Quoted 2 lines
Quoted 2 lines
VLAN, etc.) headers, or only Layer 3 (IP) or Layer 4 (TCP, UDP,
etc.)? And how
Quoted 1 line
Quoted 1 line
about ARP packets and Layer 2 control protocol packets (STP,
LACP,
Quoted 6 lines
Quoted 6 lines
etc.)?
Quoted 4 lines
Quoted 1 line
Well, if you append or adjust an odd number of bytes (e.g. a PDCP
header) from a previously aligned payload the entire packet will
then
Quoted 4 lines
Quoted 4 lines
be unaligned.
Quoted 1 line
If PDCP headers can leave the rest of the packet field unaligned,
then
Quoted 1 line
Quoted 1 line
we had better remove the alignment restrictions through all of
DPDK.
Quoted 5 lines
Quoted 2 lines
/Bruce
Re-reading the details regarding unaligned pointers in C11, as posted
by Emil
Quoted 1 line
in Bugzilla [2], I interpret it as follows: Any 16 bit or wider
pointer type a must
Quoted 1 line
point to data aligned with that type, i.e. a pointer of the type
"uint16_t *"
Quoted 1 line
must point to 16 bit aligned data, and a pointer of the type
"uint64_t *" must
Quoted 1 line
point to 64 bit aligned data. Please, someone tell me I got this
wrong, and
Quoted 3 lines
wake me up from my nightmare!
Updating DPDK's packet structures to fully support this C11
limitation with
Quoted 1 line
unaligned access would be a nightmare, as we would need to use byte
arrays
Quoted 2 lines
for all structure fields. Functions would also be unable to use other
pointer
Quoted 1 line
types than "void *" and "char *", which seems to be the actual
problem in
Quoted 1 line
the __rte_raw_cksum() function. I guess that it also would prevent
the
Quoted 3 lines
compiler from auto-vectorizing the functions.
I am usually a big proponent of academically correct solutions, but
such a
Quoted 1 line
change would be too wide ranging, so I would like to narrow it down
to the
Quoted 6 lines
actual use case, and perhaps extrapolate a bit from there.
@Emil: Do you only need to calculate the checksum of the (potentially
unaligned) embedded packet? Or do you also need to use other DPDK functions with the embedded packet, potentially accessing it at an
Quoted 13 lines
unaligned
Quoted 3 lines
address?
I'm trying to determine the scope of this C11 pointer alignment
limitation for
Quoted 1 line
your use case, i.e. whether or not other DPDK functions need to be
updated
Quoted 6 lines
to support unaligned packet access too.
[2]
https://protect2.fireeye.com/v1/url?k=31323334-501cfaf3-313273af-
454445554331-2ffe58e5caaeb74e&q=1&e=3f0544d3-8a71-4676-b4f9-
27e0952f7de0&u=https%3A%2F%2Fbugs.dpdk.org%2Fshow_bug.cgi%3Fid%
Quoted 11 lines
Quoted 1 line
3D1035
That's my interpretation of the standard as well; For example an uint16_t* must be on even addresses. If not it is undefined behavior. I think this is a bigger problem on ARM for example.
Without being that invested in dpdk, adding unaligned support for everything seems like a steep step, but I'm not sure what it entails in practice.
We are actually only interested in the checksumming.

Quoted 8 lines

Great! Then we can cancel the panic about rewriting DPDK Core completely. Although it might still need some review for similar alignment bugs, where we have been forcing the compiler shut up when trying to warn us. :-)
I have provided v3 of the patch, which should do as requested - and still allow the compiler to auto-vectorize.
@Emil, will you please test v3 of the patch?

It seems to work in these two cases:
* Even address, even length
* Even address, odd length
But it breaks in these two cases:
* Odd address, even length (although it works for small buffers, probably when the sum fits inside a uint16_t integer or something)
* Odd address, odd length
I get (and like) the main idea of the algorithm but haven't yet figured out what the problem is with odd addresses.

/Emil

Morten Brørup

2022-06-23 12:18 UTC #26 lore raw

RE: [PATCH] net: fix checksum with unaligned buffer

Quoted 561 lines

From: Emil Berg [mailto:[email protected]]
Sent: Thursday, 23 June 2022 13.39
Quoted 28 lines
From: Morten Brørup <[email protected]>
Sent: den 23 juni 2022 09:01
Quoted 25 lines
From: Emil Berg [mailto:[email protected]]
Sent: Thursday, 23 June 2022 07.22
Quoted 18 lines
From: Morten Brørup <[email protected]>
Sent: den 22 juni 2022 16:02
Quoted 15 lines
From: Emil Berg [mailto:[email protected]]
Sent: Wednesday, 22 June 2022 14.25
Quoted 12 lines
From: Morten Brørup <[email protected]>
Sent: den 22 juni 2022 13:26
Quoted 9 lines
From: Bruce Richardson [mailto:[email protected]]
Sent: Wednesday, 22 June 2022 11.18
Quoted 5 lines · Emil Berg
Quoted 5 lines
From: Morten Brørup <[email protected]>
Sent: den 21 juni 2022 11:35
Quoted 1 line
From: Bruce Richardson
[mailto:[email protected]]
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Sent: Tuesday, 21 June 2022 10.23
On Tue, Jun 21, 2022 at 10:05:15AM +0200, Morten
Brørup
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 1 line · wrote:
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
+TO: @Bruce and @Stephen: You signed off on the 16
bit
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
alignment
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
requirement. We need background info on this.
Quoted 8 lines
Quoted 8 lines
From: Emil Berg [mailto:[email protected]]
Sent: Tuesday, 21 June 2022 09.17
Quoted 4 lines
From: Morten Brørup <[email protected]>
Sent: den 20 juni 2022 12:58
From: Emil Berg
[mailto:[email protected]]
Quoted 19 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Sent: Monday, 20 June 2022 12.38
From: Morten Brørup
<[email protected]>
Quoted 15 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Sent: den 17 juni 2022 11:07
From: Morten Brørup
[mailto:[email protected]]
Sent: Friday, 17 June 2022 10.45
With this patch, the checksum can be
calculated
Quoted 7 lines
Quoted 7 lines
on
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
an
unligned
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
part
of
a packet buffer.
I.e. the buf parameter is no longer
required
Quoted 7 lines
Quoted 7 lines
to
Quoted 6 lines
Quoted 6 lines
be
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
16
bit
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
aligned.
Quoted 2 lines
The DPDK invariant that packet buffers
must
Quoted 6 lines
Quoted 6 lines
be
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
16 bit
aligned
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
remains
unchanged.
This invariant also defines how to
calculate
Quoted 18 lines
Quoted 18 lines
the
Quoted 17 lines
Quoted 17 lines
16
Quoted 16 lines
Quoted 16 lines
bit
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
checksum
Quoted 8 lines
on
an
unaligned part of a packet buffer.
Bugzilla ID: 1035
Cc: [email protected]
Signed-off-by: Morten Brørup
<[email protected]>
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
---
lib/net/rte_ip.h | 17 +++++++++++++++--
1 file changed, 15 insertions(+), 2
deletions(-)
Quoted 7 lines
Quoted 7 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
diff --git a/lib/net/rte_ip.h
b/lib/net/rte_ip.h
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
index
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
b502481670..8e301d9c26 100644
--- a/lib/net/rte_ip.h
+++ b/lib/net/rte_ip.h
@@ -162,9 +162,22 @@
__rte_raw_cksum(const
Quoted 9 lines
Quoted 9 lines
void
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
*buf,
size_t
Quoted 4 lines
Quoted 4 lines
len,
Quoted 3 lines
uint32_t sum) {
/* extend strict-aliasing rules */
typedef uint16_t
__attribute__((__may_alias__))
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
u16_p;
Quoted 8 lines
Quoted 8 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
- const u16_p *u16_buf = (const u16_p
*)buf;
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
- const u16_p *end = u16_buf + len /
sizeof(*u16_buf);
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
+ const u16_p *u16_buf;
+ const u16_p *end;
+
+ /* if buffer is unaligned, keeping
it
Quoted 8 lines
Quoted 8 lines
byte
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
order
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
independent */
Quoted 5 lines
+ if (unlikely((uintptr_t)buf & 1)) {
+ uint16_t first = 0;
+ if (unlikely(len == 0))
+ return 0;
+ ((unsigned char *)&first)[1]
=
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
*(const
Quoted 4 lines
unsigned
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
char *)buf;
+ sum += first;
+ buf = (const void
*)((uintptr_t)buf
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
+ 1);
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
+ len--;
+ }
+ u16_buf = (const u16_p *)buf;
+ end = u16_buf + len /
sizeof(*u16_buf);
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
for (; u16_buf != end; ++u16_buf)
sum += *u16_buf;
--
2.17.1
@Emil, can you please test this patch with
an
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
unaligned
buffer on
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
your
application to confirm that it produces the
expected
Quoted 5 lines
Quoted 5 lines
result.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Hi!
I tested the patch. It doesn't seem to
produce
Quoted 8 lines
Quoted 8 lines
the
Quoted 3 lines
Quoted 3 lines
same
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
results. I
Quoted 1 line
Quoted 1 line
Quoted 1 line
think the problem is that it always starts
summing
Quoted 3 lines
Quoted 3 lines
from
Quoted 2 lines
Quoted 2 lines
an
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
even address, the sum should always start
from
Quoted 4 lines
Quoted 4 lines
the
Quoted 3 lines
Quoted 3 lines
first
Quoted 2 lines
Quoted 2 lines
byte
Quoted 1 line
Quoted 1 line
Quoted 1 line
according
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
to
Quoted 2 lines
the checksum specification. Can I instead
propose
something
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Mattias
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Rönnblom sent me?
I assume that it produces the same result when
the
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
"buf"
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
parameter is
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
aligned?
And when the "buf" parameter is unaligned, I
don't
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
expect
Quoted 6 lines
Quoted 6 lines
it to
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
produce the
Quoted 4 lines
same results as the simple algorithm!
This was the whole point of the patch: I expect
the overall
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
packet
Quoted 3 lines
Quoted 3 lines
buffer to
Quoted 2 lines
be 16 bit aligned, and the checksum to be a
partial
checksum of
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
such
Quoted 2 lines
Quoted 2 lines
a 16 bit
Quoted 1 line
aligned packet buffer. When calling this
function,
Quoted 6 lines
Quoted 6 lines
I
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
assume
that
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
the
Quoted 2 lines
Quoted 2 lines
"buf" and
Quoted 1 line
"len" parameters point to a part of such a
packet
Quoted 16 lines
Quoted 16 lines
Quoted 10 lines
Quoted 10 lines
buffer.
Quoted 9 lines
Quoted 9 lines
If
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
these
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
expectations are correct, the simple algorithm
will produce
incorrect
Quoted 4 lines
Quoted 4 lines
results
Quoted 3 lines
when "buf" is unaligned.
I was asking you to test if the checksum on the
packet
Quoted 4 lines
Quoted 4 lines
is
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
correct
when your
Quoted 1 line
application modifies an unaligned part of the
packet
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
and
Quoted 7 lines
Quoted 7 lines
uses
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
this
Quoted 5 lines
Quoted 5 lines
function to
Quoted 2 lines
update the checksum.
Now I understand your use case. Your use case
seems
Quoted 5 lines
Quoted 5 lines
to
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
be
about
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
partial
Quoted 1 line
Quoted 1 line
checksums, of which some partial checksums may
start
Quoted 5 lines
Quoted 5 lines
on
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
unaligned
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
addresses in an otherwise aligned packet.
Our use case is about calculating the full
checksum
Quoted 4 lines
Quoted 4 lines
on a
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
nested
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
packet.
Quoted 1 line
Quoted 1 line
That nested packet may start on unaligned
addresses.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
The difference is basically if we want to sum
over
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
aligned
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
addresses or
Quoted 1 line
Quoted 1 line
not, handling the heading and trailing bytes
appropriately.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Your method does not work in our case since we
want
Quoted 21 lines
Quoted 21 lines
to
Quoted 10 lines
Quoted 10 lines
treat
Quoted 3 lines
Quoted 3 lines
the
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
first
Quoted 1 line
Quoted 1 line
two bytes as the first word in our case. But I do
understand
Quoted 5 lines
Quoted 5 lines
that
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
both
Quoted 3 lines
Quoted 1 line
methods are useful.
Yes, that certainly are two different use cases,
requiring
Quoted 9 lines
Quoted 9 lines
two
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
different ways of calculating the 16 bit checksum.
Quoted 3 lines
Quoted 3 lines
Note that your method breaks the API. Previously
(assuming
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
no
crashing
Quoted 1 line
Quoted 1 line
due to low optimization levels, more accepting
Quoted 8 lines
Quoted 8 lines
hardware,
Quoted 4 lines
Quoted 4 lines
or
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
a
different
Quoted 1 line
Quoted 1 line
compiler (version)) the current method would
calculate
Quoted 2 lines
Quoted 2 lines
the
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
checksum assuming the first two bytes is the
first
Quoted 4 lines
Quoted 4 lines
word.
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Depending on the point of view, my patch either
fixes
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
a bug
(where
Quoted 1 line
Quoted 1 line
Quoted 1 line
the checksum was calculated incorrectly when the buf
pointer
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
was
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
unaligned) or breaks the API (by calculating the
differently
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
when
the
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
buffer is unaligned).
Quoted 2 lines
I cannot say with certainty which one is correct,
but
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
perhaps
Quoted 5 lines
Quoted 5 lines
some
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
of
the people with a deeper DPDK track record can...
Quoted 2 lines
@Bruce and @Stephen, in 2019 you signed off on a
patch
Quoted 10 lines
Quoted 10 lines
[1]
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
introducing a 16 bit alignment requirement to the
Ethernet
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
address
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
structure.
Quoted 3 lines
It is my understanding that DPDK has an invariant
requiring
packets
Quoted 1 line
Quoted 1 line
Quoted 1 line
to be 16 bit aligned, which that patch supports. Is
this
Quoted 6 lines
Quoted 6 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
invariant
Quoted 1 line
Quoted 1 line
Quoted 1 line
documented anywhere, or am I completely wrong? If I'm
wrong,
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
then
the
Quoted 1 line
Quoted 1 line
Quoted 1 line
alignment requirement introduced in that patch needs
to
Quoted 3 lines
Quoted 3 lines
be
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
removed, as
Quoted 1 line
Quoted 1 line
Quoted 1 line
well as any similar alignment requirements elsewhere
in
Quoted 3 lines
Quoted 3 lines
DPDK.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
I don't believe it is explicitly documented as a
global
Quoted 6 lines
Quoted 6 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
invariant, but
Quoted 1 line
Quoted 1 line
Quoted 1 line
I think it should be unless there is a definite case
where
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
we
need to
Quoted 1 line
Quoted 1 line
Quoted 1 line
allow packets to be completely unaligned. Across all
Quoted 8 lines
Quoted 8 lines
packet
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
Quoted 2 lines
headers we
Quoted 1 line
Quoted 1 line
Quoted 1 line
looked at, there was no tunneling protocol where the
resulting
Quoted 4 lines
Quoted 4 lines
packet
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
was left unaligned.
That said, if there are real use cases where we need
to
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
allow
packets
Quoted 1 line
Quoted 1 line
Quoted 1 line
to start at an unaligned address, then I agree with
you
Quoted 4 lines
Quoted 4 lines
that
Quoted 3 lines
Quoted 3 lines
we
Quoted 2 lines
Quoted 2 lines
need
Quoted 1 line
Quoted 1 line
Quoted 1 line
to roll back the patch and work to ensure everything
Quoted 8 lines
Quoted 8 lines
works
Quoted 7 lines
Quoted 7 lines
with
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines
unaligned addresses.
/Bruce
@Emil, can you please describe or refer to which
tunneling
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
Quoted 1 line
protocol
you are
Quoted 3 lines
Quoted 3 lines
using, where the nested packet can be unaligned?
I am asking to determine if your use case is exotic
(maybe
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 1 line
Quoted 1 line
some
Ericsson
Quoted 2 lines
Quoted 2 lines
proprietary protocol), or more generic (rooted in some standard
protocol).
Quoted 2 lines
Quoted 2 lines
This information affects the DPDK community's opinion about how
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Quoted 5 lines
it
Quoted 4 lines
Quoted 4 lines
should
Quoted 3 lines
Quoted 3 lines
be supported by DPDK.
If possible, please provide more details about the
tunneling
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
protocol and
Quoted 1 line
Quoted 1 line
nested packets... E.g. do the nested packets also
contain
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Layer
2
Quoted 2 lines
Quoted 2 lines
(Ethernet,
Quoted 1 line
Quoted 1 line
VLAN, etc.) headers, or only Layer 3 (IP) or Layer 4
(TCP,
Quoted 9 lines
Quoted 9 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
UDP,
etc.)? And how
Quoted 1 line
Quoted 1 line
about ARP packets and Layer 2 control protocol packets
(STP,
Quoted 5 lines
Quoted 5 lines
LACP,
Quoted 4 lines
Quoted 4 lines
etc.)?
Quoted 3 lines
Quoted 1 line
Well, if you append or adjust an odd number of bytes
(e.g. a
Quoted 11 lines
Quoted 11 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 1 line
PDCP
header) from a previously aligned payload the entire packet
will
Quoted 5 lines
Quoted 5 lines
then
Quoted 4 lines
Quoted 4 lines
be unaligned.
Quoted 1 line
If PDCP headers can leave the rest of the packet field
unaligned,
Quoted 2 lines
Quoted 2 lines
then
Quoted 1 line
Quoted 1 line
we had better remove the alignment restrictions through all
of
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
DPDK.
Quoted 4 lines
Quoted 2 lines
/Bruce
Re-reading the details regarding unaligned pointers in C11,
as
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
posted
by Emil
Quoted 1 line
in Bugzilla [2], I interpret it as follows: Any 16 bit or
wider
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
pointer type a must
Quoted 1 line
point to data aligned with that type, i.e. a pointer of the
type
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
"uint16_t *"
Quoted 1 line
must point to 16 bit aligned data, and a pointer of the type
"uint64_t *" must
Quoted 1 line
point to 64 bit aligned data. Please, someone tell me I got
this
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
wrong, and
Quoted 3 lines
wake me up from my nightmare!
Updating DPDK's packet structures to fully support this C11
limitation with
Quoted 1 line
unaligned access would be a nightmare, as we would need to
use
Quoted 3 lines
Quoted 3 lines
byte
Quoted 2 lines
Quoted 2 lines
arrays
Quoted 1 line
for all structure fields. Functions would also be unable to
use
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
other
pointer
Quoted 1 line
types than "void *" and "char *", which seems to be the
actual
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
Quoted 2 lines
problem in
Quoted 1 line
the __rte_raw_cksum() function. I guess that it also would
prevent
Quoted 4 lines
Quoted 4 lines
the
Quoted 3 lines
compiler from auto-vectorizing the functions.
I am usually a big proponent of academically correct
solutions,
Quoted 3 lines
Quoted 3 lines
but
Quoted 2 lines
Quoted 2 lines
such a
Quoted 1 line
change would be too wide ranging, so I would like to narrow
it
Quoted 8 lines
Quoted 8 lines
down
Quoted 7 lines
Quoted 7 lines
to the
Quoted 6 lines
actual use case, and perhaps extrapolate a bit from there.
@Emil: Do you only need to calculate the checksum of the (potentially
unaligned) embedded packet? Or do you also need to use other DPDK functions with the embedded packet, potentially
accessing
Quoted 6 lines
Quoted 6 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
it at
an
Quoted 4 lines
Quoted 4 lines
unaligned
Quoted 3 lines
address?
I'm trying to determine the scope of this C11 pointer
alignment
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
limitation for
Quoted 1 line
your use case, i.e. whether or not other DPDK functions need
to
Quoted 6 lines
Quoted 6 lines
be
Quoted 5 lines
Quoted 5 lines
updated
Quoted 4 lines
to support unaligned packet access too.
[2]
https://protect2.fireeye.com/v1/url?k=31323334-501cfaf3-
313273af
Quoted 8 lines
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
Quoted 3 lines
- 454445554331-2ffe58e5caaeb74e&q=1&e=3f0544d3-8a71-4676-b4f9-
27e0952f7de0&u=https%3A%2F%2Fbugs.dpdk.org%2Fshow_bug.cgi%3Fid%
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
3D1035
That's my interpretation of the standard as well; For example
an
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
Quoted 1 line
uint16_t* must be on even addresses. If not it is undefined
behavior.
Quoted 3 lines
Quoted 3 lines
I think this is a bigger problem on ARM for example.
Without being that invested in dpdk, adding unaligned support
for
Quoted 9 lines
Quoted 9 lines
Quoted 1 line
Quoted 1 line
everything seems like a steep step, but I'm not sure what it
entails
Quoted 5 lines
Quoted 3 lines
in practice.
We are actually only interested in the checksumming.
Great! Then we can cancel the panic about rewriting DPDK Core
completely.
Quoted 1 line
Although it might still need some review for similar alignment
bugs,
Quoted 5 lines
Quoted 5 lines
where
Quoted 4 lines
we have been forcing the compiler shut up when trying to warn us. :-)
I have provided v3 of the patch, which should do as requested -
and
Quoted 14 lines
Quoted 11 lines
still allow
Quoted 3 lines
the compiler to auto-vectorize.
@Emil, will you please test v3 of the patch?
It seems to work in these two cases:
* Even address, even length
* Even address, odd length
But it breaks in these two cases:
* Odd address, even length (although it works for small buffers, probably when the sum fits inside a uint16_t integer or something)
Interesting observation, good analysis.
Quoted 8 lines
Quoted 1 line
* Odd address, odd length
Does this also work for small buffers?
Quoted 2 lines
I get (and like) the main idea of the algorithm but haven't yet figured out what the problem is with odd addresses.
I wonder if I messed up the algorithm for swapping back the bytes in
bsum
Quoted 1 line
after the calculation... Is the checksum also wrong when compiling
without
Quoted 3 lines
optimization?
And just to be sure: The algorithm requires that

Quoted 8 lines

__rte_raw_cksum_reduce()
Quoted 1 line
is also applied to the sum. Please confirm that you call
rte_raw_cksum() (or
Quoted 2 lines
__rte_raw_cksum() followed by __rte_raw_cksum_reduce())?
Yes, I messed up. I didn't run the reduction part. When I do the output seems to be the same.

I'm really happy to hear that! Thank you for informing me quickly.

Quoted 6 lines

It seems to be about as fast as the previous algorithm, obviously. Both valgrind and fsanitize=undefined are happy.
Some minor improvements:
* #include <stdbool.h>?

Some include it, some don't.
Looking at other header files, I can't see any pattern, but I agree that it should be included here.

Quoted 1 line

* Use RTE_PTR_ADD to make the casts cleaner?

Yes. Will do.

Quoted 2 lines

* I guess you could skip using 'bsum' and add to 'sum' instead, but that's a matter of preference

No, because we want to keep the byte order of the initial sum intact, and only swap the bytes of the buffer's sum (when the buffer is unaligned).

Quoted 2 lines

* Can't you just do bsum += *(const unsigned char *)buf; to avoid 'first', making it a bit more readable?

No, because on little endian CPUs, the second byte is the MSB of the uint16_t. It would only work on big endian CPUs.

I considered doing it differently. But for consistency, I reused the 'left' block's method; only I'm putting the spare byte in the second byte of the 16 bit word, whereas the 'first' block puts the spare byte in the first byte of the 16 bit word.

PS: A lot of smart tricks are available for IP checksumming... RFC 1071 is worth a read. :-)

Morten Brørup

2022-06-23 12:39 UTC #30 lore raw

PATCH v4 1/1 net: fix checksum with unaligned buffer

With this patch, the checksum can be calculated on an unaligned buffer. I.e. the buf parameter is no longer required to be 16 bit aligned.

The checksum is still calculated using a 16 bit aligned pointer, so the compiler can auto-vectorize the function's inner loop.

v4:
* Add copyright notice.
* Include stdbool.h (Emil Berg).
* Use RTE_PTR_ADD (Emil Berg).
* Fix one more typo in commit message. Is 'unligned' even a word? v3:
* Remove braces from single statement block.
* Fix typo in commit message.
v2:
* Do not assume that the buffer is part of an aligned packet buffer.

Bugzilla ID: 1035
Cc: [email protected]

---
 lib/net/rte_ip.h | 32 +++++++++++++++++++++++++++-----
 1 file changed, 27 insertions(+), 5 deletions(-)

lib/net/rte_ip.h +27 -5

index b502481670..738d643da0 100644
@@ -3,6 +3,7 @@
  *      The Regents of the University of California.
  * Copyright(c) 2010-2014 Intel Corporation.
  * Copyright(c) 2014 6WIND S.A.
+ * Copyright(c) 2022 SmartShare Systems.
  * All rights reserved.
  */

@@ -15,6 +16,7 @@
  * IP-related defines
  */
 
+#include <stdbool.h>
 #include <stdint.h>
 
 #ifdef RTE_EXEC_ENV_WINDOWS

@@ -162,20 +164,40 @@ __rte_raw_cksum(const void *buf, size_t len, uint32_t sum)
 {
 	/* extend strict-aliasing rules */
 	typedef uint16_t __attribute__((__may_alias__)) u16_p;
-	const u16_p *u16_buf = (const u16_p *)buf;
-	const u16_p *end = u16_buf + len / sizeof(*u16_buf);
+	const u16_p *u16_buf;
+	const u16_p *end;
+	uint32_t bsum = 0;
+	const bool unaligned = (uintptr_t)buf & 1;
+
+	/* if buffer is unaligned, keeping it byte order independent */
+	if (unlikely(unaligned)) {
+		uint16_t first = 0;
+		if (unlikely(len == 0))
+			return 0;
+		((unsigned char *)&first)[1] = *(const unsigned char *)buf;
+		bsum += first;
+		buf = RTE_PTR_ADD(buf, 1);
+		len--;
+	}
 
+	/* aligned access for compiler auto-vectorization */
+	u16_buf = (const u16_p *)buf;
+	end = u16_buf + len / sizeof(*u16_buf);
 	for (; u16_buf != end; ++u16_buf)
-		sum += *u16_buf;
+		bsum += *u16_buf;
 
 	/* if length is odd, keeping it byte order independent */
 	if (unlikely(len % 2)) {
 		uint16_t left = 0;
 		*(unsigned char *)&left = *(const unsigned char *)end;
-		sum += left;
+		bsum += left;
 	}
 
-	return sum;
+	/* if buffer is unaligned, swap the checksum bytes */
+	if (unlikely(unaligned))
+		bsum = (bsum & 0xFF00FF00) >> 8 | (bsum & 0x00FF00FF) << 8;
+
+	return sum + bsum;
 }
 
 /**

Signature · 1 line

2.17.1

Morten Brørup

2022-06-23 12:51 UTC #31 lore raw

RE: [PATCH v4] net: fix checksum with unaligned buffer

Quoted 100 lines

From: Morten Brørup [mailto:[email protected]]
Sent: Thursday, 23 June 2022 14.39
With this patch, the checksum can be calculated on an unaligned buffer. I.e. the buf parameter is no longer required to be 16 bit aligned.
The checksum is still calculated using a 16 bit aligned pointer, so the compiler can auto-vectorize the function's inner loop.
When the buffer is unaligned, the first byte of the buffer is handled separately. Furthermore, the calculated checksum of the buffer is byte shifted before being added to the initial checksum, to compensate for the
checksum having been calculated on the buffer shifted by one byte.
v4:
* Add copyright notice.
* Include stdbool.h (Emil Berg).
* Use RTE_PTR_ADD (Emil Berg).
* Fix one more typo in commit message. Is 'unligned' even a word? v3:
* Remove braces from single statement block.
* Fix typo in commit message.
v2:
* Do not assume that the buffer is part of an aligned packet buffer.
Bugzilla ID: 1035
Cc: [email protected]
Signed-off-by Morten Brørup <[email protected]>
---
 lib/net/rte_ip.h | 32 +++++++++++++++++++++++++++-----
 1 file changed, 27 insertions(+), 5 deletions(-)
lib/net/rte_ip.h +27 -5
index b502481670..738d643da0 100644
@@ -3,6 +3,7 @@
  *      The Regents of the University of California.
  * Copyright(c) 2010-2014 Intel Corporation.
  * Copyright(c) 2014 6WIND S.A.
+ * Copyright(c) 2022 SmartShare Systems.
  * All rights reserved.
  */
 
@@ -15,6 +16,7 @@
  * IP-related defines
  */
 
+#include <stdbool.h>
 #include <stdint.h>
 
 #ifdef RTE_EXEC_ENV_WINDOWS
@@ -162,20 +164,40 @@ __rte_raw_cksum(const void *buf, size_t len,
 uint32_t sum)
 {
 	/* extend strict-aliasing rules */
 	typedef uint16_t __attribute__((__may_alias__)) u16_p;
-	const u16_p *u16_buf = (const u16_p *)buf;
-	const u16_p *end = u16_buf + len / sizeof(*u16_buf);
+	const u16_p *u16_buf;
+	const u16_p *end;
+	uint32_t bsum = 0;
+	const bool unaligned = (uintptr_t)buf & 1;
+
+	/* if buffer is unaligned, keeping it byte order independent */
+	if (unlikely(unaligned)) {
+		uint16_t first = 0;
+		if (unlikely(len == 0))
+			return 0;
+		((unsigned char *)&first)[1] = *(const unsigned char *)buf;
+		bsum += first;
+		buf = RTE_PTR_ADD(buf, 1);
+		len--;
+	}
 
+	/* aligned access for compiler auto-vectorization */
+	u16_buf = (const u16_p *)buf;
+	end = u16_buf + len / sizeof(*u16_buf);
 	for (; u16_buf != end; ++u16_buf)
-		sum += *u16_buf;
+		bsum += *u16_buf;
 
 	/* if length is odd, keeping it byte order independent */
 	if (unlikely(len % 2)) {
 		uint16_t left = 0;
 		*(unsigned char *)&left = *(const unsigned char *)end;
-		sum += left;
+		bsum += left;
 	}
 
-	return sum;
+	/* if buffer is unaligned, swap the checksum bytes */
+	if (unlikely(unaligned))
+		bsum = (bsum & 0xFF00FF00) >> 8 | (bsum & 0x00FF00FF) << 8;
+
+	return sum + bsum;
 }
 

Quoted 3 lines

/**
--
2.17.1

@Emil, thank you for thoroughly reviewing the previous versions.

If your test succeeds and you are satisfied with the patch, remember to reply with a "Tested-by" tag for patchwork.

Emil Berg

2022-06-24 08:34 UTC #25 lore raw

RE: [PATCH] net: fix checksum with unaligned buffer

Quoted 503 lines

-----Original Message-----
From: Morten Brørup <[email protected]>
Sent: den 23 juni 2022 09:01
To: Emil Berg <[email protected]>; Bruce Richardson <[email protected]>
Cc: Stephen Hemminger <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: RE: [PATCH] net: fix checksum with unaligned buffer
Quoted 15 lines
From: Emil Berg [mailto:[email protected]]
Sent: Thursday, 23 June 2022 07.22
Quoted 12 lines
From: Morten Brørup <[email protected]>
Sent: den 22 juni 2022 16:02
Quoted 9 lines
From: Emil Berg [mailto:[email protected]]
Sent: Wednesday, 22 June 2022 14.25
Quoted 6 lines
From: Morten Brørup <[email protected]>
Sent: den 22 juni 2022 13:26
Quoted 3 lines
From: Bruce Richardson [mailto:[email protected]]
Sent: Wednesday, 22 June 2022 11.18
Quoted 25 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines · Emil Berg
Quoted 5 lines
Quoted 5 lines
From: Morten Brørup <[email protected]>
Sent: den 21 juni 2022 11:35
Quoted 1 line
From: Bruce Richardson
[mailto:[email protected]]
Quoted 18 lines
Quoted 18 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Sent: Tuesday, 21 June 2022 10.23
On Tue, Jun 21, 2022 at 10:05:15AM +0200, Morten Brørup
Quoted 14 lines · wrote:
Quoted 14 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
+TO: @Bruce and @Stephen: You signed off on the 16 bit
alignment
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
requirement. We need background info on this.
Quoted 11 lines
Quoted 11 lines
From: Emil Berg [mailto:[email protected]]
Sent: Tuesday, 21 June 2022 09.17
Quoted 7 lines
From: Morten Brørup <[email protected]>
Sent: den 20 juni 2022 12:58
Quoted 4 lines
From: Emil Berg [mailto:[email protected]]
Sent: Monday, 20 June 2022 12.38
From: Morten Brørup
<[email protected]>
Quoted 389 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Sent: den 17 juni 2022 11:07
From: Morten Brørup
[mailto:[email protected]]
Sent: Friday, 17 June 2022 10.45
With this patch, the checksum can be
calculated
Quoted 7 lines
Quoted 7 lines
on
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
an
unligned
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
part
of
a packet buffer.
I.e. the buf parameter is no longer required
to
Quoted 6 lines
Quoted 6 lines
be
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
16
bit
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
aligned.
Quoted 2 lines
Quoted 2 lines
The DPDK invariant that packet buffers must
be
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
16 bit
aligned
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
remains
unchanged.
This invariant also defines how to calculate
the
Quoted 10 lines
Quoted 10 lines
16
Quoted 9 lines
Quoted 9 lines
bit
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
checksum
Quoted 7 lines
Quoted 7 lines
on
an
unaligned part of a packet buffer.
Bugzilla ID: 1035
Cc: [email protected]
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Signed-off-by: Morten Brørup
<[email protected]>
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
---
lib/net/rte_ip.h | 17 +++++++++++++++--
1 file changed, 15 insertions(+), 2
deletions(-)
diff --git a/lib/net/rte_ip.h
b/lib/net/rte_ip.h
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
index
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
b502481670..8e301d9c26 100644
--- a/lib/net/rte_ip.h
+++ b/lib/net/rte_ip.h
@@ -162,9 +162,22 @@ __rte_raw_cksum(const
void
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
*buf,
size_t
Quoted 4 lines
Quoted 4 lines
len,
Quoted 3 lines
Quoted 3 lines
uint32_t sum) {
/* extend strict-aliasing rules */
typedef uint16_t
__attribute__((__may_alias__))
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
u16_p;
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
- const u16_p *u16_buf = (const u16_p
*)buf;
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
- const u16_p *end = u16_buf + len /
sizeof(*u16_buf);
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
+ const u16_p *u16_buf;
+ const u16_p *end;
+
+ /* if buffer is unaligned, keeping it
byte
Quoted 27 lines
Quoted 27 lines
Quoted 27 lines
Quoted 7 lines
order
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
independent */
Quoted 5 lines
Quoted 5 lines
+ if (unlikely((uintptr_t)buf & 1)) {
+ uint16_t first = 0;
+ if (unlikely(len == 0))
+ return 0;
+ ((unsigned char *)&first)[1] =
*(const
Quoted 4 lines
unsigned
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
char *)buf;
+ sum += first;
+ buf = (const void *)((uintptr_t)buf
+ 1);
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
+ len--;
+ }
+ u16_buf = (const u16_p *)buf;
+ end = u16_buf + len / sizeof(*u16_buf);
for (; u16_buf != end; ++u16_buf)
sum += *u16_buf;
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
--
2.17.1
@Emil, can you please test this patch with an
unaligned
buffer on
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
your
application to confirm that it produces the
expected
Quoted 5 lines
Quoted 5 lines
result.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Hi!
I tested the patch. It doesn't seem to produce
the
Quoted 3 lines
Quoted 3 lines
same
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
results. I
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
think the problem is that it always starts
summing
Quoted 3 lines
Quoted 3 lines
from
Quoted 2 lines
Quoted 2 lines
an
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
even address, the sum should always start from
the
Quoted 23 lines
Quoted 23 lines
first
Quoted 10 lines
Quoted 10 lines
byte
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
according
Quoted 3 lines
Quoted 3 lines
to
Quoted 2 lines
Quoted 2 lines
the checksum specification. Can I instead
propose
something
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Mattias
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Rönnblom sent me?
I assume that it produces the same result when the
"buf"
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
parameter is
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
aligned?
And when the "buf" parameter is unaligned, I don't
expect
Quoted 6 lines
Quoted 6 lines
it to
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
produce the
Quoted 4 lines
same results as the simple algorithm!
This was the whole point of the patch: I expect
the overall
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
packet
Quoted 3 lines
Quoted 3 lines
buffer to
Quoted 2 lines
be 16 bit aligned, and the checksum to be a
partial
checksum of
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
such
Quoted 2 lines
Quoted 2 lines
a 16 bit
Quoted 1 line
aligned packet buffer. When calling this function,
I
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
assume
that
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
the
Quoted 2 lines
Quoted 2 lines
"buf" and
Quoted 1 line
"len" parameters point to a part of such a packet
buffer.
Quoted 1 line
Quoted 1 line
If
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
these
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
expectations are correct, the simple algorithm
will produce
incorrect
Quoted 4 lines
Quoted 4 lines
results
Quoted 3 lines
when "buf" is unaligned.
I was asking you to test if the checksum on the
packet
Quoted 4 lines
Quoted 4 lines
is
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
correct
when your
Quoted 1 line
application modifies an unaligned part of the
packet
Quoted 8 lines
Quoted 8 lines
and
Quoted 7 lines
Quoted 7 lines
uses
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
this
Quoted 5 lines
Quoted 5 lines
function to
Quoted 2 lines
update the checksum.
Now I understand your use case. Your use case seems
to
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
be
about
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
partial
Quoted 1 line
Quoted 1 line
checksums, of which some partial checksums may start
on
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
unaligned
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
addresses in an otherwise aligned packet.
Our use case is about calculating the full checksum
on a
Quoted 11 lines
Quoted 11 lines
Quoted 5 lines
Quoted 5 lines
nested
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
packet.
Quoted 3 lines
Quoted 3 lines
That nested packet may start on unaligned addresses.
The difference is basically if we want to sum over
aligned
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
addresses or
Quoted 1 line
Quoted 1 line
not, handling the heading and trailing bytes
appropriately.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Your method does not work in our case since we want
to
Quoted 10 lines
Quoted 10 lines
treat
Quoted 3 lines
Quoted 3 lines
the
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
first
Quoted 1 line
Quoted 1 line
two bytes as the first word in our case. But I do
understand
Quoted 5 lines
Quoted 5 lines
that
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
both
Quoted 3 lines
Quoted 1 line
methods are useful.
Yes, that certainly are two different use cases,
requiring
Quoted 9 lines
Quoted 9 lines
two
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
different ways of calculating the 16 bit checksum.
Quoted 3 lines
Quoted 3 lines
Note that your method breaks the API. Previously
(assuming
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
no
crashing
Quoted 1 line
Quoted 1 line
due to low optimization levels, more accepting
hardware,
Quoted 4 lines
Quoted 4 lines
or
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
a
different
Quoted 1 line
Quoted 1 line
compiler (version)) the current method would
calculate
Quoted 2 lines
Quoted 2 lines
the
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
checksum assuming the first two bytes is the first
word.
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
Depending on the point of view, my patch either fixes a bug
(where
Quoted 1 line
Quoted 1 line
Quoted 1 line
the checksum was calculated incorrectly when the buf
pointer
Quoted 2 lines
Quoted 2 lines
was
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
unaligned) or breaks the API (by calculating the
differently
Quoted 11 lines
Quoted 11 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
when
the
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
buffer is unaligned).
Quoted 2 lines
I cannot say with certainty which one is correct, but
perhaps
Quoted 5 lines
Quoted 5 lines
some
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
of
the people with a deeper DPDK track record can...
Quoted 2 lines
@Bruce and @Stephen, in 2019 you signed off on a patch
[1]
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
introducing a 16 bit alignment requirement to the
Ethernet
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
address
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
structure.
Quoted 3 lines
It is my understanding that DPDK has an invariant
requiring
packets
Quoted 1 line
Quoted 1 line
Quoted 1 line
to be 16 bit aligned, which that patch supports. Is this
invariant
Quoted 1 line
Quoted 1 line
Quoted 1 line
documented anywhere, or am I completely wrong? If I'm
wrong,
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
then
the
Quoted 1 line
Quoted 1 line
Quoted 1 line
alignment requirement introduced in that patch needs to
be
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
removed, as
Quoted 1 line
Quoted 1 line
Quoted 1 line
well as any similar alignment requirements elsewhere in
DPDK.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
I don't believe it is explicitly documented as a global
invariant, but
Quoted 1 line
Quoted 1 line
Quoted 1 line
I think it should be unless there is a definite case
where
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
we
need to
Quoted 1 line
Quoted 1 line
Quoted 1 line
allow packets to be completely unaligned. Across all
packet
Quoted 10 lines
Quoted 10 lines
Quoted 2 lines
Quoted 2 lines
headers we
Quoted 1 line
Quoted 1 line
Quoted 1 line
looked at, there was no tunneling protocol where the
resulting
Quoted 7 lines
Quoted 7 lines
packet
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
was left unaligned.
That said, if there are real use cases where we need to allow
packets
Quoted 1 line
Quoted 1 line
Quoted 1 line
to start at an unaligned address, then I agree with you
that
Quoted 3 lines
Quoted 3 lines
we
Quoted 2 lines
Quoted 2 lines
need
Quoted 1 line
Quoted 1 line
Quoted 1 line
to roll back the patch and work to ensure everything
works
Quoted 16 lines
Quoted 16 lines
with
Quoted 15 lines
Quoted 15 lines
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
unaligned addresses.
/Bruce
@Emil, can you please describe or refer to which tunneling protocol
you are
Quoted 4 lines
Quoted 4 lines
using, where the nested packet can be unaligned?
I am asking to determine if your use case is exotic (maybe some
Ericsson
Quoted 2 lines
Quoted 2 lines
proprietary protocol), or more generic (rooted in some standard
Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
Quoted 3 lines
protocol).
Quoted 2 lines
Quoted 2 lines
This information affects the DPDK community's opinion about how
it
Quoted 4 lines
Quoted 4 lines
should
Quoted 3 lines
Quoted 3 lines
be supported by DPDK.
If possible, please provide more details about the
tunneling
Quoted 9 lines
Quoted 9 lines
Quoted 3 lines
Quoted 3 lines
protocol and
Quoted 2 lines
Quoted 2 lines
nested packets... E.g. do the nested packets also contain Layer
2
Quoted 5 lines
Quoted 5 lines
(Ethernet,
Quoted 2 lines
Quoted 2 lines
VLAN, etc.) headers, or only Layer 3 (IP) or Layer 4 (TCP, UDP,
etc.)? And how
Quoted 1 line
Quoted 1 line
about ARP packets and Layer 2 control protocol packets
(STP,
Quoted 7 lines
Quoted 7 lines
LACP,
Quoted 6 lines
Quoted 6 lines
etc.)?
Quoted 4 lines
Quoted 1 line
Well, if you append or adjust an odd number of bytes (e.g. a PDCP
header) from a previously aligned payload the entire packet
will
Quoted 5 lines
Quoted 5 lines
then
Quoted 4 lines
Quoted 4 lines
be unaligned.
Quoted 1 line
If PDCP headers can leave the rest of the packet field
unaligned,
Quoted 14 lines
Quoted 14 lines
then
Quoted 1 line
Quoted 1 line
we had better remove the alignment restrictions through all of
DPDK.
Quoted 5 lines
Quoted 2 lines
/Bruce
Re-reading the details regarding unaligned pointers in C11, as posted
by Emil
Quoted 1 line
in Bugzilla [2], I interpret it as follows: Any 16 bit or wider
pointer type a must
Quoted 1 line
point to data aligned with that type, i.e. a pointer of the type
"uint16_t *"
Quoted 1 line
must point to 16 bit aligned data, and a pointer of the type
Quoted 8 lines
Quoted 8 lines
"uint64_t *" must
Quoted 1 line
point to 64 bit aligned data. Please, someone tell me I got this
wrong, and
Quoted 3 lines
wake me up from my nightmare!
Updating DPDK's packet structures to fully support this C11
limitation with
Quoted 1 line
unaligned access would be a nightmare, as we would need to use
byte
Quoted 7 lines
Quoted 7 lines
arrays
Quoted 2 lines
for all structure fields. Functions would also be unable to use other
pointer
Quoted 1 line
types than "void *" and "char *", which seems to be the actual
problem in
Quoted 1 line
the __rte_raw_cksum() function. I guess that it also would
prevent
Quoted 4 lines
Quoted 4 lines
the
Quoted 3 lines
compiler from auto-vectorizing the functions.
I am usually a big proponent of academically correct solutions,
but
Quoted 2 lines
Quoted 2 lines
such a
Quoted 1 line
change would be too wide ranging, so I would like to narrow it
down
Quoted 8 lines
Quoted 8 lines
to the
Quoted 7 lines
actual use case, and perhaps extrapolate a bit from there.
@Emil: Do you only need to calculate the checksum of the (potentially
unaligned) embedded packet? Or do you also need to use other DPDK functions with the embedded packet, potentially accessing it at
Quoted 17 lines
an
Quoted 6 lines
Quoted 6 lines
unaligned
Quoted 3 lines
address?
I'm trying to determine the scope of this C11 pointer alignment
limitation for
Quoted 1 line
your use case, i.e. whether or not other DPDK functions need to
be
Quoted 9 lines
Quoted 8 lines
updated
Quoted 7 lines
to support unaligned packet access too.
[2]
https://protect2.fireeye.com/v1/url?k=31323334-501cfaf3-313273af
- 454445554331-2ffe58e5caaeb74e&q=1&e=3f0544d3-8a71-4676-b4f9-
27e0952f7de0&u=https%3A%2F%2Fbugs.dpdk.org%2Fshow_bug.cgi%3Fid%
Quoted 25 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
3D1035
That's my interpretation of the standard as well; For example an uint16_t* must be on even addresses. If not it is undefined
behavior.
Quoted 4 lines
Quoted 4 lines
I think this is a bigger problem on ARM for example.
Without being that invested in dpdk, adding unaligned support for everything seems like a steep step, but I'm not sure what it
entails
Quoted 5 lines
Quoted 3 lines
in practice.
We are actually only interested in the checksumming.
Great! Then we can cancel the panic about rewriting DPDK Core
completely.
Quoted 1 line
Although it might still need some review for similar alignment bugs,
where
Quoted 4 lines
we have been forcing the compiler shut up when trying to warn us. :-)
I have provided v3 of the patch, which should do as requested - and
still allow
Quoted 2 lines
the compiler to auto-vectorize.
Quoted 8 lines
Quoted 1 line
@Emil, will you please test v3 of the patch?
It seems to work in these two cases:
* Even address, even length
* Even address, odd length
But it breaks in these two cases:
* Odd address, even length (although it works for small buffers, probably when the sum fits inside a uint16_t integer or something)
Interesting observation, good analysis.
Quoted 1 line
* Odd address, odd length
Does this also work for small buffers?
Quoted 2 lines
I get (and like) the main idea of the algorithm but haven't yet figured out what the problem is with odd addresses.

Quoted 8 lines

I wonder if I messed up the algorithm for swapping back the bytes in bsum after the calculation... Is the checksum also wrong when compiling without optimization?
And just to be sure: The algorithm requires that __rte_raw_cksum_reduce() is also applied to the sum. Please confirm that you call rte_raw_cksum() (or __rte_raw_cksum() followed by __rte_raw_cksum_reduce())?

Yes, I messed up. I didn't run the reduction part. When I do the output seems to be the same.

It seems to be about as fast as the previous algorithm, obviously. Both valgrind and fsanitize=undefined are happy.

Some minor improvements:
* #include <stdbool.h>?
* Use RTE_PTR_ADD to make the casts cleaner?
* I guess you could skip using 'bsum' and add to 'sum' instead, but that's a matter of preference
* Can't you just do bsum += *(const unsigned char *)buf; to avoid 'first', making it a bit more readable?

Quoted 2 lines

Quoted 2 lines
/Emil

Morten Brørup

Tested-by Emil Berg

2022-06-27 10:54 UTC #33 lore raw

RE: [PATCH v4] net: fix checksum with unaligned buffer

Quoted 120 lines

From: Emil Berg [mailto:[email protected]]
Sent: Monday, 27 June 2022 09.57
Quoted 7 lines
From: Morten Brørup <[email protected]>
Sent: den 23 juni 2022 14:51
Quoted 4 lines
From: Morten Brørup [mailto:[email protected]]
Sent: Thursday, 23 June 2022 14.39
With this patch, the checksum can be calculated on an unaligned
buffer.
Quoted 6 lines
Quoted 6 lines
I.e. the buf parameter is no longer required to be 16 bit aligned.
The checksum is still calculated using a 16 bit aligned pointer, so the compiler can auto-vectorize the function's inner loop.
When the buffer is unaligned, the first byte of the buffer is
handled
Quoted 1 line
Quoted 1 line
separately. Furthermore, the calculated checksum of the buffer is
byte
Quoted 1 line
Quoted 1 line
shifted before being added to the initial checksum, to compensate
for
Quoted 1 line
Quoted 1 line
the checksum having been calculated on the buffer shifted by one
byte.
Quoted 11 lines
Quoted 11 lines
v4:
* Add copyright notice.
* Include stdbool.h (Emil Berg).
* Use RTE_PTR_ADD (Emil Berg).
* Fix one more typo in commit message. Is 'unligned' even a word? v3:
* Remove braces from single statement block.
* Fix typo in commit message.
v2:
* Do not assume that the buffer is part of an aligned packet
buffer.
Quoted 84 lines
Quoted 37 lines
Bugzilla ID: 1035
Cc: [email protected]
Signed-off-by Morten Brørup <[email protected]>
Tested-by Emil Berg <[email protected]>
---
 lib/net/rte_ip.h | 32 +++++++++++++++++++++++++++-----
 1 file changed, 27 insertions(+), 5 deletions(-)
lib/net/rte_ip.h index renamed from lib/net/rte_ip.h +0 -0
b502481670..738d643da0 100644
lib/net/rte_ip.h +3 -2
@@ -3,6 +3,7 @@
  *      The Regents of the University of California.
  * Copyright(c) 2010-2014 Intel Corporation.
  * Copyright(c) 2014 6WIND S.A.
+ * Copyright(c) 2022 SmartShare Systems.
  * All rights reserved.
  */
 
@@ -15,6 +16,7 @@
  * IP-related defines
  */
 
+#include <stdbool.h>
 #include <stdint.h>
 
 #ifdef RTE_EXEC_ENV_WINDOWS
@@ -162,20 +164,40 @@ __rte_raw_cksum(const void *buf, size_t len,
 uint32_t sum)  {
 	/* extend strict-aliasing rules */
 	typedef uint16_t __attribute__((__may_alias__)) u16_p;
-	const u16_p *u16_buf = (const u16_p *)buf;
-	const u16_p *end = u16_buf + len / sizeof(*u16_buf);
+	const u16_p *u16_buf;
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, size_t len, · line 177
Quoted 10 lines
lib/net/rte_ip.h +10 -0
+	const u16_p *end;
+	uint32_t bsum = 0;
+	const bool unaligned = (uintptr_t)buf & 1;
+
+	/* if buffer is unaligned, keeping it byte order independent */
+	if (unlikely(unaligned)) {
+		uint16_t first = 0;
+		if (unlikely(len == 0))
+			return 0;
+		((unsigned char *)&first)[1] = *(const unsigned
char *)buf;
Quoted 6 lines
lib/net/rte_ip.h +5 -0
+		bsum += first;
+		buf = RTE_PTR_ADD(buf, 1);
+		len--;
+	}
 
+	/* aligned access for compiler auto-vectorization */
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, size_t len, uint32_t sum)
Quoted 10 lines
lib/net/rte_ip.h +3 -1
+	u16_buf = (const u16_p *)buf;
+	end = u16_buf + len / sizeof(*u16_buf);
 	for (; u16_buf != end; ++u16_buf)
-		sum += *u16_buf;
+		bsum += *u16_buf;
 
 	/* if length is odd, keeping it byte order independent */
 	if (unlikely(len % 2)) {
 		uint16_t left = 0;
 		*(unsigned char *)&left = *(const unsigned char
*)end;
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, size_t len, uint32_t sum)
Quoted 8 lines
lib/net/rte_ip.h +4 -2
-		sum += left;
+		bsum += left;
 	}
 
-	return sum;
+	/* if buffer is unaligned, swap the checksum bytes */
+	if (unlikely(unaligned))
+		bsum = (bsum & 0xFF00FF00) >> 8 | (bsum &
0x00FF00FF) << 8;
Quoted 7 lines
lib/net/rte_ip.h +2 -1
+
+	return sum + bsum;
 }
 
 /**
--
2.17.1
@Emil, thank you for thoroughly reviewing the previous versions.

Quoted 7 lines

Quoted 1 line
If your test succeeds and you are satisfied with the patch, remember
to reply
Quoted 1 line
with a "Tested-by" tag for patchwork.
The test succeeded and I'm satisfied with the patch. I added 'Tested-
by: Emil Berg <[email protected]>' to the patch above, hopefully as you intended.

Thank you for testing! You don't need to put the tag inside the patch. Next time, just put the tag in your reply, like here, and Patchwork will catch it.

Tested-by Emil Berg <[email protected]>

The same goes for other tags, like Acked-by, Reviewed-by, etc..

-Morten

Mattias Rönnblom

2022-06-27 12:28 UTC #34 lore raw

Re: [PATCH v4] net: fix checksum with unaligned buffer

Quoted 77 lines · Morten Brørup

Quoted 77 lines
From: Morten Brørup [mailto:[email protected]]
Sent: Thursday, 23 June 2022 14.39
With this patch, the checksum can be calculated on an unaligned buffer. I.e. the buf parameter is no longer required to be 16 bit aligned.
The checksum is still calculated using a 16 bit aligned pointer, so the compiler can auto-vectorize the function's inner loop.
When the buffer is unaligned, the first byte of the buffer is handled separately. Furthermore, the calculated checksum of the buffer is byte shifted before being added to the initial checksum, to compensate for the
checksum having been calculated on the buffer shifted by one byte.
v4:
* Add copyright notice.
* Include stdbool.h (Emil Berg).
* Use RTE_PTR_ADD (Emil Berg).
* Fix one more typo in commit message. Is 'unligned' even a word? v3:
* Remove braces from single statement block.
* Fix typo in commit message.
v2:
* Do not assume that the buffer is part of an aligned packet buffer.
Bugzilla ID: 1035
Cc: [email protected]
Signed-off-by Morten Brørup <[email protected]>
---
  lib/net/rte_ip.h | 32 +++++++++++++++++++++++++++-----
  1 file changed, 27 insertions(+), 5 deletions(-)
lib/net/rte_ip.h +17 -2
index b502481670..738d643da0 100644
@@ -3,6 +3,7 @@
   *      The Regents of the University of California.
   * Copyright(c) 2010-2014 Intel Corporation.
   * Copyright(c) 2014 6WIND S.A.
+ * Copyright(c) 2022 SmartShare Systems.
   * All rights reserved.
   */
 
@@ -15,6 +16,7 @@
   * IP-related defines
   */
 
+#include <stdbool.h>
  #include <stdint.h>
 
  #ifdef RTE_EXEC_ENV_WINDOWS
@@ -162,20 +164,40 @@ __rte_raw_cksum(const void *buf, size_t len,
 uint32_t sum)
  {
  	/* extend strict-aliasing rules */
  	typedef uint16_t __attribute__((__may_alias__)) u16_p;
-	const u16_p *u16_buf = (const u16_p *)buf;
-	const u16_p *end = u16_buf + len / sizeof(*u16_buf);
+	const u16_p *u16_buf;
+	const u16_p *end;
+	uint32_t bsum = 0;
+	const bool unaligned = (uintptr_t)buf & 1;
+
+	/* if buffer is unaligned, keeping it byte order independent */
+	if (unlikely(unaligned)) {
+		uint16_t first = 0;
+		if (unlikely(len == 0))
+			return 0;
+		((unsigned char *)&first)[1] = *(const unsigned char *)buf;
+		bsum += first;
+		buf = RTE_PTR_ADD(buf, 1);
+		len--;
+	}
 

Quoted 1 line

lib/net/rte_ip.h __rte_raw_cksum(const void *buf, size_t len, uint32_t sum) · line 183
Quoted 1 line
lib/net/rte_ip.h +1 -0
+	/* aligned access for compiler auto-vectorization */

The compiler will be able to auto vectorize even unaligned accesses, just with different instructions. From what I can tell, there's no performance impact, at least not on the x86_64 systems I tried on.

I think you should remove the first special case conditional and use memcpy() instead of the cumbersome __may_alias__ construct to retrieve the data.

Quoted 30 lines

Quoted 22 lines

lib/net/rte_ip.h +9 -3

+	u16_buf = (const u16_p *)buf;
+	end = u16_buf + len / sizeof(*u16_buf);
  	for (; u16_buf != end; ++u16_buf)
-		sum += *u16_buf;
+		bsum += *u16_buf;
 
  	/* if length is odd, keeping it byte order independent */
  	if (unlikely(len % 2)) {
  		uint16_t left = 0;
  		*(unsigned char *)&left = *(const unsigned char *)end;
-		sum += left;
+		bsum += left;
  	}
 
-	return sum;
+	/* if buffer is unaligned, swap the checksum bytes */
+	if (unlikely(unaligned))
+		bsum = (bsum & 0xFF00FF00) >> 8 | (bsum & 0x00FF00FF) << 8;
+
+	return sum + bsum;
  }

Quoted 3 lines

/**
--
2.17.1

@Emil, thank you for thoroughly reviewing the previous versions.

If your test succeeds and you are satisfied with the patch, remember to reply with a "Tested-by" tag for patchwork.

Morten Brørup

2022-06-27 13:22 UTC #37 lore raw

RE: [PATCH v4] net: fix checksum with unaligned buffer

[1] http://inbox.dpdk.org/dev/[email protected]/
[2] http://inbox.dpdk.org/dev/[email protected]/

Quoted 117 lines

From: Emil Berg [mailto:[email protected]]
Sent: Monday, 27 June 2022 14.51
Quoted 11 lines
From: Emil Berg
Sent: den 27 juni 2022 14:46
Quoted 8 lines
From: Mattias Rönnblom <[email protected]>
Sent: den 27 juni 2022 14:28
Quoted 4 lines · Morten Brørup
Quoted 4 lines
From: Morten Brørup [mailto:[email protected]]
Sent: Thursday, 23 June 2022 14.39
With this patch, the checksum can be calculated on an unaligned
buffer.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
I.e. the buf parameter is no longer required to be 16 bit
aligned.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
The checksum is still calculated using a 16 bit aligned pointer,
so
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
the compiler can auto-vectorize the function's inner loop.
When the buffer is unaligned, the first byte of the buffer is handled separately. Furthermore, the calculated checksum of the buffer is byte shifted before being added to the initial
checksum,
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
to compensate for the checksum having been calculated on the
buffer
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
shifted by one byte.
v4:
* Add copyright notice.
* Include stdbool.h (Emil Berg).
* Use RTE_PTR_ADD (Emil Berg).
* Fix one more typo in commit message. Is 'unligned' even a
word?
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
v3:
* Remove braces from single statement block.
* Fix typo in commit message.
v2:
* Do not assume that the buffer is part of an aligned packet
buffer.
Quoted 25 lines
Quoted 25 lines
Quoted 25 lines
Quoted 25 lines
Bugzilla ID: 1035
Cc: [email protected]
Signed-off-by Morten Brørup <[email protected]>
---
  lib/net/rte_ip.h | 32 +++++++++++++++++++++++++++-----
  1 file changed, 27 insertions(+), 5 deletions(-)
lib/net/rte_ip.h index renamed from lib/net/rte_ip.h +0 -0
b502481670..738d643da0 100644
lib/net/rte_ip.h +1 -0
@@ -3,6 +3,7 @@
   *      The Regents of the University of California.
   * Copyright(c) 2010-2014 Intel Corporation.
   * Copyright(c) 2014 6WIND S.A.
+ * Copyright(c) 2022 SmartShare Systems.
   * All rights reserved.
   */
 
@@ -15,6 +16,7 @@
   * IP-related defines
   */
 
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, size_t
Quoted 5 lines
lib/net/rte_ip.h +1 -0
+#include <stdbool.h>
  #include <stdint.h>
 
  #ifdef RTE_EXEC_ENV_WINDOWS
@@ -162,20 +164,40 @@ __rte_raw_cksum(const void *buf, size_t
len,
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
lib/net/rte_ip.h +0 -0
 uint32_t sum)
  {
  	/* extend strict-aliasing rules */
  	typedef uint16_t __attribute__((__may_alias__)) u16_p;
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, size_t len, uint32_t sum)
Quoted 8 lines
lib/net/rte_ip.h +6 -2
-	const u16_p *u16_buf = (const u16_p *)buf;
-	const u16_p *end = u16_buf + len / sizeof(*u16_buf);
+	const u16_p *u16_buf;
+	const u16_p *end;
+	uint32_t bsum = 0;
+	const bool unaligned = (uintptr_t)buf & 1;
+
+	/* if buffer is unaligned, keeping it byte order
independent */
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, size_t len, uint32_t sum)
Quoted 5 lines
lib/net/rte_ip.h +5 -0
+	if (unlikely(unaligned)) {
+		uint16_t first = 0;
+		if (unlikely(len == 0))
+			return 0;
+		((unsigned char *)&first)[1] = *(const unsigned
char *)buf;
Quoted 8 lines
Quoted 8 lines
Quoted 6 lines
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, size_t len, uint32_t sum) · line 183
Quoted 6 lines
lib/net/rte_ip.h +5 -0
+		bsum += first;
+		buf = RTE_PTR_ADD(buf, 1);
+		len--;
+	}
 
+	/* aligned access for compiler auto-vectorization */
The compiler will be able to auto vectorize even unaligned
accesses,
Quoted 4 lines
Quoted 4 lines
just with different instructions. From what I can tell, there's no performance impact, at least not on the x86_64 systems I tried on.
I think you should remove the first special case conditional and
use
Quoted 1 line
Quoted 1 line
memcpy() instead of the cumbersome __may_alias__ construct to
retrieve
Quoted 6 lines
Quoted 2 lines
the data.
Here:
https://www.agner.org/optimize/instruction_tables.pdf
it lists the latency of vmovdqa (aligned) as 6 cycles and the latency
for
Quoted 1 line
vmovdqu (unaligned) as 7 cycles. So I guess there can be some
difference.

Quoted 9 lines

Quoted 1 line
Although in practice I'm not sure what difference it makes. I've not
seen any
Quoted 2 lines
difference in runtime between the two versions.
Correction to my comment:
Those stats are for some older CPU. For some newer CPUs such as Tiger Lake the stats seem to be the same regardless of aligned or unaligned.

I agree that the memcpy method is more elegant and easy to read.

However, we would need to performance test the modified checksum function with a large number of CPUs to prove that we don't introduce a performance regression on any CPU architecture still supported by DPDK. And Emil already found a CPU where it costs 1 extra cycle per 16 bytes, which adds up to a total of ca. 91 extra cycles on a 1460 byte TCP packet.

So I opted for a solution with zero changes to the inner loop, so no performance retesting is required (for the previously supported use cases, where the buffer is aligned).

I have previously submitted a couple of patches to fix some minor bugs in the mempool cache functions [1] and [2], while also refactoring the functions for readability. But after having incorporated various feedback, nobody wants to proceed with the patches, probably due to fear of performance regressions. I didn't want to risk the same with this patch.

Mattias Rönnblom

2022-06-27 17:22 UTC #38 lore raw

Re: [PATCH v4] net: fix checksum with unaligned buffer

Quoted 122 lines · Morten Brørup

Quoted 122 lines
From: Emil Berg [mailto:[email protected]]
Sent: Monday, 27 June 2022 14.51
Quoted 11 lines
From: Emil Berg
Sent: den 27 juni 2022 14:46
Quoted 8 lines
From: Mattias Rönnblom <[email protected]>
Sent: den 27 juni 2022 14:28
Quoted 4 lines · Morten Brørup
Quoted 4 lines
From: Morten Brørup [mailto:[email protected]]
Sent: Thursday, 23 June 2022 14.39
With this patch, the checksum can be calculated on an unaligned
buffer.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
I.e. the buf parameter is no longer required to be 16 bit
aligned.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
The checksum is still calculated using a 16 bit aligned pointer,
so
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
the compiler can auto-vectorize the function's inner loop.
When the buffer is unaligned, the first byte of the buffer is handled separately. Furthermore, the calculated checksum of the buffer is byte shifted before being added to the initial
checksum,
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
to compensate for the checksum having been calculated on the
buffer
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
shifted by one byte.
v4:
* Add copyright notice.
* Include stdbool.h (Emil Berg).
* Use RTE_PTR_ADD (Emil Berg).
* Fix one more typo in commit message. Is 'unligned' even a
word?
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
v3:
* Remove braces from single statement block.
* Fix typo in commit message.
v2:
* Do not assume that the buffer is part of an aligned packet
buffer.
Quoted 25 lines
Quoted 25 lines
Quoted 25 lines
Quoted 25 lines
Bugzilla ID: 1035
Cc: [email protected]
Signed-off-by Morten Brørup <[email protected]>
---
   lib/net/rte_ip.h | 32 +++++++++++++++++++++++++++-----
   1 file changed, 27 insertions(+), 5 deletions(-)
lib/net/rte_ip.h index renamed from lib/net/rte_ip.h +0 -0
b502481670..738d643da0 100644
lib/net/rte_ip.h +1 -0
@@ -3,6 +3,7 @@
    *      The Regents of the University of California.
    * Copyright(c) 2010-2014 Intel Corporation.
    * Copyright(c) 2014 6WIND S.A.
+ * Copyright(c) 2022 SmartShare Systems.
    * All rights reserved.
    */
 
@@ -15,6 +16,7 @@
    * IP-related defines
    */
 
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, size_t
Quoted 5 lines
lib/net/rte_ip.h +1 -0
+#include <stdbool.h>
   #include <stdint.h>
 
   #ifdef RTE_EXEC_ENV_WINDOWS
@@ -162,20 +164,40 @@ __rte_raw_cksum(const void *buf, size_t
len,
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
lib/net/rte_ip.h +0 -0
 uint32_t sum)
   {
   	/* extend strict-aliasing rules */
   	typedef uint16_t __attribute__((__may_alias__)) u16_p;
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, size_t len, uint32_t sum)
Quoted 8 lines
lib/net/rte_ip.h +6 -2
-	const u16_p *u16_buf = (const u16_p *)buf;
-	const u16_p *end = u16_buf + len / sizeof(*u16_buf);
+	const u16_p *u16_buf;
+	const u16_p *end;
+	uint32_t bsum = 0;
+	const bool unaligned = (uintptr_t)buf & 1;
+
+	/* if buffer is unaligned, keeping it byte order
independent */
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, size_t len, uint32_t sum)
Quoted 5 lines
lib/net/rte_ip.h +5 -0
+	if (unlikely(unaligned)) {
+		uint16_t first = 0;
+		if (unlikely(len == 0))
+			return 0;
+		((unsigned char *)&first)[1] = *(const unsigned
char *)buf;
Quoted 8 lines
Quoted 8 lines
Quoted 6 lines
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, size_t len, uint32_t sum) · line 183
Quoted 6 lines
lib/net/rte_ip.h +5 -0
+		bsum += first;
+		buf = RTE_PTR_ADD(buf, 1);
+		len--;
+	}
 
+	/* aligned access for compiler auto-vectorization */
The compiler will be able to auto vectorize even unaligned
accesses,
Quoted 4 lines
Quoted 4 lines
just with different instructions. From what I can tell, there's no performance impact, at least not on the x86_64 systems I tried on.
I think you should remove the first special case conditional and
use
Quoted 1 line
Quoted 1 line
memcpy() instead of the cumbersome __may_alias__ construct to
retrieve
Quoted 6 lines
Quoted 2 lines
the data.
Here:
https://www.agner.org/optimize/instruction_tables.pdf
it lists the latency of vmovdqa (aligned) as 6 cycles and the latency
for
Quoted 1 line
vmovdqu (unaligned) as 7 cycles. So I guess there can be some
difference.
Quoted 1 line
Although in practice I'm not sure what difference it makes. I've not
seen any
Quoted 2 lines
difference in runtime between the two versions.

Quoted 9 lines

Quoted 4 lines
Correction to my comment:
Those stats are for some older CPU. For some newer CPUs such as Tiger Lake the stats seem to be the same regardless of aligned or unaligned.
I agree that the memcpy method is more elegant and easy to read.
However, we would need to performance test the modified checksum function with a large number of CPUs to prove that we don't introduce a performance regression on any CPU architecture still supported by DPDK. And Emil already found a CPU where it costs 1 extra cycle per 16 bytes, which adds up to a total of ca. 91 extra cycles on a 1460 byte TCP packet.

I think you've misunderstood what latency means in such tables. It's a data dependency thing, not a measure of throughput. The throughput is *much* higher. My guess would be two such instruction per clock.

For your 1460 bytes example, my Zen3 AMD needs performs identical with both the current DPDK implementation, your patch, and a memcpy()-ified version of the current implementation. They all need ~130 clock cycles/packet, with warm caches. IPC is 3 instructions per cycle, but obvious not all instructions are SIMD.

The main issue with checksumming on the CPU is, in my experience, not that you don't have enough compute, but that you trash the caches.

Quoted 2 lines

So I opted for a solution with zero changes to the inner loop, so no performance retesting is required (for the previously supported use cases, where the buffer is aligned).

You will see performance degradation with this solution as well, under certain conditions. For unaligned 100 bytes of data, the current DPDK implementation and the memcpy()-fied version needs ~21 cc/packet. Your patch needs 54 cc/packet.

But the old version didn't support unaligned accesses? In many compiler flag/machine combinations it did.

Quoted 5 lines

I have previously submitted a couple of patches to fix some minor bugs in the mempool cache functions [1] and [2], while also refactoring the functions for readability. But after having incorporated various feedback, nobody wants to proceed with the patches, probably due to fear of performance regressions. I didn't want to risk the same with this patch.
[1] http://inbox.dpdk.org/dev/[email protected]/
[2] http://inbox.dpdk.org/dev/[email protected]/

Morten Brørup

2022-06-27 20:21 UTC #39 lore raw

RE: [PATCH v4] net: fix checksum with unaligned buffer

Quoted 51 lines

From: Mattias Rönnblom [mailto:[email protected]]
Sent: Monday, 27 June 2022 19.23
Quoted 47 lines · Morten Brørup
Quoted 47 lines
From: Emil Berg [mailto:[email protected]]
Sent: Monday, 27 June 2022 14.51
Quoted 11 lines
From: Emil Berg
Sent: den 27 juni 2022 14:46
Quoted 8 lines
From: Mattias Rönnblom <[email protected]>
Sent: den 27 juni 2022 14:28
Quoted 4 lines · Morten Brørup
Quoted 4 lines
From: Morten Brørup [mailto:[email protected]]
Sent: Thursday, 23 June 2022 14.39
With this patch, the checksum can be calculated on an unaligned
buffer.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
I.e. the buf parameter is no longer required to be 16 bit
aligned.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
The checksum is still calculated using a 16 bit aligned pointer,
so
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
the compiler can auto-vectorize the function's inner loop.
When the buffer is unaligned, the first byte of the buffer is handled separately. Furthermore, the calculated checksum of the buffer is byte shifted before being added to the initial
checksum,
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
to compensate for the checksum having been calculated on the
buffer
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
shifted by one byte.
v4:
* Add copyright notice.
* Include stdbool.h (Emil Berg).
* Use RTE_PTR_ADD (Emil Berg).
* Fix one more typo in commit message. Is 'unligned' even a
word?
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
v3:
* Remove braces from single statement block.
* Fix typo in commit message.
v2:
* Do not assume that the buffer is part of an aligned packet
buffer.
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Bugzilla ID: 1035
Cc: [email protected]
Signed-off-by Morten Brørup <[email protected]>

[...]

Quoted 47 lines

Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
The compiler will be able to auto vectorize even unaligned
accesses,
Quoted 4 lines
Quoted 4 lines
just with different instructions. From what I can tell, there's no performance impact, at least not on the x86_64 systems I tried on.
I think you should remove the first special case conditional and
use
Quoted 8 lines
Quoted 8 lines
Quoted 1 line
Quoted 1 line
memcpy() instead of the cumbersome __may_alias__ construct to
retrieve
Quoted 6 lines
Quoted 2 lines
the data.
Here:
https://www.agner.org/optimize/instruction_tables.pdf
it lists the latency of vmovdqa (aligned) as 6 cycles and the
latency
Quoted 4 lines
Quoted 4 lines
for
Quoted 1 line
vmovdqu (unaligned) as 7 cycles. So I guess there can be some
difference.
Quoted 1 line
Although in practice I'm not sure what difference it makes. I've
not
Quoted 6 lines
Quoted 6 lines
seen any
Quoted 2 lines
difference in runtime between the two versions.
Correction to my comment:
Those stats are for some older CPU. For some newer CPUs such as
Tiger
Quoted 1 line
Quoted 1 line
Lake the stats seem to be the same regardless of aligned or
unaligned.
Quoted 5 lines
Quoted 1 line
I agree that the memcpy method is more elegant and easy to read.
However, we would need to performance test the modified checksum
function with a large number of CPUs to prove that we don't introduce a performance regression on any CPU architecture still supported by DPDK. And Emil already found a CPU where it costs 1 extra cycle per 16 bytes, which adds up to a total of ca. 91 extra cycles on a 1460 byte TCP packet.
Quoted 1 line
I think you've misunderstood what latency means in such tables. It's a data dependency thing, not a measure of throughput. The throughput is *much* higher. My guess would be two such instruction per clock.

Quoted 5 lines

For your 1460 bytes example, my Zen3 AMD needs performs identical with both the current DPDK implementation, your patch, and a memcpy()-ified version of the current implementation. They all need ~130 clock cycles/packet, with warm caches. IPC is 3 instructions per cycle, but obvious not all instructions are SIMD.

You're right, I wasn't thinking deeper about it before extrapolating.

Great to see some real numbers! I wish someone would do the same testing on an old ARM CPU, so we could also see the other end of the scale.

Quoted 2 lines

The main issue with checksumming on the CPU is, in my experience, not that you don't have enough compute, but that you trash the caches.

Agree. I have noticed that x86 has "non-temporal" instruction variants to load/store data without trashing the cache entirely.

A variant of the checksum function using such instructions might be handy.

Variants of the memcpy function using such instructions might also be handy for some purposes, e.g. copying the contents of packets, where the original and/or copy will not accessed shortly thereafter.

Quoted 9 lines

Quoted 1 line
So I opted for a solution with zero changes to the inner loop, so no
performance retesting is required (for the previously supported use cases, where the buffer is aligned).
Quoted 1 line
You will see performance degradation with this solution as well, under certain conditions. For unaligned 100 bytes of data, the current DPDK implementation and the memcpy()-fied version needs ~21 cc/packet. Your patch needs 54 cc/packet.

Yes, it's a tradeoff. I exclusively aimed at maintaining performance for the case with aligned buffers (under all circumstances, with all CPUs etc.), and ignored how it affects the performance for the case with unaligned buffers.

Unlike this patch, the memcpy() variant has no additional branches for the unaligned case, so its performance should be generally unaffected by the buffer being aligned or not. However, I don't have sufficient in-depth CPU knowledge to say if this also applies to RISCV and older ARM CPUs still supported by DPDK.

I don't have strong feelings for this patch, so you can provide a memcpy() based alternative patch, and we will let the community decide which one they prefer.

Quoted 2 lines

But the old version didn't support unaligned accesses? In many compiler flag/machine combinations it did.

It crashed in some cases. That was the point of the bug report [1], and the reason to provide a patch.

[1] https://bugs.dpdk.org/show_bug.cgi?id=1035

Mattias Rönnblom

2022-06-28 06:28 UTC #40 lore raw

Re: [PATCH v4] net: fix checksum with unaligned buffer

Quoted 107 lines · Morten Brørup

Quoted 51 lines
From: Mattias Rönnblom [mailto:[email protected]]
Sent: Monday, 27 June 2022 19.23
Quoted 47 lines · Morten Brørup
Quoted 47 lines
From: Emil Berg [mailto:[email protected]]
Sent: Monday, 27 June 2022 14.51
Quoted 11 lines
From: Emil Berg
Sent: den 27 juni 2022 14:46
Quoted 8 lines
From: Mattias Rönnblom <[email protected]>
Sent: den 27 juni 2022 14:28
Quoted 4 lines · Morten Brørup
Quoted 4 lines
From: Morten Brørup [mailto:[email protected]]
Sent: Thursday, 23 June 2022 14.39
With this patch, the checksum can be calculated on an unaligned
buffer.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
I.e. the buf parameter is no longer required to be 16 bit
aligned.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
The checksum is still calculated using a 16 bit aligned pointer,
so
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
the compiler can auto-vectorize the function's inner loop.
When the buffer is unaligned, the first byte of the buffer is handled separately. Furthermore, the calculated checksum of the buffer is byte shifted before being added to the initial
checksum,
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
to compensate for the checksum having been calculated on the
buffer
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
shifted by one byte.
v4:
* Add copyright notice.
* Include stdbool.h (Emil Berg).
* Use RTE_PTR_ADD (Emil Berg).
* Fix one more typo in commit message. Is 'unligned' even a
word?
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
v3:
* Remove braces from single statement block.
* Fix typo in commit message.
v2:
* Do not assume that the buffer is part of an aligned packet
buffer.
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Bugzilla ID: 1035
Cc: [email protected]
Signed-off-by Morten Brørup <[email protected]>
[...]
Quoted 47 lines
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
The compiler will be able to auto vectorize even unaligned
accesses,
Quoted 4 lines
Quoted 4 lines
just with different instructions. From what I can tell, there's no performance impact, at least not on the x86_64 systems I tried on.
I think you should remove the first special case conditional and
use
Quoted 8 lines
Quoted 8 lines
Quoted 1 line
Quoted 1 line
memcpy() instead of the cumbersome __may_alias__ construct to
retrieve
Quoted 6 lines
Quoted 2 lines
the data.
Here:
https://www.agner.org/optimize/instruction_tables.pdf
it lists the latency of vmovdqa (aligned) as 6 cycles and the
latency
Quoted 4 lines
Quoted 4 lines
for
Quoted 1 line
vmovdqu (unaligned) as 7 cycles. So I guess there can be some
difference.
Quoted 1 line
Although in practice I'm not sure what difference it makes. I've
not
Quoted 6 lines
Quoted 6 lines
seen any
Quoted 2 lines
difference in runtime between the two versions.
Correction to my comment:
Those stats are for some older CPU. For some newer CPUs such as
Tiger
Quoted 1 line
Quoted 1 line
Lake the stats seem to be the same regardless of aligned or
unaligned.
Quoted 5 lines
Quoted 1 line
I agree that the memcpy method is more elegant and easy to read.
However, we would need to performance test the modified checksum
function with a large number of CPUs to prove that we don't introduce a performance regression on any CPU architecture still supported by DPDK. And Emil already found a CPU where it costs 1 extra cycle per 16 bytes, which adds up to a total of ca. 91 extra cycles on a 1460 byte TCP packet.
Quoted 1 line
I think you've misunderstood what latency means in such tables. It's a data dependency thing, not a measure of throughput. The throughput is *much* higher. My guess would be two such instruction per clock.
Quoted 5 lines
For your 1460 bytes example, my Zen3 AMD needs performs identical with both the current DPDK implementation, your patch, and a memcpy()-ified version of the current implementation. They all need ~130 clock cycles/packet, with warm caches. IPC is 3 instructions per cycle, but obvious not all instructions are SIMD.

Quoted 4 lines

You're right, I wasn't thinking deeper about it before extrapolating.
Great to see some real numbers! I wish someone would do the same testing on an old ARM CPU, so we could also see the other end of the scale.

I've ran it on an ARM A72. For the aligned 1460 bytes case I got: Current DPDK ~572 cc. Your patch: ~578 cc. Memcpy-fied: ~573 cc. They performed about the same for all unaligned/aligned and sizes I tested. This platform (or could be GCC version as well) doesn't suffer from the unaligned performance degradation your patch showed on my AMD machine.

Quoted 7 lines

Quoted 2 lines
The main issue with checksumming on the CPU is, in my experience, not that you don't have enough compute, but that you trash the caches.
Agree. I have noticed that x86 has "non-temporal" instruction variants to load/store data without trashing the cache entirely.
A variant of the checksum function using such instructions might be handy.

Yes, although you may need to prefetch the payload for good performance.

Quoted 2 lines

Variants of the memcpy function using such instructions might also be handy for some purposes, e.g. copying the contents of packets, where the original and/or copy will not accessed shortly thereafter.

Indeed and I think it's been discussed on the list. There's some work to get it right, since alignment requirement and the fact a different memory model is used for those SIMD instructions causes trouble for a generic implementation. (For x86_64.)

Quoted 5 lines

Quoted 5 lines
Quoted 1 line
So I opted for a solution with zero changes to the inner loop, so no
performance retesting is required (for the previously supported use cases, where the buffer is aligned).
Quoted 1 line

Quoted 9 lines

Quoted 4 lines
You will see performance degradation with this solution as well, under certain conditions. For unaligned 100 bytes of data, the current DPDK implementation and the memcpy()-fied version needs ~21 cc/packet. Your patch needs 54 cc/packet.
Yes, it's a tradeoff. I exclusively aimed at maintaining performance for the case with aligned buffers (under all circumstances, with all CPUs etc.), and ignored how it affects the performance for the case with unaligned buffers.
Unlike this patch, the memcpy() variant has no additional branches for the unaligned case, so its performance should be generally unaffected by the buffer being aligned or not. However, I don't have sufficient in-depth CPU knowledge to say if this also applies to RISCV and older ARM CPUs still supported by DPDK.

I don't think avoiding RISCV non-catastrophic regressions triumphs improving performance on mainstream CPUs and avoiding code quality regressions.

Quoted 2 lines

I don't have strong feelings for this patch, so you can provide a memcpy() based alternative patch, and we will let the community decide which one they prefer.

This is what my conversion looks like:

static inline uint32_t
__rte_raw_cksum(const void *buf, size_t len, uint32_t sum)

{
	const void *end;

	for (end = RTE_PTR_ADD(buf, (len/sizeof(uint16_t)) * sizeof(uint16_t));
	     buf != end; buf = RTE_PTR_ADD(buf, sizeof(uint16_t))) {
		uint16_t v;

		memcpy(&v, buf, sizeof(uint16_t));
		sum += v;
	}

	/* if length is odd, keeping it byte order independent */
	if (unlikely(len % 2)) {
		uint8_t last;
		uint16_t left = 0;

		memcpy(&last, end, 1);
		*(unsigned char *)&left = last;
		sum += left;
	}

	return sum;
}

Maybe the maintainer can have an opinion, before I provide a real patch, to save me some work. I would really like to contribute some performance autotests in that case. I mean, you could be right and the performance could be totally off on some platform.

Quoted 7 lines

Quoted 2 lines
But the old version didn't support unaligned accesses? In many compiler flag/machine combinations it did.
It crashed in some cases. That was the point of the bug report [1], and the reason to provide a patch.
[1] https://bugs.dpdk.org/show_bug.cgi?id=1035

Emil Berg

2022-06-29 14:55 UTC #32 lore raw

RE: [PATCH v4] net: fix checksum with unaligned buffer

Quoted 113 lines

-----Original Message-----
From: Morten Brørup <[email protected]>
Sent: den 23 juni 2022 14:51
To: Emil Berg <[email protected]>; [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: RE: [PATCH v4] net: fix checksum with unaligned buffer
Quoted 61 lines
From: Morten Brørup [mailto:[email protected]]
Sent: Thursday, 23 June 2022 14.39
With this patch, the checksum can be calculated on an unaligned buffer. I.e. the buf parameter is no longer required to be 16 bit aligned.
The checksum is still calculated using a 16 bit aligned pointer, so the compiler can auto-vectorize the function's inner loop.
When the buffer is unaligned, the first byte of the buffer is handled separately. Furthermore, the calculated checksum of the buffer is byte shifted before being added to the initial checksum, to compensate for the checksum having been calculated on the buffer shifted by one byte.
v4:
* Add copyright notice.
* Include stdbool.h (Emil Berg).
* Use RTE_PTR_ADD (Emil Berg).
* Fix one more typo in commit message. Is 'unligned' even a word? v3:
* Remove braces from single statement block.
* Fix typo in commit message.
v2:
* Do not assume that the buffer is part of an aligned packet buffer.
Bugzilla ID: 1035
Cc: [email protected]
Signed-off-by Morten Brørup <[email protected]>
Tested-by Emil Berg <[email protected]>
---
 lib/net/rte_ip.h | 32 +++++++++++++++++++++++++++-----
 1 file changed, 27 insertions(+), 5 deletions(-)
lib/net/rte_ip.h index renamed from lib/net/rte_ip.h +0 -0
b502481670..738d643da0 100644
lib/net/rte_ip.h +3 -2
@@ -3,6 +3,7 @@
  *      The Regents of the University of California.
  * Copyright(c) 2010-2014 Intel Corporation.
  * Copyright(c) 2014 6WIND S.A.
+ * Copyright(c) 2022 SmartShare Systems.
  * All rights reserved.
  */
 
@@ -15,6 +16,7 @@
  * IP-related defines
  */
 
+#include <stdbool.h>
 #include <stdint.h>
 
 #ifdef RTE_EXEC_ENV_WINDOWS
@@ -162,20 +164,40 @@ __rte_raw_cksum(const void *buf, size_t len,
 uint32_t sum)  {
 	/* extend strict-aliasing rules */
 	typedef uint16_t __attribute__((__may_alias__)) u16_p;
-	const u16_p *u16_buf = (const u16_p *)buf;
-	const u16_p *end = u16_buf + len / sizeof(*u16_buf);
+	const u16_p *u16_buf;
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, size_t len, · line 177
Quoted 10 lines
lib/net/rte_ip.h +10 -0
+	const u16_p *end;
+	uint32_t bsum = 0;
+	const bool unaligned = (uintptr_t)buf & 1;
+
+	/* if buffer is unaligned, keeping it byte order independent */
+	if (unlikely(unaligned)) {
+		uint16_t first = 0;
+		if (unlikely(len == 0))
+			return 0;
+		((unsigned char *)&first)[1] = *(const unsigned
char *)buf;
Quoted 6 lines
lib/net/rte_ip.h +5 -0
+		bsum += first;
+		buf = RTE_PTR_ADD(buf, 1);
+		len--;
+	}
 
+	/* aligned access for compiler auto-vectorization */
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, size_t len, uint32_t sum)
Quoted 10 lines
lib/net/rte_ip.h +3 -1
+	u16_buf = (const u16_p *)buf;
+	end = u16_buf + len / sizeof(*u16_buf);
 	for (; u16_buf != end; ++u16_buf)
-		sum += *u16_buf;
+		bsum += *u16_buf;
 
 	/* if length is odd, keeping it byte order independent */
 	if (unlikely(len % 2)) {
 		uint16_t left = 0;
 		*(unsigned char *)&left = *(const unsigned char
*)end;
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, size_t len, uint32_t sum)
Quoted 8 lines
lib/net/rte_ip.h +4 -2
-		sum += left;
+		bsum += left;
 	}
 
-	return sum;
+	/* if buffer is unaligned, swap the checksum bytes */
+	if (unlikely(unaligned))
+		bsum = (bsum & 0xFF00FF00) >> 8 | (bsum &
0x00FF00FF) << 8;
Quoted 4 lines
lib/net/rte_ip.h +2 -0
+
+	return sum + bsum;
 }
 

Quoted 8 lines

Quoted 3 lines
lib/net/rte_ip.h +0 -1
 /**
--
2.17.1
@Emil, thank you for thoroughly reviewing the previous versions.
If your test succeeds and you are satisfied with the patch, remember to reply with a "Tested-by" tag for patchwork.

The test succeeded and I'm satisfied with the patch. I added 'Tested-by: Emil Berg <[email protected]>' to the patch above, hopefully as you intended.

Emil Berg

2022-06-29 14:55 UTC #35 lore raw

RE: [PATCH v4] net: fix checksum with unaligned buffer

Quoted 96 lines

-----Original Message-----
From: Mattias Rönnblom <[email protected]>
Sent: den 27 juni 2022 14:28
To: Morten Brørup <[email protected]>; Emil Berg <[email protected]>; [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH v4] net: fix checksum with unaligned buffer
Quoted 64 lines · Morten Brørup
Quoted 64 lines
From: Morten Brørup [mailto:[email protected]]
Sent: Thursday, 23 June 2022 14.39
With this patch, the checksum can be calculated on an unaligned buffer. I.e. the buf parameter is no longer required to be 16 bit aligned.
The checksum is still calculated using a 16 bit aligned pointer, so the compiler can auto-vectorize the function's inner loop.
When the buffer is unaligned, the first byte of the buffer is handled separately. Furthermore, the calculated checksum of the buffer is byte shifted before being added to the initial checksum, to compensate for the checksum having been calculated on the buffer shifted by one byte.
v4:
* Add copyright notice.
* Include stdbool.h (Emil Berg).
* Use RTE_PTR_ADD (Emil Berg).
* Fix one more typo in commit message. Is 'unligned' even a word? v3:
* Remove braces from single statement block.
* Fix typo in commit message.
v2:
* Do not assume that the buffer is part of an aligned packet buffer.
Bugzilla ID: 1035
Cc: [email protected]
Signed-off-by Morten Brørup <[email protected]>
---
  lib/net/rte_ip.h | 32 +++++++++++++++++++++++++++-----
  1 file changed, 27 insertions(+), 5 deletions(-)
lib/net/rte_ip.h index renamed from lib/net/rte_ip.h +0 -0
b502481670..738d643da0 100644
lib/net/rte_ip.h +5 -2
@@ -3,6 +3,7 @@
   *      The Regents of the University of California.
   * Copyright(c) 2010-2014 Intel Corporation.
   * Copyright(c) 2014 6WIND S.A.
+ * Copyright(c) 2022 SmartShare Systems.
   * All rights reserved.
   */
 
@@ -15,6 +16,7 @@
   * IP-related defines
   */
 
+#include <stdbool.h>
  #include <stdint.h>
 
  #ifdef RTE_EXEC_ENV_WINDOWS
@@ -162,20 +164,40 @@ __rte_raw_cksum(const void *buf, size_t len,
 uint32_t sum)
  {
  	/* extend strict-aliasing rules */
  	typedef uint16_t __attribute__((__may_alias__)) u16_p;
-	const u16_p *u16_buf = (const u16_p *)buf;
-	const u16_p *end = u16_buf + len / sizeof(*u16_buf);
+	const u16_p *u16_buf;
+	const u16_p *end;
+	uint32_t bsum = 0;
Quoted 8 lines
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, size_t len, uint32_t sum)
Quoted 8 lines
lib/net/rte_ip.h +8 -0
+	const bool unaligned = (uintptr_t)buf & 1;
+
+	/* if buffer is unaligned, keeping it byte order independent */
+	if (unlikely(unaligned)) {
+		uint16_t first = 0;
+		if (unlikely(len == 0))
+			return 0;
+		((unsigned char *)&first)[1] = *(const unsigned
char *)buf;
Quoted 6 lines
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, size_t len, uint32_t sum) · line 183
Quoted 6 lines
lib/net/rte_ip.h +5 -0
+		bsum += first;
+		buf = RTE_PTR_ADD(buf, 1);
+		len--;
+	}
 
+	/* aligned access for compiler auto-vectorization */
The compiler will be able to auto vectorize even unaligned accesses, just with different instructions. From what I can tell, there's no performance impact, at least not on the x86_64 systems I tried on.

Quoted 4 lines

I think you should remove the first special case conditional and use memcpy() instead of the cumbersome __may_alias__ construct to retrieve the data.

Here:
https://www.agner.org/optimize/instruction_tables.pdf
it lists the latency of vmovdqa (aligned) as 6 cycles and the latency for vmovdqu (unaligned) as 7 cycles. So I guess there can be some difference. Although in practice I'm not sure what difference it makes. I've not seen any difference in runtime between the two versions.

Quoted 33 lines

Quoted 10 lines
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, size_t len, uint32_t sum)
Quoted 10 lines
lib/net/rte_ip.h +3 -1
+	u16_buf = (const u16_p *)buf;
+	end = u16_buf + len / sizeof(*u16_buf);
  	for (; u16_buf != end; ++u16_buf)
-		sum += *u16_buf;
+		bsum += *u16_buf;
 
  	/* if length is odd, keeping it byte order independent */
  	if (unlikely(len % 2)) {
  		uint16_t left = 0;
  		*(unsigned char *)&left = *(const unsigned char
*)end;
Quoted 8 lines
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, size_t len, uint32_t sum)
Quoted 8 lines
lib/net/rte_ip.h +4 -2
-		sum += left;
+		bsum += left;
  	}
 
-	return sum;
+	/* if buffer is unaligned, swap the checksum bytes */
+	if (unlikely(unaligned))
+		bsum = (bsum & 0xFF00FF00) >> 8 | (bsum &
0x00FF00FF) << 8;
Quoted 11 lines
Quoted 7 lines
lib/net/rte_ip.h +2 -0
+
+	return sum + bsum;
  }
 
/**
--
2.17.1
@Emil, thank you for thoroughly reviewing the previous versions.
If your test succeeds and you are satisfied with the patch, remember to
reply with a "Tested-by" tag for patchwork.
Quoted 1 line

Emil Berg

2022-06-29 14:55 UTC #36 lore raw

RE: [PATCH v4] net: fix checksum with unaligned buffer

Quoted 113 lines

-----Original Message-----
From: Emil Berg
Sent: den 27 juni 2022 14:46
To: Mattias Rönnblom <[email protected]>; Morten Brørup <[email protected]>; [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]
Subject: RE: [PATCH v4] net: fix checksum with unaligned buffer
Quoted 94 lines
-----Original Message-----
From: Mattias Rönnblom <[email protected]>
Sent: den 27 juni 2022 14:28
To: Morten Brørup <[email protected]>; Emil Berg <[email protected]>; [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH v4] net: fix checksum with unaligned buffer
Quoted 64 lines · Morten Brørup
Quoted 64 lines
From: Morten Brørup [mailto:[email protected]]
Sent: Thursday, 23 June 2022 14.39
With this patch, the checksum can be calculated on an unaligned buffer. I.e. the buf parameter is no longer required to be 16 bit aligned.
The checksum is still calculated using a 16 bit aligned pointer, so the compiler can auto-vectorize the function's inner loop.
When the buffer is unaligned, the first byte of the buffer is handled separately. Furthermore, the calculated checksum of the buffer is byte shifted before being added to the initial checksum, to compensate for the checksum having been calculated on the buffer shifted by one byte.
v4:
* Add copyright notice.
* Include stdbool.h (Emil Berg).
* Use RTE_PTR_ADD (Emil Berg).
* Fix one more typo in commit message. Is 'unligned' even a word? v3:
* Remove braces from single statement block.
* Fix typo in commit message.
v2:
* Do not assume that the buffer is part of an aligned packet buffer.
Bugzilla ID: 1035
Cc: [email protected]
Signed-off-by Morten Brørup <[email protected]>
---
  lib/net/rte_ip.h | 32 +++++++++++++++++++++++++++-----
  1 file changed, 27 insertions(+), 5 deletions(-)
lib/net/rte_ip.h index renamed from lib/net/rte_ip.h +0 -0
b502481670..738d643da0 100644
lib/net/rte_ip.h +5 -2
@@ -3,6 +3,7 @@
   *      The Regents of the University of California.
   * Copyright(c) 2010-2014 Intel Corporation.
   * Copyright(c) 2014 6WIND S.A.
+ * Copyright(c) 2022 SmartShare Systems.
   * All rights reserved.
   */
 
@@ -15,6 +16,7 @@
   * IP-related defines
   */
 
+#include <stdbool.h>
  #include <stdint.h>
 
  #ifdef RTE_EXEC_ENV_WINDOWS
@@ -162,20 +164,40 @@ __rte_raw_cksum(const void *buf, size_t len,
 uint32_t sum)
  {
  	/* extend strict-aliasing rules */
  	typedef uint16_t __attribute__((__may_alias__)) u16_p;
-	const u16_p *u16_buf = (const u16_p *)buf;
-	const u16_p *end = u16_buf + len / sizeof(*u16_buf);
+	const u16_p *u16_buf;
+	const u16_p *end;
+	uint32_t bsum = 0;
Quoted 8 lines
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, size_t len, uint32_t sum)
Quoted 8 lines
lib/net/rte_ip.h +8 -0
+	const bool unaligned = (uintptr_t)buf & 1;
+
+	/* if buffer is unaligned, keeping it byte order independent */
+	if (unlikely(unaligned)) {
+		uint16_t first = 0;
+		if (unlikely(len == 0))
+			return 0;
+		((unsigned char *)&first)[1] = *(const unsigned
char *)buf;
Quoted 6 lines
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, size_t len, uint32_t sum) · line 183
Quoted 6 lines
lib/net/rte_ip.h +5 -0
+		bsum += first;
+		buf = RTE_PTR_ADD(buf, 1);
+		len--;
+	}
 
+	/* aligned access for compiler auto-vectorization */
The compiler will be able to auto vectorize even unaligned accesses, just with different instructions. From what I can tell, there's no performance impact, at least not on the x86_64 systems I tried on.
Quoted 4 lines
I think you should remove the first special case conditional and use memcpy() instead of the cumbersome __may_alias__ construct to retrieve the data.

Quoted 7 lines

Here:
https://www.agner.org/optimize/instruction_tables.pdf
it lists the latency of vmovdqa (aligned) as 6 cycles and the latency for vmovdqu (unaligned) as 7 cycles. So I guess there can be some difference. Although in practice I'm not sure what difference it makes. I've not seen any difference in runtime between the two versions.

Correction to my comment:
Those stats are for some older CPU. For some newer CPUs such as Tiger Lake the stats seem to be the same regardless of aligned or unaligned.

Quoted 34 lines

Quoted 34 lines
Quoted 10 lines
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, size_t len, uint32_t sum)
Quoted 10 lines
lib/net/rte_ip.h +3 -1
+	u16_buf = (const u16_p *)buf;
+	end = u16_buf + len / sizeof(*u16_buf);
  	for (; u16_buf != end; ++u16_buf)
-		sum += *u16_buf;
+		bsum += *u16_buf;
 
  	/* if length is odd, keeping it byte order independent */
  	if (unlikely(len % 2)) {
  		uint16_t left = 0;
  		*(unsigned char *)&left = *(const unsigned char
*)end;
Quoted 8 lines
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, size_t len, uint32_t sum)
Quoted 8 lines
lib/net/rte_ip.h +4 -2
-		sum += left;
+		bsum += left;
  	}
 
-	return sum;
+	/* if buffer is unaligned, swap the checksum bytes */
+	if (unlikely(unaligned))
+		bsum = (bsum & 0xFF00FF00) >> 8 | (bsum &
0x00FF00FF) << 8;
Quoted 4 lines
Quoted 4 lines
lib/net/rte_ip.h +2 -0
+
+	return sum + bsum;
  }
 
Quoted 8 lines
Quoted 3 lines
/**
--
2.17.1
@Emil, thank you for thoroughly reviewing the previous versions.
If your test succeeds and you are satisfied with the patch, remember to
reply with a "Tested-by" tag for patchwork.
Quoted 1 line

Morten Brørup

2022-06-30 16:32 UTC #41 lore raw

RE: [PATCH v4] net: fix checksum with unaligned buffer

[1] http://inbox.dpdk.org/dev/[email protected]/T/#u

Quoted 156 lines

From: Mattias Rönnblom [mailto:[email protected]]
Sent: Tuesday, 28 June 2022 08.28
Quoted 10 lines · Morten Brørup
Quoted 10 lines
From: Mattias Rönnblom [mailto:[email protected]]
Sent: Monday, 27 June 2022 19.23
Quoted 6 lines · Morten Brørup
Quoted 6 lines
From: Emil Berg [mailto:[email protected]]
Sent: Monday, 27 June 2022 14.51
Quoted 3 lines
From: Emil Berg
Sent: den 27 juni 2022 14:46
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
From: Mattias Rönnblom <[email protected]>
Sent: den 27 juni 2022 14:28
Quoted 4 lines · Morten Brørup
Quoted 4 lines
From: Morten Brørup [mailto:[email protected]]
Sent: Thursday, 23 June 2022 14.39
With this patch, the checksum can be calculated on an
unaligned
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
buffer.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
I.e. the buf parameter is no longer required to be 16 bit
aligned.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
The checksum is still calculated using a 16 bit aligned
pointer,
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
so
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
the compiler can auto-vectorize the function's inner loop.
When the buffer is unaligned, the first byte of the buffer is handled separately. Furthermore, the calculated checksum of
the
Quoted 22 lines
Quoted 22 lines
Quoted 22 lines
Quoted 22 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
buffer is byte shifted before being added to the initial
checksum,
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
to compensate for the checksum having been calculated on the
buffer
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
shifted by one byte.
v4:
* Add copyright notice.
* Include stdbool.h (Emil Berg).
* Use RTE_PTR_ADD (Emil Berg).
* Fix one more typo in commit message. Is 'unligned' even a
word?
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
v3:
* Remove braces from single statement block.
* Fix typo in commit message.
v2:
* Do not assume that the buffer is part of an aligned packet
buffer.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Bugzilla ID: 1035
Cc: [email protected]
Quoted 8 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Signed-off-by Morten Brørup <[email protected]>
[...]
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines
The compiler will be able to auto vectorize even unaligned
accesses,
Quoted 1 line
Quoted 1 line
just with different instructions. From what I can tell, there's
no
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
performance impact, at least not on the x86_64 systems I tried
on.
Quoted 24 lines
Quoted 24 lines
Quoted 11 lines
Quoted 11 lines
Quoted 2 lines
Quoted 2 lines
I think you should remove the first special case conditional and
use
Quoted 1 line
Quoted 1 line
memcpy() instead of the cumbersome __may_alias__ construct to
retrieve
Quoted 6 lines
Quoted 2 lines
the data.
Here:
https://www.agner.org/optimize/instruction_tables.pdf
it lists the latency of vmovdqa (aligned) as 6 cycles and the
latency
Quoted 4 lines
Quoted 4 lines
for
Quoted 1 line
vmovdqu (unaligned) as 7 cycles. So I guess there can be some
difference.
Quoted 1 line
Although in practice I'm not sure what difference it makes. I've
not
Quoted 6 lines
Quoted 6 lines
seen any
Quoted 2 lines
difference in runtime between the two versions.
Correction to my comment:
Those stats are for some older CPU. For some newer CPUs such as
Tiger
Quoted 8 lines
Quoted 8 lines
Quoted 1 line
Quoted 1 line
Lake the stats seem to be the same regardless of aligned or
unaligned.
Quoted 5 lines
Quoted 1 line
I agree that the memcpy method is more elegant and easy to read.
However, we would need to performance test the modified checksum
function with a large number of CPUs to prove that we don't
introduce a
Quoted 1 line
Quoted 1 line
performance regression on any CPU architecture still supported by
DPDK.
Quoted 1 line
Quoted 1 line
And Emil already found a CPU where it costs 1 extra cycle per 16
bytes,
Quoted 5 lines
Quoted 5 lines
which adds up to a total of ca. 91 extra cycles on a 1460 byte TCP packet.
Quoted 1 line
I think you've misunderstood what latency means in such tables. It's
a
Quoted 1 line
Quoted 1 line
data dependency thing, not a measure of throughput. The throughput
is
Quoted 3 lines
Quoted 3 lines
*much* higher. My guess would be two such instruction per clock.
For your 1460 bytes example, my Zen3 AMD needs performs identical
with
Quoted 1 line
Quoted 1 line
both the current DPDK implementation, your patch, and a memcpy()-
ified
Quoted 2 lines
Quoted 2 lines
version of the current implementation. They all need ~130 clock cycles/packet, with warm caches. IPC is 3 instructions per cycle,
but
Quoted 5 lines
Quoted 1 line
obvious not all instructions are SIMD.
You're right, I wasn't thinking deeper about it before extrapolating.
Great to see some real numbers! I wish someone would do the same
testing on an old ARM CPU, so we could also see the other end of the scale.
Quoted 1 line
I've ran it on an ARM A72. For the aligned 1460 bytes case I got:
Current DPDK ~572 cc. Your patch: ~578 cc. Memcpy-fied: ~573 cc. They performed about the same for all unaligned/aligned and sizes I tested. This platform (or could be GCC version as well) doesn't suffer from the unaligned performance degradation your patch showed on my AMD machine.
Quoted 1 line
Quoted 1 line
The main issue with checksumming on the CPU is, in my experience,
not
Quoted 3 lines
Quoted 1 line
that you don't have enough compute, but that you trash the caches.
Agree. I have noticed that x86 has "non-temporal" instruction
variants to load/store data without trashing the cache entirely.
Quoted 2 lines
A variant of the checksum function using such instructions might be
handy.
Quoted 1 line
Yes, although you may need to prefetch the payload for good performance.
Quoted 1 line
Variants of the memcpy function using such instructions might also be
handy for some purposes, e.g. copying the contents of packets, where the original and/or copy will not accessed shortly thereafter.
Quoted 1 line

Quoted 5 lines

Indeed and I think it's been discussed on the list. There's some work to
get it right, since alignment requirement and the fact a different memory model is used for those SIMD instructions causes trouble for a generic implementation. (For x86_64.)

I just posted an RFC [1] for such memcpy() and memset() functions, so let's see how it fans out.

Quoted 27 lines

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
So I opted for a solution with zero changes to the inner loop, so
no
Quoted 5 lines
Quoted 5 lines
performance retesting is required (for the previously supported use cases, where the buffer is aligned).
Quoted 1 line
You will see performance degradation with this solution as well,
under
Quoted 1 line
Quoted 1 line
certain conditions. For unaligned 100 bytes of data, the current
DPDK
Quoted 1 line
Quoted 1 line
implementation and the memcpy()-fied version needs ~21 cc/packet.
Your
Quoted 3 lines
Quoted 1 line
patch needs 54 cc/packet.
Yes, it's a tradeoff. I exclusively aimed at maintaining performance
for the case with aligned buffers (under all circumstances, with all CPUs etc.), and ignored how it affects the performance for the case with unaligned buffers.
Quoted 2 lines
Unlike this patch, the memcpy() variant has no additional branches
for the unaligned case, so its performance should be generally unaffected by the buffer being aligned or not. However, I don't have sufficient in-depth CPU knowledge to say if this also applies to RISCV and older ARM CPUs still supported by DPDK.
Quoted 1 line

Quoted 3 lines

I don't think avoiding RISCV non-catastrophic regressions triumphs improving performance on mainstream CPUs and avoiding code quality regressions.

Quoted 35 lines

Quoted 2 lines
I don't have strong feelings for this patch, so you can provide a
memcpy() based alternative patch, and we will let the community decide which one they prefer.
Quoted 1 line
This is what my conversion looks like:
static inline uint32_t
__rte_raw_cksum(const void *buf, size_t len, uint32_t sum)
{
	const void *end;

	for (end = RTE_PTR_ADD(buf, (len/sizeof(uint16_t)) *
sizeof(uint16_t));
	     buf != end; buf = RTE_PTR_ADD(buf, sizeof(uint16_t))) {
		uint16_t v;

		memcpy(&v, buf, sizeof(uint16_t));
		sum += v;
	}

	/* if length is odd, keeping it byte order independent */
	if (unlikely(len % 2)) {
		uint8_t last;
		uint16_t left = 0;

		memcpy(&last, end, 1);
		*(unsigned char *)&left = last;
		sum += left;
	}

	return sum;
}

Quoted 6 lines

Maybe the maintainer can have an opinion, before I provide a real patch,
to save me some work. I would really like to contribute some performance
autotests in that case. I mean, you could be right and the performance could be totally off on some platform.

Looking at the meson files, A72 seems to be the weakest type of CPU actively supported by DPDK. So your testing shows better performance for your solution than mine under all conditions.

If you post a patch, I'll close this one as superseded.

Quoted 10 lines

Quoted 2 lines
Quoted 2 lines
But the old version didn't support unaligned accesses? In many
compiler
Quoted 3 lines
Quoted 1 line
flag/machine combinations it did.
It crashed in some cases. That was the point of the bug report [1],
and the reason to provide a patch.
Quoted 3 lines
[1] https://bugs.dpdk.org/show_bug.cgi?id=1035

Stephen Hemminger

NAK

2022-06-30 17:42 UTC #66 lore raw

Re: [PATCH v4] net: fix checksum with unaligned buffer

lib/net/rte_ip.h line 8

Quoted 11 lines · On Thu, 23 Jun 2022 14:39:00 +0200 Morten Brørup <[email protected]>…

lib/net/rte_ip.h +1 -0

index b502481670..738d643da0 100644
@@ -3,6 +3,7 @@
  *      The Regents of the University of California.
  * Copyright(c) 2010-2014 Intel Corporation.
  * Copyright(c) 2014 6WIND S.A.
+ * Copyright(c) 2022 SmartShare Systems.
  * All rights reserved.
  */

NAK
Doing a small incremental fix should not change copyright.

Stephen Hemminger

2022-06-30 17:45 UTC #67 lore raw

Re: [PATCH v4] net: fix checksum with unaligned buffer

https://www.felixcloutier.com/x86/movdqa:vmovdqa32:vmovdqa64

lib/net/rte_ip.h __rte_raw_cksum(const void *buf, size_t len, uint32_t sum) · line 176

Quoted 5 lines · On Thu, 23 Jun 2022 14:39:00 +0200 Morten Brørup <[email protected]>…

lib/net/rte_ip.h +5 -0

+	/* if buffer is unaligned, keeping it byte order independent */
+	if (unlikely(unaligned)) {
+		uint16_t first = 0;
+		if (unlikely(len == 0))
+			return 0;

Why is length == 0 unique to unaligned case?

lib/net/rte_ip.h __rte_raw_cksum(const void *buf, size_t len, uint32_t sum) · line 177

Quoted 1 line

lib/net/rte_ip.h +1 -0

+		((unsigned char *)&first)[1] = *(const unsigned char *)buf;

Use a proper union instead of casting to avoid aliasing warnings.

lib/net/rte_ip.h __rte_raw_cksum(const void *buf, size_t len, uint32_t sum) · line 181

Quoted 4 lines

lib/net/rte_ip.h +4 -0

+		bsum += first;
+		buf = RTE_PTR_ADD(buf, 1);
+		len--;
+	}

Many CPU's (such as x86) won't care about alignment and therefore the extra code to handle this is not worth doing.

Perhaps DPDK needs a macro (like Linux kernel) for efficient unaligned access.

In Linux kernel it is CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS

Emil Berg

2022-07-01 08:10 UTC #68 lore raw

RE: [PATCH v4] net: fix checksum with unaligned buffer

Quoted 27 lines

-----Original Message-----
From: Stephen Hemminger <[email protected]>
Sent: den 30 juni 2022 19:46
To: Morten Brørup <[email protected]>
Cc: Emil Berg <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH v4] net: fix checksum with unaligned buffer
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, size_t len, uint32_t sum) · line 176
Quoted 5 lines · On Thu, 23 Jun 2022 14:39:00 +0200 Morten Brørup <[email protected]>…
lib/net/rte_ip.h +5 -0
+	/* if buffer is unaligned, keeping it byte order independent */
+	if (unlikely(unaligned)) {
+		uint16_t first = 0;
+		if (unlikely(len == 0))
+			return 0;
Why is length == 0 unique to unaligned case?
Review
Quoted 1 line
(quoted diff) +1 -0
+		((unsigned char *)&first)[1] = *(const unsigned
char *)buf;
Use a proper union instead of casting to avoid aliasing warnings.

Quoted 8 lines

lib/net/rte_ip.h __rte_raw_cksum(const void *buf, size_t len, uint32_t sum) · line 181
Quoted 4 lines
lib/net/rte_ip.h +4 -0
+		bsum += first;
+		buf = RTE_PTR_ADD(buf, 1);
+		len--;
+	}
Many CPU's (such as x86) won't care about alignment and therefore the extra code to handle this is not worth doing.

x86 does care about alignment. An example is the vmovdqa instruction, where 'a' stands for 'aligned'. The description in the link below says: "When the source or destination operand is a memory operand, the operand must be aligned on a 16-byte boundary or a general-protection exception (#GP) will be generated. "

Quoted 4 lines

Perhaps DPDK needs a macro (like Linux kernel) for efficient unaligned access.
In Linux kernel it is CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS

Morten Brørup

2022-07-01 16:50 UTC #69 lore raw

RE: [PATCH v4] net: fix checksum with unaligned buffer

Quoted 16 lines

From: Emil Berg [mailto:[email protected]]
Sent: Friday, 1 July 2022 06.11
Quoted 13 lines
From: Stephen Hemminger <[email protected]>
Sent: den 30 juni 2022 19:46
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, size_t len, uint32_t sum) · line 176
Quoted 5 lines · On Thu, 23 Jun 2022 14:39:00 +0200 Morten Brørup <[email protected]>…
lib/net/rte_ip.h +5 -0
+	/* if buffer is unaligned, keeping it byte order independent */
+	if (unlikely(unaligned)) {
+		uint16_t first = 0;
+		if (unlikely(len == 0))
+			return 0;
Why is length == 0 unique to unaligned case?

Because the aligned case handles it gracefully. The unaligned case subtracts one from 'len', which (being unsigned) would become a very large number, causing 'end' to become way off.

Quoted 5 lines

Quoted 5 lines
Review
Quoted 2 lines
(quoted diff) +1 -0
+		((unsigned char *)&first)[1] = *(const unsigned
char *)buf;
Use a proper union instead of casting to avoid aliasing warnings.

I copied what is done by 'left', so it resembles the existing code in the function, making it easier to review.

It is part of the endian neutral checksum handling. Tricky stuff! :-)

Quoted 10 lines

Quoted 9 lines
lib/net/rte_ip.h __rte_raw_cksum(const void *buf, size_t len, uint32_t sum) · line 181
Quoted 5 lines
lib/net/rte_ip.h +4 -0
+		bsum += first;
+		buf = RTE_PTR_ADD(buf, 1);
+		len--;
+	}
Many CPU's (such as x86) won't care about alignment and therefore the extra code to handle this is not worth doing.

Quoted 8 lines

x86 does care about alignment. An example is the vmovdqa instruction, where 'a' stands for 'aligned'. The description in the link below says:
"When the source or destination operand is a memory operand, the operand must be aligned on a 16-byte boundary or a general-protection exception (#GP) will be generated. "
https://www.felixcloutier.com/x86/movdqa:vmovdqa32:vmovdqa64

Also, this misconception is exactly the bug [1] this patch fixes.

[1] https://bugs.dpdk.org/show_bug.cgi?id=1035

Quoted 5 lines

Quoted 1 line
Perhaps DPDK needs a macro (like Linux kernel) for efficient
unaligned
Quoted 3 lines
access.
In Linux kernel it is CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS

I recently stumbled across RTE_ARCH_STRICT_ALIGN in /lib/eal/include/rte_common.h.

But I guess it is something else.

Anyway, this function has ugly alignment problems (also before the patch), and has gone through a couple of iterations to silence warnings from the compiler. These warnings should have been addressed instead of silenced. Mattias has suggested a far better solution [2] than mine, which also correctly addresses the compiler alignment warnings, so we will probably end up with his solution instead.

[2] http://inbox.dpdk.org/dev/AM8PR07MB7666AD7BF7B780CC5062C14598BD9@AM8PR07MB7666.eurprd07.prod.outlook.com/T/#m1a76490541fce4a85b12d9390f2f4fac5a9f4660

Stephen Hemminger

2022-07-01 17:04 UTC #70 lore raw

Re: [PATCH v4] net: fix checksum with unaligned buffer

Quoted 6 lines · On Fri, 1 Jul 2022 18:50:34 +0200 Morten Brørup <[email protected]>…

But I guess it is something else.
Anyway, this function has ugly alignment problems (also before the patch), and has gone through a couple of iterations to silence warnings from the compiler. These warnings should have been addressed instead of silenced. Mattias has suggested a far better solution [2] than mine, which also correctly addresses the compiler alignment warnings, so we will probably end up with his solution instead.
[2] http://inbox.dpdk.org/dev/AM8PR07MB7666AD7BF7B780CC5062C14598BD9@AM8PR07MB7666.eurprd07.prod.outlook.com/T/#m1a76490541fce4a85b12d9390f2f4fac5a9f4660

Maybe some mix of the memcpy for unaligned and odd length and faster (unrolled?) version for the case of aligned and exact multiple? Or just take code from FreeBSD?

Morten Brørup

2022-07-01 20:46 UTC #71 lore raw

RE: [PATCH v4] net: fix checksum with unaligned buffer

Quoted 22 lines

From: Stephen Hemminger [mailto:[email protected]]
Sent: Friday, 1 July 2022 19.05
Quoted 3 lines · On Fri, 1 Jul 2022 18:50:34 +0200 Morten Brørup <[email protected]>…
But I guess it is something else.
Anyway, this function has ugly alignment problems (also before the
patch), and has gone through a couple of iterations to silence warnings from the compiler. These warnings should have been addressed instead of silenced. Mattias has suggested a far better solution [2] than mine, which also correctly addresses the compiler alignment warnings, so we will probably end up with his solution instead.
Quoted 2 lines
[2]
http://inbox.dpdk.org/dev/AM8PR07MB7666AD7BF7B780CC5062C14598BD9@AM8PR0
7MB7666.eurprd07.prod.outlook.com/T/#m1a76490541fce4a85b12d9390f2f4fac5 a9f4660
Quoted 1 line

Quoted 3 lines

Maybe some mix of the memcpy for unaligned and odd length and faster (unrolled?) version for the case of aligned and exact multiple? Or just take code from FreeBSD?

I just took a look at the BSD code, and it starts with the same "if ptr is unaligned" as my patch, and then does some manual loop unrolling, which we expect the compiler to do. Mattias has demonstrated that his solution has better performance, not only on modern X86 CPUs, also on an A72 CPU, so I prefer his solution. And the difference between using "vmovdqa" and "vmovdq" instructions here seem to be insignificant.

Stanisław Kardach

2022-07-07 15:21 UTC #42 lore raw

Re: [PATCH v4] net: fix checksum with unaligned buffer

Quoted 195 lines · Morten Brørup

Quoted 157 lines
From: Mattias Rönnblom [mailto:[email protected]]
Sent: Tuesday, 28 June 2022 08.28
Quoted 10 lines · Morten Brørup
Quoted 10 lines
From: Mattias Rönnblom [mailto:[email protected]]
Sent: Monday, 27 June 2022 19.23
Quoted 6 lines · Morten Brørup
Quoted 6 lines
From: Emil Berg [mailto:[email protected]]
Sent: Monday, 27 June 2022 14.51
Quoted 3 lines
From: Emil Berg
Sent: den 27 juni 2022 14:46
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
From: Mattias Rönnblom <[email protected]>
Sent: den 27 juni 2022 14:28
Quoted 4 lines · Morten Brørup
Quoted 4 lines
From: Morten Brørup [mailto:[email protected]]
Sent: Thursday, 23 June 2022 14.39
With this patch, the checksum can be calculated on an
unaligned
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
buffer.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
I.e. the buf parameter is no longer required to be 16 bit
aligned.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
The checksum is still calculated using a 16 bit aligned
pointer,
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
so
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
the compiler can auto-vectorize the function's inner loop.
When the buffer is unaligned, the first byte of the buffer is handled separately. Furthermore, the calculated checksum of
the
Quoted 22 lines
Quoted 22 lines
Quoted 22 lines
Quoted 22 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
buffer is byte shifted before being added to the initial
checksum,
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
to compensate for the checksum having been calculated on the
buffer
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
shifted by one byte.
v4:
* Add copyright notice.
* Include stdbool.h (Emil Berg).
* Use RTE_PTR_ADD (Emil Berg).
* Fix one more typo in commit message. Is 'unligned' even a
word?
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
v3:
* Remove braces from single statement block.
* Fix typo in commit message.
v2:
* Do not assume that the buffer is part of an aligned packet
buffer.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Bugzilla ID: 1035
Cc: [email protected]
Quoted 8 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Signed-off-by Morten Brørup <[email protected]>
[...]
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines
The compiler will be able to auto vectorize even unaligned
accesses,
Quoted 1 line
Quoted 1 line
just with different instructions. From what I can tell, there's
no
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
performance impact, at least not on the x86_64 systems I tried
on.
Quoted 24 lines
Quoted 24 lines
Quoted 11 lines
Quoted 11 lines
Quoted 2 lines
Quoted 2 lines
I think you should remove the first special case conditional and
use
Quoted 1 line
Quoted 1 line
memcpy() instead of the cumbersome __may_alias__ construct to
retrieve
Quoted 6 lines
Quoted 2 lines
the data.
Here:
https://www.agner.org/optimize/instruction_tables.pdf
it lists the latency of vmovdqa (aligned) as 6 cycles and the
latency
Quoted 4 lines
Quoted 4 lines
for
Quoted 1 line
vmovdqu (unaligned) as 7 cycles. So I guess there can be some
difference.
Quoted 1 line
Although in practice I'm not sure what difference it makes. I've
not
Quoted 6 lines
Quoted 6 lines
seen any
Quoted 2 lines
difference in runtime between the two versions.
Correction to my comment:
Those stats are for some older CPU. For some newer CPUs such as
Tiger
Quoted 8 lines
Quoted 8 lines
Quoted 1 line
Quoted 1 line
Lake the stats seem to be the same regardless of aligned or
unaligned.
Quoted 5 lines
Quoted 1 line
I agree that the memcpy method is more elegant and easy to read.
However, we would need to performance test the modified checksum
function with a large number of CPUs to prove that we don't
introduce a
Quoted 1 line
Quoted 1 line
performance regression on any CPU architecture still supported by
DPDK.
Quoted 1 line
Quoted 1 line
And Emil already found a CPU where it costs 1 extra cycle per 16
bytes,
Quoted 5 lines
Quoted 5 lines
which adds up to a total of ca. 91 extra cycles on a 1460 byte TCP packet.
Quoted 1 line
I think you've misunderstood what latency means in such tables. It's
a
Quoted 1 line
Quoted 1 line
data dependency thing, not a measure of throughput. The throughput
is
Quoted 3 lines
Quoted 3 lines
*much* higher. My guess would be two such instruction per clock.
For your 1460 bytes example, my Zen3 AMD needs performs identical
with
Quoted 1 line
Quoted 1 line
both the current DPDK implementation, your patch, and a memcpy()-
ified
Quoted 2 lines
Quoted 2 lines
version of the current implementation. They all need ~130 clock cycles/packet, with warm caches. IPC is 3 instructions per cycle,
but
Quoted 5 lines
Quoted 1 line
obvious not all instructions are SIMD.
You're right, I wasn't thinking deeper about it before extrapolating.
Great to see some real numbers! I wish someone would do the same
testing on an old ARM CPU, so we could also see the other end of the scale.
Quoted 1 line
I've ran it on an ARM A72. For the aligned 1460 bytes case I got:
Current DPDK ~572 cc. Your patch: ~578 cc. Memcpy-fied: ~573 cc. They performed about the same for all unaligned/aligned and sizes I tested. This platform (or could be GCC version as well) doesn't suffer from the unaligned performance degradation your patch showed on my AMD machine.
Quoted 1 line
Quoted 1 line
The main issue with checksumming on the CPU is, in my experience,
not
Quoted 3 lines
Quoted 1 line
that you don't have enough compute, but that you trash the caches.
Agree. I have noticed that x86 has "non-temporal" instruction
variants to load/store data without trashing the cache entirely.
Quoted 2 lines
A variant of the checksum function using such instructions might be
handy.
Quoted 1 line
Yes, although you may need to prefetch the payload for good performance.
Quoted 1 line
Variants of the memcpy function using such instructions might also be
handy for some purposes, e.g. copying the contents of packets, where the original and/or copy will not accessed shortly thereafter.
Quoted 1 line
Quoted 5 lines
Indeed and I think it's been discussed on the list. There's some work to
get it right, since alignment requirement and the fact a different memory model is used for those SIMD instructions causes trouble for a generic implementation. (For x86_64.)
I just posted an RFC [1] for such memcpy() and memset() functions, so let's see how it fans out.
[1] http://inbox.dpdk.org/dev/[email protected]/T/#u
Quoted 27 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
So I opted for a solution with zero changes to the inner loop, so
no
Quoted 5 lines
Quoted 5 lines
performance retesting is required (for the previously supported use cases, where the buffer is aligned).
Quoted 1 line
You will see performance degradation with this solution as well,
under
Quoted 1 line
Quoted 1 line
certain conditions. For unaligned 100 bytes of data, the current
DPDK
Quoted 1 line
Quoted 1 line
implementation and the memcpy()-fied version needs ~21 cc/packet.
Your
Quoted 3 lines
Quoted 1 line
patch needs 54 cc/packet.
Yes, it's a tradeoff. I exclusively aimed at maintaining performance
for the case with aligned buffers (under all circumstances, with all CPUs etc.), and ignored how it affects the performance for the case with unaligned buffers.
Quoted 2 lines
Unlike this patch, the memcpy() variant has no additional branches
for the unaligned case, so its performance should be generally unaffected by the buffer being aligned or not. However, I don't have sufficient in-depth CPU knowledge to say if this also applies to RISCV and older ARM CPUs still supported by DPDK.
Quoted 1 line

Quoted 4 lines

Quoted 3 lines
I don't think avoiding RISCV non-catastrophic regressions triumphs improving performance on mainstream CPUs and avoiding code quality regressions.
+1

+1. In general RISC-V spec leaves the unaligned load/store handling to implementation (it might fault, it might not). The U74 core that I have at hand allows unaligned reads/writes. Though it's not a platform for performance evaluation (time measurement causes a trap to firmware), so I won't say anything on that.

--
Best Regards,
Stanisław Kardach

Mattias Rönnblom

2022-07-07 18:35 UTC #43 lore raw

PATCH 1/2 app/test: add cksum performance test

From: Mattias Rönnblom <[email protected]>

Add performance test for the rte_raw_cksum() function, which delegates the actual work to __rte_raw_cksum(), which in turn is used by other functions in need of Internet checksum calculation.

---
 MAINTAINERS                |   1 +
 app/test/meson.build       |   1 +
 app/test/test_cksum_perf.c | 118 +++++++++++++++++++++++++++++++++++++
 3 files changed, 120 insertions(+)
 create mode 100644 app/test/test_cksum_perf.c

MAINTAINERS +1 -0

index c923712946..2a4c99e05a 100644
@@ -1414,6 +1414,7 @@ Network headers
 M: Olivier Matz <[email protected]>
 F: lib/net/
 F: app/test/test_cksum.c
+F: app/test/test_cksum_perf.c
 
 Packet CRC
 M: Jasvinder Singh <[email protected]>

app/test/meson.build +1 -0

index 431c5bd318..191db03d1d 100644
@@ -18,6 +18,7 @@ test_sources = files(
         'test_bpf.c',
         'test_byteorder.c',
         'test_cksum.c',
+        'test_cksum_perf.c',
         'test_cmdline.c',
         'test_cmdline_cirbuf.c',
         'test_cmdline_etheraddr.c',

app/test/test_cksum_perf.c new file +118 -0

118 lines not on this page · raw message

Signature · 1 line

2.25.1

Mattias Rönnblom

2022-07-07 18:35 UTC #44 lore raw

PATCH 2/2 net: have checksum routines accept unaligned data

From: Mattias Rönnblom <[email protected]>

__rte_raw_cksum() (used by rte_raw_cksum() among others) accessed its data through an uint16_t pointer, which allowed the compiler to assume the data was 16-bit aligned. This in turn would, with certain architectures and compiler flag combinations, result in code with SIMD load or store instructions with restrictions on data alignment.

This patch keeps the old algorithm, but data is read using memcpy() instead of direct pointer access, forcing the compiler to always generate code that handles unaligned input. The __may_alias__ GCC attribute is no longer needed.

The data on which the Internet checksum functions operates are almost always 16-bit aligned, but there are exceptions. In particular, the PDCP protocol header may (literally) have an odd size.

Performance impact seems to range from none to a very slight regression.

Bugzilla ID: 1035
Cc: [email protected]

---
 lib/net/rte_ip.h | 19 ++++++++++++-------
 1 file changed, 12 insertions(+), 7 deletions(-)

lib/net/rte_ip.h +12 -7

index b502481670..a9e6251f14 100644
@@ -160,18 +160,23 @@ rte_ipv4_hdr_len(const struct rte_ipv4_hdr *ipv4_hdr)
 static inline uint32_t
 __rte_raw_cksum(const void *buf, size_t len, uint32_t sum)
 {
-	/* extend strict-aliasing rules */
-	typedef uint16_t __attribute__((__may_alias__)) u16_p;
-	const u16_p *u16_buf = (const u16_p *)buf;
-	const u16_p *end = u16_buf + len / sizeof(*u16_buf);
+	const void *end;
 
-	for (; u16_buf != end; ++u16_buf)
-		sum += *u16_buf;
+	for (end = RTE_PTR_ADD(buf, (len/sizeof(uint16_t)) * sizeof(uint16_t));
+	     buf != end; buf = RTE_PTR_ADD(buf, sizeof(uint16_t))) {
+		uint16_t v;
+
+		memcpy(&v, buf, sizeof(uint16_t));
+		sum += v;
+	}
 
 	/* if length is odd, keeping it byte order independent */
 	if (unlikely(len % 2)) {
+		uint8_t last;
 		uint16_t left = 0;
-		*(unsigned char *)&left = *(const unsigned char *)end;
+
+		memcpy(&last, end, 1);
+		*(unsigned char *)&left = last;
 		sum += left;
 	}

Signature · 1 line

2.25.1

Morten Brørup

Reviewed-by on v1 2/2

2022-07-07 21:44 UTC #45 lore raw

RE: [PATCH 2/2] net: have checksum routines accept unaligned data

Quoted 36 lines

From: Mattias Rönnblom [mailto:[email protected]]
Sent: Thursday, 7 July 2022 20.35
From: Mattias Rönnblom <[email protected]>
__rte_raw_cksum() (used by rte_raw_cksum() among others) accessed its data through an uint16_t pointer, which allowed the compiler to assume the data was 16-bit aligned. This in turn would, with certain architectures and compiler flag combinations, result in code with SIMD load or store instructions with restrictions on data alignment.
This patch keeps the old algorithm, but data is read using memcpy() instead of direct pointer access, forcing the compiler to always generate code that handles unaligned input. The __may_alias__ GCC attribute is no longer needed.
The data on which the Internet checksum functions operates are almost always 16-bit aligned, but there are exceptions. In particular, the PDCP protocol header may (literally) have an odd size.
Performance impact seems to range from none to a very slight regression.
Bugzilla ID: 1035
Cc: [email protected]
Signed-off-by Mattias Rönnblom <[email protected]>
---
 lib/net/rte_ip.h | 19 ++++++++++++-------
 1 file changed, 12 insertions(+), 7 deletions(-)
lib/net/rte_ip.h +0 -0
index b502481670..a9e6251f14 100644

lib/net/rte_ip.h rte_ipv4_hdr_len(const struct rte_ipv4_hdr · line 164

Quoted 9 lines

lib/net/rte_ip.h +1 -4

@@ -160,18 +160,23 @@ rte_ipv4_hdr_len(const struct rte_ipv4_hdr
 *ipv4_hdr)
 static inline uint32_t
 __rte_raw_cksum(const void *buf, size_t len, uint32_t sum)
 {
-	/* extend strict-aliasing rules */
-	typedef uint16_t __attribute__((__may_alias__)) u16_p;
-	const u16_p *u16_buf = (const u16_p *)buf;
-	const u16_p *end = u16_buf + len / sizeof(*u16_buf);
+	const void *end;

I would set "end" here instead, possibly making the pointer const too. And add spaces around '/'. const void * const end = RTE_PTR_ADD(buf, (len / sizeof(uint16_t)) * sizeof(uint16_t));

Quoted 10 lines

lib/net/rte_ip.h +6 -2

-	for (; u16_buf != end; ++u16_buf)
-		sum += *u16_buf;
+	for (end = RTE_PTR_ADD(buf, (len/sizeof(uint16_t)) *
 sizeof(uint16_t));
+	     buf != end; buf = RTE_PTR_ADD(buf, sizeof(uint16_t))) {
+		uint16_t v;
+
+		memcpy(&v, buf, sizeof(uint16_t));
+		sum += v;

lib/net/rte_ip.h rte_ipv4_hdr_len(const struct rte_ipv4_hdr *ipv4_hdr) · line 179

Quoted 10 lines

lib/net/rte_ip.h +5 -1

+	}
 
 	/* if length is odd, keeping it byte order independent */
 	if (unlikely(len % 2)) {
+		uint8_t last;
 		uint16_t left = 0;
-		*(unsigned char *)&left = *(const unsigned char *)end;
+
+		memcpy(&last, end, 1);
+		*(unsigned char *)&left = last;

Couldn't you just memcpy(&left, end, 1), and omit the temporary variable "last"?

Quoted 6 lines

lib/net/rte_ip.h +0 -1
 		sum += left;
 	}
 
--
2.25.1

With our without my suggested changes, it looks good.

Reviewed-by Morten Brørup <[email protected]>

Mattias Rönnblom

2022-07-08 12:44 UTC #46 lore raw

Re: [PATCH 2/2] net: have checksum routines accept unaligned data

Quoted 40 lines · Morten Brørup

Quoted 40 lines
From: Mattias Rönnblom [mailto:[email protected]]
Sent: Thursday, 7 July 2022 20.35
From: Mattias Rönnblom <[email protected]>
__rte_raw_cksum() (used by rte_raw_cksum() among others) accessed its data through an uint16_t pointer, which allowed the compiler to assume the data was 16-bit aligned. This in turn would, with certain architectures and compiler flag combinations, result in code with SIMD load or store instructions with restrictions on data alignment.
This patch keeps the old algorithm, but data is read using memcpy() instead of direct pointer access, forcing the compiler to always generate code that handles unaligned input. The __may_alias__ GCC attribute is no longer needed.
The data on which the Internet checksum functions operates are almost always 16-bit aligned, but there are exceptions. In particular, the PDCP protocol header may (literally) have an odd size.
Performance impact seems to range from none to a very slight regression.
Bugzilla ID: 1035
Cc: [email protected]
Signed-off-by Mattias Rönnblom <[email protected]>
---
  lib/net/rte_ip.h | 19 ++++++++++++-------
  1 file changed, 12 insertions(+), 7 deletions(-)
lib/net/rte_ip.h +0 -0
index b502481670..a9e6251f14 100644
@@ -160,18 +160,23 @@ rte_ipv4_hdr_len(const struct rte_ipv4_hdr
 *ipv4_hdr)
  static inline uint32_t
  __rte_raw_cksum(const void *buf, size_t len, uint32_t sum)
  {

Quoted 9 lines

lib/net/rte_ip.h rte_ipv4_hdr_len(const struct rte_ipv4_hdr *ipv4_hdr) · line 163
Quoted 5 lines
lib/net/rte_ip.h +1 -4
-	/* extend strict-aliasing rules */
-	typedef uint16_t __attribute__((__may_alias__)) u16_p;
-	const u16_p *u16_buf = (const u16_p *)buf;
-	const u16_p *end = u16_buf + len / sizeof(*u16_buf);
+	const void *end;
I would set "end" here instead, possibly making the pointer const too. And add spaces around '/'. const void * const end = RTE_PTR_ADD(buf, (len / sizeof(uint16_t)) * sizeof(uint16_t));

I don't think that makes the code more readable.

Quoted 12 lines

lib/net/rte_ip.h +7 -2

-	for (; u16_buf != end; ++u16_buf)
-		sum += *u16_buf;
+	for (end = RTE_PTR_ADD(buf, (len/sizeof(uint16_t)) *
 sizeof(uint16_t));
+	     buf != end; buf = RTE_PTR_ADD(buf, sizeof(uint16_t))) {
+		uint16_t v;
+
+		memcpy(&v, buf, sizeof(uint16_t));
+		sum += v;
+	}

Quoted 11 lines

lib/net/rte_ip.h rte_ipv4_hdr_len(const struct rte_ipv4_hdr *ipv4_hdr) · line 179
Quoted 8 lines
lib/net/rte_ip.h +4 -1
  	/* if length is odd, keeping it byte order independent */
  	if (unlikely(len % 2)) {
+		uint8_t last;
  		uint16_t left = 0;
-		*(unsigned char *)&left = *(const unsigned char *)end;
+
+		memcpy(&last, end, 1);
+		*(unsigned char *)&left = last;
Couldn't you just memcpy(&left, end, 1), and omit the temporary variable "last"?

Good point.

I don't like how this code is clever vis-à-vis byte order, but then I also don't have a better suggestion.

Quoted 11 lines

Quoted 6 lines
lib/net/rte_ip.h +0 -1
  		sum += left;
  	}
 
--
2.25.1
With our without my suggested changes, it looks good.
Reviewed-by Morten Brørup <[email protected]>

Thanks!

Mattias Rönnblom

2022-07-08 12:58 UTC #48 lore raw

PATCH v2 2/2 net: have checksum routines accept unaligned data

The data on which the Internet checksum functions operates are almost always 16-bit aligned, but there are exceptions. In particular, the PDCP protocol header may (literally) have an odd size.

Performance impact seems to range from none to a very slight regression.

Bugzilla ID: 1035
Cc: [email protected]

---

v2:
* Simplified the odd-length conditional (Morten Brørup).

Reviewed-by Morten Brørup <[email protected]>

---
 lib/net/rte_ip.h | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)

lib/net/rte_ip.h +10 -7

index b502481670..a0334d931e 100644
@@ -160,18 +160,21 @@ rte_ipv4_hdr_len(const struct rte_ipv4_hdr *ipv4_hdr)
 static inline uint32_t
 __rte_raw_cksum(const void *buf, size_t len, uint32_t sum)
 {
-	/* extend strict-aliasing rules */
-	typedef uint16_t __attribute__((__may_alias__)) u16_p;
-	const u16_p *u16_buf = (const u16_p *)buf;
-	const u16_p *end = u16_buf + len / sizeof(*u16_buf);
+	const void *end;
 
-	for (; u16_buf != end; ++u16_buf)
-		sum += *u16_buf;
+	for (end = RTE_PTR_ADD(buf, (len/sizeof(uint16_t)) * sizeof(uint16_t));
+	     buf != end; buf = RTE_PTR_ADD(buf, sizeof(uint16_t))) {
+		uint16_t v;
+
+		memcpy(&v, buf, sizeof(uint16_t));
+		sum += v;
+	}
 
 	/* if length is odd, keeping it byte order independent */
 	if (unlikely(len % 2)) {
 		uint16_t left = 0;
-		*(unsigned char *)&left = *(const unsigned char *)end;
+
+		memcpy(&left, end, 1);
 		sum += left;
 	}

Signature · 1 line

2.25.1

Mattias Rönnblom

2022-07-08 12:58 UTC #47 lore raw

PATCH v2 1/2 app/test: add cksum performance test

Add performance test for the rte_raw_cksum() function, which delegates the actual work to __rte_raw_cksum(), which in turn is used by other functions in need of Internet checksum calculation.

---

v2:
* Added __rte_unused to unused volatile variable, to keep the Intel compiler happy.

---
 MAINTAINERS                |   1 +
 app/test/meson.build       |   1 +
 app/test/test_cksum_perf.c | 118 +++++++++++++++++++++++++++++++++++++
 3 files changed, 120 insertions(+)
 create mode 100644 app/test/test_cksum_perf.c

MAINTAINERS +1 -0

index c923712946..2a4c99e05a 100644
@@ -1414,6 +1414,7 @@ Network headers
 M: Olivier Matz <[email protected]>
 F: lib/net/
 F: app/test/test_cksum.c
+F: app/test/test_cksum_perf.c
 
 Packet CRC
 M: Jasvinder Singh <[email protected]>

app/test/meson.build +1 -0

index 431c5bd318..191db03d1d 100644
@@ -18,6 +18,7 @@ test_sources = files(
         'test_bpf.c',
         'test_byteorder.c',
         'test_cksum.c',
+        'test_cksum_perf.c',
         'test_cmdline.c',
         'test_cmdline_cirbuf.c',
         'test_cmdline_etheraddr.c',

app/test/test_cksum_perf.c new file +118 -0

118 lines not on this page · raw message

Signature · 1 line

2.25.1

Morten Brørup

2022-07-08 13:02 UTC #62 lore raw

RE: [PATCH 2/2] net: have checksum routines accept unaligned data

[1] https://doc.dpdk.org/guides/contributing/coding_style.html

Quoted 55 lines

From: Mattias Rönnblom [mailto:[email protected]]
Sent: Friday, 8 July 2022 14.44
Quoted 6 lines · Morten Brørup
Quoted 6 lines
From: Mattias Rönnblom [mailto:[email protected]]
Sent: Thursday, 7 July 2022 20.35
From: Mattias Rönnblom <[email protected]>
__rte_raw_cksum() (used by rte_raw_cksum() among others) accessed
its
Quoted 1 line
Quoted 1 line
data through an uint16_t pointer, which allowed the compiler to
assume
Quoted 2 lines
Quoted 2 lines
the data was 16-bit aligned. This in turn would, with certain architectures and compiler flag combinations, result in code with
SIMD
Quoted 8 lines
Quoted 8 lines
load or store instructions with restrictions on data alignment.
This patch keeps the old algorithm, but data is read using memcpy() instead of direct pointer access, forcing the compiler to always generate code that handles unaligned input. The __may_alias__ GCC attribute is no longer needed.
The data on which the Internet checksum functions operates are
almost
Quoted 14 lines
Quoted 14 lines
always 16-bit aligned, but there are exceptions. In particular, the PDCP protocol header may (literally) have an odd size.
Performance impact seems to range from none to a very slight regression.
Bugzilla ID: 1035
Cc: [email protected]
Signed-off-by Mattias Rönnblom <[email protected]>
---
  lib/net/rte_ip.h | 19 ++++++++++++-------
  1 file changed, 12 insertions(+), 7 deletions(-)
Quoted 16 lines
Quoted 5 lines
lib/net/rte_ip.h +0 -0
index b502481670..a9e6251f14 100644
lib/net/rte_ip.h rte_ipv4_hdr_len(const struct rte_ipv4_hdr · line 164
Quoted 9 lines
lib/net/rte_ip.h +1 -4
@@ -160,18 +160,23 @@ rte_ipv4_hdr_len(const struct rte_ipv4_hdr
 *ipv4_hdr)
  static inline uint32_t
  __rte_raw_cksum(const void *buf, size_t len, uint32_t sum)
  {
-	/* extend strict-aliasing rules */
-	typedef uint16_t __attribute__((__may_alias__)) u16_p;
-	const u16_p *u16_buf = (const u16_p *)buf;
-	const u16_p *end = u16_buf + len / sizeof(*u16_buf);
+	const void *end;
I would set "end" here instead, possibly making the pointer const

Quoted 6 lines

too. And add spaces around '/'.
Quoted 1 line
const void * const end = RTE_PTR_ADD(buf, (len / sizeof(uint16_t)) *
sizeof(uint16_t));
Quoted 1 line
I don't think that makes the code more readable.

It's only a matter of taste... Your code, your decision. :-)

I think the spaces are required by the coding standard; not sure, though.

Quoted 23 lines

Quoted 13 lines

lib/net/rte_ip.h +7 -2

-	for (; u16_buf != end; ++u16_buf)
-		sum += *u16_buf;
+	for (end = RTE_PTR_ADD(buf, (len/sizeof(uint16_t)) *
 sizeof(uint16_t));
+	     buf != end; buf = RTE_PTR_ADD(buf, sizeof(uint16_t))) {
+		uint16_t v;
+
+		memcpy(&v, buf, sizeof(uint16_t));
+		sum += v;
+	}

Quoted 10 lines

lib/net/rte_ip.h rte_ipv4_hdr_len(const struct rte_ipv4_hdr *ipv4_hdr) · line 179
Quoted 8 lines
lib/net/rte_ip.h +4 -1
  	/* if length is odd, keeping it byte order independent */
  	if (unlikely(len % 2)) {
+		uint8_t last;
  		uint16_t left = 0;
-		*(unsigned char *)&left = *(const unsigned char *)end;
+
+		memcpy(&last, end, 1);
+		*(unsigned char *)&left = last;
Couldn't you just memcpy(&left, end, 1), and omit the temporary

Quoted 7 lines

variable "last"?
Quoted 1 line
Good point.
I don't like how this code is clever vis-à-vis byte order, but then I also don't have a better suggestion.

The byte ordering cleverness has its roots in RFC 1071.

Stephen suggested using a union, although in a slightly different context. I'm not sure it will be more readable here, because it will require #ifdef to support byte ordering. Just thought I'd mention it, for your consideration.

Your patch v2 just reached my inbox, and it looks good. No further response to this email is expected.

Mattias Rönnblom

2022-07-08 13:52 UTC #63 lore raw

Re: [PATCH 2/2] net: have checksum routines accept unaligned data

Quoted 57 lines · Morten Brørup

Quoted 57 lines
From: Mattias Rönnblom [mailto:[email protected]]
Sent: Friday, 8 July 2022 14.44
Quoted 6 lines · Morten Brørup
Quoted 6 lines
From: Mattias Rönnblom [mailto:[email protected]]
Sent: Thursday, 7 July 2022 20.35
From: Mattias Rönnblom <[email protected]>
__rte_raw_cksum() (used by rte_raw_cksum() among others) accessed
its
Quoted 1 line
Quoted 1 line
data through an uint16_t pointer, which allowed the compiler to
assume
Quoted 2 lines
Quoted 2 lines
the data was 16-bit aligned. This in turn would, with certain architectures and compiler flag combinations, result in code with
SIMD
Quoted 8 lines
Quoted 8 lines
load or store instructions with restrictions on data alignment.
This patch keeps the old algorithm, but data is read using memcpy() instead of direct pointer access, forcing the compiler to always generate code that handles unaligned input. The __may_alias__ GCC attribute is no longer needed.
The data on which the Internet checksum functions operates are
almost
Quoted 14 lines
Quoted 14 lines
always 16-bit aligned, but there are exceptions. In particular, the PDCP protocol header may (literally) have an odd size.
Performance impact seems to range from none to a very slight regression.
Bugzilla ID: 1035
Cc: [email protected]
Signed-off-by Mattias Rönnblom <[email protected]>
---
   lib/net/rte_ip.h | 19 ++++++++++++-------
   1 file changed, 12 insertions(+), 7 deletions(-)
Quoted 16 lines
Quoted 5 lines
lib/net/rte_ip.h +0 -0
index b502481670..a9e6251f14 100644
lib/net/rte_ip.h rte_ipv4_hdr_len(const struct rte_ipv4_hdr · line 164
Quoted 9 lines
lib/net/rte_ip.h +1 -4
@@ -160,18 +160,23 @@ rte_ipv4_hdr_len(const struct rte_ipv4_hdr
 *ipv4_hdr)
   static inline uint32_t
   __rte_raw_cksum(const void *buf, size_t len, uint32_t sum)
   {
-	/* extend strict-aliasing rules */
-	typedef uint16_t __attribute__((__may_alias__)) u16_p;
-	const u16_p *u16_buf = (const u16_p *)buf;
-	const u16_p *end = u16_buf + len / sizeof(*u16_buf);
+	const void *end;
I would set "end" here instead, possibly making the pointer const
too. And add spaces around '/'.
Quoted 1 line
const void * const end = RTE_PTR_ADD(buf, (len / sizeof(uint16_t)) *

Quoted 9 lines

Quoted 4 lines
sizeof(uint16_t));
Quoted 1 line
I don't think that makes the code more readable.
It's only a matter of taste... Your code, your decision. :-)
I think the spaces are required by the coding standard; not sure, though.

If it isn't in the coding standard, it should be. But if you add spaces, you have to break the line, to fit into 80 characters. A net loss, IMO.

Quoted 37 lines

Quoted 23 lines
Quoted 13 lines
Quoted 13 lines
lib/net/rte_ip.h +7 -2
-	for (; u16_buf != end; ++u16_buf)
-		sum += *u16_buf;
+	for (end = RTE_PTR_ADD(buf, (len/sizeof(uint16_t)) *
 sizeof(uint16_t));
+	     buf != end; buf = RTE_PTR_ADD(buf, sizeof(uint16_t))) {
+		uint16_t v;
+
+		memcpy(&v, buf, sizeof(uint16_t));
+		sum += v;
+	}
 
Quoted 10 lines
lib/net/rte_ip.h rte_ipv4_hdr_len(const struct rte_ipv4_hdr *ipv4_hdr) · line 179
Quoted 8 lines
lib/net/rte_ip.h +4 -1
   	/* if length is odd, keeping it byte order independent */
   	if (unlikely(len % 2)) {
+		uint8_t last;
   		uint16_t left = 0;
-		*(unsigned char *)&left = *(const unsigned char *)end;
+
+		memcpy(&last, end, 1);
+		*(unsigned char *)&left = last;
Couldn't you just memcpy(&left, end, 1), and omit the temporary
Quoted 7 lines
variable "last"?
Quoted 1 line
Good point.
I don't like how this code is clever vis-à-vis byte order, but then I also don't have a better suggestion.
The byte ordering cleverness has its roots in RFC 1071.
Stephen suggested using a union, although in a slightly different context. I'm not sure it will be more readable here, because it will require #ifdef to support byte ordering. Just thought I'd mention it, for your consideration.
Your patch v2 just reached my inbox, and it looks good. No further response to this email is expected.

Bruce Richardson

2022-07-08 14:10 UTC #64 lore raw

Re: [PATCH 2/2] net: have checksum routines accept unaligned data

Quoted 63 lines · Mattias Rönnblom

Quoted 62 lines · Morten Brørup
Quoted 61 lines
From: Mattias Rönnblom [mailto:[email protected]]
Sent: Friday, 8 July 2022 14.44
Quoted 6 lines · Morten Brørup
Quoted 6 lines
From: Mattias Rönnblom [mailto:[email protected]]
Sent: Thursday, 7 July 2022 20.35
From: Mattias Rönnblom <[email protected]>
__rte_raw_cksum() (used by rte_raw_cksum() among others) accessed
its
Quoted 1 line
Quoted 1 line
data through an uint16_t pointer, which allowed the compiler to
assume
Quoted 2 lines
Quoted 2 lines
the data was 16-bit aligned. This in turn would, with certain architectures and compiler flag combinations, result in code with
SIMD
Quoted 8 lines
Quoted 8 lines
load or store instructions with restrictions on data alignment.
This patch keeps the old algorithm, but data is read using memcpy() instead of direct pointer access, forcing the compiler to always generate code that handles unaligned input. The __may_alias__ GCC attribute is no longer needed.
The data on which the Internet checksum functions operates are
almost
Quoted 14 lines
Quoted 14 lines
always 16-bit aligned, but there are exceptions. In particular, the PDCP protocol header may (literally) have an odd size.
Performance impact seems to range from none to a very slight regression.
Bugzilla ID: 1035
Cc: [email protected]
Signed-off-by Mattias Rönnblom <[email protected]>
---
   lib/net/rte_ip.h | 19 ++++++++++++-------
   1 file changed, 12 insertions(+), 7 deletions(-)
Quoted 16 lines
Quoted 5 lines
lib/net/rte_ip.h +0 -0
index b502481670..a9e6251f14 100644
lib/net/rte_ip.h rte_ipv4_hdr_len(const struct rte_ipv4_hdr · line 164
Quoted 9 lines
lib/net/rte_ip.h +1 -4
@@ -160,18 +160,23 @@ rte_ipv4_hdr_len(const struct rte_ipv4_hdr
 *ipv4_hdr)
   static inline uint32_t
   __rte_raw_cksum(const void *buf, size_t len, uint32_t sum)
   {
-	/* extend strict-aliasing rules */
-	typedef uint16_t __attribute__((__may_alias__)) u16_p;
-	const u16_p *u16_buf = (const u16_p *)buf;
-	const u16_p *end = u16_buf + len / sizeof(*u16_buf);
+	const void *end;
I would set "end" here instead, possibly making the pointer const
too. And add spaces around '/'.
Quoted 1 line
const void * const end = RTE_PTR_ADD(buf, (len / sizeof(uint16_t)) *
sizeof(uint16_t));
Quoted 1 line
I don't think that makes the code more readable.

Quoted 8 lines

Quoted 4 lines
It's only a matter of taste... Your code, your decision. :-)
I think the spaces are required by the coding standard; not sure, though.
If it isn't in the coding standard, it should be. But if you add spaces, you have to break the line, to fit into 80 characters. A net loss, IMO.

Just FYI, lines up to 100 chars are ok now. Automated checkpatch checks as shown in patchwork only flag lines longer than that.

/Bruce

Morten Brørup

2022-07-08 14:30 UTC #65 lore raw

RE: [PATCH 2/2] net: have checksum routines accept unaligned data

Quoted 12 lines

From: Bruce Richardson [mailto:[email protected]]
Sent: Friday, 8 July 2022 16.10
Quoted 8 lines · Mattias Rönnblom
Quoted 7 lines · Morten Brørup
Quoted 7 lines
From: Mattias Rönnblom [mailto:[email protected]]
Sent: Friday, 8 July 2022 14.44
Quoted 3 lines · Morten Brørup
Quoted 3 lines
From: Mattias Rönnblom [mailto:[email protected]]
Sent: Thursday, 7 July 2022 20.35

[...]

Quoted 27 lines

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
lib/net/rte_ip.h rte_ipv4_hdr_len(const struct rte_ipv4_hdr *ipv4_hdr) · line 163
Quoted 8 lines
   static inline uint32_t
   __rte_raw_cksum(const void *buf, size_t len, uint32_t sum)
   {
lib/net/rte_ip.h +1 -4
-	/* extend strict-aliasing rules */
-	typedef uint16_t __attribute__((__may_alias__)) u16_p;
-	const u16_p *u16_buf = (const u16_p *)buf;
-	const u16_p *end = u16_buf + len / sizeof(*u16_buf);
+	const void *end;
I would set "end" here instead, possibly making the pointer const
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
too. And add spaces around '/'.
Quoted 1 line
const void * const end = RTE_PTR_ADD(buf, (len /
sizeof(uint16_t)) *
Quoted 8 lines
Quoted 8 lines
Quoted 4 lines
sizeof(uint16_t));
Quoted 1 line
I don't think that makes the code more readable.
It's only a matter of taste... Your code, your decision. :-)
I think the spaces are required by the coding standard; not sure,
though.
Quoted 3 lines
Quoted 1 line
If it isn't in the coding standard, it should be. But if you add
spaces,
Quoted 1 line
you have to break the line, to fit into 80 characters. A net loss,

Quoted 8 lines

IMO.
Quoted 1 line
Just FYI, lines up to 100 chars are ok now. Automated checkpatch checks as
shown in patchwork only flag lines longer than that.
/Bruce

The coding style [1] recommends max 80 characters, with an (easy to miss) note below, saying that up to 100 characters are acceptable. I think it should be simplified, replacing the 80 characters recommendation by a 100 characters limit, so the additional note can be removed. Why still recommend 80, when we really mean 100?

Ferruh Yigit

2022-07-08 14:44 UTC #49 lore raw

Re: [PATCH v2 2/2] net: have checksum routines accept unaligned data

Quoted 56 lines · Mattias Rönnblom

__rte_raw_cksum() (used by rte_raw_cksum() among others) accessed its data through an uint16_t pointer, which allowed the compiler to assume the data was 16-bit aligned. This in turn would, with certain architectures and compiler flag combinations, result in code with SIMD load or store instructions with restrictions on data alignment.
This patch keeps the old algorithm, but data is read using memcpy() instead of direct pointer access, forcing the compiler to always generate code that handles unaligned input. The __may_alias__ GCC attribute is no longer needed.
The data on which the Internet checksum functions operates are almost always 16-bit aligned, but there are exceptions. In particular, the PDCP protocol header may (literally) have an odd size.
Performance impact seems to range from none to a very slight regression.
Bugzilla ID: 1035
Cc: [email protected]
---
v2:
* Simplified the odd-length conditional (Morten Brørup).
Reviewed-by Morten Brørup <[email protected]>
Signed-off-by Mattias Rönnblom <[email protected]>
---
  lib/net/rte_ip.h | 17 ++++++++++-------
  1 file changed, 10 insertions(+), 7 deletions(-)
lib/net/rte_ip.h +8 -6
index b502481670..a0334d931e 100644
@@ -160,18 +160,21 @@ rte_ipv4_hdr_len(const struct rte_ipv4_hdr *ipv4_hdr)
  static inline uint32_t
  __rte_raw_cksum(const void *buf, size_t len, uint32_t sum)
  {
-	/* extend strict-aliasing rules */
-	typedef uint16_t __attribute__((__may_alias__)) u16_p;
-	const u16_p *u16_buf = (const u16_p *)buf;
-	const u16_p *end = u16_buf + len / sizeof(*u16_buf);
+	const void *end;
 
-	for (; u16_buf != end; ++u16_buf)
-		sum += *u16_buf;
+	for (end = RTE_PTR_ADD(buf, (len/sizeof(uint16_t)) * sizeof(uint16_t));
+	     buf != end; buf = RTE_PTR_ADD(buf, sizeof(uint16_t))) {
+		uint16_t v;
+
+		memcpy(&v, buf, sizeof(uint16_t));
+		sum += v;
+	}

lib/net/rte_ip.h rte_ipv4_hdr_len(const struct rte_ipv4_hdr *ipv4_hdr) · line 180

Quoted 10 lines

lib/net/rte_ip.h +2 -1

 
  	/* if length is odd, keeping it byte order independent */
  	if (unlikely(len % 2)) {
  		uint16_t left = 0;
-		*(unsigned char *)&left = *(const unsigned char *)end;
+
+		memcpy(&left, end, 1);
  		sum += left;
  	}

Hi Mattias,

I got following result [1] with patches on [2].
Can you shed light to some questions I have,
1) For 1500 why 'Unaligned' access gives better performance than 'Aligned' access?
2) Why 21/101 bytes almost doubles 20/100 bytes perf?
3) Why 1501 bytes perf better than 1500 bytes perf?

Btw, I don't see any noticeable performance difference between with and without patch.

[1]
RTE>>cksum_perf_autotest
### rte_raw_cksum() performance ###
Alignment Block size TSC cycles/block TSC cycles/byte

Aligned           20                25.1             1.25
Unaligned         20                25.1             1.25
Aligned           21                51.5             2.45
Unaligned         21                51.5             2.45
Aligned          100                28.2             0.28
Unaligned        100                28.2             0.28
Aligned          101                54.5             0.54
Unaligned        101                54.5             0.54
Aligned         1500               188.9             0.13
Unaligned       1500               138.7             0.09
Aligned         1501               114.1             0.08
Unaligned       1501               110.1             0.07

Test OK
RTE>>

[2]
AMD EPYC 7543P

Olivier Matz

2022-07-11 09:47 UTC #52 lore raw

Re: [PATCH v2 1/2] app/test: add cksum performance test

Hi Mattias,

Please see few comments below.

Quoted 54 lines · Mattias Rönnblom

Add performance test for the rte_raw_cksum() function, which delegates the actual work to __rte_raw_cksum(), which in turn is used by other functions in need of Internet checksum calculation.

---

v2:
* Added __rte_unused to unused volatile variable, to keep the Intel compiler happy.

---
 MAINTAINERS                |   1 +
 app/test/meson.build       |   1 +
 app/test/test_cksum_perf.c | 118 +++++++++++++++++++++++++++++++++++++
 3 files changed, 120 insertions(+)
 create mode 100644 app/test/test_cksum_perf.c

MAINTAINERS +1 -0

index c923712946..2a4c99e05a 100644
@@ -1414,6 +1414,7 @@ Network headers
 M: Olivier Matz <[email protected]>
 F: lib/net/
 F: app/test/test_cksum.c
+F: app/test/test_cksum_perf.c
 
 Packet CRC
 M: Jasvinder Singh <[email protected]>

app/test/meson.build +1 -0

index 431c5bd318..191db03d1d 100644
@@ -18,6 +18,7 @@ test_sources = files(
         'test_bpf.c',
         'test_byteorder.c',
         'test_cksum.c',
+        'test_cksum_perf.c',
         'test_cmdline.c',
         'test_cmdline_cirbuf.c',
         'test_cmdline_etheraddr.c',

app/test/test_cksum_perf.c new file +6 -0

new file mode 100644
index 0000000000..bff73cb3bb
@@ -0,0 +1,118 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Ericsson AB
+ */
+
+#include <stdio.h>
+

app/test/test_cksum_perf.c line 16

Quoted 10 lines

app/test/test_cksum_perf.c new file +10 -0

+#include <rte_common.h>
+#include <rte_cycles.h>
+#include <rte_ip.h>
+#include <rte_malloc.h>
+#include <rte_random.h>
+
+#include "test.h"
+
+#define NUM_BLOCKS (10)
+#define ITERATIONS (1000000)

Parenthesis can be safely removed

app/test/test_cksum_perf.c line 24

Quoted 8 lines

app/test/test_cksum_perf.c new file +8 -0

+
+static const size_t data_sizes[] = { 20, 21, 100, 101, 1500, 1501 };
+
+static __rte_noinline uint16_t
+do_rte_raw_cksum(const void *buf, size_t len)
+{
+	return rte_raw_cksum(buf, len);
+}

I don't understand the need to have this wrapper, especially marked __rte_noinline. What is the objective?

Note that when I remove the __rte_noinline, the performance is better for size 20 and 21.

app/test/test_cksum_perf.c line 27

Quoted 3 lines

app/test/test_cksum_perf.c new file +3 -0
+
+static void
+init_block(void *buf, size_t len)

Can buf be a (char *) instead?
It would avoid a cast below.

Quoted 38 lines

app/test/test_cksum_perf.c new file +38 -0

+{
+	size_t i;
+
+	for (i = 0; i < len; i++)
+		((char *)buf)[i] = (uint8_t)rte_rand();
+}
+
+static int
+test_cksum_perf_size_alignment(size_t block_size, bool aligned)
+{
+	char *data[NUM_BLOCKS];
+	char *blocks[NUM_BLOCKS];
+	unsigned int i;
+	uint64_t start;
+	uint64_t end;
+	/* Floating point to handle low (pseudo-)TSC frequencies */
+	double block_latency;
+	double byte_latency;
+	volatile __rte_unused uint64_t sum = 0;
+
+	for (i = 0; i < NUM_BLOCKS; i++) {
+		data[i] = rte_malloc(NULL, block_size + 1, 0);
+
+		if (data[i] == NULL) {
+			printf("Failed to allocate memory for block\n");
+			return TEST_FAILED;
+		}
+
+		init_block(data[i], block_size + 1);
+
+		blocks[i] = aligned ? data[i] : data[i] + 1;
+	}
+
+	start = rte_rdtsc();
+
+	for (i = 0; i < ITERATIONS; i++) {
+		unsigned int j;
+		for (j = 0; j < NUM_BLOCKS; j++)

app/test/test_cksum_perf.c line 75

Quoted 10 lines

app/test/test_cksum_perf.c new file +10 -0

+			sum += do_rte_raw_cksum(blocks[j], block_size);
+	}
+
+	end = rte_rdtsc();
+
+	block_latency = (end - start) / (double)(ITERATIONS * NUM_BLOCKS);
+	byte_latency = block_latency / block_size;
+
+	printf("%-9s %10zd %19.1f %16.2f\n", aligned ? "Aligned" : "Unaligned",
+	       block_size, block_latency, byte_latency);

When I run the test on my dev machine, I get the following results, which are quite reproductible:

Aligned           20       10.4      0.52     (range is 0.48 - 0.52)
Unaligned         20        7.9      0.39     (range is 0.39 - 0.40)

...

If I increase the number of iterations, the first results change significantly:

Aligned           20        8.2      0.42     (range is 0.41 - 0.42)
Unaligned         20        8.0      0.40     (always this value)

To have more precise tests with small size, would it make sense to target a test time instead of an iteration count? Something like this:

	#define ITERATIONS 1000000
	uint64_t iterations = 0;

	...

	do {
		for (i = 0; i < ITERATIONS; i++) {
			unsigned int j;
			for (j = 0; j < NUM_BLOCKS; j++)
				sum += do_rte_raw_cksum(blocks[j], block_size);
		}
		iterations += ITERATIONS;
		end = rte_rdtsc();
	} while ((end - start) < rte_get_tsc_hz());

	block_latency = (end - start) / (double)(iterations * NUM_BLOCKS);

After this change, the aligned and unaligned cases have the same performance on my machine.

Quoted 33 lines

app/test/test_cksum_perf.c new file +33 -0

+
+	for (i = 0; i < NUM_BLOCKS; i++)
+		rte_free(data[i]);
+
+	return TEST_SUCCESS;
+}
+
+static int
+test_cksum_perf_size(size_t block_size)
+{
+	int rc;
+
+	rc = test_cksum_perf_size_alignment(block_size, true);
+	if (rc != TEST_SUCCESS)
+		return rc;
+
+	rc = test_cksum_perf_size_alignment(block_size, false);
+
+	return rc;
+}
+
+static int
+test_cksum_perf(void)
+{
+	uint16_t i;
+
+	printf("### rte_raw_cksum() performance ###\n");
+	printf("Alignment  Block size    TSC cycles/block  TSC cycles/byte\n");
+
+	for (i = 0; i < RTE_DIM(data_sizes); i++) {
+		int rc;
+
+		rc = test_cksum_perf_size(data_sizes[i]);

app/test/test_cksum_perf.c line 118

Quoted 10 lines

app/test/test_cksum_perf.c new file +10 -0

+		if (rc != TEST_SUCCESS)
+			return rc;
+	}
+
+	return TEST_SUCCESS;
+}
+
+
+REGISTER_TEST_COMMAND(cksum_perf_autotest, test_cksum_perf);
+

The last empty line can be removed.

Quoted 3 lines

Signature · 1 line
2.25.1

Olivier Matz

2022-07-11 09:53 UTC #50 lore raw

Re: [PATCH v2 2/2] net: have checksum routines accept unaligned data

Hi,

Quoted 15 lines · Mattias Rönnblom

__rte_raw_cksum() (used by rte_raw_cksum() among others) accessed its data through an uint16_t pointer, which allowed the compiler to assume the data was 16-bit aligned. This in turn would, with certain architectures and compiler flag combinations, result in code with SIMD load or store instructions with restrictions on data alignment.
This patch keeps the old algorithm, but data is read using memcpy() instead of direct pointer access, forcing the compiler to always generate code that handles unaligned input. The __may_alias__ GCC attribute is no longer needed.
The data on which the Internet checksum functions operates are almost always 16-bit aligned, but there are exceptions. In particular, the PDCP protocol header may (literally) have an odd size.

Quoted 7 lines

Performance impact seems to range from none to a very slight regression.
Bugzilla ID: 1035
Cc: [email protected]
---

Using memcpy() looks to be a good solution fix the issue, while avoiding a branch and the __may_alias__.

I just have one minor comment below.

Quoted 18 lines

v2:
* Simplified the odd-length conditional (Morten Brørup).
Reviewed-by Morten Brørup <[email protected]>
Signed-off-by Mattias Rönnblom <[email protected]>
---
 lib/net/rte_ip.h | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)
lib/net/rte_ip.h +0 -0
index b502481670..a0334d931e 100644
@@ -160,18 +160,21 @@ rte_ipv4_hdr_len(const struct rte_ipv4_hdr *ipv4_hdr)
 static inline uint32_t
 __rte_raw_cksum(const void *buf, size_t len, uint32_t sum)

lib/net/rte_ip.h rte_ipv4_hdr_len(const struct rte_ipv4_hdr *ipv4_hdr) · line 165

Quoted 10 lines

lib/net/rte_ip.h +2 -6

 {
-	/* extend strict-aliasing rules */
-	typedef uint16_t __attribute__((__may_alias__)) u16_p;
-	const u16_p *u16_buf = (const u16_p *)buf;
-	const u16_p *end = u16_buf + len / sizeof(*u16_buf);
+	const void *end;
 
-	for (; u16_buf != end; ++u16_buf)
-		sum += *u16_buf;
+	for (end = RTE_PTR_ADD(buf, (len/sizeof(uint16_t)) * sizeof(uint16_t));

What do you think about this form:

	for (end = RTE_PTR_ADD(buf, RTE_ALIGN_FLOOR(len, sizeof(uint16_t)));

This also has the good property to solve the debate about the spaces around the '/' :)

Quoted 19 lines

lib/net/rte_ip.h +8 -1

+	     buf != end; buf = RTE_PTR_ADD(buf, sizeof(uint16_t))) {
+		uint16_t v;
+
+		memcpy(&v, buf, sizeof(uint16_t));
+		sum += v;
+	}
 
 	/* if length is odd, keeping it byte order independent */
 	if (unlikely(len % 2)) {
 		uint16_t left = 0;
-		*(unsigned char *)&left = *(const unsigned char *)end;
+
+		memcpy(&left, end, 1);
 		sum += left;
 	}

Signature · 1 line

2.25.1

Mattias Rönnblom

2022-07-11 10:42 UTC #53 lore raw

Re: [PATCH v2 1/2] app/test: add cksum performance test

Quoted 72 lines · Olivier Matz

Hi Mattias,
Please see few comments below.
Quoted 54 lines · Mattias Rönnblom
Add performance test for the rte_raw_cksum() function, which delegates the actual work to __rte_raw_cksum(), which in turn is used by other functions in need of Internet checksum calculation.
Signed-off-by Mattias Rönnblom <[email protected]>
---
v2:
* Added __rte_unused to unused volatile variable, to keep the Intel compiler happy.
---
  MAINTAINERS                |   1 +
  app/test/meson.build       |   1 +
  app/test/test_cksum_perf.c | 118 +++++++++++++++++++++++++++++++++++++
  3 files changed, 120 insertions(+)
create mode 100644 app/test/test_cksum_perf.c
MAINTAINERS +1 -0
index c923712946..2a4c99e05a 100644
@@ -1414,6 +1414,7 @@ Network headers
  M: Olivier Matz <[email protected]>
  F: lib/net/
  F: app/test/test_cksum.c
+F: app/test/test_cksum_perf.c
 
  Packet CRC
  M: Jasvinder Singh <[email protected]>
app/test/meson.build +1 -0
index 431c5bd318..191db03d1d 100644
@@ -18,6 +18,7 @@ test_sources = files(
          'test_bpf.c',
          'test_byteorder.c',
          'test_cksum.c',
+        'test_cksum_perf.c',
          'test_cmdline.c',
          'test_cmdline_cirbuf.c',
          'test_cmdline_etheraddr.c',
app/test/test_cksum_perf.c new file +6 -0
new file mode 100644
index 0000000000..bff73cb3bb
@@ -0,0 +1,118 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Ericsson AB
+ */
+
+#include <stdio.h>
+
app/test/test_cksum_perf.c line 16
Quoted 10 lines
app/test/test_cksum_perf.c new file +10 -0
+#include <rte_common.h>
+#include <rte_cycles.h>
+#include <rte_ip.h>
+#include <rte_malloc.h>
+#include <rte_random.h>
+
+#include "test.h"
+
+#define NUM_BLOCKS (10)
+#define ITERATIONS (1000000)
Parenthesis can be safely removed

Quoted 12 lines

app/test/test_cksum_perf.c line 24
Quoted 8 lines
app/test/test_cksum_perf.c new file +8 -0
+
+static const size_t data_sizes[] = { 20, 21, 100, 101, 1500, 1501 };
+
+static __rte_noinline uint16_t
+do_rte_raw_cksum(const void *buf, size_t len)
+{
+	return rte_raw_cksum(buf, len);
+}
I don't understand the need to have this wrapper, especially marked __rte_noinline. What is the objective?

The intention is to disallow the compiler to perform unrolling and integrating/interleave one cksum operating with the next buffer's in a way that wouldn't be feasable in a real application.

It will result in an overestimation of the cost for small cksums, so it's still misleading, but in another direction. :)

Quoted 10 lines

Note that when I remove the __rte_noinline, the performance is better for size 20 and 21.
app/test/test_cksum_perf.c line 27
Quoted 3 lines
app/test/test_cksum_perf.c new file +3 -0
+
+static void
+init_block(void *buf, size_t len)
Can buf be a (char *) instead?
It would avoid a cast below.

Yes.

Quoted 54 lines

Quoted 38 lines

app/test/test_cksum_perf.c new file +38 -0

+{
+	size_t i;
+
+	for (i = 0; i < len; i++)
+		((char *)buf)[i] = (uint8_t)rte_rand();
+}
+
+static int
+test_cksum_perf_size_alignment(size_t block_size, bool aligned)
+{
+	char *data[NUM_BLOCKS];
+	char *blocks[NUM_BLOCKS];
+	unsigned int i;
+	uint64_t start;
+	uint64_t end;
+	/* Floating point to handle low (pseudo-)TSC frequencies */
+	double block_latency;
+	double byte_latency;
+	volatile __rte_unused uint64_t sum = 0;
+
+	for (i = 0; i < NUM_BLOCKS; i++) {
+		data[i] = rte_malloc(NULL, block_size + 1, 0);
+
+		if (data[i] == NULL) {
+			printf("Failed to allocate memory for block\n");
+			return TEST_FAILED;
+		}
+
+		init_block(data[i], block_size + 1);
+
+		blocks[i] = aligned ? data[i] : data[i] + 1;
+	}
+
+	start = rte_rdtsc();
+
+	for (i = 0; i < ITERATIONS; i++) {
+		unsigned int j;
+		for (j = 0; j < NUM_BLOCKS; j++)

app/test/test_cksum_perf.c line 75

Quoted 10 lines

app/test/test_cksum_perf.c new file +10 -0

+			sum += do_rte_raw_cksum(blocks[j], block_size);
+	}
+
+	end = rte_rdtsc();
+
+	block_latency = (end - start) / (double)(ITERATIONS * NUM_BLOCKS);
+	byte_latency = block_latency / block_size;
+
+	printf("%-9s %10zd %19.1f %16.2f\n", aligned ? "Aligned" : "Unaligned",
+	       block_size, block_latency, byte_latency);

When I run the test on my dev machine, I get the following results, which are quite reproductible:

Aligned           20       10.4      0.52     (range is 0.48 - 0.52)
Unaligned         20        7.9      0.39     (range is 0.39 - 0.40)

Quoted 7 lines

...
If I increase the number of iterations, the first results change significantly:
Aligned           20        8.2      0.42     (range is 0.41 - 0.42)
Unaligned         20        8.0      0.40     (always this value)

I suspect you have frequency scaling enabled on your system. This is generally not advisable, you want to some level of determinism in when benchmarking. Especially on short runs like this is (and must be).

I thought about doing something about this, but it seemed like an issue that should be addressed on a framework level, rather than on a per-perf autotest level.

If you want your CPU core to scale up, you can just insert

rte_delay_block_us(100000);

before the actual test is run.

Should I add this? I *think* 100 ms should be enough, but maybe someone with more in-depth knowledge of the frequency governors can comment on this.

Quoted 5 lines

To have more precise tests with small size, would it make sense to target a test time instead of an iteration count? Something like this:

The time lost when running on a lower frequency (plus the hiccups when the frequency is changed) will be amortized as you add to the length of the test run, which will partly solved the problem. A better solution is to not start the test before the core runs on the max frequency.

Again, this is assuming DVFS is what you suffer from here. I guess in theory it could be TLB miss as well.

Quoted 18 lines

	#define ITERATIONS 1000000
	uint64_t iterations = 0;

	...

	do {
		for (i = 0; i < ITERATIONS; i++) {
			unsigned int j;
			for (j = 0; j < NUM_BLOCKS; j++)
				sum += do_rte_raw_cksum(blocks[j], block_size);
		}
		iterations += ITERATIONS;
		end = rte_rdtsc();
	} while ((end - start) < rte_get_tsc_hz());

	block_latency = (end - start) / (double)(iterations * NUM_BLOCKS);

Quoted 4 lines

After this change, the aligned and unaligned cases have the same performance on my machine.

RTE>>cksum_perf_autotest
### rte_raw_cksum() performance ###
Alignment Block size TSC cycles/block TSC cycles/byte

Aligned           20                16.1             0.81
Unaligned         20                16.1             0.81

... with the 100 ms busy-wait delay (and frequency scaling enabled) on my AMD machine.

Quoted 37 lines

app/test/test_cksum_perf.c new file +37 -0

+
+	for (i = 0; i < NUM_BLOCKS; i++)
+		rte_free(data[i]);
+
+	return TEST_SUCCESS;
+}
+
+static int
+test_cksum_perf_size(size_t block_size)
+{
+	int rc;
+
+	rc = test_cksum_perf_size_alignment(block_size, true);
+	if (rc != TEST_SUCCESS)
+		return rc;
+
+	rc = test_cksum_perf_size_alignment(block_size, false);
+
+	return rc;
+}
+
+static int
+test_cksum_perf(void)
+{
+	uint16_t i;
+
+	printf("### rte_raw_cksum() performance ###\n");
+	printf("Alignment  Block size    TSC cycles/block  TSC cycles/byte\n");
+
+	for (i = 0; i < RTE_DIM(data_sizes); i++) {
+		int rc;
+
+		rc = test_cksum_perf_size(data_sizes[i]);
+		if (rc != TEST_SUCCESS)
+			return rc;
+	}
+

Quoted 9 lines

app/test/test_cksum_perf.c line 118
Quoted 6 lines
app/test/test_cksum_perf.c new file +6 -0
+	return TEST_SUCCESS;
+}
+
+
+REGISTER_TEST_COMMAND(cksum_perf_autotest, test_cksum_perf);
+
The last empty line can be removed.

OK.

Thanks for the review. I will send a v3 as soon as we've settled the DVFS issue.

Quoted 3 lines

Quoted 3 lines
Signature · 1 line
2.25.1

Mattias Rönnblom

2022-07-11 10:53 UTC #51 lore raw

Re: [PATCH v2 2/2] net: have checksum routines accept unaligned data

Quoted 62 lines · Olivier Matz

Hi,
Quoted 15 lines · Mattias Rönnblom
__rte_raw_cksum() (used by rte_raw_cksum() among others) accessed its data through an uint16_t pointer, which allowed the compiler to assume the data was 16-bit aligned. This in turn would, with certain architectures and compiler flag combinations, result in code with SIMD load or store instructions with restrictions on data alignment.
This patch keeps the old algorithm, but data is read using memcpy() instead of direct pointer access, forcing the compiler to always generate code that handles unaligned input. The __may_alias__ GCC attribute is no longer needed.
The data on which the Internet checksum functions operates are almost always 16-bit aligned, but there are exceptions. In particular, the PDCP protocol header may (literally) have an odd size.
Quoted 7 lines
Performance impact seems to range from none to a very slight regression.
Bugzilla ID: 1035
Cc: [email protected]
---
Using memcpy() looks to be a good solution fix the issue, while avoiding a branch and the __may_alias__.
I just have one minor comment below.
Quoted 18 lines
v2:
* Simplified the odd-length conditional (Morten Brørup).
Reviewed-by Morten Brørup <[email protected]>
Signed-off-by Mattias Rönnblom <[email protected]>
---
  lib/net/rte_ip.h | 17 ++++++++++-------
  1 file changed, 10 insertions(+), 7 deletions(-)
lib/net/rte_ip.h +0 -0
index b502481670..a0334d931e 100644
@@ -160,18 +160,21 @@ rte_ipv4_hdr_len(const struct rte_ipv4_hdr *ipv4_hdr)
  static inline uint32_t
  __rte_raw_cksum(const void *buf, size_t len, uint32_t sum)
lib/net/rte_ip.h rte_ipv4_hdr_len(const struct rte_ipv4_hdr *ipv4_hdr) · line 165
Quoted 10 lines
lib/net/rte_ip.h +2 -6
  {
-	/* extend strict-aliasing rules */
-	typedef uint16_t __attribute__((__may_alias__)) u16_p;
-	const u16_p *u16_buf = (const u16_p *)buf;
-	const u16_p *end = u16_buf + len / sizeof(*u16_buf);
+	const void *end;
 
-	for (; u16_buf != end; ++u16_buf)
-		sum += *u16_buf;
+	for (end = RTE_PTR_ADD(buf, (len/sizeof(uint16_t)) * sizeof(uint16_t));
What do you think about this form:

Quoted 5 lines

	for (end = RTE_PTR_ADD(buf, RTE_ALIGN_FLOOR(len, sizeof(uint16_t)));
This also has the good property to solve the debate about the spaces around the '/' :)

Shorter, more readable. Looks good to me.

Thanks.

Quoted 20 lines

lib/net/rte_ip.h +8 -1

+	     buf != end; buf = RTE_PTR_ADD(buf, sizeof(uint16_t))) {
+		uint16_t v;
+
+		memcpy(&v, buf, sizeof(uint16_t));
+		sum += v;
+	}
 
  	/* if length is odd, keeping it byte order independent */
  	if (unlikely(len % 2)) {
  		uint16_t left = 0;
-		*(unsigned char *)&left = *(const unsigned char *)end;
+
+		memcpy(&left, end, 1);
  		sum += left;
  	}

Signature · 1 line

2.25.1

Olivier Matz

2022-07-11 11:33 UTC #54 lore raw

Re: [PATCH v2 1/2] app/test: add cksum performance test

Quoted 86 lines · Mattias Rönnblom

Quoted 72 lines · Olivier Matz
Hi Mattias,
Please see few comments below.
Quoted 54 lines · Mattias Rönnblom
Add performance test for the rte_raw_cksum() function, which delegates the actual work to __rte_raw_cksum(), which in turn is used by other functions in need of Internet checksum calculation.
Signed-off-by Mattias Rönnblom <[email protected]>
---
v2:
* Added __rte_unused to unused volatile variable, to keep the Intel compiler happy.
---
  MAINTAINERS                |   1 +
  app/test/meson.build       |   1 +
  app/test/test_cksum_perf.c | 118 +++++++++++++++++++++++++++++++++++++
  3 files changed, 120 insertions(+)
create mode 100644 app/test/test_cksum_perf.c
MAINTAINERS +1 -0
index c923712946..2a4c99e05a 100644
@@ -1414,6 +1414,7 @@ Network headers
  M: Olivier Matz <[email protected]>
  F: lib/net/
  F: app/test/test_cksum.c
+F: app/test/test_cksum_perf.c
 
  Packet CRC
  M: Jasvinder Singh <[email protected]>
app/test/meson.build +1 -0
index 431c5bd318..191db03d1d 100644
@@ -18,6 +18,7 @@ test_sources = files(
          'test_bpf.c',
          'test_byteorder.c',
          'test_cksum.c',
+        'test_cksum_perf.c',
          'test_cmdline.c',
          'test_cmdline_cirbuf.c',
          'test_cmdline_etheraddr.c',
app/test/test_cksum_perf.c new file +6 -0
new file mode 100644
index 0000000000..bff73cb3bb
@@ -0,0 +1,118 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2022 Ericsson AB
+ */
+
+#include <stdio.h>
+
app/test/test_cksum_perf.c line 16
Quoted 10 lines
app/test/test_cksum_perf.c new file +10 -0
+#include <rte_common.h>
+#include <rte_cycles.h>
+#include <rte_ip.h>
+#include <rte_malloc.h>
+#include <rte_random.h>
+
+#include "test.h"
+
+#define NUM_BLOCKS (10)
+#define ITERATIONS (1000000)
Parenthesis can be safely removed
Quoted 12 lines
app/test/test_cksum_perf.c line 24
Quoted 8 lines
app/test/test_cksum_perf.c new file +8 -0
+
+static const size_t data_sizes[] = { 20, 21, 100, 101, 1500, 1501 };
+
+static __rte_noinline uint16_t
+do_rte_raw_cksum(const void *buf, size_t len)
+{
+	return rte_raw_cksum(buf, len);
+}
I don't understand the need to have this wrapper, especially marked __rte_noinline. What is the objective?

Quoted 6 lines

The intention is to disallow the compiler to perform unrolling and integrating/interleave one cksum operating with the next buffer's in a way that wouldn't be feasable in a real application.
It will result in an overestimation of the cost for small cksums, so it's still misleading, but in another direction. :)

OK, got it. I think it's fine like you did then.

Quoted 85 lines

Quoted 11 lines
Note that when I remove the __rte_noinline, the performance is better for size 20 and 21.
app/test/test_cksum_perf.c line 27
Quoted 3 lines
app/test/test_cksum_perf.c new file +3 -0
+
+static void
+init_block(void *buf, size_t len)
Can buf be a (char *) instead?
It would avoid a cast below.
Yes.
Quoted 54 lines
Quoted 38 lines
app/test/test_cksum_perf.c new file +38 -0
+{
+	size_t i;
+
+	for (i = 0; i < len; i++)
+		((char *)buf)[i] = (uint8_t)rte_rand();
+}
+
+static int
+test_cksum_perf_size_alignment(size_t block_size, bool aligned)
+{
+	char *data[NUM_BLOCKS];
+	char *blocks[NUM_BLOCKS];
+	unsigned int i;
+	uint64_t start;
+	uint64_t end;
+	/* Floating point to handle low (pseudo-)TSC frequencies */
+	double block_latency;
+	double byte_latency;
+	volatile __rte_unused uint64_t sum = 0;
+
+	for (i = 0; i < NUM_BLOCKS; i++) {
+		data[i] = rte_malloc(NULL, block_size + 1, 0);
+
+		if (data[i] == NULL) {
+			printf("Failed to allocate memory for block\n");
+			return TEST_FAILED;
+		}
+
+		init_block(data[i], block_size + 1);
+
+		blocks[i] = aligned ? data[i] : data[i] + 1;
+	}
+
+	start = rte_rdtsc();
+
+	for (i = 0; i < ITERATIONS; i++) {
+		unsigned int j;
+		for (j = 0; j < NUM_BLOCKS; j++)
app/test/test_cksum_perf.c line 75
Quoted 10 lines
app/test/test_cksum_perf.c new file +10 -0
+			sum += do_rte_raw_cksum(blocks[j], block_size);
+	}
+
+	end = rte_rdtsc();
+
+	block_latency = (end - start) / (double)(ITERATIONS * NUM_BLOCKS);
+	byte_latency = block_latency / block_size;
+
+	printf("%-9s %10zd %19.1f %16.2f\n", aligned ? "Aligned" : "Unaligned",
+	       block_size, block_latency, byte_latency);
When I run the test on my dev machine, I get the following results, which are quite reproductible:
Aligned           20       10.4      0.52     (range is 0.48 - 0.52)
Unaligned         20        7.9      0.39     (range is 0.39 - 0.40)
Quoted 7 lines
...
If I increase the number of iterations, the first results change significantly:
Aligned           20        8.2      0.42     (range is 0.41 - 0.42)
Unaligned         20        8.0      0.40     (always this value)
I suspect you have frequency scaling enabled on your system. This is generally not advisable, you want to some level of determinism in when benchmarking. Especially on short runs like this is (and must be).
I thought about doing something about this, but it seemed like an issue that should be addressed on a framework level, rather than on a per-perf autotest level.

Quoted 5 lines

If you want your CPU core to scale up, you can just insert
rte_delay_block_us(100000);
before the actual test is run.

Your hypothesis is correct. When I disable frequency scaling, the results are now the same with 100K, 1M or 10M iterations.

However, adding a rte_delay_us_block() does not really solve the issue, probably because it calls rte_pause() in the loop.

Quoted 2 lines

Should I add this? I *think* 100 ms should be enough, but maybe someone with more in-depth knowledge of the frequency governors can comment on this.

Anyway, I think we don't need to add the blocking delay, we can legitimally expect that freq scaling is disabled when we run performance tests.

Quoted 103 lines

Quoted 6 lines
To have more precise tests with small size, would it make sense to target a test time instead of an iteration count? Something like this:
The time lost when running on a lower frequency (plus the hiccups when the frequency is changed) will be amortized as you add to the length of the test run, which will partly solved the problem. A better solution is to not start the test before the core runs on the max frequency.
Again, this is assuming DVFS is what you suffer from here. I guess in theory it could be TLB miss as well.
Quoted 18 lines
	#define ITERATIONS 1000000
	uint64_t iterations = 0;

	...

	do {
		for (i = 0; i < ITERATIONS; i++) {
			unsigned int j;
			for (j = 0; j < NUM_BLOCKS; j++)
				sum += do_rte_raw_cksum(blocks[j], block_size);
		}
		iterations += ITERATIONS;
		end = rte_rdtsc();
	} while ((end - start) < rte_get_tsc_hz());

	block_latency = (end - start) / (double)(iterations * NUM_BLOCKS);
Quoted 4 lines
After this change, the aligned and unaligned cases have the same performance on my machine.
RTE>>cksum_perf_autotest
### rte_raw_cksum() performance ###
Alignment Block size TSC cycles/block TSC cycles/byte
Aligned           20                16.1             0.81
Unaligned         20                16.1             0.81
... with the 100 ms busy-wait delay (and frequency scaling enabled) on my AMD machine.
Quoted 37 lines
Quoted 37 lines
app/test/test_cksum_perf.c new file +37 -0
+
+	for (i = 0; i < NUM_BLOCKS; i++)
+		rte_free(data[i]);
+
+	return TEST_SUCCESS;
+}
+
+static int
+test_cksum_perf_size(size_t block_size)
+{
+	int rc;
+
+	rc = test_cksum_perf_size_alignment(block_size, true);
+	if (rc != TEST_SUCCESS)
+		return rc;
+
+	rc = test_cksum_perf_size_alignment(block_size, false);
+
+	return rc;
+}
+
+static int
+test_cksum_perf(void)
+{
+	uint16_t i;
+
+	printf("### rte_raw_cksum() performance ###\n");
+	printf("Alignment  Block size    TSC cycles/block  TSC cycles/byte\n");
+
+	for (i = 0; i < RTE_DIM(data_sizes); i++) {
+		int rc;
+
+		rc = test_cksum_perf_size(data_sizes[i]);
+		if (rc != TEST_SUCCESS)
+			return rc;
+	}
+
Quoted 9 lines
app/test/test_cksum_perf.c line 118
Quoted 6 lines
app/test/test_cksum_perf.c new file +6 -0
+	return TEST_SUCCESS;
+}
+
+
+REGISTER_TEST_COMMAND(cksum_perf_autotest, test_cksum_perf);
+
The last empty line can be removed.
OK.
Thanks for the review. I will send a v3 as soon as we've settled the DVFS issue.
Quoted 3 lines
Quoted 3 lines
Signature · 1 line
2.25.1

Mattias Rönnblom

2022-07-11 12:14 UTC #56 lore raw

PATCH v3 2/2 net: have checksum routines accept unaligned data

The data on which the Internet checksum functions operates are almost always 16-bit aligned, but there are exceptions. In particular, the PDCP protocol header may (literally) have an odd size.

Performance impact seems to range from none to a very slight regression.

Bugzilla ID: 1035
Cc: [email protected]

---

v3:
* Use RTE_ALIGN_FLOOR() in the pointer arithmetic (Olivier Matz).
v2:
* Simplified the odd-length conditional (Morten Brørup).

Reviewed-by Morten Brørup <[email protected]>

---
 lib/net/rte_ip.h | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)

lib/net/rte_ip.h +10 -7

index b502481670..ecd250e9be 100644
@@ -160,18 +160,21 @@ rte_ipv4_hdr_len(const struct rte_ipv4_hdr *ipv4_hdr)
 static inline uint32_t
 __rte_raw_cksum(const void *buf, size_t len, uint32_t sum)
 {
-	/* extend strict-aliasing rules */
-	typedef uint16_t __attribute__((__may_alias__)) u16_p;
-	const u16_p *u16_buf = (const u16_p *)buf;
-	const u16_p *end = u16_buf + len / sizeof(*u16_buf);
+	const void *end;
 
-	for (; u16_buf != end; ++u16_buf)
-		sum += *u16_buf;
+	for (end = RTE_PTR_ADD(buf, RTE_ALIGN_FLOOR(len, sizeof(uint16_t)));
+	     buf != end; buf = RTE_PTR_ADD(buf, sizeof(uint16_t))) {
+		uint16_t v;
+
+		memcpy(&v, buf, sizeof(uint16_t));
+		sum += v;
+	}
 
 	/* if length is odd, keeping it byte order independent */
 	if (unlikely(len % 2)) {
 		uint16_t left = 0;
-		*(unsigned char *)&left = *(const unsigned char *)end;
+
+		memcpy(&left, end, 1);
 		sum += left;
 	}

Signature · 1 line

2.25.1

Mattias Rönnblom

2022-07-11 12:14 UTC #55 lore raw

PATCH v3 1/2 app/test: add cksum performance test

Add performance test for the rte_raw_cksum() function, which delegates the actual work to __rte_raw_cksum(), which in turn is used by other functions in need of Internet checksum calculation.

---

v3:
* Changed init function buffer parameter type, to avoid cast.
* Code formatting improved.
v2:
* Added __rte_unused to unused volatile variable, to keep the Intel compiler happy.

---
 MAINTAINERS                |   1 +
 app/test/meson.build       |   1 +
 app/test/test_cksum_perf.c | 117 +++++++++++++++++++++++++++++++++++++
 3 files changed, 119 insertions(+)
 create mode 100644 app/test/test_cksum_perf.c

MAINTAINERS +1 -0

index c923712946..2a4c99e05a 100644
@@ -1414,6 +1414,7 @@ Network headers
 M: Olivier Matz <[email protected]>
 F: lib/net/
 F: app/test/test_cksum.c
+F: app/test/test_cksum_perf.c
 
 Packet CRC
 M: Jasvinder Singh <[email protected]>

app/test/meson.build +1 -0

index 431c5bd318..191db03d1d 100644
@@ -18,6 +18,7 @@ test_sources = files(
         'test_bpf.c',
         'test_byteorder.c',
         'test_cksum.c',
+        'test_cksum_perf.c',
         'test_cmdline.c',
         'test_cmdline_cirbuf.c',
         'test_cmdline_etheraddr.c',

app/test/test_cksum_perf.c new file +117 -0

117 lines not on this page · raw message

Signature · 1 line

2.25.1

Olivier Matz

Acked-by on v3 1/2

2022-07-11 13:20 UTC #61 lore raw

Re: [PATCH v3 1/2] app/test: add cksum performance test

Quoted 5 lines · Mattias Rönnblom

Add performance test for the rte_raw_cksum() function, which delegates the actual work to __rte_raw_cksum(), which in turn is used by other functions in need of Internet checksum calculation.
Signed-off-by Mattias Rönnblom <[email protected]>

Acked-by Olivier Matz <[email protected]>

Thank you!

Olivier Matz

Acked-by on v3 2/2

2022-07-11 13:25 UTC #57 lore raw

Re: [PATCH v3 2/2] net: have checksum routines accept unaligned data

Quoted 15 lines · Mattias Rönnblom

__rte_raw_cksum() (used by rte_raw_cksum() among others) accessed its data through an uint16_t pointer, which allowed the compiler to assume the data was 16-bit aligned. This in turn would, with certain architectures and compiler flag combinations, result in code with SIMD load or store instructions with restrictions on data alignment.
This patch keeps the old algorithm, but data is read using memcpy() instead of direct pointer access, forcing the compiler to always generate code that handles unaligned input. The __may_alias__ GCC attribute is no longer needed.
The data on which the Internet checksum functions operates are almost always 16-bit aligned, but there are exceptions. In particular, the PDCP protocol header may (literally) have an odd size.

Quoted 5 lines

Performance impact seems to range from none to a very slight regression.
Bugzilla ID: 1035
Cc: [email protected]

Fixes 6006818cfb26 ("net: new checksum functions")

Quoted 10 lines

---
v3:
* Use RTE_ALIGN_FLOOR() in the pointer arithmetic (Olivier Matz).
v2:
* Simplified the odd-length conditional (Morten Brørup).
Reviewed-by Morten Brørup <[email protected]>
Signed-off-by Mattias Rönnblom <[email protected]>

Acked-by Olivier Matz <[email protected]>

Thank you!

Mattias Rönnblom

2022-08-08 09:25 UTC #58 lore raw

Re: [PATCH v3 2/2] net: have checksum routines accept unaligned data

Quoted 31 lines · Olivier Matz

Quoted 15 lines · Mattias Rönnblom
__rte_raw_cksum() (used by rte_raw_cksum() among others) accessed its data through an uint16_t pointer, which allowed the compiler to assume the data was 16-bit aligned. This in turn would, with certain architectures and compiler flag combinations, result in code with SIMD load or store instructions with restrictions on data alignment.
This patch keeps the old algorithm, but data is read using memcpy() instead of direct pointer access, forcing the compiler to always generate code that handles unaligned input. The __may_alias__ GCC attribute is no longer needed.
The data on which the Internet checksum functions operates are almost always 16-bit aligned, but there are exceptions. In particular, the PDCP protocol header may (literally) have an odd size.
Quoted 5 lines
Performance impact seems to range from none to a very slight regression.
Bugzilla ID: 1035
Cc: [email protected]
Fixes 6006818cfb26 ("net: new checksum functions")
Quoted 7 lines
---
v3:
* Use RTE_ALIGN_FLOOR() in the pointer arithmetic (Olivier Matz).
v2:
* Simplified the odd-length conditional (Morten Brørup).

Quoted 7 lines

Quoted 3 lines
Reviewed-by Morten Brørup <[email protected]>
Signed-off-by Mattias Rönnblom <[email protected]>
Acked-by Olivier Matz <[email protected]>
Thank you!

Will this be merged into 22.11? Into the stable branches?

Mattias Rönnblom

2022-09-20 12:09 UTC #59 lore raw

Re: [PATCH v3 2/2] net: have checksum routines accept unaligned data