Skip to contentSkip to conversation

app/testpmd: fix IP checksum calculation

PATCH Superseded George Prekas 3 participants 21 messages Dec 3, 2020 – Jan 18, 2021 Arm/Amlogic Meson SoC support

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 app/testpmd: fix IP checksum calculation

Insert a compiler barrier to make sure that the IP checksum calculation happens after setting all the fields of the IP header.

---
 app/test-pmd/flowgen.c | 1 +
 1 file changed, 1 insertion(+)
app/test-pmd/flowgen.c +1 -0
index acf3e2460..893b4b0b8 100644
@@ -150,6 +150,7 @@ pkt_burst_flow_gen(struct fwd_stream *fs)
 							   next_flow);
 		ip_hdr->total_length	= RTE_CPU_TO_BE_16(pkt_size -
 							   sizeof(*eth_hdr));
+		rte_compiler_barrier();
 		ip_hdr->hdr_checksum	= ip_sum((unaligned_uint16_t *)ip_hdr,
 						 sizeof(*ip_hdr));
 
Signature · 1 line
2.17.1
#1 lore raw
Quoted 4 lines · On Thu, 3 Dec 2020 07:59:54 -0600 George Prekas <[email protected]> wrote:

Insert a compiler barrier to make sure that the IP checksum calculation happens after setting all the fields of the IP header.

I don't think this is necessary. All other OS's don't have to do this. The CPU is going to maintain proper memory order.

#2 lore raw
Quoted 9 lines · Stephen Hemminger
Quoted 4 lines · On Thu, 3 Dec 2020 07:59:54 -0600 George Prekas <[email protected]> wrote:

Insert a compiler barrier to make sure that the IP checksum calculation happens after setting all the fields of the IP header.

I don't think this is necessary. All other OS's don't have to do this. The CPU is going to maintain proper memory order.

Hi Stephen,

This is not a CPU or OS issue. This is a compiler issue. The compiler is free to reorder statements if it does not detect dependencies between them. Without this compiler barrier, the calculated IP checksum is wrong.

#3 lore raw
Quoted 8 lines · On Thu, 3 Dec 2020 10:35:50 -0600 George Prekas <[email protected]> wrote:
Quoted 7 lines · Stephen Hemminger
Quoted 4 lines · On Thu, 3 Dec 2020 07:59:54 -0600 George Prekas <[email protected]> wrote:

Insert a compiler barrier to make sure that the IP checksum calculation happens after setting all the fields of the IP header.

Quoted 10 lines
Quoted 2 lines

I don't think this is necessary. All other OS's don't have to do this. The CPU is going to maintain proper memory order.

Hi Stephen,

This is not a CPU or OS issue. This is a compiler issue. The compiler is free to reorder statements if it does not detect dependencies between them. Without this compiler barrier, the calculated IP checksum is wrong.

But the compiler should be detecting any aliasing here, if no you have a bad compiler.

#4 lore raw
In reply to George Prekas
Quoted 3 lines · George Prekas

Insert a compiler barrier to make sure that the IP checksum calculation happens after setting all the fields of the IP header.

Can you please provide the compiler details, and if there is any specific instruction on how to reproduce this failure?

Quoted 18 lines
---
  app/test-pmd/flowgen.c | 1 +
  1 file changed, 1 insertion(+)
app/test-pmd/flowgen.c +1 -0
index acf3e2460..893b4b0b8 100644
@@ -150,6 +150,7 @@ pkt_burst_flow_gen(struct fwd_stream *fs)
  							   next_flow);
  		ip_hdr->total_length	= RTE_CPU_TO_BE_16(pkt_size -
  							   sizeof(*eth_hdr));
+		rte_compiler_barrier();
  		ip_hdr->hdr_checksum	= ip_sum((unaligned_uint16_t *)ip_hdr,
  						 sizeof(*ip_hdr));
 
#5 lore raw
Quoted 11 lines · Ferruh Yigit

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.

Quoted 3 lines · George Prekas

Insert a compiler barrier to make sure that the IP checksum calculation happens after setting all the fields of the IP header.

Can you please provide the compiler details, and if there is any specific instruction on how to reproduce this failure?

This happens with GCC 9 and GCC 10. It works fine on GCC 8.

Stephen was right that a compiler barrier here is not the right solution. After spending some time on it, I realized that it is an aliasing problem when casting the IP header to uint16_t*. As far as I understand, this is not allowed by the C standard. As far as I know, there are 3 ways to fix this problem: Use a union, use memcpy, or set the compiler flag -fno-strict-aliasing. I assume that the last option is the least intrusive. I've submitted a second version of the patch with it.

Let me know of your opinion.

Quoted 20 lines
Quoted 19 lines

---
  app/test-pmd/flowgen.c | 1 +

  1 file changed, 1 insertion(+)
app/test-pmd/flowgen.c +0 -0
index acf3e2460..893b4b0b8 100644
@@ -150,6 +150,7 @@ pkt_burst_flow_gen(struct fwd_stream *fs)
 next_flow);

              ip_hdr->total_length    = RTE_CPU_TO_BE_16(pkt_size - sizeof(*eth_hdr));
+             rte_compiler_barrier();
              ip_hdr->hdr_checksum    = ip_sum((unaligned_uint16_t *)ip_hdr, sizeof(*ip_hdr));

#6 lore raw

PATCH v2 1/1 app/testpmd: fix IP checksum calculation

In reply to George Prekas

Strict-aliasing rules are violated by cast to uint16_t* in flowgen.c and the calculated IP checksum is wrong on GCC 9 and GCC 10.

---
v2:
* Instead of a compiler barrier, use a compiler flag.

---
 app/test-pmd/meson.build | 1 +
 1 file changed, 1 insertion(+)
app/test-pmd/meson.build +1 -0
index 7e9c7bdd6..5d24e807f 100644
@@ -4,6 +4,7 @@
 # override default name to drop the hyphen
 name = 'testpmd'
 cflags += '-Wno-deprecated-declarations'
+cflags += '-fno-strict-aliasing'
 sources = files('5tswap.c',
 	'cmdline.c',
 	'cmdline_flow.c',
Signature · 1 line
2.17.1
#7 lore raw
Quoted 16 lines · George Prekas

Strict-aliasing rules are violated by cast to uint16_t* in flowgen.c and the calculated IP checksum is wrong on GCC 9 and GCC 10.

---
v2:
* Instead of a compiler barrier, use a compiler flag.

---
  app/test-pmd/meson.build | 1 +
  1 file changed, 1 insertion(+)
app/test-pmd/meson.build +0 -0
index 7e9c7bdd6..5d24e807f 100644

app/test-pmd/meson.build line 10

Quoted 7 lines
app/test-pmd/meson.build +1 -0
@@ -4,6 +4,7 @@
  # override default name to drop the hyphen
  name = 'testpmd'
  cflags += '-Wno-deprecated-declarations'
+cflags += '-fno-strict-aliasing'
  sources = files('5tswap.c',
  	'cmdline.c',
  	'cmdline_flow.c',

Happy New Year!

Any updates on this?

Thanks,
George

#8 lore raw
In reply to George Prekas
Quoted 17 lines · George Prekas

Strict-aliasing rules are violated by cast to uint16_t* in flowgen.c and the calculated IP checksum is wrong on GCC 9 and GCC 10.

---
v2:
* Instead of a compiler barrier, use a compiler flag.

---
  app/test-pmd/meson.build | 1 +
  1 file changed, 1 insertion(+)
app/test-pmd/meson.build +0 -0
index 7e9c7bdd6..5d24e807f 100644

app/test-pmd/meson.build line 10

Quoted 7 lines
app/test-pmd/meson.build +1 -0
@@ -4,6 +4,7 @@
  # override default name to drop the hyphen
  name = 'testpmd'
  cflags += '-Wno-deprecated-declarations'
+cflags += '-fno-strict-aliasing'
  sources = files('5tswap.c',
  	'cmdline.c',
  	'cmdline_flow.c',

Hi George,

I am trying to understand this, the relevant code is as below:
ip_hdr->hdr_checksum = ip_sum((unaligned_uint16_t *)ip_hdr, sizeof(*ip_hdr));

You are suspicious of strict aliasing rule violation, with more details:
The concern is the "struct rte_ipv4_hdr *ip_hdr;" aliased to "const unaligned_uint16_t *hdr", and compiler can optimize out the calculations using data pointed by 'hdr' pointer, since the 'hdr' pointer is not used to alter the data and compiler may think data is not changed at all.

1) But the pointer "hdr" is assigned in the loop, from another pointer whose content is changing, why this is not helping to figure out that the data 'hdr' pointing is changed.

2) I tried to debug this, but I am not able to reproduce the issue, 'ip_sum()' called each time and checksum calculated correctly. Using gcc 10.2.1-9. Can you able to confirm the case with debug, or from the assembly/object file?

And if the issue is strict aliasing rule violation as you said, compiler flag is an option but not sure how much it reduces the compiler optimization benefit, I guess other options also not so good, memcpy brings too much work on runtime and union requires bigger change and makes code complex. I wonder if making 'ip_sum()' a non inline function can help, can you please give a try since you can reproduce it?

#9 lore raw
Quoted 46 lines · On Wed, 6 Jan 2021 18:02:49 +0000 Ferruh Yigit <[email protected]> wrote:
Quoted 17 lines · George Prekas

Strict-aliasing rules are violated by cast to uint16_t* in flowgen.c and the calculated IP checksum is wrong on GCC 9 and GCC 10.

---
v2:
* Instead of a compiler barrier, use a compiler flag.

---
  app/test-pmd/meson.build | 1 +
  1 file changed, 1 insertion(+)
app/test-pmd/meson.build +0 -0
index 7e9c7bdd6..5d24e807f 100644

app/test-pmd/meson.build line 10

Quoted 7 lines
app/test-pmd/meson.build +1 -0
@@ -4,6 +4,7 @@
  # override default name to drop the hyphen
  name = 'testpmd'
  cflags += '-Wno-deprecated-declarations'
+cflags += '-fno-strict-aliasing'
  sources = files('5tswap.c',
  	'cmdline.c',
  	'cmdline_flow.c',

Hi George,

I am trying to understand this, the relevant code is as below:
ip_hdr->hdr_checksum = ip_sum((unaligned_uint16_t *)ip_hdr, sizeof(*ip_hdr));

You are suspicious of strict aliasing rule violation, with more details:
The concern is the "struct rte_ipv4_hdr *ip_hdr;" aliased to "const unaligned_uint16_t *hdr", and compiler can optimize out the calculations using data pointed by 'hdr' pointer, since the 'hdr' pointer is not used to alter the data and compiler may think data is not changed at all.

1) But the pointer "hdr" is assigned in the loop, from another pointer whose content is changing, why this is not helping to figure out that the data 'hdr' pointing is changed.

2) I tried to debug this, but I am not able to reproduce the issue, 'ip_sum()' called each time and checksum calculated correctly. Using gcc 10.2.1-9. Can you able to confirm the case with debug, or from the assembly/object file?

Quoted 6 lines

And if the issue is strict aliasing rule violation as you said, compiler flag is an option but not sure how much it reduces the compiler optimization benefit, I guess other options also not so good, memcpy brings too much work on runtime and union requires bigger change and makes code complex.
I wonder if making 'ip_sum()' a non inline function can help, can you please give a try since you can reproduce it?

If it is an aliasing problem, it should be fixed with a union instead of a compiler flag.

#10 lore raw
In reply to Ferruh Yigit
Quoted 46 lines · Ferruh Yigit
Quoted 17 lines · George Prekas

Strict-aliasing rules are violated by cast to uint16_t* in flowgen.c and the calculated IP checksum is wrong on GCC 9 and GCC 10.

---
v2:
* Instead of a compiler barrier, use a compiler flag.
---
  app/test-pmd/meson.build | 1 +

  1 file changed, 1 insertion(+)
app/test-pmd/meson.build +0 -0
index 7e9c7bdd6..5d24e807f 100644
@@ -4,6 +4,7 @@
   # override default name to drop the hyphen
Quoted 7 lines

  name = 'testpmd'
  cflags += '-Wno-deprecated-declarations'
+cflags += '-fno-strict-aliasing'
  sources = files('5tswap.c',
      'cmdline.c',
      'cmdline_flow.c',

Hi George,

I am trying to understand this, the relevant code is as below:
ip_hdr->hdr_checksum = ip_sum((unaligned_uint16_t *)ip_hdr, sizeof(*ip_hdr));

You are suspicious of strict aliasing rule violation, with more details:
The concern is the "struct rte_ipv4_hdr *ip_hdr;" aliased to "const unaligned_uint16_t *hdr", and compiler can optimize out the calculations using data pointed by 'hdr' pointer, since the 'hdr' pointer is not used to alter the data and compiler may think data is not changed at all.

1) But the pointer "hdr" is assigned in the loop, from another pointer whose content is changing, why this is not helping to figure out that the data 'hdr' pointing is changed.

2) I tried to debug this, but I am not able to reproduce the issue, 'ip_sum()' called each time and checksum calculated correctly. Using gcc 10.2.1-9. Can you able to confirm the case with debug, or from the assembly/object file?

Quoted 6 lines

And if the issue is strict aliasing rule violation as you said, compiler flag is an option but not sure how much it reduces the compiler optimization benefit, I guess other options also not so good, memcpy brings too much work on runtime and union requires bigger change and makes code complex.
I wonder if making 'ip_sum()' a non inline function can help, can you please give a try since you can reproduce it?

Hi Ferruh,

Thanks for looking into it.

I am copy-pasting at the end of this email a minimal reproduction. It calculates a checksum and prints it. The correct value is f8d9. If you compile it with -O0 or -O3 -fno-strict-aliasing, you will get the correct value. If you compile it with gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0 and -O3, you will get f8e8. You can also try it on https://godbolt.org/ and see how different versions behave.

My understanding is that the code violates the C standard (https://stackoverflow.com/a/99010).

--- cut here ---

#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

struct rte_ipv4_hdr {
	uint8_t  version_ihl;
	uint8_t  type_of_service;
	uint16_t total_length;
	uint16_t packet_id;
	uint16_t fragment_offset;
	uint8_t  time_to_live;
	uint8_t  next_proto_id;
	uint16_t hdr_checksum;
	uint32_t src_addr;
	uint32_t dst_addr;
};

static inline uint16_t ip_sum(const uint16_t *hdr, int hdr_len)

{
	uint32_t sum = 0;

	while (hdr_len > 1)
	{
		sum += *hdr++;
		if (sum & 0x80000000)
			sum = (sum & 0xFFFF) + (sum >> 16);
		hdr_len -= 2;
	}

	while (sum >> 16)
		sum = (sum & 0xFFFF) + (sum >> 16);

	return ~sum;
}

static void pkt_burst_flow_gen(void)

{
	struct rte_ipv4_hdr *ip_hdr = (struct rte_ipv4_hdr *) malloc(4096);
	memset(ip_hdr, 0, sizeof(*ip_hdr));
	ip_hdr->version_ihl	= 1;
	ip_hdr->type_of_service	= 2;
	ip_hdr->fragment_offset	= 3;
	ip_hdr->time_to_live	= 4;
	ip_hdr->next_proto_id	= 5;
	ip_hdr->packet_id	= 6;
	ip_hdr->src_addr	= 7;
	ip_hdr->dst_addr	= 8;
	ip_hdr->total_length	= 9;
	ip_hdr->hdr_checksum	= ip_sum((uint16_t *)ip_hdr, sizeof(*ip_hdr));
	printf("%x\n", ip_hdr->hdr_checksum);
}

int main(void)

{
	pkt_burst_flow_gen();
	return 0;
}
#11 lore raw
Quoted 56 lines · George Prekas
Quoted 46 lines · Ferruh Yigit
Quoted 17 lines · George Prekas

Strict-aliasing rules are violated by cast to uint16_t* in flowgen.c and the calculated IP checksum is wrong on GCC 9 and GCC 10.

---
v2:
* Instead of a compiler barrier, use a compiler flag.

---
   app/test-pmd/meson.build | 1 +
   1 file changed, 1 insertion(+)
app/test-pmd/meson.build +0 -0
index 7e9c7bdd6..5d24e807f 100644

app/test-pmd/meson.build line 10

Quoted 7 lines
app/test-pmd/meson.build +1 -0
@@ -4,6 +4,7 @@
   # override default name to drop the hyphen
   name = 'testpmd'
   cflags += '-Wno-deprecated-declarations'
+cflags += '-fno-strict-aliasing'
   sources = files('5tswap.c',
       'cmdline.c',
       'cmdline_flow.c',

Hi George,

I am trying to understand this, the relevant code is as below:
ip_hdr->hdr_checksum = ip_sum((unaligned_uint16_t *)ip_hdr, sizeof(*ip_hdr));

You are suspicious of strict aliasing rule violation, with more details:
The concern is the "struct rte_ipv4_hdr *ip_hdr;" aliased to "const unaligned_uint16_t *hdr", and compiler can optimize out the calculations using data pointed by 'hdr' pointer, since the 'hdr' pointer is not used to alter the data and compiler may think data is not changed at all.

1) But the pointer "hdr" is assigned in the loop, from another pointer whose content is changing, why this is not helping to figure out that the data 'hdr' pointing is changed.

2) I tried to debug this, but I am not able to reproduce the issue, 'ip_sum()' called each time and checksum calculated correctly. Using gcc 10.2.1-9. Can you able to confirm the case with debug, or from the assembly/object file?

Quoted 6 lines

And if the issue is strict aliasing rule violation as you said, compiler flag is an option but not sure how much it reduces the compiler optimization benefit, I guess other options also not so good, memcpy brings too much work on runtime and union requires bigger change and makes code complex.
I wonder if making 'ip_sum()' a non inline function can help, can you please give a try since you can reproduce it?

Quoted 8 lines

Hi Ferruh,

Thanks for looking into it.

I am copy-pasting at the end of this email a minimal reproduction. It calculates a checksum and prints it. The correct value is f8d9. If you compile it with -O0 or -O3 -fno-strict-aliasing, you will get the correct value. If you compile it with gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0 and -O3, you will get f8e8. You can also try it on https://godbolt.org/ and see how different versions behave.

My understanding is that the code violates the C standard (https://stackoverflow.com/a/99010).

Thanks for the sample code below, I copied to the godbolt:
https://godbolt.org/z/6fMK19

In gcc 10, the checksum calculation is done during compilation (when optimization is enabled) and the value is returned directly:
mov $0xffed,%esi

Since a calculation is happening I assume the compiler knows about the aliasing and OK with it.

But that optimized calculation seems wrong, when it is disabled [1] the checksum is correct again.

[1] all following seems helping to disable compile time calculation
- disabling optimization
- putting a compiler barrier
- putting a 'printf' inside 'ip_sum()'
- fno-strict-aliasing

gcc 8 & 9 is not doing this compile time calculation, hence they are not affected.

This feels like an optimization issue in gcc10, but not sure exactly on the root cause, and how to disable it properly in our case.

Quoted 61 lines

--- cut here ---

#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

struct rte_ipv4_hdr {
	uint8_t  version_ihl;
	uint8_t  type_of_service;
	uint16_t total_length;
	uint16_t packet_id;
	uint16_t fragment_offset;
	uint8_t  time_to_live;
	uint8_t  next_proto_id;
	uint16_t hdr_checksum;
	uint32_t src_addr;
	uint32_t dst_addr;
};

static inline uint16_t ip_sum(const uint16_t *hdr, int hdr_len)

{
	uint32_t sum = 0;

	while (hdr_len > 1)
	{
		sum += *hdr++;
		if (sum & 0x80000000)
			sum = (sum & 0xFFFF) + (sum >> 16);
		hdr_len -= 2;
	}

	while (sum >> 16)
		sum = (sum & 0xFFFF) + (sum >> 16);

	return ~sum;
}

static void pkt_burst_flow_gen(void)

{
	struct rte_ipv4_hdr *ip_hdr = (struct rte_ipv4_hdr *) malloc(4096);
	memset(ip_hdr, 0, sizeof(*ip_hdr));
	ip_hdr->version_ihl	= 1;
	ip_hdr->type_of_service	= 2;
	ip_hdr->fragment_offset	= 3;
	ip_hdr->time_to_live	= 4;
	ip_hdr->next_proto_id	= 5;
	ip_hdr->packet_id	= 6;
	ip_hdr->src_addr	= 7;
	ip_hdr->dst_addr	= 8;
	ip_hdr->total_length	= 9;
	ip_hdr->hdr_checksum	= ip_sum((uint16_t *)ip_hdr, sizeof(*ip_hdr));
	printf("%x\n", ip_hdr->hdr_checksum);
}

int main(void)

{
	pkt_burst_flow_gen();
	return 0;
}
#12 lore raw
Quoted 82 lines · Ferruh Yigit
Quoted 56 lines · George Prekas
Quoted 46 lines · Ferruh Yigit
Quoted 17 lines · George Prekas

Strict-aliasing rules are violated by cast to uint16_t* in flowgen.c and the calculated IP checksum is wrong on GCC 9 and GCC 10.

---
v2:
* Instead of a compiler barrier, use a compiler flag.
---
   app/test-pmd/meson.build | 1 +

   1 file changed, 1 insertion(+)
app/test-pmd/meson.build +0 -0
index 7e9c7bdd6..5d24e807f 100644
@@ -4,6 +4,7 @@
    # override default name to drop the hyphen
Quoted 7 lines

   name = 'testpmd'
   cflags += '-Wno-deprecated-declarations'
+cflags += '-fno-strict-aliasing'
   sources = files('5tswap.c',
       'cmdline.c',
       'cmdline_flow.c',

Hi George,

I am trying to understand this, the relevant code is as below:
ip_hdr->hdr_checksum = ip_sum((unaligned_uint16_t *)ip_hdr, sizeof(*ip_hdr));

You are suspicious of strict aliasing rule violation, with more details:
The concern is the "struct rte_ipv4_hdr *ip_hdr;" aliased to "const unaligned_uint16_t *hdr", and compiler can optimize out the calculations using data pointed by 'hdr' pointer, since the 'hdr' pointer is not used to alter the data and compiler may think data is not changed at all.

1) But the pointer "hdr" is assigned in the loop, from another pointer whose content is changing, why this is not helping to figure out that the data 'hdr' pointing is changed.

2) I tried to debug this, but I am not able to reproduce the issue, 'ip_sum()' called each time and checksum calculated correctly. Using gcc 10.2.1-9. Can you able to confirm the case with debug, or from the assembly/object file?

Quoted 6 lines

And if the issue is strict aliasing rule violation as you said, compiler flag is an option but not sure how much it reduces the compiler optimization benefit, I guess other options also not so good, memcpy brings too much work on runtime and union requires bigger change and makes code complex.
I wonder if making 'ip_sum()' a non inline function can help, can you please give a try since you can reproduce it?

Quoted 8 lines

Hi Ferruh,

Thanks for looking into it.

I am copy-pasting at the end of this email a minimal reproduction. It calculates a checksum and prints it. The correct value is f8d9. If you compile it with -O0 or -O3 -fno-strict-aliasing, you will get the correct value. If you compile it with gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0 and -O3, you will get f8e8. You can also try it on https://godbolt.org/ and see how different versions behave.

My understanding is that the code violates the C standard (https://stackoverflow.com/a/99010).

Thanks for the sample code below, I copied to the godbolt:
https://godbolt.org/z/6fMK19

In gcc 10, the checksum calculation is done during compilation (when optimization is enabled) and the value is returned directly:
mov    $0xffed,%esi

Since a calculation is happening I assume the compiler knows about the aliasing and OK with it.

But that optimized calculation seems wrong, when it is disabled [1] the checksum is correct again.

[1] all following seems helping to disable compile time calculation
- disabling optimization
- putting a compiler barrier
- putting a 'printf' inside 'ip_sum()'
- fno-strict-aliasing

Quoted 4 lines

gcc 8 & 9 is not doing this compile time calculation, hence they are not affected.

This feels like an optimization issue in gcc10, but not sure exactly on the root cause, and how to disable it properly in our case.

As checked with the Harry, latest finding is gcc 10 left out any _non_ uint16_t type variable in sturct during its compile time calculation. Not sure if it is because of broken aliasing or gcc defect, I will report the issue.

Meanwhile for short time solution, can you please try force uninline the 'ip_sum()' and try?

Quoted 62 lines
Quoted 61 lines

--- cut here ---

#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

struct rte_ipv4_hdr {
    uint8_t  version_ihl;
    uint8_t  type_of_service;
    uint16_t total_length;
    uint16_t packet_id;
    uint16_t fragment_offset;
    uint8_t  time_to_live;
    uint8_t  next_proto_id;
    uint16_t hdr_checksum;
    uint32_t src_addr;
    uint32_t dst_addr;
};

static inline uint16_t ip_sum(const uint16_t *hdr, int hdr_len) {
    uint32_t sum = 0;

    while (hdr_len > 1)
    {
        sum += *hdr++;
        if (sum & 0x80000000)
            sum = (sum & 0xFFFF) + (sum >> 16);
        hdr_len -= 2;
    }

    while (sum >> 16)
        sum = (sum & 0xFFFF) + (sum >> 16);

    return ~sum;
}

static void pkt_burst_flow_gen(void)
{
    struct rte_ipv4_hdr *ip_hdr = (struct rte_ipv4_hdr *) malloc(4096); memset(ip_hdr, 0, sizeof(*ip_hdr));
    ip_hdr->version_ihl    = 1;
    ip_hdr->type_of_service    = 2;
    ip_hdr->fragment_offset    = 3;
    ip_hdr->time_to_live    = 4;
    ip_hdr->next_proto_id    = 5;
    ip_hdr->packet_id    = 6;
    ip_hdr->src_addr    = 7;
    ip_hdr->dst_addr    = 8;
    ip_hdr->total_length    = 9;
    ip_hdr->hdr_checksum    = ip_sum((uint16_t *)ip_hdr, sizeof(*ip_hdr)); printf("%x\n", ip_hdr->hdr_checksum);
}

int main(void)
{
    pkt_burst_flow_gen();
    return 0;
}

#13 lore raw
In reply to Ferruh Yigit
Quoted 67 lines · Ferruh Yigit
Quoted 54 lines · George Prekas
Quoted 46 lines · Ferruh Yigit
Quoted 17 lines · George Prekas

Strict-aliasing rules are violated by cast to uint16_t* in flowgen.c and the calculated IP checksum is wrong on GCC 9 and GCC 10.

---
v2:
* Instead of a compiler barrier, use a compiler flag.
---
   app/test-pmd/meson.build | 1 +

   1 file changed, 1 insertion(+)
app/test-pmd/meson.build +0 -0
index 7e9c7bdd6..5d24e807f 100644
@@ -4,6 +4,7 @@
    # override default name to drop the hyphen
Quoted 7 lines

   name = 'testpmd'
   cflags += '-Wno-deprecated-declarations'
+cflags += '-fno-strict-aliasing'
   sources = files('5tswap.c',
       'cmdline.c',
       'cmdline_flow.c',

Hi George,

I am trying to understand this, the relevant code is as below:
ip_hdr->hdr_checksum = ip_sum((unaligned_uint16_t *)ip_hdr, sizeof(*ip_hdr));

You are suspicious of strict aliasing rule violation, with more details:
The concern is the "struct rte_ipv4_hdr *ip_hdr;" aliased to "const unaligned_uint16_t *hdr", and compiler can optimize out the calculations using data pointed by 'hdr' pointer, since the 'hdr' pointer is not used to alter the data and compiler may think data is not changed at all.

1) But the pointer "hdr" is assigned in the loop, from another pointer whose content is changing, why this is not helping to figure out that the data 'hdr' pointing is changed.

2) I tried to debug this, but I am not able to reproduce the issue, 'ip_sum()' called each time and checksum calculated correctly. Using gcc 10.2.1-9. Can you able to confirm the case with debug, or from the assembly/object file?

Quoted 6 lines

And if the issue is strict aliasing rule violation as you said, compiler flag is an option but not sure how much it reduces the compiler optimization benefit, I guess other options also not so good, memcpy brings too much work on runtime and union requires bigger change and makes code complex.
I wonder if making 'ip_sum()' a non inline function can help, can you please give a try since you can reproduce it?

Quoted 8 lines

Hi Ferruh,

Thanks for looking into it.

I am copy-pasting at the end of this email a minimal reproduction. It calculates a checksum and prints it. The correct value is f8d9. If you compile it with -O0 or -O3 -fno-strict-aliasing, you will get the correct value. If you compile it with gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0 and -O3, you will get f8e8. You can also try it on https://godbolt.org/ and see how different versions behave.

My understanding is that the code violates the C standard (https://stackoverflow.com/a/99010).

Thanks for the sample code below, I copied to the godbolt:
https://godbolt.org/z/6fMK19

Quoted 6 lines

In gcc 10, the checksum calculation is done during compilation (when optimization is enabled) and the value is returned directly:
mov    $0xffed,%esi

Since a calculation is happening I assume the compiler knows about the aliasing and OK with it.

According to https://gcc.gnu.org/bugs/: "if compiling with -fno-strict-aliasing -fwrapv -fno-aggressive-loop-optimizations makes a difference ... then your code is probably not correct"

Quoted 11 lines

But that optimized calculation seems wrong, when it is disabled [1] the checksum is correct again.

[1] all following seems helping to disable compile time calculation
- disabling optimization
- putting a compiler barrier
- putting a 'printf' inside 'ip_sum()'
- fno-strict-aliasing

gcc 8 & 9 is not doing this compile time calculation, hence they are not affected.

I just checked gcc 8.3 and gcc 9.3 on godbolt and I got f8e8 (which is wrong; the correct is f8d9).

Quoted 3 lines

This feels like an optimization issue in gcc10, but not sure exactly on the root cause, and how to disable it properly in our case.

I've tried with __attribute__ ((noinline)) and it fixes the problem. But keep in mind that we are dealing with broken C code. This attribute just prevents the optimization that reveals the problem. It does not guarantee that the problem will not reappear in a future compiler version.

I've also tried to use a union as suggested by Stephen Hemminger and it works correctly but it requires significant code changes: you have to copy paste the IP header structure inside a union and access it only through the union.

As a side note, here is a piece of opinion from Linus Torvalds regarding strict aliasing:
https://lkml.org/lkml/2018/6/5/769

DPDK already uses -fno-strict-aliasing for librte_node and librte_vhost.

Quoted 63 lines
Quoted 62 lines

--- cut here ---

#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

struct rte_ipv4_hdr {
      uint8_t  version_ihl;
      uint8_t  type_of_service;
      uint16_t total_length;
      uint16_t packet_id;
      uint16_t fragment_offset;
      uint8_t  time_to_live;
      uint8_t  next_proto_id;
      uint16_t hdr_checksum;
      uint32_t src_addr;
      uint32_t dst_addr;
};

static inline uint16_t ip_sum(const uint16_t *hdr, int hdr_len) {
      uint32_t sum = 0;

      while (hdr_len > 1)
      {
              sum += *hdr++;
              if (sum & 0x80000000)
                      sum = (sum & 0xFFFF) + (sum >> 16); hdr_len -= 2;
      }

      while (sum >> 16)
              sum = (sum & 0xFFFF) + (sum >> 16);

      return ~sum;
}

static void pkt_burst_flow_gen(void)
{
      struct rte_ipv4_hdr *ip_hdr = (struct rte_ipv4_hdr *) malloc(4096); memset(ip_hdr, 0, sizeof(*ip_hdr));
      ip_hdr->version_ihl     = 1;
      ip_hdr->type_of_service = 2;
      ip_hdr->fragment_offset = 3;
      ip_hdr->time_to_live    = 4;
      ip_hdr->next_proto_id   = 5;
      ip_hdr->packet_id       = 6;
      ip_hdr->src_addr        = 7;
      ip_hdr->dst_addr        = 8;
      ip_hdr->total_length    = 9;
      ip_hdr->hdr_checksum    = ip_sum((uint16_t *)ip_hdr, sizeof(*ip_hdr)); printf("%x\n", ip_hdr->hdr_checksum);
}

int main(void)
{
      pkt_burst_flow_gen();
      return 0;
}

#14 lore raw
Quoted 72 lines · George Prekas
Quoted 71 lines · Ferruh Yigit
Quoted 54 lines · George Prekas
Quoted 46 lines · Ferruh Yigit
Quoted 17 lines · George Prekas

Strict-aliasing rules are violated by cast to uint16_t* in flowgen.c and the calculated IP checksum is wrong on GCC 9 and GCC 10.

---
v2:
* Instead of a compiler barrier, use a compiler flag.

---
    app/test-pmd/meson.build | 1 +
    1 file changed, 1 insertion(+)
app/test-pmd/meson.build +0 -0
index 7e9c7bdd6..5d24e807f 100644

app/test-pmd/meson.build line 10

Quoted 7 lines
app/test-pmd/meson.build +1 -0
@@ -4,6 +4,7 @@
    # override default name to drop the hyphen
    name = 'testpmd'
    cflags += '-Wno-deprecated-declarations'
+cflags += '-fno-strict-aliasing'
    sources = files('5tswap.c',
        'cmdline.c',
        'cmdline_flow.c',

Hi George,

I am trying to understand this, the relevant code is as below:
ip_hdr->hdr_checksum = ip_sum((unaligned_uint16_t *)ip_hdr, sizeof(*ip_hdr));

You are suspicious of strict aliasing rule violation, with more details:
The concern is the "struct rte_ipv4_hdr *ip_hdr;" aliased to "const unaligned_uint16_t *hdr", and compiler can optimize out the calculations using data pointed by 'hdr' pointer, since the 'hdr' pointer is not used to alter the data and compiler may think data is not changed at all.

1) But the pointer "hdr" is assigned in the loop, from another pointer whose content is changing, why this is not helping to figure out that the data 'hdr' pointing is changed.

2) I tried to debug this, but I am not able to reproduce the issue, 'ip_sum()' called each time and checksum calculated correctly. Using gcc 10.2.1-9. Can you able to confirm the case with debug, or from the assembly/object file?

Quoted 6 lines

And if the issue is strict aliasing rule violation as you said, compiler flag is an option but not sure how much it reduces the compiler optimization benefit, I guess other options also not so good, memcpy brings too much work on runtime and union requires bigger change and makes code complex.
I wonder if making 'ip_sum()' a non inline function can help, can you please give a try since you can reproduce it?

Quoted 8 lines

Hi Ferruh,

Thanks for looking into it.

I am copy-pasting at the end of this email a minimal reproduction. It calculates a checksum and prints it. The correct value is f8d9. If you compile it with -O0 or -O3 -fno-strict-aliasing, you will get the correct value. If you compile it with gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0 and -O3, you will get f8e8. You can also try it on https://godbolt.org/ and see how different versions behave.

My understanding is that the code violates the C standard (https://stackoverflow.com/a/99010).

Thanks for the sample code below, I copied to the godbolt:
https://godbolt.org/z/6fMK19

In gcc 10, the checksum calculation is done during compilation (when optimization is enabled) and the value is returned directly:
mov    $0xffed,%esi

Quoted 7 lines
Quoted 2 lines

Since a calculation is happening I assume the compiler knows about the aliasing and OK with it.

According to https://gcc.gnu.org/bugs/: "if compiling with -fno-strict-aliasing -fwrapv -fno-aggressive-loop-optimizations makes a difference ... then your code is probably not correct"

Yep, I saw it while submitting the gcc ticket, and it seems it was right:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98582

Quoted 10 lines
Quoted 10 lines

But that optimized calculation seems wrong, when it is disabled [1] the checksum is correct again.

[1] all following seems helping to disable compile time calculation
- disabling optimization
- putting a compiler barrier
- putting a 'printf' inside 'ip_sum()'
- fno-strict-aliasing

Quoted 5 lines
Quoted 1 line

gcc 8 & 9 is not doing this compile time calculation, hence they are not affected.

I just checked gcc 8.3 and gcc 9.3 on godbolt and I got f8e8 (which is wrong; the correct is f8d9).

True, I missed that they generate wrong value.

Quoted 9 lines
Quoted 3 lines

This feels like an optimization issue in gcc10, but not sure exactly on the root cause, and how to disable it properly in our case.

I've tried with __attribute__ ((noinline)) and it fixes the problem. But keep in mind that we are dealing with broken C code. This attribute just prevents the optimization that reveals the problem. It does not guarantee that the problem will not reappear in a future compiler version.

Quoted 8 lines

I've also tried to use a union as suggested by Stephen Hemminger and it works correctly but it requires significant code changes: you have to copy paste the IP header structure inside a union and access it only through the union.

As a side note, here is a piece of opinion from Linus Torvalds regarding strict aliasing:
https://lkml.org/lkml/2018/6/5/769

DPDK already uses -fno-strict-aliasing for librte_node and librte_vhost.

In the above ticket, 'may_alias' attribute is also suggested, which is working for the sample, can you please try with it too? It may be better to allow non compatible aliasing only for single function, instead of whole binary.

typedef uint16_t alias_int16_t __attribute__((may_alias));

Quoted 64 lines
Quoted 64 lines
Quoted 63 lines

--- cut here ---

#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

struct rte_ipv4_hdr {
      uint8_t  version_ihl;
      uint8_t  type_of_service;
      uint16_t total_length;
      uint16_t packet_id;
      uint16_t fragment_offset;
      uint8_t  time_to_live;
      uint8_t  next_proto_id;
      uint16_t hdr_checksum;
      uint32_t src_addr;
      uint32_t dst_addr;
};

static inline uint16_t ip_sum(const uint16_t *hdr, int hdr_len) {
      uint32_t sum = 0;

      while (hdr_len > 1)
      {
              sum += *hdr++;
              if (sum & 0x80000000)
                      sum = (sum & 0xFFFF) + (sum >> 16); hdr_len -= 2;
      }

      while (sum >> 16)
              sum = (sum & 0xFFFF) + (sum >> 16);

      return ~sum;
}

static void pkt_burst_flow_gen(void)
{
      struct rte_ipv4_hdr *ip_hdr = (struct rte_ipv4_hdr *) malloc(4096); memset(ip_hdr, 0, sizeof(*ip_hdr));
      ip_hdr->version_ihl     = 1;
      ip_hdr->type_of_service = 2;
      ip_hdr->fragment_offset = 3;
      ip_hdr->time_to_live    = 4;
      ip_hdr->next_proto_id   = 5;
      ip_hdr->packet_id       = 6;
      ip_hdr->src_addr        = 7;
      ip_hdr->dst_addr        = 8;
      ip_hdr->total_length    = 9;
      ip_hdr->hdr_checksum    = ip_sum((uint16_t *)ip_hdr, sizeof(*ip_hdr)); printf("%x\n", ip_hdr->hdr_checksum);
}

int main(void)
{
      pkt_burst_flow_gen();
      return 0;
}

#15 lore raw
Quoted 118 lines · Ferruh Yigit
Quoted 72 lines · George Prekas
Quoted 71 lines · Ferruh Yigit
Quoted 54 lines · George Prekas
Quoted 46 lines · Ferruh Yigit
Quoted 17 lines · George Prekas

Strict-aliasing rules are violated by cast to uint16_t* in flowgen.c and the calculated IP checksum is wrong on GCC 9 and GCC 10.

---
v2:
* Instead of a compiler barrier, use a compiler flag.
---
    app/test-pmd/meson.build | 1 +

    1 file changed, 1 insertion(+)
app/test-pmd/meson.build +0 -0
index 7e9c7bdd6..5d24e807f 100644
@@ -4,6 +4,7 @@
     # override default name to drop the hyphen
Quoted 7 lines

    name = 'testpmd'
    cflags += '-Wno-deprecated-declarations'
+cflags += '-fno-strict-aliasing'
    sources = files('5tswap.c',
        'cmdline.c',
        'cmdline_flow.c',

Hi George,

I am trying to understand this, the relevant code is as below:
ip_hdr->hdr_checksum = ip_sum((unaligned_uint16_t *)ip_hdr, sizeof(*ip_hdr));

You are suspicious of strict aliasing rule violation, with more details:
The concern is the "struct rte_ipv4_hdr *ip_hdr;" aliased to "const unaligned_uint16_t *hdr", and compiler can optimize out the calculations using data pointed by 'hdr' pointer, since the 'hdr' pointer is not used to alter the data and compiler may think data is not changed at all.

1) But the pointer "hdr" is assigned in the loop, from another pointer whose content is changing, why this is not helping to figure out that the data 'hdr' pointing is changed.

2) I tried to debug this, but I am not able to reproduce the issue, 'ip_sum()' called each time and checksum calculated correctly. Using gcc 10.2.1-9. Can you able to confirm the case with debug, or from the assembly/object file?

Quoted 6 lines

And if the issue is strict aliasing rule violation as you said, compiler flag is an option but not sure how much it reduces the compiler optimization benefit, I guess other options also not so good, memcpy brings too much work on runtime and union requires bigger change and makes code complex.
I wonder if making 'ip_sum()' a non inline function can help, can you please give a try since you can reproduce it?

Quoted 8 lines

Hi Ferruh,

Thanks for looking into it.

I am copy-pasting at the end of this email a minimal reproduction. It calculates a checksum and prints it. The correct value is f8d9. If you compile it with -O0 or -O3 -fno-strict-aliasing, you will get the correct value. If you compile it with gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0 and -O3, you will get f8e8. You can also try it on https://godbolt.org/ and see how different versions behave.

My understanding is that the code violates the C standard (https://stackoverflow.com/a/99010).

Thanks for the sample code below, I copied to the godbolt:
https://godbolt.org/z/6fMK19

In gcc 10, the checksum calculation is done during compilation (when optimization is enabled) and the value is returned directly:
mov    $0xffed,%esi

Quoted 7 lines
Quoted 2 lines

Since a calculation is happening I assume the compiler knows about the aliasing and OK with it.

According to https://gcc.gnu.org/bugs/: "if compiling with -fno-strict-aliasing -fwrapv -fno-aggressive-loop-optimizations makes a difference ... then your code is probably not correct"

Yep, I saw it while submitting the gcc ticket, and it seems it was right:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98582

Quoted 10 lines
Quoted 10 lines

But that optimized calculation seems wrong, when it is disabled [1] the checksum is correct again.

[1] all following seems helping to disable compile time calculation
- disabling optimization
- putting a compiler barrier
- putting a 'printf' inside 'ip_sum()'
- fno-strict-aliasing

Quoted 5 lines
Quoted 1 line

gcc 8 & 9 is not doing this compile time calculation, hence they are not affected.

I just checked gcc 8.3 and gcc 9.3 on godbolt and I got f8e8 (which is wrong; the correct is f8d9).

True, I missed that they generate wrong value.

Quoted 16 lines
Quoted 3 lines

This feels like an optimization issue in gcc10, but not sure exactly on the root cause, and how to disable it properly in our case.

I've tried with __attribute__ ((noinline)) and it fixes the problem. But keep in mind that we are dealing with broken C code. This attribute just prevents the optimization that reveals the problem. It does not guarantee that the problem will not reappear in a future compiler version.

I've also tried to use a union as suggested by Stephen Hemminger and it works correctly but it requires significant code changes: you have to copy paste the IP header structure inside a union and access it only through the union.

As a side note, here is a piece of opinion from Linus Torvalds regarding strict aliasing:
https://lkml.org/lkml/2018/6/5/769

Quoted 8 lines
Quoted 1 line

DPDK already uses -fno-strict-aliasing for librte_node and librte_vhost.

In the above ticket, 'may_alias' attribute is also suggested, which is working for the sample, can you please try with it too?
It may be better to allow non compatible aliasing only for single function, instead of whole binary.

typedef uint16_t alias_int16_t __attribute__((may_alias));

I've tested with may_alias and it works correctly between 2 AWS EC2 instances. may_alias is used in other places in DPDK as well. I've posted a new version of the patch.

Quoted 66 lines
Quoted 65 lines
Quoted 65 lines
Quoted 64 lines

--- cut here ---

#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

struct rte_ipv4_hdr {
       uint8_t  version_ihl;
       uint8_t  type_of_service;
       uint16_t total_length;
       uint16_t packet_id;
       uint16_t fragment_offset;
       uint8_t  time_to_live;
       uint8_t  next_proto_id;
       uint16_t hdr_checksum;
       uint32_t src_addr;
       uint32_t dst_addr;
};

static inline uint16_t ip_sum(const uint16_t *hdr, int hdr_len) {
       uint32_t sum = 0;

       while (hdr_len > 1)
       {
               sum += *hdr++;
               if (sum & 0x80000000)
                       sum = (sum & 0xFFFF) + (sum >> 16); hdr_len -= 2;
       }

       while (sum >> 16)
               sum = (sum & 0xFFFF) + (sum >> 16);

       return ~sum;
}

static void pkt_burst_flow_gen(void)
{
       struct rte_ipv4_hdr *ip_hdr = (struct rte_ipv4_hdr *) malloc(4096); memset(ip_hdr, 0, sizeof(*ip_hdr));
       ip_hdr->version_ihl     = 1;
       ip_hdr->type_of_service = 2;
       ip_hdr->fragment_offset = 3;
       ip_hdr->time_to_live    = 4;
       ip_hdr->next_proto_id   = 5;
       ip_hdr->packet_id       = 6;
       ip_hdr->src_addr        = 7;
       ip_hdr->dst_addr        = 8;
       ip_hdr->total_length    = 9;
       ip_hdr->hdr_checksum    = ip_sum((uint16_t *)ip_hdr, sizeof(*ip_hdr)); printf("%x\n", ip_hdr->hdr_checksum);
}

int main(void)
{
       pkt_burst_flow_gen();
       return 0;
}

#16 lore raw
In reply to George Prekas
Quoted 117 lines · On Wed, 6 Jan 2021 23:39:39 -0600 George Prekas <[email protected]> wrote:
Quoted 46 lines · Ferruh Yigit
Quoted 17 lines · George Prekas

Strict-aliasing rules are violated by cast to uint16_t* in flowgen.c and the calculated IP checksum is wrong on GCC 9 and GCC 10.

---
v2:
* Instead of a compiler barrier, use a compiler flag.
---
  app/test-pmd/meson.build | 1 +

  1 file changed, 1 insertion(+)
app/test-pmd/meson.build +0 -0
index 7e9c7bdd6..5d24e807f 100644
@@ -4,6 +4,7 @@
   # override default name to drop the hyphen
Quoted 7 lines

  name = 'testpmd'
  cflags += '-Wno-deprecated-declarations'
+cflags += '-fno-strict-aliasing'
  sources = files('5tswap.c',
      'cmdline.c',
      'cmdline_flow.c',

Hi George,

I am trying to understand this, the relevant code is as below:
ip_hdr->hdr_checksum = ip_sum((unaligned_uint16_t *)ip_hdr, sizeof(*ip_hdr));

You are suspicious of strict aliasing rule violation, with more details:
The concern is the "struct rte_ipv4_hdr *ip_hdr;" aliased to "const unaligned_uint16_t *hdr", and compiler can optimize out the calculations using data pointed by 'hdr' pointer, since the 'hdr' pointer is not used to alter the data and compiler may think data is not changed at all.

1) But the pointer "hdr" is assigned in the loop, from another pointer whose content is changing, why this is not helping to figure out that the data 'hdr' pointing is changed.

2) I tried to debug this, but I am not able to reproduce the issue, 'ip_sum()' called each time and checksum calculated correctly. Using gcc 10.2.1-9. Can you able to confirm the case with debug, or from the assembly/object file?

Quoted 6 lines

And if the issue is strict aliasing rule violation as you said, compiler flag is an option but not sure how much it reduces the compiler optimization benefit, I guess other options also not so good, memcpy brings too much work on runtime and union requires bigger change and makes code complex.
I wonder if making 'ip_sum()' a non inline function can help, can you please give a try since you can reproduce it?

Hi Ferruh,

Thanks for looking into it.

I am copy-pasting at the end of this email a minimal reproduction. It calculates a checksum and prints it. The correct value is f8d9. If you compile it with -O0 or -O3 -fno-strict-aliasing, you will get the correct value. If you compile it with gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0 and -O3, you will get f8e8. You can also try it on https://godbolt.org/ and see how different versions behave.

My understanding is that the code violates the C standard (https://stackoverflow.com/a/99010).

--- cut here ---

#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

struct rte_ipv4_hdr {
	uint8_t  version_ihl;
	uint8_t  type_of_service;
	uint16_t total_length;
	uint16_t packet_id;
	uint16_t fragment_offset;
	uint8_t  time_to_live;
	uint8_t  next_proto_id;
	uint16_t hdr_checksum;
	uint32_t src_addr;
	uint32_t dst_addr;
};

static inline uint16_t ip_sum(const uint16_t *hdr, int hdr_len)

{
	uint32_t sum = 0;

	while (hdr_len > 1)
	{
		sum += *hdr++;
		if (sum & 0x80000000)
			sum = (sum & 0xFFFF) + (sum >> 16);
		hdr_len -= 2;
	}

	while (sum >> 16)
		sum = (sum & 0xFFFF) + (sum >> 16);

	return ~sum;
}

static void pkt_burst_flow_gen(void)

{
	struct rte_ipv4_hdr *ip_hdr = (struct rte_ipv4_hdr *) malloc(4096);
	memset(ip_hdr, 0, sizeof(*ip_hdr));
	ip_hdr->version_ihl	= 1;
	ip_hdr->type_of_service	= 2;
	ip_hdr->fragment_offset	= 3;
	ip_hdr->time_to_live	= 4;
	ip_hdr->next_proto_id	= 5;
	ip_hdr->packet_id	= 6;
	ip_hdr->src_addr	= 7;
	ip_hdr->dst_addr	= 8;
	ip_hdr->total_length	= 9;
	ip_hdr->hdr_checksum	= ip_sum((uint16_t *)ip_hdr, sizeof(*ip_hdr));
	printf("%x\n", ip_hdr->hdr_checksum);
}
Quoted 5 lines

int main(void)

{
	pkt_burst_flow_gen();
	return 0;
}

If I change your code like this to use union, Gcc 10 is still broken. It is a compiler bug. It maybe because optimizer is not smart enough to know that memset has cleared the header.

#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

struct rte_ipv4_hdr {
	uint8_t  version_ihl;
	uint8_t  type_of_service;
	uint16_t total_length;
	uint16_t packet_id;
	uint16_t fragment_offset;
	uint8_t  time_to_live;
	uint8_t  next_proto_id;
	uint16_t hdr_checksum;
	uint32_t src_addr;
	uint32_t dst_addr;
};

static inline uint16_t ip_sum(const uint16_t *hdr, int hdr_len)

{
	uint32_t sum = 0;

	while (hdr_len > 1)
	{
		sum += *hdr++;
		if (sum & 0x80000000)
			sum = (sum & 0xFFFF) + (sum >> 16);
		hdr_len -= 2;
	}

	while (sum >> 16)
		sum = (sum & 0xFFFF) + (sum >> 16);

	return ~sum;
}

static void pkt_burst_flow_gen(void)

{
	union {
		struct rte_ipv4_hdr ip;
		uint16_t data[10];
	} *hdr;

	hdr = malloc(sizeof(*hdr));

	memset(hdr, 0, sizeof(*hdr));
	hdr->ip.version_ihl	= 1;
	hdr->ip.type_of_service	= 2;
	hdr->ip.fragment_offset	= 3;
	hdr->ip.time_to_live	= 4;
	hdr->ip.next_proto_id	= 5;
	hdr->ip.packet_id	= 6;
	hdr->ip.src_addr	= 7;
	hdr->ip.dst_addr	= 8;
	hdr->ip.total_length	= 9;
	hdr->ip.hdr_checksum	= ip_sum(hdr->data, sizeof(*hdr));
	printf("%x\n", hdr->ip.hdr_checksum);
}

int main(void)

{
	pkt_burst_flow_gen();
	return 0;
}
#17 lore raw
Quoted 120 lines · Stephen Hemminger
Quoted 117 lines · On Wed, 6 Jan 2021 23:39:39 -0600 George Prekas <[email protected]> wrote:
Quoted 46 lines · Ferruh Yigit
Quoted 17 lines · George Prekas

Strict-aliasing rules are violated by cast to uint16_t* in flowgen.c and the calculated IP checksum is wrong on GCC 9 and GCC 10.

---
v2:
* Instead of a compiler barrier, use a compiler flag.

---
   app/test-pmd/meson.build | 1 +
   1 file changed, 1 insertion(+)
app/test-pmd/meson.build +0 -0
index 7e9c7bdd6..5d24e807f 100644

app/test-pmd/meson.build line 10

Quoted 7 lines
app/test-pmd/meson.build +1 -0
@@ -4,6 +4,7 @@
   # override default name to drop the hyphen
   name = 'testpmd'
   cflags += '-Wno-deprecated-declarations'
+cflags += '-fno-strict-aliasing'
   sources = files('5tswap.c',
       'cmdline.c',
       'cmdline_flow.c',

Hi George,

I am trying to understand this, the relevant code is as below:
ip_hdr->hdr_checksum = ip_sum((unaligned_uint16_t *)ip_hdr, sizeof(*ip_hdr));

You are suspicious of strict aliasing rule violation, with more details:
The concern is the "struct rte_ipv4_hdr *ip_hdr;" aliased to "const unaligned_uint16_t *hdr", and compiler can optimize out the calculations using data pointed by 'hdr' pointer, since the 'hdr' pointer is not used to alter the data and compiler may think data is not changed at all.

1) But the pointer "hdr" is assigned in the loop, from another pointer whose content is changing, why this is not helping to figure out that the data 'hdr' pointing is changed.

2) I tried to debug this, but I am not able to reproduce the issue, 'ip_sum()' called each time and checksum calculated correctly. Using gcc 10.2.1-9. Can you able to confirm the case with debug, or from the assembly/object file?

Quoted 6 lines

And if the issue is strict aliasing rule violation as you said, compiler flag is an option but not sure how much it reduces the compiler optimization benefit, I guess other options also not so good, memcpy brings too much work on runtime and union requires bigger change and makes code complex.
I wonder if making 'ip_sum()' a non inline function can help, can you please give a try since you can reproduce it?

Hi Ferruh,

Thanks for looking into it.

I am copy-pasting at the end of this email a minimal reproduction. It calculates a checksum and prints it. The correct value is f8d9. If you compile it with -O0 or -O3 -fno-strict-aliasing, you will get the correct value. If you compile it with gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0 and -O3, you will get f8e8. You can also try it on https://godbolt.org/ and see how different versions behave.

My understanding is that the code violates the C standard (https://stackoverflow.com/a/99010).

--- cut here ---

#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

struct rte_ipv4_hdr {
	uint8_t  version_ihl;
	uint8_t  type_of_service;
	uint16_t total_length;
	uint16_t packet_id;
	uint16_t fragment_offset;
	uint8_t  time_to_live;
	uint8_t  next_proto_id;
	uint16_t hdr_checksum;
	uint32_t src_addr;
	uint32_t dst_addr;
};

static inline uint16_t ip_sum(const uint16_t *hdr, int hdr_len)

{
	uint32_t sum = 0;

	while (hdr_len > 1)
	{
		sum += *hdr++;
		if (sum & 0x80000000)
			sum = (sum & 0xFFFF) + (sum >> 16);
		hdr_len -= 2;
	}

	while (sum >> 16)
		sum = (sum & 0xFFFF) + (sum >> 16);

	return ~sum;
}

static void pkt_burst_flow_gen(void)

{
	struct rte_ipv4_hdr *ip_hdr = (struct rte_ipv4_hdr *) malloc(4096);
	memset(ip_hdr, 0, sizeof(*ip_hdr));
	ip_hdr->version_ihl	= 1;
	ip_hdr->type_of_service	= 2;
	ip_hdr->fragment_offset	= 3;
	ip_hdr->time_to_live	= 4;
	ip_hdr->next_proto_id	= 5;
	ip_hdr->packet_id	= 6;
	ip_hdr->src_addr	= 7;
	ip_hdr->dst_addr	= 8;
	ip_hdr->total_length	= 9;
	ip_hdr->hdr_checksum	= ip_sum((uint16_t *)ip_hdr, sizeof(*ip_hdr));
	printf("%x\n", ip_hdr->hdr_checksum);
}
Quoted 7 lines
Quoted 5 lines

int main(void)

{
	pkt_burst_flow_gen();
	return 0;
}

If I change your code like this to use union, Gcc 10 is still broken.

This worked fine for me: https://godbolt.org/z/vdsxh9

Quoted 69 lines

It is a compiler bug. It maybe because optimizer is not smart enough to know that memset has cleared the header.

#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

struct rte_ipv4_hdr {
	uint8_t  version_ihl;
	uint8_t  type_of_service;
	uint16_t total_length;
	uint16_t packet_id;
	uint16_t fragment_offset;
	uint8_t  time_to_live;
	uint8_t  next_proto_id;
	uint16_t hdr_checksum;
	uint32_t src_addr;
	uint32_t dst_addr;
};

static inline uint16_t ip_sum(const uint16_t *hdr, int hdr_len)

{
	uint32_t sum = 0;

	while (hdr_len > 1)
	{
		sum += *hdr++;
		if (sum & 0x80000000)
			sum = (sum & 0xFFFF) + (sum >> 16);
		hdr_len -= 2;
	}

	while (sum >> 16)
		sum = (sum & 0xFFFF) + (sum >> 16);

	return ~sum;
}

static void pkt_burst_flow_gen(void)

{
	union {
		struct rte_ipv4_hdr ip;
		uint16_t data[10];
	} *hdr;

	hdr = malloc(sizeof(*hdr));

	memset(hdr, 0, sizeof(*hdr));
	hdr->ip.version_ihl	= 1;
	hdr->ip.type_of_service	= 2;
	hdr->ip.fragment_offset	= 3;
	hdr->ip.time_to_live	= 4;
	hdr->ip.next_proto_id	= 5;
	hdr->ip.packet_id	= 6;
	hdr->ip.src_addr	= 7;
	hdr->ip.dst_addr	= 8;
	hdr->ip.total_length	= 9;
	hdr->ip.hdr_checksum	= ip_sum(hdr->data, sizeof(*hdr));
	printf("%x\n", hdr->ip.hdr_checksum);
}

int main(void)

{
	pkt_burst_flow_gen();
	return 0;
}
#18 lore raw
Quoted 10 lines · On Thu, 7 Jan 2021 15:59:59 +0000 Ferruh Yigit <[email protected]> wrote:
Quoted 8 lines
Quoted 6 lines

int main(void)

{
	pkt_burst_flow_gen();
	return 0;
}  

If I change your code like this to use union, Gcc 10 is still broken.

This worked fine for me: https://godbolt.org/z/vdsxh9

I was looking for wrong result.
The union version gives same answer with and without optimization.

#19 lore raw

PATCH v3 1/1 app/testpmd: fix IP checksum calculation

In reply to George Prekas

Strict-aliasing rules are violated by cast to uint16_t* in flowgen.c and the calculated IP checksum is wrong. Use attribute __may_alias__ to fix the problem.

---
v3:
* Instead of a compiler flag, use a compiler attribute. v2:
* Instead of a compiler barrier, use a compiler flag.

---
 app/test-pmd/flowgen.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
app/test-pmd/flowgen.c +5 -2
index acf3e2460..cabfc688f 100644
@@ -53,8 +53,11 @@ static struct rte_ether_addr cfg_ether_dst =
 
 #define IP_DEFTTL  64   /* from RFC 1340. */
 
+/* Use this type to inform GCC that ip_sum violates aliasing rules. */
+typedef unaligned_uint16_t alias_int16_t __attribute__((__may_alias__));
+
 static inline uint16_t
-ip_sum(const unaligned_uint16_t *hdr, int hdr_len)
+ip_sum(const alias_int16_t *hdr, int hdr_len)
 {
 	uint32_t sum = 0;
 
@@ -150,7 +153,7 @@ pkt_burst_flow_gen(struct fwd_stream *fs)
 							   next_flow);
 		ip_hdr->total_length	= RTE_CPU_TO_BE_16(pkt_size -
 							   sizeof(*eth_hdr));
-		ip_hdr->hdr_checksum	= ip_sum((unaligned_uint16_t *)ip_hdr,
+		ip_hdr->hdr_checksum	= ip_sum((const alias_int16_t *)ip_hdr,
 						 sizeof(*ip_hdr));
 
 		/* Initialize UDP header. */
Signature · 1 line
2.17.1