symbol conflicts between netinet/in.h, arpa/inet.h, and rte_ip.h

Hello,

I ran into some weird symbol conflicts between system netinet/in.h and DPDK
rte_ip.h. They have a lot of duplicated definitions for stuff like IPPROTO_IP
and so on. This breaks when you want to use inet_pton from arpa/inet.h, because it includes netinet/in.h to define struct in_addr.

Thus with all the conflicts it's impossible to use a DPDK IP struct instead of
all the system's sockaddr stuff, to store a value from the system copy of inet_pton. This would be a common operation if, for example, you want to configure all the IP addresses on your box from a JSON file, which is what I
was doing.

The DPDK kludged around it internally by using a file called cmdline_parse_ipaddr.c with private copies of these functions. But it in my
opinion very unwisely marked all of the functions as static except for cmdline_parse_ipaddr, which only works on the DPDK's proprietary argument handling, and not with anything the user might have which is a different format.

So, it would be a big help for users if the macros in librte_net files would
check if the symbols already existed, or if they had subheader files available
to grab only non conflicting symbols, or if they would make a proper .h and
factor all the inet_pton and inet_ntop inside the cmdline lib into a place
where users can access them. It would also be a help if they had a less ugly
equivalent to struct sockaddr, which let you work with IP addresses a bit more
easily, such as something like this:

struct ip4_addr {
   uint32_t addr;
};

typedef struct ip4_addr ip4_addr;

struct ip6_addr {
   uint8_t addr[16];
};

typedef struct ip6_addr ip6_addr;

struct ip_addr {
   uint8_t family;
   uint8_t prefix;
   union {
       struct ip4_addr ipv4;
       struct ip6_addr ipv6;
   };
};

I had to create a bunch of duplicate code to handle it in my project, since
the DPDK marked its copies of all these functions as "secret" and didn't make
a .h for them. If any of it is useful I am happy to donate it, although I don't think I've got quite enough experience with this specifc part of the
DPDK to code it up all by myself.

Thanks,
Matthew.

Hello,

I ran into some weird symbol conflicts between system netinet/in.h and DPDK rte_ip.h. They have a lot of duplicated definitions for stuff like IPPROTO_IP and so on. This breaks when you want to use inet_pton from arpa/inet.h, because it includes netinet/in.h to define struct in_addr.

Thus with all the conflicts it's impossible to use a DPDK IP struct instead of all the system's sockaddr stuff, to store a value from the system copy of inet_pton. This would be a common operation if, for example, you want to configure all the IP addresses on your box from a JSON file, which is what I was doing.

The DPDK kludged around it internally by using a file called cmdline_parse_ipaddr.c with private copies of these functions. But it in my opinion very unwisely marked all of the functions as static except for cmdline_parse_ipaddr, which only works on the DPDK's proprietary argument handling, and not with anything the user might have which is a different format.

So, it would be a big help for users if the macros in librte_net files would check if the symbols already existed, or if they had subheader files available to grab only non conflicting symbols, or if they would make a proper .h and factor all the inet_pton and inet_ntop inside the cmdline lib into a place where users can access them. It would also be a help if they had a less ugly equivalent to struct sockaddr, which let you work with IP addresses a bit more easily, such as something like this:

In my experience from years of fighting with more or less this exact same problem -- the fight is now thankfully over but the scars remain -- you either want to expose a complete set of types and provide support for everything, or you want to expose nothing.

-----Original Message-----
From: dev [mailto:[email protected]] On Behalf Of Antti Kantee
Sent: Friday, July 25, 2014 6:56 AM
To: Matthew Hall; [email protected]
Subject: Re: [dpdk-dev] symbol conflicts between netinet/in.h, arpa/inet.h, and rte_ip.h

Quoted 6 lines · Matthew Hall

Hello,

I ran into some weird symbol conflicts between system netinet/in.h and DPDK rte_ip.h. They have a lot of duplicated definitions for stuff like IPPROTO_IP and so on. This breaks when you want to use inet_pton from arpa/inet.h, because it includes netinet/in.h to define struct in_addr.

I would namespace the definitions in DPDK, i.e. make them DPDK_IPPROTO_FOO etc.

Quoted 6 lines

Thus with all the conflicts it's impossible to use a DPDK IP struct instead of all the system's sockaddr stuff, to store a value from the system copy of inet_pton. This would be a common operation if, for example, you want to configure all the IP addresses on your box from a JSON file, which is what I was doing.

Quoted 6 lines

The DPDK kludged around it internally by using a file called cmdline_parse_ipaddr.c with private copies of these functions. But it in my opinion very unwisely marked all of the functions as static except for cmdline_parse_ipaddr, which only works on the DPDK's proprietary argument handling, and not with anything the user might have which is a different format.

In my experience from years of fighting with more or less this exact same problem -- the fight is now thankfully over but the scars remain -- you either want to expose a complete set of types and provide support for everything, or you want to expose nothing. Approaches where you use cute definitions and reuse some host routines is like asking for an audience with Tyranthraxus when armed with a kitten. It's that doubly so if you don't have to and do it anyway.

Quoted 7 lines

So, it would be a big help for users if the macros in librte_net files would check if the symbols already existed, or if they had subheader files available to grab only non conflicting symbols, or if they would make a proper .h and factor all the inet_pton and inet_ntop inside the cmdline lib into a place where users can access them. It would also be a help if they had a less ugly equivalent to struct sockaddr, which let you work with IP addresses a bit more easily, such as something like this:

Again, I recommend steering away from any tightrope approaches that "know" which types are non-conflicting, or pick out half-and-half from the host and IP stack. "Do, or do not, there is no half-and-half"

Hello,

We also notice these conflicts, we just planned to fix it in our new feature development. The proposal is like:

#ifndef _NETINET_IN_H
#ifndef _NETINET_IN_H_

#define IPPROTO_IP 0
... ...
#define IPPROTO_MAX 256

#endif
#endif

Do you think it is a good idea?

Quoted 5 lines

-----Original Message-----
From: dev [mailto:[email protected]] On Behalf Of Antti Kantee
Sent: Friday, July 25, 2014 6:56 AM
To: Matthew Hall; [email protected]
Subject: Re: [dpdk-dev] symbol conflicts between netinet/in.h,

arpa/inet.h, and rte_ip.h

Quoted 5 lines · Matthew Hall

Quoted 3 lines

Hello,

I ran into some weird symbol conflicts between system netinet/in.h

and DPDK

Quoted 1 line

Quoted 1 line

rte_ip.h. They have a lot of duplicated definitions for stuff like

IPPROTO_IP

Quoted 1 line

Quoted 1 line

and so on. This breaks when you want to use inet_pton from

arpa/inet.h,

Quoted 6 lines

Quoted 1 line

because it includes netinet/in.h to define struct in_addr.

I would namespace the definitions in DPDK, i.e. make them DPDK_IPPROTO_FOO etc.

Quoted 1 line

Thus with all the conflicts it's impossible to use a DPDK IP struct

instead of

Quoted 1 line

Quoted 1 line

all the system's sockaddr stuff, to store a value from the system

copy of

Quoted 1 line

Quoted 1 line

inet_pton. This would be a common operation if, for example, you

want to

Quoted 1 line

Quoted 1 line

configure all the IP addresses on your box from a JSON file, which

is what I

Quoted 4 lines

Quoted 4 lines

was doing.

The DPDK kludged around it internally by using a file called cmdline_parse_ipaddr.c with private copies of these functions. But

it in my

Quoted 1 line

Quoted 1 line

opinion very unwisely marked all of the functions as static except

for

Quoted 1 line

Quoted 1 line

cmdline_parse_ipaddr, which only works on the DPDK's proprietary

argument

Quoted 1 line

Quoted 1 line

handling, and not with anything the user might have which is a

different

Quoted 4 lines

Quoted 1 line

format.

In my experience from years of fighting with more or less this exact same problem -- the fight is now thankfully over but the scars remain

--

Quoted 2 lines

you either want to expose a complete set of types and provide support for everything, or you want to expose nothing. Approaches where you

use

Quoted 2 lines

cute definitions and reuse some host routines is like asking for an audience with Tyranthraxus when armed with a kitten. It's that

doubly

Quoted 3 lines

so if you don't have to and do it anyway.

Quoted 1 line

So, it would be a big help for users if the macros in librte_net

files would

Quoted 1 line

Quoted 1 line

check if the symbols already existed, or if they had subheader

files available

Quoted 1 line

Quoted 1 line

to grab only non conflicting symbols, or if they would make a

proper .h and

Quoted 1 line

Quoted 1 line

factor all the inet_pton and inet_ntop inside the cmdline lib into

a place

Quoted 1 line

Quoted 1 line

where users can access them. It would also be a help if they had a

less ugly

Quoted 1 line

Quoted 1 line

equivalent to struct sockaddr, which let you work with IP addresses

a bit more

Quoted 4 lines

Quoted 1 line

easily, such as something like this:

Again, I recommend steering away from any tightrope approaches that "know" which types are non-conflicting, or pick out half-and-half

from

Quoted 1 line

the host and IP stack. "Do, or do not, there is no half-and-half"

I don't know if it will work right on both Linux and BSD and/or if they also 100% agree with the libc / glibc values compiled into the system's .so files. That's the risk that you run if you don't have more complete support in the DPDK itself for these features.

--
Sent from my mobile device.

Quoted 88 lines · Jingjing

Hello,

We also notice these conflicts, we just planned to fix it in our new feature development. The proposal is like:

#ifndef _NETINET_IN_H
#ifndef _NETINET_IN_H_

#define IPPROTO_IP 0
... ...
#define IPPROTO_MAX 256

#endif
#endif

Do you think it is a good idea?

Quoted 5 lines

-----Original Message-----
From: dev [mailto:[email protected]] On Behalf Of Antti Kantee
Sent: Friday, July 25, 2014 6:56 AM
To: Matthew Hall; [email protected]
Subject: Re: [dpdk-dev] symbol conflicts between netinet/in.h,

arpa/inet.h, and rte_ip.h

Quoted 5 lines · Matthew Hall

Quoted 3 lines

Hello,

I ran into some weird symbol conflicts between system netinet/in.h

and DPDK

Quoted 1 line

Quoted 1 line

rte_ip.h. They have a lot of duplicated definitions for stuff like

IPPROTO_IP

Quoted 1 line

Quoted 1 line

and so on. This breaks when you want to use inet_pton from

arpa/inet.h,

Quoted 6 lines

Quoted 1 line

because it includes netinet/in.h to define struct in_addr.

I would namespace the definitions in DPDK, i.e. make them DPDK_IPPROTO_FOO etc.

Quoted 1 line

Thus with all the conflicts it's impossible to use a DPDK IP struct

instead of

Quoted 1 line

Quoted 1 line

all the system's sockaddr stuff, to store a value from the system

copy of

Quoted 1 line

Quoted 1 line

inet_pton. This would be a common operation if, for example, you

want to

Quoted 1 line

Quoted 1 line

configure all the IP addresses on your box from a JSON file, which

is what I

Quoted 4 lines

Quoted 4 lines

was doing.

The DPDK kludged around it internally by using a file called cmdline_parse_ipaddr.c with private copies of these functions. But

it in my

Quoted 1 line

Quoted 1 line

opinion very unwisely marked all of the functions as static except

for

Quoted 1 line

Quoted 1 line

cmdline_parse_ipaddr, which only works on the DPDK's proprietary

argument

Quoted 1 line

Quoted 1 line

handling, and not with anything the user might have which is a

different

Quoted 4 lines

Quoted 1 line

format.

In my experience from years of fighting with more or less this exact same problem -- the fight is now thankfully over but the scars remain

--

Quoted 2 lines

you either want to expose a complete set of types and provide support for everything, or you want to expose nothing. Approaches where you

use

Quoted 2 lines

cute definitions and reuse some host routines is like asking for an audience with Tyranthraxus when armed with a kitten. It's that

doubly

Quoted 3 lines

so if you don't have to and do it anyway.

Quoted 1 line

So, it would be a big help for users if the macros in librte_net

files would

Quoted 1 line

Quoted 1 line

check if the symbols already existed, or if they had subheader

files available

Quoted 1 line

Quoted 1 line

to grab only non conflicting symbols, or if they would make a

proper .h and

Quoted 1 line

Quoted 1 line

factor all the inet_pton and inet_ntop inside the cmdline lib into

a place

Quoted 1 line

Quoted 1 line

where users can access them. It would also be a help if they had a

less ugly

Quoted 1 line

Quoted 1 line

equivalent to struct sockaddr, which let you work with IP addresses

a bit more

Quoted 4 lines

Quoted 1 line

easily, such as something like this:

Again, I recommend steering away from any tightrope approaches that "know" which types are non-conflicting, or pick out half-and-half

from

Quoted 1 line

the host and IP stack. "Do, or do not, there is no half-and-half"

Quoted 4 lines

I ran into some weird symbol conflicts between system netinet/in.h and DPDK rte_ip.h. They have a lot of duplicated definitions for stuff like IPPROTO_IP and so on. This breaks when you want to use inet_pton from arpa/inet.h, because it includes netinet/in.h to define struct in_addr.

Again, I recommend steering away from any tightrope approaches that "know" which types are non-conflicting, or pick out half-and-half from the host and IP stack. "Do, or do not, there is no half-and-half"

Quoted 5 lines · Matthew Hall

Quoted 4 lines

I ran into some weird symbol conflicts between system netinet/in.h and DPDK rte_ip.h. They have a lot of duplicated definitions for stuff like IPPROTO_IP and so on. This breaks when you want to use inet_pton from arpa/inet.h, because it includes netinet/in.h to define struct in_addr.

[...]

Quoted 3 lines

Again, I recommend steering away from any tightrope approaches that "know" which types are non-conflicting, or pick out half-and-half from the host and IP stack. "Do, or do not, there is no half-and-half"

The general problem here is that DPDK is conflicting with libc. So the obvious question would be: "why DPDK needs to redefine libc stuff"? I don't see any obvious answer since bare metal is planned to be removed. (see http://dpdk.org/ml/archives/dev/2014-June/003868.html)

Quoted 2 lines · Thomas Monjalon

Quoted 2 lines · Matthew Hall

Quoted 1 line

I ran into some weird symbol conflicts between system netinet/in.h

and DPDK

Quoted 1 line

Quoted 1 line

Quoted 1 line

rte_ip.h. They have a lot of duplicated definitions for stuff like

IPPROTO_IP

Quoted 1 line

Quoted 1 line

Quoted 1 line

and so on. This breaks when you want to use inet_pton from

arpa/inet.h,

Quoted 4 lines

Quoted 1 line

Quoted 1 line

because it includes netinet/in.h to define struct in_addr.

[...]

Quoted 2 lines

Again, I recommend steering away from any tightrope approaches that "know" which types are non-conflicting, or pick out half-and-half

from

Quoted 4 lines

Quoted 1 line

the host and IP stack. "Do, or do not, there is no half-and-half"

The general problem here is that DPDK is conflicting with libc. So the obvious question would be: "why DPDK needs to redefine libc

stuff"?

Quoted 1 line

I don't see any obvious answer since bare metal is planned to be

removed.

Quoted 1 line

(see http://dpdk.org/ml/archives/dev/2014-June/003868.html)

One reason is if you want DPDK to be a portable network programming environment. Especially in that case you do not want definitions based

on hackish assumptions of some particular version of some particular host implementation. However, I'm not trying to argue if DPDK should or
shouldn't be that, just that you should either dramatically improve the

current implementation or nuke it.