fixes for FreeBSD compilation

Bruce Richardson (2):
examples/performance-thread: fix FreeBSD compilation net/ark: fix FreeBSD compilation

 drivers/net/ark/Makefile                                |  2 ++
 examples/performance-thread/l3fwd-thread/main.c         |  4 ++++
 examples/performance-thread/pthread_shim/main.c         |  4 ++++
 examples/performance-thread/pthread_shim/pthread_shim.c | 15 +++++++++++++--
 examples/performance-thread/pthread_shim/pthread_shim.h |  9 +++++++++
 5 files changed, 32 insertions(+), 2 deletions(-)

2.11.0

This set of sample apps did not compile on FreeBSD due to use of a number of Linux/glibc-specific APIs, or APIs which existed in different headers on FreeBSD. Specifically, the following APIs has problems:
* sched_getcpu() is a glibc extension
* pthread_yield() returns int on Linux, but void on FreeBSD
* APIs for managing cpu affinity are in pthread_np.h on FreeBSD, rather than in pthread.h
* the type for managing cpu sets is cpuset_t on FreeBSD rather than cpu_set_t as on Linux.

---
 examples/performance-thread/l3fwd-thread/main.c         |  4 ++++
 examples/performance-thread/pthread_shim/main.c         |  4 ++++
 examples/performance-thread/pthread_shim/pthread_shim.c | 15 +++++++++++++--
 examples/performance-thread/pthread_shim/pthread_shim.h |  9 +++++++++
 4 files changed, 30 insertions(+), 2 deletions(-)

index f6154ac2a..2d98473eb 100644
@@ -90,6 +90,10 @@
 #define APP_LOOKUP_METHOD             APP_LOOKUP_LPM
 #endif
 
+#ifndef __GLIBC__ /* sched_getcpu() is glibc specific */
+#define sched_getcpu() rte_lcore_id()
+#endif
+
 static int
 check_ptype(int portid)
 {

index f03572181..f7074006e 100644
@@ -59,6 +59,10 @@
 #define DEBUG_APP 0
 #define HELLOW_WORLD_MAX_LTHREADS 10
 
+#ifndef __GLIBC__ /* sched_getcpu() is glibc-specific */
+#define sched_getcpu() rte_lcore_id()
+#endif
+
 __thread int print_count;
 __thread pthread_mutex_t print_lock;
 

index 0d6100c90..1b2e1de03 100644
@@ -576,15 +576,26 @@ int pthread_rwlock_wrlock(pthread_rwlock_t *a)
 	return _sys_pthread_funcs.f_pthread_rwlock_wrlock(a);
 }
 
-int pthread_yield(void)
+#ifdef RTE_EXEC_ENV_LINUXAPP
+int
+pthread_yield(void)
 {
 	if (override) {
 		lthread_yield();
 		return 0;
 	}
 	return _sys_pthread_funcs.f_pthread_yield();
-
 }
+#else
+void
+pthread_yield(void)
+{
+	if (override)
+		lthread_yield();
+	else
+		_sys_pthread_funcs.f_pthread_yield();
+}
+#endif
 
 pthread_t pthread_self(void)
 {

index 78bbb5ac0..527640bd6 100644
@@ -36,6 +36,15 @@
 #include <pthread.h>
 
 /*
+ * on BSD, the non-std calls are in pthread_np.h,
+ * and cpuset_t has only one "_" rather than two.
+ */
+#ifdef RTE_EXEC_ENV_BSDAPP
+#include <pthread_np.h>
+#define cpu_set_t cpuset_t
+#endif
+
+/*
  * This pthread shim is an example that demonstrates how legacy code
  * that makes use of POSIX pthread services can make use of lthreads
  * with reduced porting effort.

2.11.0

20/04/2017 18:32, Bruce Richardson:

It can be fixed by using rte_cpuset_t defined in
lib/librte_eal/common/include/rte_lcore.h

On FreeBSD it's not necessary to use -ldl to link apps which use dlopen. This error only showed up with a shared library gcc build, not standard build using static libs.

---
 drivers/net/ark/Makefile | 2 ++
 1 file changed, 2 insertions(+)

index b3d462ff5..ca64b1957 100644
@@ -59,6 +59,8 @@ SRCS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark_udm.c
 
 # this lib depends upon:
 LDLIBS += -lpthread
+ifdef CONFIG_RTE_EXEC_ENV_LINUXAPP
 LDLIBS += -ldl
+endif
 
 include $(RTE_SDK)/mk/rte.lib.mk

2.11.0

Thanks Bruce.

Fixes a number of issues with compiling on BSD, and a general fix to clean up the compilation of the performance-thread app.

V2 changes:
* adjust fix for perf thread compilation to use rte_lcore.h instead of extra #ifdefs.
* added in two extra patches, enabling examples_clean to run on BSD, and removing the "common" build dir in perf thread example.

Bruce Richardson (4):
net/ark: fix FreeBSD compilation
examples: fix examples_clean build target on FreeBSD examples/performance-thread: fix FreeBSD compilation examples/performance-thread: use a single build output dir

 drivers/net/ark/Makefile                           |  2 ++
 examples/exception_path/Makefile                   |  1 +
 examples/netmap_compat/bridge/Makefile             |  1 +
 examples/performance-thread/common/common.mk       | 20 ++++++++++---------
 examples/performance-thread/l3fwd-thread/main.c    |  4 ++++
 examples/performance-thread/pthread_shim/main.c    |  8 ++++++--
 .../performance-thread/pthread_shim/pthread_shim.c | 23 ++++++++++++++++------
 .../performance-thread/pthread_shim/pthread_shim.h |  3 ++-
 examples/qos_sched/Makefile                        |  1 +
 9 files changed, 45 insertions(+), 18 deletions(-)

2.11.0

On FreeBSD it's not necessary to use -ldl to link apps which use dlopen. This error only showed up with a shared library gcc build, not standard build using static libs.

---
 drivers/net/ark/Makefile | 2 ++
 1 file changed, 2 insertions(+)

index b3d462ff5..ca64b1957 100644
@@ -59,6 +59,8 @@ SRCS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark_udm.c
 
 # this lib depends upon:
 LDLIBS += -lpthread
+ifdef CONFIG_RTE_EXEC_ENV_LINUXAPP
 LDLIBS += -ldl
+endif
 
 include $(RTE_SDK)/mk/rte.lib.mk

2.11.0

The "examples_clean" top-level build target calls "make clean" for each individual example. However, for a number of those which were linux-only examples, the "if" condition only had a dummy "all" target i.e. no "clean" target, causing an error with examples_clean.

---
 examples/exception_path/Makefile       | 1 +
 examples/netmap_compat/bridge/Makefile | 1 +
 examples/qos_sched/Makefile            | 1 +
 3 files changed, 3 insertions(+)

index 959914a2a..4b6e07175 100644
@@ -42,6 +42,7 @@ ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp")
 $(info This application can only operate in a linuxapp environment, \
 please change the definition of the RTE_TARGET environment variable)
 all:
+clean:
 else
 
 # binary name

index 50d96e819..1d4ddfffc 100644
@@ -42,6 +42,7 @@ ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp")
 $(info This application can only operate in a linuxapp environment, \
 please change the definition of the RTE_TARGET environment variable)
 all:
+clean:
 else
 
 # binary name

index f59645f52..e41ac500f 100644
@@ -42,6 +42,7 @@ ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp")
 $(info This application can only operate in a linuxapp environment, \
 please change the definition of the RTE_TARGET environment variable)
 all:
+clean:
 else
 
 # binary name

2.11.0

This set of sample apps did not compile on FreeBSD due to use of a number of Linux/glibc-specific APIs, or APIs which existed in different headers on FreeBSD. Specifically, the following APIs has problems:
* sched_getcpu() is a glibc extension, so use rte_lcore_id() on BSD
* pthread_yield() returns int on Linux, but void on FreeBSD, so we have to create two slightly different copies of the function.
* the type for managing cpu sets is cpuset_t on FreeBSD rather than cpu_set_t as on Linux, so use rte_cpuset_t from rte_lcore.h.
* APIs for managing cpu affinity are in pthread_np.h on FreeBSD, rather than in pthread.h, also fixed by including rte_lcore.h

---
V2: use rte_lcore.h to fix cpu_set_t type and pthread_np.h inclusion, rather than ifdefs in the app itself.

---
 examples/performance-thread/l3fwd-thread/main.c    |  4 ++++
 examples/performance-thread/pthread_shim/main.c    |  8 ++++++--
 .../performance-thread/pthread_shim/pthread_shim.c | 23 ++++++++++++++++------
 .../performance-thread/pthread_shim/pthread_shim.h |  3 ++-
 4 files changed, 29 insertions(+), 9 deletions(-)

index f6154ac2a..2d98473eb 100644
@@ -90,6 +90,10 @@
 #define APP_LOOKUP_METHOD             APP_LOOKUP_LPM
 #endif
 
+#ifndef __GLIBC__ /* sched_getcpu() is glibc specific */
+#define sched_getcpu() rte_lcore_id()
+#endif
+
 static int
 check_ptype(int portid)
 {

index f03572181..850b009d3 100644
@@ -59,6 +59,10 @@
 #define DEBUG_APP 0
 #define HELLOW_WORLD_MAX_LTHREADS 10
 
+#ifndef __GLIBC__ /* sched_getcpu() is glibc-specific */
+#define sched_getcpu() rte_lcore_id()
+#endif
+
 __thread int print_count;
 __thread pthread_mutex_t print_lock;
 

@@ -175,12 +179,12 @@ static void initial_lthread(void *args __attribute__((unused)))
 		 * use an attribute to pass the desired lcore
 		 */
 		pthread_attr_t attr;
-		cpu_set_t cpuset;
+		rte_cpuset_t cpuset;
 
 		CPU_ZERO(&cpuset);
 		CPU_SET(lcore, &cpuset);
 		pthread_attr_init(&attr);
-		pthread_attr_setaffinity_np(&attr, sizeof(cpu_set_t), &cpuset);
+		pthread_attr_setaffinity_np(&attr, sizeof(rte_cpuset_t), &cpuset);
 
 		/* create the thread */
 		pthread_create(&tid[i], &attr, helloworld_pthread, (void *) i);

index 0d6100c90..71673119a 100644
@@ -159,7 +159,7 @@ int (*f_pthread_setschedparam)
 int (*f_pthread_yield)
 	(void);
 int (*f_pthread_setaffinity_np)
-	(pthread_t thread, size_t cpusetsize, const cpu_set_t *cpuset);
+	(pthread_t thread, size_t cpusetsize, const rte_cpuset_t *cpuset);
 int (*f_nanosleep)
 	(const struct timespec *req, struct timespec *rem);
 } _sys_pthread_funcs = {

@@ -390,11 +390,11 @@ pthread_create(pthread_t *__restrict tid,
 
 		if (attr != NULL) {
 			/* determine CPU being requested */
-			cpu_set_t cpuset;
+			rte_cpuset_t cpuset;
 
 			CPU_ZERO(&cpuset);
 			pthread_attr_getaffinity_np(attr,
-						sizeof(cpu_set_t),
+						sizeof(rte_cpuset_t),
 						&cpuset);
 
 			if (CPU_COUNT(&cpuset) != 1)

@@ -576,15 +576,26 @@ int pthread_rwlock_wrlock(pthread_rwlock_t *a)
 	return _sys_pthread_funcs.f_pthread_rwlock_wrlock(a);
 }
 
-int pthread_yield(void)
+#ifdef RTE_EXEC_ENV_LINUXAPP
+int
+pthread_yield(void)
 {
 	if (override) {
 		lthread_yield();
 		return 0;
 	}
 	return _sys_pthread_funcs.f_pthread_yield();
-
 }
+#else
+void
+pthread_yield(void)
+{
+	if (override)
+		lthread_yield();
+	else
+		_sys_pthread_funcs.f_pthread_yield();
+}
+#endif
 
 pthread_t pthread_self(void)
 {

@@ -686,7 +697,7 @@ int nanosleep(const struct timespec *req, struct timespec *rem)
 
 int
 pthread_setaffinity_np(pthread_t thread, size_t cpusetsize,
-		       const cpu_set_t *cpuset)
+		       const rte_cpuset_t *cpuset)
 {
 	if (override) {
 		/* we only allow affinity with a single CPU */

index 78bbb5ac0..10f878940 100644
@@ -33,7 +33,8 @@
 
 #ifndef _PTHREAD_SHIM_H_
 #define _PTHREAD_SHIM_H_
-#include <pthread.h>
+
+#include <rte_lcore.h>
 
 /*
  * This pthread shim is an example that demonstrates how legacy code

2.11.0

When building any of the perf-thread examples, the output .o files were placed in two separate directories for each app: the regular build dir and a "common" build directory. This was due to the way the files to be built were specified, using a relative path. Switching to use VPATH to find the files causes Make to put all .o's into the one build directory.

---
 examples/performance-thread/common/common.mk | 20 +++++++++++---------
 1 file changed, 11 insertions(+), 9 deletions(-)

index d3de5fc60..f6cab7718 100644
@@ -30,13 +30,15 @@
 #   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
 #   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
-# list the C files belonhing to the lthread subsystem, these are common to all lthread apps
-SRCS-y +=	../common/lthread.c \
-			../common/lthread_sched.c \
-			../common/lthread_cond.c \
-			../common/lthread_tls.c \
-			../common/lthread_mutex.c \
-			../common/lthread_diag.c \
-			../common/arch/x86/ctx.c
+# list the C files belonging to the lthread subsystem, these are common to all
+# lthread apps. Any makefile including this should set VPATH to include this
+# directory path
+#
+
+MKFILE_PATH=$(abspath $(dir $(lastword $(MAKEFILE_LIST))))
+
+VPATH := $(MKFILE_PATH) $(MKFILE_PATH)/arch/x86
+
+SRCS-y += lthread.c lthread_sched.c lthread_cond.c lthread_tls.c lthread_mutex.c lthread_diag.c ctx.c
 
-INCLUDES += -I$(RTE_SDK)/examples/performance-thread/common/ -I$(RTE_SDK)/examples/performance-thread/common/arch/x86/
+INCLUDES += -I$(MKFILE_PATH) -I$(MKFILE_PATH)/arch/x86/

2.11.0

21/04/2017 15:50, Bruce Richardson:

Applied, thanks