Skip to contentSkip to conversation

Performance thread example application

PATCH Superseded ibetts 3 participants 8 messages Oct 29, 2015 – Nov 13, 2015

v2 of 10 Superseded Latest is v10 →

lore
Share & tools
lore

ibetts

author
#0 lore raw

v2 cover Performance thread example application

From: Ian Betts <[email protected]>

This example comprises layer 3 forwarding derivative intended to facilitate characterization of performance with different threading models, specifically:-

1. EAL threads running on different physical cores
2. EAL threads running on the same physical core
3. Lightweight threads running in an EAL thread

Purpose and justification

Since dpdk 2.0 it has been possible to assign multiple EAL threads to a physical core ( case 2 above ).
Currently no example application has focused on demonstrating the performance constraints of differing threading models.

Whilst purpose built applications that fully comprehend the DPDK single threaded programming model will always yield superior performance, the desire to preserve ROI in legacy code written for multithreaded operating environments makes lightweight threads (case 3 above) worthy of consideration.

As well as aiding with legacy code reuse, it is anticipated that lightweight threads will make it possible to scale a multithreaded application with fine granularity allowing an application to more easily take advantage of headroom on EAL cores, or conversely occupy more cores, as dictated by system load.

To explore performance with lightweight threads a simple cooperative scheduler subsystem is being included in this example application. If the expected benefits and use cases prove to be of value, it is anticipated that this lightweight thread subsystem would become a library in some future DPDK release.

A simple pthread shim in the form of a hello world example is also included.

Ian Betts (5):
doc: add performance-thread sample application guide
examples: add cooperative scheduler subsytem for
performance-thread app
examples: add l3fwd-thread in performance-thread sample app
examples: add pthread-shim in performance-thread sample app
config: add build files for performance-thread

 config/common_linuxapp                             |    5 +
 config/defconfig_i686-native-linuxapp-gcc          |    5 +
 config/defconfig_i686-native-linuxapp-icc          |    5 +
 config/defconfig_x86_x32-native-linuxapp-gcc       |    5 +
 doc/guides/rel_notes/release_2_2.rst               |   11 +
 .../sample_app_ug/img/performance_thread_1.svg     |  799 +++++
 .../sample_app_ug/img/performance_thread_2.svg     |  865 +++++
 doc/guides/sample_app_ug/index.rst                 |    1 +
 doc/guides/sample_app_ug/performance_thread.rst    | 1243 +++++++
 examples/Makefile                                  |    1 +
 examples/performance-thread/Makefile               |   45 +
 .../performance-thread/common/arch/x86/atomic.h    |   59 +
 examples/performance-thread/common/arch/x86/ctx.c  |   93 +
 examples/performance-thread/common/arch/x86/ctx.h  |   57 +
 examples/performance-thread/common/common.mk       |   40 +
 examples/performance-thread/common/lthread.c       |  546 +++
 examples/performance-thread/common/lthread.h       |   99 +
 examples/performance-thread/common/lthread_api.h   |  822 +++++
 examples/performance-thread/common/lthread_cond.c  |  241 ++
 examples/performance-thread/common/lthread_cond.h  |   77 +
 examples/performance-thread/common/lthread_diag.c  |  321 ++
 examples/performance-thread/common/lthread_diag.h  |  129 +
 .../performance-thread/common/lthread_diag_api.h   |  319 ++
 examples/performance-thread/common/lthread_int.h   |  212 ++
 examples/performance-thread/common/lthread_mutex.c |  256 ++
 examples/performance-thread/common/lthread_mutex.h |   52 +
 .../performance-thread/common/lthread_objcache.h   |  160 +
 examples/performance-thread/common/lthread_pool.h  |  333 ++
 examples/performance-thread/common/lthread_queue.h |  303 ++
 examples/performance-thread/common/lthread_sched.c |  598 ++++
 examples/performance-thread/common/lthread_sched.h |  152 +
 examples/performance-thread/common/lthread_timer.h |   47 +
 examples/performance-thread/common/lthread_tls.c   |  242 ++
 examples/performance-thread/common/lthread_tls.h   |   64 +
 examples/performance-thread/l3fwd-thread/Makefile  |   57 +
 examples/performance-thread/l3fwd-thread/main.c    | 3628 ++++++++++++++++++++
 examples/performance-thread/pthread_shim/Makefile  |   60 +
 examples/performance-thread/pthread_shim/main.c    |  284 ++
 .../performance-thread/pthread_shim/pthread_shim.c |  714 ++++
 .../performance-thread/pthread_shim/pthread_shim.h |  113 +
 40 files changed, 13063 insertions(+)
 create mode 100644 doc/guides/sample_app_ug/img/performance_thread_1.svg
 create mode 100644 doc/guides/sample_app_ug/img/performance_thread_2.svg
 create mode 100644 doc/guides/sample_app_ug/performance_thread.rst
 create mode 100644 examples/performance-thread/Makefile
 create mode 100644 examples/performance-thread/common/arch/x86/atomic.h
 create mode 100644 examples/performance-thread/common/arch/x86/ctx.c
 create mode 100644 examples/performance-thread/common/arch/x86/ctx.h
 create mode 100644 examples/performance-thread/common/common.mk
 create mode 100644 examples/performance-thread/common/lthread.c
 create mode 100644 examples/performance-thread/common/lthread.h
 create mode 100644 examples/performance-thread/common/lthread_api.h
 create mode 100644 examples/performance-thread/common/lthread_cond.c
 create mode 100644 examples/performance-thread/common/lthread_cond.h
 create mode 100644 examples/performance-thread/common/lthread_diag.c
 create mode 100644 examples/performance-thread/common/lthread_diag.h
 create mode 100644 examples/performance-thread/common/lthread_diag_api.h
 create mode 100644 examples/performance-thread/common/lthread_int.h
 create mode 100644 examples/performance-thread/common/lthread_mutex.c
 create mode 100644 examples/performance-thread/common/lthread_mutex.h
 create mode 100644 examples/performance-thread/common/lthread_objcache.h
 create mode 100644 examples/performance-thread/common/lthread_pool.h
 create mode 100644 examples/performance-thread/common/lthread_queue.h
 create mode 100644 examples/performance-thread/common/lthread_sched.c
 create mode 100644 examples/performance-thread/common/lthread_sched.h
 create mode 100644 examples/performance-thread/common/lthread_timer.h
 create mode 100644 examples/performance-thread/common/lthread_tls.c
 create mode 100644 examples/performance-thread/common/lthread_tls.h
 create mode 100644 examples/performance-thread/l3fwd-thread/Makefile
 create mode 100644 examples/performance-thread/l3fwd-thread/main.c
 create mode 100644 examples/performance-thread/pthread_shim/Makefile
 create mode 100644 examples/performance-thread/pthread_shim/main.c
 create mode 100644 examples/performance-thread/pthread_shim/pthread_shim.c
 create mode 100644 examples/performance-thread/pthread_shim/pthread_shim.h
Signature · 1 line
1.7.9.5
author
#1 lore raw

PATCH v2 1/5 doc: add performance-thread sample application guide

From: Ian Betts <[email protected]>

This commit adds documentation for the performance-thread sample application.

Changes in this version:
* fix typos
* changes to tx parameter
* add diagrams
* add CPU load stats

---
 doc/guides/rel_notes/release_2_2.rst               |   11 +
 .../sample_app_ug/img/performance_thread_1.svg     |  799 +++++++++++++
 .../sample_app_ug/img/performance_thread_2.svg     |  865 ++++++++++++++
 doc/guides/sample_app_ug/index.rst                 |    1 +
 doc/guides/sample_app_ug/performance_thread.rst    | 1243 ++++++++++++++++++++
 5 files changed, 2919 insertions(+)
 create mode 100644 doc/guides/sample_app_ug/img/performance_thread_1.svg
 create mode 100644 doc/guides/sample_app_ug/img/performance_thread_2.svg
 create mode 100644 doc/guides/sample_app_ug/performance_thread.rst
doc/guides/rel_notes/release_2_2.rst +11 -0
index 128f956..2c031de 100644
@@ -75,6 +75,12 @@ Libraries
 Examples
 ~~~~~~~~
 
+* **examples: Introducing a performance thread example**
+
+  This an l3fwd derivative focused to enable characterization of performance
+  with different threading models, including multiple EAL threads per physical
+  core, and multiple Lightweight threads running in an EAL thread.
+  The examples includes a simple cooperative scheduler.
 
 Other
 ~~~~~
@@ -82,6 +88,11 @@ Other
 
 Known Issues
 ------------
+* When running the performance thread application in configurations with more than
+  two EAL threads per phsycial core, then forwarding throughput may drop suddenly
+  to a low level. Stopping and restarting traffic restores correct operation.
+  This problem does not occur when running with multiple lightweight threads per
+  physical core.
 
 
 API Changes
doc/guides/sample_app_ug/img/performance_thread_1.svg new file +799 -0

799 lines not on this page · raw message

doc/guides/sample_app_ug/img/performance_thread_2.svg new file +865 -0

865 lines not on this page · raw message

doc/guides/sample_app_ug/index.rst +1 -0
index 9beedd9..70d4a5c 100644
@@ -73,6 +73,7 @@ Sample Applications User Guide
     vm_power_management
     tep_termination
     proc_info
+    performance_thread
 
 **Figures**
 
doc/guides/sample_app_ug/performance_thread.rst new file +1243 -0

1,243 lines not on this page · raw message

Signature · 1 line
1.7.9.5
#2 lore raw

Re: [PATCH v2 1/5] doc: add performance-thread sample application guide

Quoted 8 lines

-----Original Message-----
From: dev [mailto:[email protected]] On Behalf Of ibetts
Sent: Thursday, October 29, 2015 3:09 PM
To: [email protected]
Cc: Betts, Ian
Subject: [dpdk-dev] [PATCH v2 1/5] doc: add performance-thread sample application guide

Quoted 4 lines

From: Ian Betts <[email protected]>

This commit adds documentation for the performance-thread sample application.

Hi Ian,

Thanks for this. It is really excellent documentation.

See below for a few minor format/rendering issues.

doc/guides/sample_app_ug/performance_thread.rst line 2

Quoted 10 lines
doc/guides/sample_app_ug/performance_thread.rst new file +2 -0
new file mode 100644
index 0000000..c735931
@@ -0,0 +1,1243 @@
+..  BSD LICENSE
+    Copyright(c) 2010-2014 Intel Corporation. All rights reserved.

The year should be 2015.

doc/guides/sample_app_ug/performance_thread.rst line 4

Quoted 2 lines
doc/guides/sample_app_ug/performance_thread.rst new file +2 -0
+       export RTE_SDK=/path/to/rte_sdk cd ${RTE_SDK}/examples/performance-thread/l3fwd-thread
+

These two commands should be on separate lines. The same applies to the later shim example.

Quoted 8 lines
doc/guides/sample_app_ug/performance_thread.rst new file +5 -0
+The application has a number of command line options:
+
+.. code-block:: console
+
+    ./build/l3fwd-thread [EAL options] -- -p PORTMASK [-P] --

rx(port,queue,lcore,thread)[,(port,queue,lcore,thread)] -- tx(lcore,thread)[,(lcore,thread)] [--enable-jumbo [--max-pkt-len PKTLEN]] [--no-numa][--hash-entry-num][--ipv6] [--no-lthreads] [--stat-lcore lcore]

Thanks for following the guidelines. This is the only code line that is over 80 chars. I'd suggest formatting it like this:

The application has a number of command line options::

    ./build/l3fwd-thread [EAL options] --
        -p PORTMASK [-P]
        --rx(port,queue,lcore,thread)[,(port,queue,lcore,thread)]
        --tx(lcore,thread)[,(lcore,thread)]
        [--enable-jumbo] [--max-pkt-len PKTLEN]]  [--no-numa]
        [--hash-entry-num] [--ipv6] [--no-lthreads] [--stat-lcore lcore]

Note also, in most cases you can avoid " .. code-block:: console" and just use "::" for single line code blocks.

doc/guides/sample_app_ug/performance_thread.rst line 114

Quoted 10 lines
doc/guides/sample_app_ug/performance_thread.rst new file +7 -0
+
+where,
+
+*   -p PORTMASK: Hexadecimal bitmask of ports to configure
+
+*   -P: optional, sets all ports to promiscuous mode so that packets are
+     accepted regardless of the packet's Ethernet MAC destination

address.

doc/guides/sample_app_ug/performance_thread.rst new file +2 -0
+     Without this option, only packets with the Ethernet MAC destination
+     address set to the Ethernet address of the port are accepted.

This would looks better with fixed width fonts for the parameters:

Where:

* ``-p PORTMASK``: Hexadecimal bitmask of ports to configure

* ``-P``: optional, sets all ports to promiscuous mode so that packets are
accepted regardless of the packet's Ethernet MAC destination address. Without this option, only packets with the Ethernet MAC destination address set to the Ethernet address of the port are accepted.

The indentation is wrong in a few of these items as well.

Quoted 10 lines

+The l3fwd-threads application allows you to start packet processing in two threading
+models: L-Threads (default) and EAL Threads (when the "--no-lthreads" parameter
+is used). For consistency all parameters are used in the same way for both models.

doc/guides/sample_app_ug/performance_thread.rst new file +4 -0
+
+* rx  parameters
+
+.. _table_l3fwd_rx_parameters:

doc/guides/sample_app_ug/performance_thread.rst line 153

Quoted 10 lines
doc/guides/sample_app_ug/performance_thread.rst new file +10 -0
+
++--------+------------------------------------------------------+
+| port   | rx port                                              |
++--------+------------------------------------------------------+
+| queue  | rx queue that will be read on the specified rx port  |
++--------+------------------------------------------------------+
+| lcore  | core to use for the thread                           |
++--------+------------------------------------------------------+
+| thread | thread id (continuously from 0 to N)                 |
++--------+------------------------------------------------------+

The tables should be indented 3 spaces to align with the bullet list.

Quoted 15 lines
doc/guides/sample_app_ug/performance_thread.rst new file +4 -0
+Running with L-threads
+~~~~~~~~~~~~~~~~~~~~~~
+
+When the L-thread model is used (default option), lcore and thread

parameters in
+--rx/--tx are used to affine threads to the selected scheduler using the rules:

doc/guides/sample_app_ug/performance_thread.rst new file +8 -0
+
+e.g.
+
+
+    .. code-block:: console
+
+l3fwd-thread -c ff -n 2 -- -P -p 3 \
+        --rx="(0,0,0,0)(1,0,1,1)" \

doc/guides/sample_app_ug/performance_thread.rst line 189

Quoted 10 lines
doc/guides/sample_app_ug/performance_thread.rst new file +10 -0
+        --tx="(2,0)(3,1)"
+
+Places every l-thread on different lcore
+
+    .. code-block:: console
+
+l3fwd-thread -c ff -n 2 -- -P -p 3 \
+        --rx="(0,0,0,0)(1,0,0,1)" \
+        --tx="(1,0)(2,1)"
+

The examples should be indented

doc/guides/sample_app_ug/performance_thread.rst

Quoted 8 lines
doc/guides/sample_app_ug/performance_thread.rst new file +8 -0
+Places rx lthreads on lcore 0 and tx l-threads on lcore 1 and 2
+
+and so on.
+
+Running with EAL threads
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+When the --no-lthreads parameter is used, L-threading model is turned off

Parameters should be quoted with backquotes ``--no-lthreads``.

Quoted 20 lines

and EAL
+threads are used for all processing. EAL Threads are enumerated in the same way as L-threads,
+but the --lcores EAL parameter is used to affine thread to the selected cpu-set (scheduler).

doc/guides/sample_app_ug/performance_thread.rst new file +15 -0
+
+Thus it is possible to place every Rx and TX thread on different lcores
+
+e.g.
+
+    .. code-block:: console
+
+l3fwd-thread -c ff -n 2 -- -P -p 3 \
+        --rx="(0,0,0,0)(1,0,1,1)" \
+        --tx="(2,0)(3,1)" \
+	--no-lthreads
+
+Places every EAL thread on different lcore.
+
+To affine two ore more EAL threads to one cpu-set, eal --lcores parameter
Quoted 1 line

is used

Affinitize is probably better here and s/ore/or/. There are a few other minor spellings to fix as well.

Quoted 11 lines
doc/guides/sample_app_ug/performance_thread.rst new file +2 -0
+
+For selected scenarios the command line configuration of the application

for L-threads
+and its corresponding EAL threads command line can be realized as follows:

doc/guides/sample_app_ug/performance_thread.rst new file +6 -0
+
+a) Start every thread on different scheduler (1:1)
+
+    .. code-block:: console
+
+l3fwd-thread -c ff -n 2 -- -P -p 3 \

doc/guides/sample_app_ug/performance_thread.rst line 246

Quoted 10 lines
doc/guides/sample_app_ug/performance_thread.rst new file +10 -0
+	--rx="(0,0,0,0)(1,0,1,1)" \
+	--tx="(2,0)(3,1)"
+
+    .. code-block:: console
+
+l3fwd-thread -c ff -n 2 -- -P -p 3 \
+		--rx="(0,0,0,0)(1,0,1,1)" \
+		--tx="(2,0)(3,1)" \
+		--no-lthreads
+

The examples should be indented. Also there are two examples here but no explanation of the difference, unlike the following examples.

I'll review the rest of the docs and the text and send on some more comments.

John.

author
#3 lore raw

PATCH v2 2/5 examples: add cooperative scheduler subsytem for performance-thread app

In reply to ibetts

From: Ian Betts <[email protected]>

This commit adds a cooperative scheduler subsystem in the performance-thread sample application.

It is used in the performance-thread sample application by the l3fwd-thread application to enable multiple lightweight threads to be run in an EAL thread.

Changes in this version:
* minor bug fixes in lthread subsystem
* improved comments
* add missing copyright in ctx.c

---
 .../performance-thread/common/arch/x86/atomic.h    |   59 ++
 examples/performance-thread/common/arch/x86/ctx.c  |   93 +++
 examples/performance-thread/common/arch/x86/ctx.h  |   57 ++
 examples/performance-thread/common/common.mk       |   40 +
 examples/performance-thread/common/lthread.c       |  546 +++++++++++++
 examples/performance-thread/common/lthread.h       |   99 +++
 examples/performance-thread/common/lthread_api.h   |  822 ++++++++++++++++++++
 examples/performance-thread/common/lthread_cond.c  |  241 ++++++
 examples/performance-thread/common/lthread_cond.h  |   77 ++
 examples/performance-thread/common/lthread_diag.c  |  321 ++++++++
 examples/performance-thread/common/lthread_diag.h  |  129 +++
 .../performance-thread/common/lthread_diag_api.h   |  319 ++++++++
 examples/performance-thread/common/lthread_int.h   |  212 +++++
 examples/performance-thread/common/lthread_mutex.c |  256 ++++++
 examples/performance-thread/common/lthread_mutex.h |   52 ++
 .../performance-thread/common/lthread_objcache.h   |  160 ++++
 examples/performance-thread/common/lthread_pool.h  |  333 ++++++++
 examples/performance-thread/common/lthread_queue.h |  303 ++++++++
 examples/performance-thread/common/lthread_sched.c |  598 ++++++++++++++
 examples/performance-thread/common/lthread_sched.h |  152 ++++
 examples/performance-thread/common/lthread_timer.h |   47 ++
 examples/performance-thread/common/lthread_tls.c   |  242 ++++++
 examples/performance-thread/common/lthread_tls.h   |   64 ++
 23 files changed, 5222 insertions(+)
 create mode 100644 examples/performance-thread/common/arch/x86/atomic.h
 create mode 100644 examples/performance-thread/common/arch/x86/ctx.c
 create mode 100644 examples/performance-thread/common/arch/x86/ctx.h
 create mode 100644 examples/performance-thread/common/common.mk
 create mode 100644 examples/performance-thread/common/lthread.c
 create mode 100644 examples/performance-thread/common/lthread.h
 create mode 100644 examples/performance-thread/common/lthread_api.h
 create mode 100644 examples/performance-thread/common/lthread_cond.c
 create mode 100644 examples/performance-thread/common/lthread_cond.h
 create mode 100644 examples/performance-thread/common/lthread_diag.c
 create mode 100644 examples/performance-thread/common/lthread_diag.h
 create mode 100644 examples/performance-thread/common/lthread_diag_api.h
 create mode 100644 examples/performance-thread/common/lthread_int.h
 create mode 100644 examples/performance-thread/common/lthread_mutex.c
 create mode 100644 examples/performance-thread/common/lthread_mutex.h
 create mode 100644 examples/performance-thread/common/lthread_objcache.h
 create mode 100644 examples/performance-thread/common/lthread_pool.h
 create mode 100644 examples/performance-thread/common/lthread_queue.h
 create mode 100644 examples/performance-thread/common/lthread_sched.c
 create mode 100644 examples/performance-thread/common/lthread_sched.h
 create mode 100644 examples/performance-thread/common/lthread_timer.h
 create mode 100644 examples/performance-thread/common/lthread_tls.c
 create mode 100644 examples/performance-thread/common/lthread_tls.h
examples/performance-thread/common/arch/x86/atomic.h new file +59 -0
new file mode 100644
index 0000000..968dbe2
@@ -0,0 +1,59 @@
+/*
+ *-
+ *   BSD LICENSE
+ *
+ *   Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
+ *   All rights reserved.
+ *
+ *   Redistribution and use in source and binary forms, with or without
+ *   modification, are permitted provided that the following conditions
+ *   are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided with the
+ *       distribution.
+ *     * Neither the name of Intel Corporation nor the names of its
+ *       contributors may be used to endorse or promote products derived
+ *       from this software without specific prior written permission.
+ *
+ *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef ATOMIC_H_
+#define ATOMIC_H_
+
+#include <stdint.h>
+
+/*
+ * Atomically set a value and return the old value
+ */
+static inline uint64_t
+atomic64_xchg(uint64_t *ptr, uint64_t val) __attribute__ ((always_inline));
+static inline uint64_t
+atomic64_xchg(uint64_t *ptr, uint64_t val)
+{
+	asm volatile (
+				"lock;"
+				"xchgq %0,%1;"
+				 : "=r" ((uint64_t) val)
+				 : "m" (*(uint64_t *) ptr), "0" (val)
+				 : "memory");
+
+	return val;
+}
+
+
+#endif /* ATOMIC_H_ */
examples/performance-thread/common/arch/x86/ctx.c new file +93 -0

93 lines not on this page · raw message

examples/performance-thread/common/arch/x86/ctx.h new file +57 -0

57 lines not on this page · raw message

examples/performance-thread/common/common.mk new file +40 -0
new file mode 100644
index 0000000..6e0bc87
@@ -0,0 +1,40 @@
+#   BSD LICENSE
+#
+#   Copyright(c) 2014 6WIND S.A.
+#
+#   Redistribution and use in source and binary forms, with or without
+#   modification, are permitted provided that the following conditions
+#   are met:
+#
+#     * Redistributions of source code must retain the above copyright
+#       notice, this list of conditions and the following disclaimer.
+#     * Redistributions in binary form must reproduce the above copyright
+#       notice, this list of conditions and the following disclaimer in
+#       the documentation and/or other materials provided with the
+#       distribution.
+#     * Neither the name of 6WIND S.A. nor the names of its
+#       contributors may be used to endorse or promote products derived
+#       from this software without specific prior written permission.
+#
+#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+# list the C files belonhing to the lthread subsystem, these are common to all lthread apps
+SRCS-y +=	../common/lthread.c \
+			../common/lthread_sched.c \
+			../common/lthread_cond.c \
+			../common/lthread_tls.c \
+			../common/lthread_mutex.c \
+			../common/lthread_diag.c \
+			../common/arch/x86/ctx.c
+
+INCLUDES += -I$(RTE_SDK)/examples/performance-thread/common/ -I$(RTE_SDK)/examples/performance-thread/common/arch/x86/
examples/performance-thread/common/lthread.c new file +546 -0

546 lines not on this page · raw message

examples/performance-thread/common/lthread.h new file +99 -0

99 lines not on this page · raw message

examples/performance-thread/common/lthread_api.h new file +822 -0

822 lines not on this page · raw message

examples/performance-thread/common/lthread_cond.c new file +240 -0

240 lines not on this page · raw message

examples/performance-thread/common/lthread_cond.h new file +77 -0

77 lines not on this page · raw message

examples/performance-thread/common/lthread_diag.c new file +321 -0

321 lines not on this page · raw message

examples/performance-thread/common/lthread_diag.h new file +129 -0

129 lines not on this page · raw message

examples/performance-thread/common/lthread_diag_api.h new file +319 -0

319 lines not on this page · raw message

examples/performance-thread/common/lthread_int.h new file +212 -0

198 lines not on this page · raw message

examples/performance-thread/common/lthread_mutex.c new file +255 -0

255 lines not on this page · raw message

examples/performance-thread/common/lthread_mutex.h new file +52 -0

52 lines not on this page · raw message

examples/performance-thread/common/lthread_objcache.h new file +160 -0

160 lines not on this page · raw message

examples/performance-thread/common/lthread_pool.h new file +333 -0

333 lines not on this page · raw message

examples/performance-thread/common/lthread_queue.h new file +303 -0

303 lines not on this page · raw message

examples/performance-thread/common/lthread_sched.c new file +598 -0

598 lines not on this page · raw message

examples/performance-thread/common/lthread_sched.h new file +152 -0

152 lines not on this page · raw message

examples/performance-thread/common/lthread_timer.h new file +47 -0

47 lines not on this page · raw message

examples/performance-thread/common/lthread_tls.c new file +242 -0

242 lines not on this page · raw message

examples/performance-thread/common/lthread_tls.h new file +64 -0

64 lines not on this page · raw message

The rest of this diff is not shown. View the raw message.

Signature · 1 line
1.7.9.5

Part of the diff is not shown here. View the raw message.

author
#4 lore raw

PATCH v2 3/5 examples: add l3fwd-thread in performance-thread sample app

In reply to ibetts

From: Ian Betts <[email protected]>

This commit adds an l3fwd derivative that allows multiple EAL threads to be run on a single physical core or multiple lightwieght threads to be run in an EAL thread.

Its purpose is to facilitate characterization of performance with different threading models.

It depends on a simple cooperative scheduler included in this patchset.

Changes in this version:-
* Fix flaw causing incorrect routing
* Changes to tx parameter
* add CPU load stats

---
 examples/performance-thread/l3fwd-thread/Makefile |   57 +
 examples/performance-thread/l3fwd-thread/main.c   | 3628 +++++++++++++++++++++
 2 files changed, 3685 insertions(+)
 create mode 100644 examples/performance-thread/l3fwd-thread/Makefile
 create mode 100644 examples/performance-thread/l3fwd-thread/main.c
examples/performance-thread/l3fwd-thread/Makefile new file +57 -0
new file mode 100644
index 0000000..d8fe5e6
@@ -0,0 +1,57 @@
+#   BSD LICENSE
+#
+#   Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
+#   All rights reserved.
+#
+#   Redistribution and use in source and binary forms, with or without
+#   modification, are permitted provided that the following conditions
+#   are met:
+#
+#     * Redistributions of source code must retain the above copyright
+#       notice, this list of conditions and the following disclaimer.
+#     * Redistributions in binary form must reproduce the above copyright
+#       notice, this list of conditions and the following disclaimer in
+#       the documentation and/or other materials provided with the
+#       distribution.
+#     * Neither the name of Intel Corporation nor the names of its
+#       contributors may be used to endorse or promote products derived
+#       from this software without specific prior written permission.
+#
+#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+ifeq ($(RTE_SDK),)
+$(error "Please define RTE_SDK environment variable")
+endif
+
+# Default target, can be overridden by command line or environment
+RTE_TARGET ?= x86_64-native-linuxapp-gcc
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+# binary name
+APP = l3fwd-thread
+
+# all source are stored in SRCS-y
+SRCS-y := main.c
+
+include $(RTE_SDK)/examples/performance-thread/common/common.mk
+
+CFLAGS += -O3 -g $(USER_FLAGS) $(INCLUDES) $(WERROR_FLAGS)
+
+# workaround for a gcc bug with noreturn attribute
+# http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12603
+#ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y)
+CFLAGS_main.o += -Wno-return-type
+#endif
+
+include $(RTE_SDK)/mk/rte.extapp.mk
examples/performance-thread/l3fwd-thread/main.c new file +3628 -0

2,943 lines not on this page · raw message

The rest of this diff is not shown. View the raw message.

Signature · 1 line
1.7.9.5

Part of the diff is not shown here. View the raw message.

author
#5 lore raw

PATCH v2 4/5 examples: add pthread-shim in performance-thread sample app

In reply to ibetts

From: Ian Betts <[email protected]>

This commit adds a simple pthread_shim example for the cooperative included with this patchset.

The shim demonstrates a way in which legacy code writtem for pthreads could be adapted to lighweight threads.

---
 examples/performance-thread/pthread_shim/Makefile  |   60 ++
 examples/performance-thread/pthread_shim/main.c    |  284 ++++++++
 .../performance-thread/pthread_shim/pthread_shim.c |  714 ++++++++++++++++++++
 .../performance-thread/pthread_shim/pthread_shim.h |  113 ++++
 4 files changed, 1171 insertions(+)
 create mode 100644 examples/performance-thread/pthread_shim/Makefile
 create mode 100644 examples/performance-thread/pthread_shim/main.c
 create mode 100644 examples/performance-thread/pthread_shim/pthread_shim.c
 create mode 100644 examples/performance-thread/pthread_shim/pthread_shim.h
examples/performance-thread/pthread_shim/Makefile new file +60 -0
new file mode 100644
index 0000000..73fd8ec
@@ -0,0 +1,60 @@
+#   BSD LICENSE
+#
+#   Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
+#   All rights reserved.
+#
+#   Redistribution and use in source and binary forms, with or without
+#   modification, are permitted provided that the following conditions
+#   are met:
+#
+#     * Redistributions of source code must retain the above copyright
+#       notice, this list of conditions and the following disclaimer.
+#     * Redistributions in binary form must reproduce the above copyright
+#       notice, this list of conditions and the following disclaimer in
+#       the documentation and/or other materials provided with the
+#       distribution.
+#     * Neither the name of Intel Corporation nor the names of its
+#       contributors may be used to endorse or promote products derived
+#       from this software without specific prior written permission.
+#
+#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+ifeq ($(RTE_SDK),)
+$(error "Please define RTE_SDK environment variable")
+endif
+
+# Default target, can be overridden by command line or environment
+RTE_TARGET ?= x86_64-native-linuxapp-gcc
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+# binary name
+APP = lthread_pthread_shim
+
+# all source are stored in SRCS-y
+SRCS-y := main.c  pthread_shim.c
+INCLUDES := -I$(RTE_SDK)/$(RTE_TARGET)/include -I$(SRCDIR)
+include $(RTE_SDK)/examples/performance-thread/common/common.mk
+
+CFLAGS=    -g -O3 $(USER_FLAGS) $(INCLUDES)
+CFLAGS += $(WERROR_FLAGS)
+
+LDFLAGS += -lpthread
+
+# workaround for a gcc bug with noreturn attribute
+# http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12603
+ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y)
+CFLAGS_main.o += -Wno-return-type
+endif
+
+include $(RTE_SDK)/mk/rte.extapp.mk
examples/performance-thread/pthread_shim/main.c new file +284 -0

284 lines not on this page · raw message

examples/performance-thread/pthread_shim/pthread_shim.c new file +714 -0

714 lines not on this page · raw message

examples/performance-thread/pthread_shim/pthread_shim.h new file +113 -0

113 lines not on this page · raw message

Signature · 1 line
1.7.9.5
author
#6 lore raw

PATCH v2 5/5 config: add build files for performance-thread

In reply to ibetts

From: Ian Betts <[email protected]>

This commit adds the build controls for the
performance-thread sample

---
 config/common_linuxapp                       |    5 +++
 config/defconfig_i686-native-linuxapp-gcc    |    5 +++
 config/defconfig_i686-native-linuxapp-icc    |    5 +++
 config/defconfig_x86_x32-native-linuxapp-gcc |    5 +++
 examples/Makefile                            |    1 +
 examples/performance-thread/Makefile         |   45 ++++++++++++++++++++++++++
 6 files changed, 66 insertions(+)
 create mode 100644 examples/performance-thread/Makefile
config/common_linuxapp +5 -0
index 0de43d5..02a9f0b 100644
@@ -466,3 +466,8 @@ CONFIG_RTE_APP_TEST=y
 CONFIG_RTE_TEST_PMD=y
 CONFIG_RTE_TEST_PMD_RECORD_CORE_CYCLES=n
 CONFIG_RTE_TEST_PMD_RECORD_BURST_STATS=n
+
+#
+# Compile the performance thread sample application
+#
+CONFIG_RTE_PERFORMANCE_THREAD=y
config/defconfig_i686-native-linuxapp-gcc +5 -0
index a90de9b..06c9ad0 100644
@@ -49,3 +49,8 @@ CONFIG_RTE_LIBRTE_KNI=n
 # Vectorized PMD is not supported on 32-bit
 #
 CONFIG_RTE_IXGBE_INC_VECTOR=n
+
+#
+# Performance thread sample application is not supported on 32-bit
+#
+CONFIG_RTE_PERFORMANCE_THREAD=n
config/defconfig_i686-native-linuxapp-icc +5 -0
index c021321..e4610d4 100644
@@ -49,3 +49,8 @@ CONFIG_RTE_LIBRTE_KNI=n
 # Vectorized PMD is not supported on 32-bit
 #
 CONFIG_RTE_IXGBE_INC_VECTOR=n
+
+#
+# Performance thread sample application is not supported on 32-bit
+#
+CONFIG_RTE_PERFORMANCE_THREAD=n
config/defconfig_x86_x32-native-linuxapp-gcc +5 -0
index fb0afc4..5680328 100644
@@ -44,3 +44,8 @@ CONFIG_RTE_TOOLCHAIN_GCC=y
 # KNI is not supported on 32-bit
 #
 CONFIG_RTE_LIBRTE_KNI=n
+
+#
+# Performance thread sample application is not supported on 32-bit
+#
+CONFIG_RTE_PERFORMANCE_THREAD=n
examples/Makefile +1 -0
index b4eddbd..9dffac4 100644
@@ -74,5 +74,6 @@ DIRS-$(CONFIG_RTE_LIBRTE_XEN_DOM0) += vhost_xen
 DIRS-y += vmdq
 DIRS-y += vmdq_dcb
 DIRS-$(CONFIG_RTE_LIBRTE_POWER) += vm_power_manager
+DIRS-$(CONFIG_RTE_PERFORMANCE_THREAD) += performance-thread
 
 include $(RTE_SDK)/mk/rte.extsubdir.mk
examples/performance-thread/Makefile new file +45 -0

45 lines not on this page · raw message

Signature · 1 line
1.7.9.5
#7 lore raw
In reply to ibetts
Quoted 47 lines

-----Original Message-----
From: dev [mailto:[email protected]] On Behalf Of ibetts
Sent: Thursday, October 29, 2015 16:09
To: [email protected]
Cc: Betts, Ian
Subject: [dpdk-dev] [PATCH v2 0/5] Performance thread example application

From: Ian Betts <[email protected]>

This example comprises layer 3 forwarding derivative intended to facilitate characterization of performance with different threading models, specifically:-

1. EAL threads running on different physical cores 2. EAL threads running on the same physical core 3. Lightweight threads running in an EAL thread

Purpose and justification

Since dpdk 2.0 it has been possible to assign multiple EAL threads to a physical core ( case 2 above ).
Currently no example application has focused on demonstrating the performance constraints of differing threading models.

Whilst purpose built applications that fully comprehend the DPDK single threaded programming model will always yield superior performance, the desire to preserve ROI in legacy code written for multithreaded operating environments makes lightweight threads (case 3 above) worthy of consideration.

As well as aiding with legacy code reuse, it is anticipated that lightweight threads will make it possible to scale a multithreaded application with fine granularity allowing an application to more easily take advantage of headroom on EAL cores, or conversely occupy more cores, as dictated by system load.

To explore performance with lightweight threads a simple cooperative scheduler subsystem is being included in this example application. If the expected benefits and use cases prove to be of value, it is anticipated that this lightweight thread subsystem would become a library in some future DPDK release.

A simple pthread shim in the form of a hello world example is also included.

Quoted 8 lines

Ian Betts (5):
doc: add performance-thread sample application guide
examples: add cooperative scheduler subsytem for
performance-thread app
examples: add l3fwd-thread in performance-thread sample app
examples: add pthread-shim in performance-thread sample app
config: add build files for performance-thread

Series Acked-by: Tomasz Kulasek <[email protected]>