Skip to contentSkip to conversation

Add an option to use LTO for DPDK build

PATCH In review Andrzej Ostruszka 13 participants 110 messages Sep 17, 2019 – Nov 8, 2019 2 messages not in the archive

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v2 cover Add an option to use LTO for DPDK build

This patch series adds an option to make use of link time optimization (if compiler has support for it). It is split as follows:
- 1st patch (build) is the enablement
- remaining patches are fixes for the warnings produced by the compiler and they are split by directory/subsystem so their maintainers can easily find and verify the changes - please note that there are two groups:
* errors (or possible errors) - with title "fix possible use ..."
* false positives - warnings that _I_ think are not valid and the changes are made only to silence the compiler.

v2 Changes:
-----------
- fixed building of shared libraries:
- gcc does not scan top level assembler statements so it missed that function implementations for specific versions were being used and was removing them
- fixed meson.build config files:
- moved from 'enable_lto' project option to built-in 'b_lto'
- documented that 'default_library' should be 'shared':
with 'default_library=static' (the default) the meson build with LTO is broken - this is because, SHARED_LIB is fixed in rte_config.h used by meson which leads to no alias for default version in generated static libraries (MAP_STATIC_SYMBOL() is empty)
- app/test: added log for failed bonding "config get"

Andrzej Ostruszka (10):
build: add an option to enable LTO build
eventdev: fix possible use of uninitialized var app/eventdev: fix maybe-uninitialized warnings for LTO build event/octeontx2: fix maybe-uninitialized warnings for LTO build app/test: fix maybe-uninitialized warnings for LTO build net/dpaa2: fix possible use of uninitialized vars net/e1000: fix maybe-uninitialized warnings for LTO build net/i40e: fix maybe-uninitialized warnings for LTO build net/ifc: fix maybe-uninitialized warnings for LTO build net/qede: fix maybe-uninitialized warnings for LTO build

 .travis.yml                                   |  7 ++++
 app/test-eventdev/test_perf_common.c          |  2 +-
 app/test-eventdev/test_pipeline_common.c      |  4 +-
 app/test/test_hash_readwrite.c                |  2 +-
 app/test/test_link_bonding_mode4.c            | 10 ++++-
 app/test/test_memzone.c                       |  3 +-
 config/common_base                            |  5 +++
 config/meson.build                            | 15 ++++++++
 doc/guides/prog_guide/lto.rst                 | 37 +++++++++++++++++++
 doc/guides/rel_notes/release_19_11.rst        |  8 ++++
 drivers/event/octeontx2/otx2_tim_worker.h     |  2 +-
 drivers/net/dpaa2/base/dpaa2_hw_dpni.c        |  1 +
 drivers/net/dpaa2/mc/dpkg.c                   |  2 +-
 drivers/net/dpaa2/mc/dpni.c                   |  9 +++--
 drivers/net/e1000/base/e1000_82543.c          |  2 +-
 drivers/net/e1000/base/e1000_ich8lan.c        |  2 +-
 drivers/net/e1000/base/e1000_phy.c            |  2 +-
 drivers/net/i40e/i40e_ethdev.c                |  2 +-
 drivers/net/ifc/ifcvf_vdpa.c                  | 14 ++++---
 drivers/net/qede/base/ecore_mcp.c             | 13 ++++---
 lib/librte_distributor/rte_distributor.c      | 18 ++++-----
 lib/librte_distributor/rte_distributor_v20.c  | 18 ++++-----
 lib/librte_eventdev/rte_event_timer_adapter.c |  8 ++--
 lib/librte_lpm/rte_lpm.c                      | 28 +++++++-------
 lib/librte_lpm/rte_lpm6.c                     | 16 ++++----
 lib/librte_timer/rte_timer.c                  | 20 +++++-----
 mk/toolchain/clang/rte.toolchain-compat.mk    |  4 ++
 mk/toolchain/clang/rte.vars.mk                |  8 ++++
 mk/toolchain/gcc/rte.toolchain-compat.mk      |  4 ++
 mk/toolchain/gcc/rte.vars.mk                  | 12 ++++++
 mk/toolchain/icc/rte.vars.mk                  |  8 ++++
 31 files changed, 205 insertions(+), 81 deletions(-)
 create mode 100644 doc/guides/prog_guide/lto.rst
Signature · 1 line
2.17.1
#1 lore raw

PATCH v2 1/10 build: add an option to enable LTO build

This patch adds an option to enable link time optimization. In addition to LTO option itself (-flto) fat-lto-objects are being used. This is because during the build pmdinfogen scans the generated ELF objects to find this_pmd_name* symbol in symbol table. Without fat-lto-objects gcc produces ELF only with extra symbols for internal use during linking and clang does not produce ELF at all (only LLVM IR bitcode).

---
 .travis.yml                                  |  7 ++++
 config/common_base                           |  5 +++
 config/meson.build                           | 15 ++++++++
 doc/guides/prog_guide/lto.rst                | 37 ++++++++++++++++++++
 doc/guides/rel_notes/release_19_11.rst       |  8 +++++
 lib/librte_distributor/rte_distributor.c     | 18 +++++-----
 lib/librte_distributor/rte_distributor_v20.c | 18 +++++-----
 lib/librte_lpm/rte_lpm.c                     | 28 +++++++--------
 lib/librte_lpm/rte_lpm6.c                    | 16 ++++-----
 lib/librte_timer/rte_timer.c                 | 20 +++++------
 mk/toolchain/clang/rte.toolchain-compat.mk   |  4 +++
 mk/toolchain/clang/rte.vars.mk               |  8 +++++
 mk/toolchain/gcc/rte.toolchain-compat.mk     |  4 +++
 mk/toolchain/gcc/rte.vars.mk                 | 12 +++++++
 mk/toolchain/icc/rte.vars.mk                 |  8 +++++
 15 files changed, 158 insertions(+), 50 deletions(-)
 create mode 100644 doc/guides/prog_guide/lto.rst
.travis.yml +7 -0
index 781f9f666..70d221852 100644
@@ -31,6 +31,7 @@ env:
   - DEF_LIB="static" OPTS="-Denable_kmods=false"
   - DEF_LIB="shared" OPTS="-Denable_kmods=false"
   - DEF_LIB="shared" RUN_TESTS=1 BUILD_DOCS=1
+  - DEF_LIB="shared" OPTS="-Db_lto=true"
 
 matrix:
   include:
@@ -100,6 +101,12 @@ matrix:
       apt:
         packages:
           - *extra_packages
+  - env: DEF_LIB="shared" OPTS="-Db_lto=true" EXTRA_PACKAGES=1
+    compiler: gcc
+    addons:
+      apt:
+        packages:
+          - *extra_packages
 
 
 script: ./.ci/${TRAVIS_OS_NAME}-build.sh
config/common_base +5 -0
index 8ef75c203..73a55fdec 100644
@@ -49,6 +49,11 @@ CONFIG_RTE_FORCE_INTRINSICS=n
 #
 CONFIG_RTE_ARCH_STRICT_ALIGN=n
 
+#
+# Enable link time optimization
+#
+CONFIG_RTE_ENABLE_LTO=n
+
 #
 # Compile to share library
 #
config/meson.build +15 -0
index 2bafea530..97bbc323b 100644
@@ -196,3 +196,18 @@ add_project_arguments('-D_GNU_SOURCE', language: 'c')
 if is_freebsd
 	add_project_arguments('-D__BSD_VISIBLE', language: 'c')
 endif
+
+if get_option('b_lto')
+	if cc.has_argument('-ffat-lto-objects')
+		add_project_arguments('-ffat-lto-objects', language: 'c')
+	else
+		error('compiler does not support fat LTO objects - please turn LTO off')
+	endif
+	if cc.get_id() == 'gcc'
+		# workaround for bug 81440
+		if cc.version().version_compare('<8.0')
+			add_project_arguments('-Wno-lto-type-mismatch', language: 'c')
+			add_project_link_arguments('-Wno-lto-type-mismatch', language: 'c')
+		endif
+	endif
+endif
doc/guides/prog_guide/lto.rst new file +37 -0
new file mode 100644
index 000000000..b2b36e51c
@@ -0,0 +1,37 @@
+Link Time Optimization
+======================
+
+The DPDK framework supports compilation with link time optimization
+turned on.  This depends obviously on the capabilities of the compiler
+to do "whole program" optimization at link time and is available only
+for compilers that support that feature (gcc, clang and icc).  To be
+more specific compiler have to support creation of ELF objects
+containing both normal code and internal representation
+(fat-lto-objects).  This is required since during build some code is
+generated by parsing produced ELF objects (pmdinfogen).
+
+The amount of performance gain that one can get from LTO depends on the
+compiler and the code that is being compiled.  However LTO is also
+useful for additional code analysis done by the compiler.  In particular
+due to interprocedural analysis compiler can produce additional warnings
+about variables that might be used uninitialized.  Some of these
+warnings might be "false positives" though and you might need to
+explicitly initialize variable in order to silence the compiler.
+
+Link time optimization can be enabled for whole DPDK framework by
+setting:
+
+.. code-block:: console
+    CONFIG_ENABLE_LTO=y
+
+in config file for the case of make based build and by:
+
+.. code-block:: console
+    meson build -Db_lto=true -Ddefault_library=shared
+    ninja -C build
+
+for the case of meson based build (only shared libraries are supported
+when building with meson and LTO enabled).
+
+Please note that turning LTO on causes considerable extension of
+compilation time.
doc/guides/rel_notes/release_19_11.rst +8 -0
index 8490d897c..97b4f4083 100644
@@ -56,6 +56,14 @@ New Features
      Also, make sure to start the actual text at the margin.
      =========================================================
 
+**Added build support for Link Time Optimization.**
+
+ LTO is an optimization technique used by the compiler to perform whole
+ program analysis and optimization at link time.  In order to do that
+ compilers store their internal representation of the source code that
+ the linker uses at the final stage of compilation process.
+
+ See :doc:`../prog_guide/lto` for more information:
 
 Removed Items
 -------------
lib/librte_distributor/rte_distributor.c +9 -9
index 21eb1fb0a..848250f4a 100644
@@ -32,7 +32,7 @@ EAL_REGISTER_TAILQ(rte_dist_burst_tailq)
 
 /**** Burst Packet APIs called by workers ****/
 
-void
+void __vsym
 rte_distributor_request_pkt_v1705(struct rte_distributor *d,
 		unsigned int worker_id, struct rte_mbuf **oldpkt,
 		unsigned int count)
@@ -84,7 +84,7 @@ MAP_STATIC_SYMBOL(void rte_distributor_request_pkt(struct rte_distributor *d,
 		unsigned int count),
 		rte_distributor_request_pkt_v1705);
 
-int
+int __vsym
 rte_distributor_poll_pkt_v1705(struct rte_distributor *d,
 		unsigned int worker_id, struct rte_mbuf **pkts)
 {
@@ -124,7 +124,7 @@ MAP_STATIC_SYMBOL(int rte_distributor_poll_pkt(struct rte_distributor *d,
 		unsigned int worker_id, struct rte_mbuf **pkts),
 		rte_distributor_poll_pkt_v1705);
 
-int
+int __vsym
 rte_distributor_get_pkt_v1705(struct rte_distributor *d,
 		unsigned int worker_id, struct rte_mbuf **pkts,
 		struct rte_mbuf **oldpkt, unsigned int return_count)
@@ -159,7 +159,7 @@ MAP_STATIC_SYMBOL(int rte_distributor_get_pkt(struct rte_distributor *d,
 		struct rte_mbuf **oldpkt, unsigned int return_count),
 		rte_distributor_get_pkt_v1705);
 
-int
+int __vsym
 rte_distributor_return_pkt_v1705(struct rte_distributor *d,
 		unsigned int worker_id, struct rte_mbuf **oldpkt, int num)
 {
@@ -335,7 +335,7 @@ release(struct rte_distributor *d, unsigned int wkr)
 
 
 /* process a set of packets to distribute them to workers */
-int
+int __vsym
 rte_distributor_process_v1705(struct rte_distributor *d,
 		struct rte_mbuf **mbufs, unsigned int num_mbufs)
 {
@@ -476,7 +476,7 @@ MAP_STATIC_SYMBOL(int rte_distributor_process(struct rte_distributor *d,
 		rte_distributor_process_v1705);
 
 /* return to the caller, packets returned from workers */
-int
+int __vsym
 rte_distributor_returned_pkts_v1705(struct rte_distributor *d,
 		struct rte_mbuf **mbufs, unsigned int max_mbufs)
 {
@@ -526,7 +526,7 @@ total_outstanding(const struct rte_distributor *d)
  * Flush the distributor, so that there are no outstanding packets in flight or
  * queued up.
  */
-int
+int __vsym
 rte_distributor_flush_v1705(struct rte_distributor *d)
 {
 	unsigned int flushed;
@@ -561,7 +561,7 @@ MAP_STATIC_SYMBOL(int rte_distributor_flush(struct rte_distributor *d),
 		rte_distributor_flush_v1705);
 
 /* clears the internal returns array in the distributor */
-void
+void __vsym
 rte_distributor_clear_returns_v1705(struct rte_distributor *d)
 {
 	unsigned int wkr;
@@ -581,7 +581,7 @@ MAP_STATIC_SYMBOL(void rte_distributor_clear_returns(struct rte_distributor *d),
 		rte_distributor_clear_returns_v1705);
 
 /* creates a distributor instance */
-struct rte_distributor *
+struct rte_distributor * __vsym
 rte_distributor_create_v1705(const char *name,
 		unsigned int socket_id,
 		unsigned int num_workers,
lib/librte_distributor/rte_distributor_v20.c +9 -9
index cdc0969a8..31c766421 100644
@@ -27,7 +27,7 @@ EAL_REGISTER_TAILQ(rte_distributor_tailq)
 
 /**** APIs called by workers ****/
 
-void
+void __vsym
 rte_distributor_request_pkt_v20(struct rte_distributor_v20 *d,
 		unsigned worker_id, struct rte_mbuf *oldpkt)
 {
@@ -40,7 +40,7 @@ rte_distributor_request_pkt_v20(struct rte_distributor_v20 *d,
 }
 VERSION_SYMBOL(rte_distributor_request_pkt, _v20, 2.0);
 
-struct rte_mbuf *
+struct rte_mbuf * __vsym
 rte_distributor_poll_pkt_v20(struct rte_distributor_v20 *d,
 		unsigned worker_id)
 {
@@ -54,7 +54,7 @@ rte_distributor_poll_pkt_v20(struct rte_distributor_v20 *d,
 }
 VERSION_SYMBOL(rte_distributor_poll_pkt, _v20, 2.0);
 
-struct rte_mbuf *
+struct rte_mbuf * __vsym
 rte_distributor_get_pkt_v20(struct rte_distributor_v20 *d,
 		unsigned worker_id, struct rte_mbuf *oldpkt)
 {
@@ -66,7 +66,7 @@ rte_distributor_get_pkt_v20(struct rte_distributor_v20 *d,
 }
 VERSION_SYMBOL(rte_distributor_get_pkt, _v20, 2.0);
 
-int
+int __vsym
 rte_distributor_return_pkt_v20(struct rte_distributor_v20 *d,
 		unsigned worker_id, struct rte_mbuf *oldpkt)
 {
@@ -191,7 +191,7 @@ process_returns(struct rte_distributor_v20 *d)
 }
 
 /* process a set of packets to distribute them to workers */
-int
+int __vsym
 rte_distributor_process_v20(struct rte_distributor_v20 *d,
 		struct rte_mbuf **mbufs, unsigned num_mbufs)
 {
@@ -296,7 +296,7 @@ rte_distributor_process_v20(struct rte_distributor_v20 *d,
 VERSION_SYMBOL(rte_distributor_process, _v20, 2.0);
 
 /* return to the caller, packets returned from workers */
-int
+int __vsym
 rte_distributor_returned_pkts_v20(struct rte_distributor_v20 *d,
 		struct rte_mbuf **mbufs, unsigned max_mbufs)
 {
@@ -334,7 +334,7 @@ total_outstanding(const struct rte_distributor_v20 *d)
 
 /* flush the distributor, so that there are no outstanding packets in flight or
  * queued up. */
-int
+int __vsym
 rte_distributor_flush_v20(struct rte_distributor_v20 *d)
 {
 	const unsigned flushed = total_outstanding(d);
@@ -347,7 +347,7 @@ rte_distributor_flush_v20(struct rte_distributor_v20 *d)
 VERSION_SYMBOL(rte_distributor_flush, _v20, 2.0);
 
 /* clears the internal returns array in the distributor */
-void
+void __vsym
 rte_distributor_clear_returns_v20(struct rte_distributor_v20 *d)
 {
 	d->returns.start = d->returns.count = 0;
@@ -358,7 +358,7 @@ rte_distributor_clear_returns_v20(struct rte_distributor_v20 *d)
 VERSION_SYMBOL(rte_distributor_clear_returns, _v20, 2.0);
 
 /* creates a distributor instance */
-struct rte_distributor_v20 *
+struct rte_distributor_v20 * __vsym
 rte_distributor_create_v20(const char *name,
 		unsigned socket_id,
 		unsigned num_workers)
lib/librte_lpm/rte_lpm.c +14 -14
index 3a929a1b1..a2fba8d61 100644
@@ -89,7 +89,7 @@ depth_to_range(uint8_t depth)
 /*
  * Find an existing lpm table and return a pointer to it.
  */
-struct rte_lpm_v20 *
+struct rte_lpm_v20 * __vsym
 rte_lpm_find_existing_v20(const char *name)
 {
 	struct rte_lpm_v20 *l = NULL;
@@ -115,7 +115,7 @@ rte_lpm_find_existing_v20(const char *name)
 }
 VERSION_SYMBOL(rte_lpm_find_existing, _v20, 2.0);
 
-struct rte_lpm *
+struct rte_lpm * __vsym
 rte_lpm_find_existing_v1604(const char *name)
 {
 	struct rte_lpm *l = NULL;
@@ -146,7 +146,7 @@ MAP_STATIC_SYMBOL(struct rte_lpm *rte_lpm_find_existing(const char *name),
 /*
  * Allocates memory for LPM object
  */
-struct rte_lpm_v20 *
+struct rte_lpm_v20 * __vsym
 rte_lpm_create_v20(const char *name, int socket_id, int max_rules,
 		__rte_unused int flags)
 {
@@ -219,7 +219,7 @@ rte_lpm_create_v20(const char *name, int socket_id, int max_rules,
 }
 VERSION_SYMBOL(rte_lpm_create, _v20, 2.0);
 
-struct rte_lpm *
+struct rte_lpm * __vsym
 rte_lpm_create_v1604(const char *name, int socket_id,
 		const struct rte_lpm_config *config)
 {
@@ -328,7 +328,7 @@ MAP_STATIC_SYMBOL(
 /*
  * Deallocates memory for given LPM table.
  */
-void
+void __vsym
 rte_lpm_free_v20(struct rte_lpm_v20 *lpm)
 {
 	struct rte_lpm_list *lpm_list;
@@ -357,7 +357,7 @@ rte_lpm_free_v20(struct rte_lpm_v20 *lpm)
 }
 VERSION_SYMBOL(rte_lpm_free, _v20, 2.0);
 
-void
+void __vsym
 rte_lpm_free_v1604(struct rte_lpm *lpm)
 {
 	struct rte_lpm_list *lpm_list;
@@ -1176,7 +1176,7 @@ add_depth_big_v1604(struct rte_lpm *lpm, uint32_t ip_masked, uint8_t depth,
 /*
  * Add a route
  */
-int
+int __vsym
 rte_lpm_add_v20(struct rte_lpm_v20 *lpm, uint32_t ip, uint8_t depth,
 		uint8_t next_hop)
 {
@@ -1217,7 +1217,7 @@ rte_lpm_add_v20(struct rte_lpm_v20 *lpm, uint32_t ip, uint8_t depth,
 }
 VERSION_SYMBOL(rte_lpm_add, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm_add_v1604(struct rte_lpm *lpm, uint32_t ip, uint8_t depth,
 		uint32_t next_hop)
 {
@@ -1263,7 +1263,7 @@ MAP_STATIC_SYMBOL(int rte_lpm_add(struct rte_lpm *lpm, uint32_t ip,
 /*
  * Look for a rule in the high-level rules table
  */
-int
+int __vsym
 rte_lpm_is_rule_present_v20(struct rte_lpm_v20 *lpm, uint32_t ip, uint8_t depth,
 uint8_t *next_hop)
 {
@@ -1290,7 +1290,7 @@ uint8_t *next_hop)
 }
 VERSION_SYMBOL(rte_lpm_is_rule_present, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm_is_rule_present_v1604(struct rte_lpm *lpm, uint32_t ip, uint8_t depth,
 uint32_t *next_hop)
 {
@@ -1843,7 +1843,7 @@ delete_depth_big_v1604(struct rte_lpm *lpm, uint32_t ip_masked,
 /*
  * Deletes a rule
  */
-int
+int __vsym
 rte_lpm_delete_v20(struct rte_lpm_v20 *lpm, uint32_t ip, uint8_t depth)
 {
 	int32_t rule_to_delete_index, sub_rule_index;
@@ -1897,7 +1897,7 @@ rte_lpm_delete_v20(struct rte_lpm_v20 *lpm, uint32_t ip, uint8_t depth)
 }
 VERSION_SYMBOL(rte_lpm_delete, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm_delete_v1604(struct rte_lpm *lpm, uint32_t ip, uint8_t depth)
 {
 	int32_t rule_to_delete_index, sub_rule_index;
@@ -1956,7 +1956,7 @@ MAP_STATIC_SYMBOL(int rte_lpm_delete(struct rte_lpm *lpm, uint32_t ip,
 /*
  * Delete all rules from the LPM table.
  */
-void
+void __vsym
 rte_lpm_delete_all_v20(struct rte_lpm_v20 *lpm)
 {
 	/* Zero rule information. */
@@ -1973,7 +1973,7 @@ rte_lpm_delete_all_v20(struct rte_lpm_v20 *lpm)
 }
 VERSION_SYMBOL(rte_lpm_delete_all, _v20, 2.0);
 
-void
+void __vsym
 rte_lpm_delete_all_v1604(struct rte_lpm *lpm)
 {
 	/* Zero rule information. */
lib/librte_lpm/rte_lpm6.c +8 -8
index 9b8aeb972..49a7fea1d 100644
@@ -811,7 +811,7 @@ add_step(struct rte_lpm6 *lpm, struct rte_lpm6_tbl_entry *tbl,
 /*
  * Add a route
  */
-int
+int __vsym
 rte_lpm6_add_v20(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
 		uint8_t next_hop)
 {
@@ -861,7 +861,7 @@ simulate_add(struct rte_lpm6 *lpm, const uint8_t *masked_ip, uint8_t depth)
 	return 0;
 }
 
-int
+int __vsym
 rte_lpm6_add_v1705(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
 		uint32_t next_hop)
 {
@@ -954,7 +954,7 @@ lookup_step(const struct rte_lpm6 *lpm, const struct rte_lpm6_tbl_entry *tbl,
 /*
  * Looks up an IP
  */
-int
+int __vsym
 rte_lpm6_lookup_v20(const struct rte_lpm6 *lpm, uint8_t *ip, uint8_t *next_hop)
 {
 	uint32_t next_hop32 = 0;
@@ -972,7 +972,7 @@ rte_lpm6_lookup_v20(const struct rte_lpm6 *lpm, uint8_t *ip, uint8_t *next_hop)
 }
 VERSION_SYMBOL(rte_lpm6_lookup, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm6_lookup_v1705(const struct rte_lpm6 *lpm, uint8_t *ip,
 		uint32_t *next_hop)
 {
@@ -1007,7 +1007,7 @@ MAP_STATIC_SYMBOL(int rte_lpm6_lookup(const struct rte_lpm6 *lpm, uint8_t *ip,
 /*
  * Looks up a group of IP addresses
  */
-int
+int __vsym
 rte_lpm6_lookup_bulk_func_v20(const struct rte_lpm6 *lpm,
 		uint8_t ips[][RTE_LPM6_IPV6_ADDR_SIZE],
 		int16_t * next_hops, unsigned n)
@@ -1048,7 +1048,7 @@ rte_lpm6_lookup_bulk_func_v20(const struct rte_lpm6 *lpm,
 }
 VERSION_SYMBOL(rte_lpm6_lookup_bulk_func, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm6_lookup_bulk_func_v1705(const struct rte_lpm6 *lpm,
 		uint8_t ips[][RTE_LPM6_IPV6_ADDR_SIZE],
 		int32_t *next_hops, unsigned int n)
@@ -1098,7 +1098,7 @@ MAP_STATIC_SYMBOL(int rte_lpm6_lookup_bulk_func(const struct rte_lpm6 *lpm,
 /*
  * Look for a rule in the high-level rules table
  */
-int
+int __vsym
 rte_lpm6_is_rule_present_v20(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
 		uint8_t *next_hop)
 {
@@ -1118,7 +1118,7 @@ rte_lpm6_is_rule_present_v20(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
 }
 VERSION_SYMBOL(rte_lpm6_is_rule_present, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm6_is_rule_present_v1705(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
 		uint32_t *next_hop)
 {
lib/librte_timer/rte_timer.c +10 -10
index bdcf05d06..e560ace06 100644
@@ -131,7 +131,7 @@ rte_timer_data_dealloc(uint32_t id)
 	return 0;
 }
 
-void
+void __vsym
 rte_timer_subsystem_init_v20(void)
 {
 	unsigned lcore_id;
@@ -153,7 +153,7 @@ VERSION_SYMBOL(rte_timer_subsystem_init, _v20, 2.0);
  * secondary processes should be empty, the zeroth entry can be shared by
  * multiple processes.
  */
-int
+int __vsym
 rte_timer_subsystem_init_v1905(void)
 {
 	const struct rte_memzone *mz;
@@ -551,7 +551,7 @@ __rte_timer_reset(struct rte_timer *tim, uint64_t expire,
 }
 
 /* Reset and start the timer associated with the timer handle tim */
-int
+int __vsym
 rte_timer_reset_v20(struct rte_timer *tim, uint64_t ticks,
 		    enum rte_timer_type type, unsigned int tim_lcore,
 		    rte_timer_cb_t fct, void *arg)
@@ -574,7 +574,7 @@ rte_timer_reset_v20(struct rte_timer *tim, uint64_t ticks,
 }
 VERSION_SYMBOL(rte_timer_reset, _v20, 2.0);
 
-int
+int __vsym
 rte_timer_reset_v1905(struct rte_timer *tim, uint64_t ticks,
 		      enum rte_timer_type type, unsigned int tim_lcore,
 		      rte_timer_cb_t fct, void *arg)
@@ -657,14 +657,14 @@ __rte_timer_stop(struct rte_timer *tim, int local_is_locked,
 }
 
 /* Stop the timer associated with the timer handle tim */
-int
+int __vsym
 rte_timer_stop_v20(struct rte_timer *tim)
 {
 	return __rte_timer_stop(tim, 0, &default_timer_data);
 }
 VERSION_SYMBOL(rte_timer_stop, _v20, 2.0);
 
-int
+int __vsym
 rte_timer_stop_v1905(struct rte_timer *tim)
 {
 	return rte_timer_alt_stop(default_data_id, tim);
@@ -817,14 +817,14 @@ __rte_timer_manage(struct rte_timer_data *timer_data)
 	priv_timer[lcore_id].running_tim = NULL;
 }
 
-void
+void __vsym
 rte_timer_manage_v20(void)
 {
 	__rte_timer_manage(&default_timer_data);
 }
 VERSION_SYMBOL(rte_timer_manage, _v20, 2.0);
 
-int
+int __vsym
 rte_timer_manage_v1905(void)
 {
 	struct rte_timer_data *timer_data;
@@ -1074,14 +1074,14 @@ __rte_timer_dump_stats(struct rte_timer_data *timer_data __rte_unused, FILE *f)
 #endif
 }
 
-void
+void __vsym
 rte_timer_dump_stats_v20(FILE *f)
 {
 	__rte_timer_dump_stats(&default_timer_data, f);
 }
 VERSION_SYMBOL(rte_timer_dump_stats, _v20, 2.0);
 
-int
+int __vsym
 rte_timer_dump_stats_v1905(FILE *f)
 {
 	return rte_timer_alt_dump_stats(default_data_id, f);
mk/toolchain/clang/rte.toolchain-compat.mk +4 -0
index e6189b498..78f96c648 100644
@@ -20,3 +20,7 @@ CLANG_MINOR_VERSION := $(shell echo $(CLANG_VERSION) | cut -f2 -d.)
 ifeq ($(shell test $(CLANG_MAJOR_VERSION)$(CLANG_MINOR_VERSION) -lt 35 && echo 1), 1)
 	CC_SUPPORTS_Z := false
 endif
+
+ifeq ($(shell test $(CLANG_MAJOR_VERSION)$(CLANG_MINOR_VERSION) -lt 60 && echo 1), 1)
+	CONFIG_RTE_ENABLE_LTO=n
+endif
mk/toolchain/clang/rte.vars.mk +8 -0
index 3c49dc568..3b1fa05f9 100644
@@ -48,6 +48,14 @@ endif
 # process cpu flags
 include $(RTE_SDK)/mk/toolchain/$(RTE_TOOLCHAIN)/rte.toolchain-compat.mk
 
+ifeq ($(CONFIG_RTE_ENABLE_LTO),y)
+# 'fat-lto' is used since pmdinfogen needs to have 'this_pmd_nameX'
+# exported in symbol table and without this option only internal
+# representation is present.
+TOOLCHAIN_CFLAGS += -flto -ffat-lto-objects
+TOOLCHAIN_LDFLAGS += -flto
+endif
+
 # workaround clang bug with warning "missing field initializer" for "= {0}"
 WERROR_FLAGS += -Wno-missing-field-initializers
 
mk/toolchain/gcc/rte.toolchain-compat.mk +4 -0
index ea40a11c0..ad4fad83c 100644
@@ -88,6 +88,10 @@ else
 		MACHINE_CFLAGS := $(filter-out -march% -mtune% -msse%,$(MACHINE_CFLAGS))
 	endif
 
+	ifeq ($(shell test $(GCC_VERSION) -lt 45 && echo 1), 1)
+		CONFIG_RTE_ENABLE_LTO=n
+	endif
+
 	# Disable thunderx PMD for gcc < 4.7
 	ifeq ($(shell test $(GCC_VERSION) -lt 47 && echo 1), 1)
 		CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD=d
mk/toolchain/gcc/rte.vars.mk +12 -0
index b852fcfd7..9fc704193 100644
@@ -62,6 +62,18 @@ endif
 # process cpu flags
 include $(RTE_SDK)/mk/toolchain/$(RTE_TOOLCHAIN)/rte.toolchain-compat.mk
 
+ifeq ($(CONFIG_RTE_ENABLE_LTO),y)
+# 'fat-lto' is used since pmdinfogen needs to have 'this_pmd_nameX'
+# exported in symbol table and without this option only internal
+# representation is present.
+TOOLCHAIN_CFLAGS += -flto -ffat-lto-objects
+TOOLCHAIN_LDFLAGS += -flto
+# workaround for GCC bug 81440
+ifeq ($(shell test $(GCC_VERSION) -lt 80 && echo 1), 1)
+WERROR_FLAGS += -Wno-lto-type-mismatch
+endif
+endif
+
 # workaround GCC bug with warning "missing initializer" for "= {0}"
 ifeq ($(shell test $(GCC_VERSION) -lt 47 && echo 1), 1)
 WERROR_FLAGS += -Wno-missing-field-initializers
mk/toolchain/icc/rte.vars.mk +8 -0
index aa1422bf1..8aa87aa1e 100644
@@ -54,5 +54,13 @@ endif
 # process cpu flags
 include $(RTE_SDK)/mk/toolchain/$(RTE_TOOLCHAIN)/rte.toolchain-compat.mk
 
+ifeq ($(CONFIG_RTE_ENABLE_LTO),y)
+# 'fat-lto' is used since pmdinfogen needs to have 'this_pmd_nameX'
+# exported in symbol table and without this option only internal
+# representation is present.
+TOOLCHAIN_CFLAGS += -flto -ffat-lto-objects
+TOOLCHAIN_LDFLAGS += -flto
+endif
+
 export CC AS AR LD OBJCOPY OBJDUMP STRIP READELF
 export TOOLCHAIN_CFLAGS TOOLCHAIN_LDFLAGS TOOLCHAIN_ASFLAGS
Signature · 1 line
2.17.1
Acked-by on v2 1/10
#2 lore raw

Re: [dpdk-dev] [PATCH v2 01/10] build: add an option to enable LTO build

Quoted 9 lines · Andrzej Ostruszka

This patch adds an option to enable link time optimization. In addition to LTO option itself (-flto) fat-lto-objects are being used. This is because during the build pmdinfogen scans the generated ELF objects to find this_pmd_name* symbol in symbol table. Without fat-lto-objects gcc produces ELF only with extra symbols for internal use during linking and clang does not produce ELF at all (only LLVM IR bitcode).

For meson changes part:

#3 lore raw

Re: [dpdk-dev] [PATCH v2 01/10] build: add an option to enable LTO build

In reply to Andrzej Ostruszka

this is cool, good work.
comments below.

Quoted 58 lines · Andrzej Ostruszka

This patch adds an option to enable link time optimization. In addition to LTO option itself (-flto) fat-lto-objects are being used. This is because during the build pmdinfogen scans the generated ELF objects to find this_pmd_name* symbol in symbol table. Without fat-lto-objects gcc produces ELF only with extra symbols for internal use during linking and clang does not produce ELF at all (only LLVM IR bitcode).

---
 .travis.yml                                  |  7 ++++
 config/common_base                           |  5 +++
 config/meson.build                           | 15 ++++++++
 doc/guides/prog_guide/lto.rst                | 37 ++++++++++++++++++++
 doc/guides/rel_notes/release_19_11.rst       |  8 +++++
 lib/librte_distributor/rte_distributor.c     | 18 +++++-----
 lib/librte_distributor/rte_distributor_v20.c | 18 +++++-----
 lib/librte_lpm/rte_lpm.c                     | 28 +++++++--------
 lib/librte_lpm/rte_lpm6.c                    | 16 ++++-----
 lib/librte_timer/rte_timer.c                 | 20 +++++------
 mk/toolchain/clang/rte.toolchain-compat.mk   |  4 +++
 mk/toolchain/clang/rte.vars.mk               |  8 +++++
 mk/toolchain/gcc/rte.toolchain-compat.mk     |  4 +++
 mk/toolchain/gcc/rte.vars.mk                 | 12 +++++++
 mk/toolchain/icc/rte.vars.mk                 |  8 +++++
 15 files changed, 158 insertions(+), 50 deletions(-)
 create mode 100644 doc/guides/prog_guide/lto.rst
.travis.yml +7 -0
index 781f9f666..70d221852 100644
@@ -31,6 +31,7 @@ env:
   - DEF_LIB="static" OPTS="-Denable_kmods=false"
   - DEF_LIB="shared" OPTS="-Denable_kmods=false"
   - DEF_LIB="shared" RUN_TESTS=1 BUILD_DOCS=1
+  - DEF_LIB="shared" OPTS="-Db_lto=true"
 
 matrix:
   include:
@@ -100,6 +101,12 @@ matrix:
       apt:
         packages:
           - *extra_packages
+  - env: DEF_LIB="shared" OPTS="-Db_lto=true" EXTRA_PACKAGES=1
+    compiler: gcc
+    addons:
+      apt:
+        packages:
+          - *extra_packages
 
 
 script: ./.ci/${TRAVIS_OS_NAME}-build.sh
config/common_base +0 -0
index 8ef75c203..73a55fdec 100644
@@ -49,6 +49,11 @@ CONFIG_RTE_FORCE_INTRINSICS=n
 #

config/common_base CONFIG_RTE_FORCE_INTRINSICS=n · line 59

Quoted 10 lines
config/common_base +5 -0
 CONFIG_RTE_ARCH_STRICT_ALIGN=n
 
+#
+# Enable link time optimization
+#
+CONFIG_RTE_ENABLE_LTO=n
+
 #
 # Compile to share library
 #

Why would we make this optional in this way and expand the matrix of different ways to build DPDK. To ask another way, why wouldn't a user turn on GSO.

Quoted 90 lines
config/meson.build +15 -0
index 2bafea530..97bbc323b 100644
@@ -196,3 +196,18 @@ add_project_arguments('-D_GNU_SOURCE', language: 'c')
 if is_freebsd
 	add_project_arguments('-D__BSD_VISIBLE', language: 'c')
 endif
+
+if get_option('b_lto')
+	if cc.has_argument('-ffat-lto-objects')
+		add_project_arguments('-ffat-lto-objects', language: 'c')
+	else
+		error('compiler does not support fat LTO objects - please turn LTO off')
+	endif
+	if cc.get_id() == 'gcc'
+		# workaround for bug 81440
+		if cc.version().version_compare('<8.0')
+			add_project_arguments('-Wno-lto-type-mismatch', language: 'c')
+			add_project_link_arguments('-Wno-lto-type-mismatch', language: 'c')
+		endif
+	endif
+endif
doc/guides/prog_guide/lto.rst new file +37 -0
new file mode 100644
index 000000000..b2b36e51c
@@ -0,0 +1,37 @@
+Link Time Optimization
+======================
+
+The DPDK framework supports compilation with link time optimization
+turned on.  This depends obviously on the capabilities of the compiler
+to do "whole program" optimization at link time and is available only
+for compilers that support that feature (gcc, clang and icc).  To be
+more specific compiler have to support creation of ELF objects
+containing both normal code and internal representation
+(fat-lto-objects).  This is required since during build some code is
+generated by parsing produced ELF objects (pmdinfogen).
+
+The amount of performance gain that one can get from LTO depends on the
+compiler and the code that is being compiled.  However LTO is also
+useful for additional code analysis done by the compiler.  In particular
+due to interprocedural analysis compiler can produce additional warnings
+about variables that might be used uninitialized.  Some of these
+warnings might be "false positives" though and you might need to
+explicitly initialize variable in order to silence the compiler.
+
+Link time optimization can be enabled for whole DPDK framework by
+setting:
+
+.. code-block:: console
+    CONFIG_ENABLE_LTO=y
+
+in config file for the case of make based build and by:
+
+.. code-block:: console
+    meson build -Db_lto=true -Ddefault_library=shared
+    ninja -C build
+
+for the case of meson based build (only shared libraries are supported
+when building with meson and LTO enabled).
+
+Please note that turning LTO on causes considerable extension of
+compilation time.
doc/guides/rel_notes/release_19_11.rst +8 -0
index 8490d897c..97b4f4083 100644
@@ -56,6 +56,14 @@ New Features
      Also, make sure to start the actual text at the margin.
      =========================================================
 
+**Added build support for Link Time Optimization.**
+
+ LTO is an optimization technique used by the compiler to perform whole
+ program analysis and optimization at link time.  In order to do that
+ compilers store their internal representation of the source code that
+ the linker uses at the final stage of compilation process.
+
+ See :doc:`../prog_guide/lto` for more information:
 
 Removed Items
 -------------
lib/librte_distributor/rte_distributor.c +0 -0
index 21eb1fb0a..848250f4a 100644

lib/librte_distributor/rte_distributor.c EAL_REGISTER_TAILQ(rte_dist_burst_tailq) · line 35

Quoted 5 lines
lib/librte_distributor/rte_distributor.c +1 -1
@@ -32,7 +32,7 @@ EAL_REGISTER_TAILQ(rte_dist_burst_tailq)
 
 /**** Burst Packet APIs called by workers ****/
 
-void
+void __vsym

all these additional __vsym annotations looks like they belong in a seperate patch, as they are fixing a bug and are not directly related to adding LTO the build system.

Quoted 545 lines
lib/librte_distributor/rte_distributor.c +8 -8
 rte_distributor_request_pkt_v1705(struct rte_distributor *d,
 		unsigned int worker_id, struct rte_mbuf **oldpkt,
 		unsigned int count)
@@ -84,7 +84,7 @@ MAP_STATIC_SYMBOL(void rte_distributor_request_pkt(struct rte_distributor *d,
 		unsigned int count),
 		rte_distributor_request_pkt_v1705);
 
-int
+int __vsym
 rte_distributor_poll_pkt_v1705(struct rte_distributor *d,
 		unsigned int worker_id, struct rte_mbuf **pkts)
 {
@@ -124,7 +124,7 @@ MAP_STATIC_SYMBOL(int rte_distributor_poll_pkt(struct rte_distributor *d,
 		unsigned int worker_id, struct rte_mbuf **pkts),
 		rte_distributor_poll_pkt_v1705);
 
-int
+int __vsym
 rte_distributor_get_pkt_v1705(struct rte_distributor *d,
 		unsigned int worker_id, struct rte_mbuf **pkts,
 		struct rte_mbuf **oldpkt, unsigned int return_count)
@@ -159,7 +159,7 @@ MAP_STATIC_SYMBOL(int rte_distributor_get_pkt(struct rte_distributor *d,
 		struct rte_mbuf **oldpkt, unsigned int return_count),
 		rte_distributor_get_pkt_v1705);
 
-int
+int __vsym
 rte_distributor_return_pkt_v1705(struct rte_distributor *d,
 		unsigned int worker_id, struct rte_mbuf **oldpkt, int num)
 {
@@ -335,7 +335,7 @@ release(struct rte_distributor *d, unsigned int wkr)
 
 
 /* process a set of packets to distribute them to workers */
-int
+int __vsym
 rte_distributor_process_v1705(struct rte_distributor *d,
 		struct rte_mbuf **mbufs, unsigned int num_mbufs)
 {
@@ -476,7 +476,7 @@ MAP_STATIC_SYMBOL(int rte_distributor_process(struct rte_distributor *d,
 		rte_distributor_process_v1705);
 
 /* return to the caller, packets returned from workers */
-int
+int __vsym
 rte_distributor_returned_pkts_v1705(struct rte_distributor *d,
 		struct rte_mbuf **mbufs, unsigned int max_mbufs)
 {
@@ -526,7 +526,7 @@ total_outstanding(const struct rte_distributor *d)
  * Flush the distributor, so that there are no outstanding packets in flight or
  * queued up.
  */
-int
+int __vsym
 rte_distributor_flush_v1705(struct rte_distributor *d)
 {
 	unsigned int flushed;
@@ -561,7 +561,7 @@ MAP_STATIC_SYMBOL(int rte_distributor_flush(struct rte_distributor *d),
 		rte_distributor_flush_v1705);
 
 /* clears the internal returns array in the distributor */
-void
+void __vsym
 rte_distributor_clear_returns_v1705(struct rte_distributor *d)
 {
 	unsigned int wkr;
@@ -581,7 +581,7 @@ MAP_STATIC_SYMBOL(void rte_distributor_clear_returns(struct rte_distributor *d),
 		rte_distributor_clear_returns_v1705);
 
 /* creates a distributor instance */
-struct rte_distributor *
+struct rte_distributor * __vsym
 rte_distributor_create_v1705(const char *name,
 		unsigned int socket_id,
 		unsigned int num_workers,
lib/librte_distributor/rte_distributor_v20.c +9 -9
index cdc0969a8..31c766421 100644
@@ -27,7 +27,7 @@ EAL_REGISTER_TAILQ(rte_distributor_tailq)
 
 /**** APIs called by workers ****/
 
-void
+void __vsym
 rte_distributor_request_pkt_v20(struct rte_distributor_v20 *d,
 		unsigned worker_id, struct rte_mbuf *oldpkt)
 {
@@ -40,7 +40,7 @@ rte_distributor_request_pkt_v20(struct rte_distributor_v20 *d,
 }
 VERSION_SYMBOL(rte_distributor_request_pkt, _v20, 2.0);
 
-struct rte_mbuf *
+struct rte_mbuf * __vsym
 rte_distributor_poll_pkt_v20(struct rte_distributor_v20 *d,
 		unsigned worker_id)
 {
@@ -54,7 +54,7 @@ rte_distributor_poll_pkt_v20(struct rte_distributor_v20 *d,
 }
 VERSION_SYMBOL(rte_distributor_poll_pkt, _v20, 2.0);
 
-struct rte_mbuf *
+struct rte_mbuf * __vsym
 rte_distributor_get_pkt_v20(struct rte_distributor_v20 *d,
 		unsigned worker_id, struct rte_mbuf *oldpkt)
 {
@@ -66,7 +66,7 @@ rte_distributor_get_pkt_v20(struct rte_distributor_v20 *d,
 }
 VERSION_SYMBOL(rte_distributor_get_pkt, _v20, 2.0);
 
-int
+int __vsym
 rte_distributor_return_pkt_v20(struct rte_distributor_v20 *d,
 		unsigned worker_id, struct rte_mbuf *oldpkt)
 {
@@ -191,7 +191,7 @@ process_returns(struct rte_distributor_v20 *d)
 }
 
 /* process a set of packets to distribute them to workers */
-int
+int __vsym
 rte_distributor_process_v20(struct rte_distributor_v20 *d,
 		struct rte_mbuf **mbufs, unsigned num_mbufs)
 {
@@ -296,7 +296,7 @@ rte_distributor_process_v20(struct rte_distributor_v20 *d,
 VERSION_SYMBOL(rte_distributor_process, _v20, 2.0);
 
 /* return to the caller, packets returned from workers */
-int
+int __vsym
 rte_distributor_returned_pkts_v20(struct rte_distributor_v20 *d,
 		struct rte_mbuf **mbufs, unsigned max_mbufs)
 {
@@ -334,7 +334,7 @@ total_outstanding(const struct rte_distributor_v20 *d)
 
 /* flush the distributor, so that there are no outstanding packets in flight or
  * queued up. */
-int
+int __vsym
 rte_distributor_flush_v20(struct rte_distributor_v20 *d)
 {
 	const unsigned flushed = total_outstanding(d);
@@ -347,7 +347,7 @@ rte_distributor_flush_v20(struct rte_distributor_v20 *d)
 VERSION_SYMBOL(rte_distributor_flush, _v20, 2.0);
 
 /* clears the internal returns array in the distributor */
-void
+void __vsym
 rte_distributor_clear_returns_v20(struct rte_distributor_v20 *d)
 {
 	d->returns.start = d->returns.count = 0;
@@ -358,7 +358,7 @@ rte_distributor_clear_returns_v20(struct rte_distributor_v20 *d)
 VERSION_SYMBOL(rte_distributor_clear_returns, _v20, 2.0);
 
 /* creates a distributor instance */
-struct rte_distributor_v20 *
+struct rte_distributor_v20 * __vsym
 rte_distributor_create_v20(const char *name,
 		unsigned socket_id,
 		unsigned num_workers)
lib/librte_lpm/rte_lpm.c +14 -14
index 3a929a1b1..a2fba8d61 100644
@@ -89,7 +89,7 @@ depth_to_range(uint8_t depth)
 /*
  * Find an existing lpm table and return a pointer to it.
  */
-struct rte_lpm_v20 *
+struct rte_lpm_v20 * __vsym
 rte_lpm_find_existing_v20(const char *name)
 {
 	struct rte_lpm_v20 *l = NULL;
@@ -115,7 +115,7 @@ rte_lpm_find_existing_v20(const char *name)
 }
 VERSION_SYMBOL(rte_lpm_find_existing, _v20, 2.0);
 
-struct rte_lpm *
+struct rte_lpm * __vsym
 rte_lpm_find_existing_v1604(const char *name)
 {
 	struct rte_lpm *l = NULL;
@@ -146,7 +146,7 @@ MAP_STATIC_SYMBOL(struct rte_lpm *rte_lpm_find_existing(const char *name),
 /*
  * Allocates memory for LPM object
  */
-struct rte_lpm_v20 *
+struct rte_lpm_v20 * __vsym
 rte_lpm_create_v20(const char *name, int socket_id, int max_rules,
 		__rte_unused int flags)
 {
@@ -219,7 +219,7 @@ rte_lpm_create_v20(const char *name, int socket_id, int max_rules,
 }
 VERSION_SYMBOL(rte_lpm_create, _v20, 2.0);
 
-struct rte_lpm *
+struct rte_lpm * __vsym
 rte_lpm_create_v1604(const char *name, int socket_id,
 		const struct rte_lpm_config *config)
 {
@@ -328,7 +328,7 @@ MAP_STATIC_SYMBOL(
 /*
  * Deallocates memory for given LPM table.
  */
-void
+void __vsym
 rte_lpm_free_v20(struct rte_lpm_v20 *lpm)
 {
 	struct rte_lpm_list *lpm_list;
@@ -357,7 +357,7 @@ rte_lpm_free_v20(struct rte_lpm_v20 *lpm)
 }
 VERSION_SYMBOL(rte_lpm_free, _v20, 2.0);
 
-void
+void __vsym
 rte_lpm_free_v1604(struct rte_lpm *lpm)
 {
 	struct rte_lpm_list *lpm_list;
@@ -1176,7 +1176,7 @@ add_depth_big_v1604(struct rte_lpm *lpm, uint32_t ip_masked, uint8_t depth,
 /*
  * Add a route
  */
-int
+int __vsym
 rte_lpm_add_v20(struct rte_lpm_v20 *lpm, uint32_t ip, uint8_t depth,
 		uint8_t next_hop)
 {
@@ -1217,7 +1217,7 @@ rte_lpm_add_v20(struct rte_lpm_v20 *lpm, uint32_t ip, uint8_t depth,
 }
 VERSION_SYMBOL(rte_lpm_add, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm_add_v1604(struct rte_lpm *lpm, uint32_t ip, uint8_t depth,
 		uint32_t next_hop)
 {
@@ -1263,7 +1263,7 @@ MAP_STATIC_SYMBOL(int rte_lpm_add(struct rte_lpm *lpm, uint32_t ip,
 /*
  * Look for a rule in the high-level rules table
  */
-int
+int __vsym
 rte_lpm_is_rule_present_v20(struct rte_lpm_v20 *lpm, uint32_t ip, uint8_t depth,
 uint8_t *next_hop)
 {
@@ -1290,7 +1290,7 @@ uint8_t *next_hop)
 }
 VERSION_SYMBOL(rte_lpm_is_rule_present, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm_is_rule_present_v1604(struct rte_lpm *lpm, uint32_t ip, uint8_t depth,
 uint32_t *next_hop)
 {
@@ -1843,7 +1843,7 @@ delete_depth_big_v1604(struct rte_lpm *lpm, uint32_t ip_masked,
 /*
  * Deletes a rule
  */
-int
+int __vsym
 rte_lpm_delete_v20(struct rte_lpm_v20 *lpm, uint32_t ip, uint8_t depth)
 {
 	int32_t rule_to_delete_index, sub_rule_index;
@@ -1897,7 +1897,7 @@ rte_lpm_delete_v20(struct rte_lpm_v20 *lpm, uint32_t ip, uint8_t depth)
 }
 VERSION_SYMBOL(rte_lpm_delete, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm_delete_v1604(struct rte_lpm *lpm, uint32_t ip, uint8_t depth)
 {
 	int32_t rule_to_delete_index, sub_rule_index;
@@ -1956,7 +1956,7 @@ MAP_STATIC_SYMBOL(int rte_lpm_delete(struct rte_lpm *lpm, uint32_t ip,
 /*
  * Delete all rules from the LPM table.
  */
-void
+void __vsym
 rte_lpm_delete_all_v20(struct rte_lpm_v20 *lpm)
 {
 	/* Zero rule information. */
@@ -1973,7 +1973,7 @@ rte_lpm_delete_all_v20(struct rte_lpm_v20 *lpm)
 }
 VERSION_SYMBOL(rte_lpm_delete_all, _v20, 2.0);
 
-void
+void __vsym
 rte_lpm_delete_all_v1604(struct rte_lpm *lpm)
 {
 	/* Zero rule information. */
lib/librte_lpm/rte_lpm6.c +8 -8
index 9b8aeb972..49a7fea1d 100644
@@ -811,7 +811,7 @@ add_step(struct rte_lpm6 *lpm, struct rte_lpm6_tbl_entry *tbl,
 /*
  * Add a route
  */
-int
+int __vsym
 rte_lpm6_add_v20(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
 		uint8_t next_hop)
 {
@@ -861,7 +861,7 @@ simulate_add(struct rte_lpm6 *lpm, const uint8_t *masked_ip, uint8_t depth)
 	return 0;
 }
 
-int
+int __vsym
 rte_lpm6_add_v1705(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
 		uint32_t next_hop)
 {
@@ -954,7 +954,7 @@ lookup_step(const struct rte_lpm6 *lpm, const struct rte_lpm6_tbl_entry *tbl,
 /*
  * Looks up an IP
  */
-int
+int __vsym
 rte_lpm6_lookup_v20(const struct rte_lpm6 *lpm, uint8_t *ip, uint8_t *next_hop)
 {
 	uint32_t next_hop32 = 0;
@@ -972,7 +972,7 @@ rte_lpm6_lookup_v20(const struct rte_lpm6 *lpm, uint8_t *ip, uint8_t *next_hop)
 }
 VERSION_SYMBOL(rte_lpm6_lookup, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm6_lookup_v1705(const struct rte_lpm6 *lpm, uint8_t *ip,
 		uint32_t *next_hop)
 {
@@ -1007,7 +1007,7 @@ MAP_STATIC_SYMBOL(int rte_lpm6_lookup(const struct rte_lpm6 *lpm, uint8_t *ip,
 /*
  * Looks up a group of IP addresses
  */
-int
+int __vsym
 rte_lpm6_lookup_bulk_func_v20(const struct rte_lpm6 *lpm,
 		uint8_t ips[][RTE_LPM6_IPV6_ADDR_SIZE],
 		int16_t * next_hops, unsigned n)
@@ -1048,7 +1048,7 @@ rte_lpm6_lookup_bulk_func_v20(const struct rte_lpm6 *lpm,
 }
 VERSION_SYMBOL(rte_lpm6_lookup_bulk_func, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm6_lookup_bulk_func_v1705(const struct rte_lpm6 *lpm,
 		uint8_t ips[][RTE_LPM6_IPV6_ADDR_SIZE],
 		int32_t *next_hops, unsigned int n)
@@ -1098,7 +1098,7 @@ MAP_STATIC_SYMBOL(int rte_lpm6_lookup_bulk_func(const struct rte_lpm6 *lpm,
 /*
  * Look for a rule in the high-level rules table
  */
-int
+int __vsym
 rte_lpm6_is_rule_present_v20(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
 		uint8_t *next_hop)
 {
@@ -1118,7 +1118,7 @@ rte_lpm6_is_rule_present_v20(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
 }
 VERSION_SYMBOL(rte_lpm6_is_rule_present, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm6_is_rule_present_v1705(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
 		uint32_t *next_hop)
 {
lib/librte_timer/rte_timer.c +10 -10
index bdcf05d06..e560ace06 100644
@@ -131,7 +131,7 @@ rte_timer_data_dealloc(uint32_t id)
 	return 0;
 }
 
-void
+void __vsym
 rte_timer_subsystem_init_v20(void)
 {
 	unsigned lcore_id;
@@ -153,7 +153,7 @@ VERSION_SYMBOL(rte_timer_subsystem_init, _v20, 2.0);
  * secondary processes should be empty, the zeroth entry can be shared by
  * multiple processes.
  */
-int
+int __vsym
 rte_timer_subsystem_init_v1905(void)
 {
 	const struct rte_memzone *mz;
@@ -551,7 +551,7 @@ __rte_timer_reset(struct rte_timer *tim, uint64_t expire,
 }
 
 /* Reset and start the timer associated with the timer handle tim */
-int
+int __vsym
 rte_timer_reset_v20(struct rte_timer *tim, uint64_t ticks,
 		    enum rte_timer_type type, unsigned int tim_lcore,
 		    rte_timer_cb_t fct, void *arg)
@@ -574,7 +574,7 @@ rte_timer_reset_v20(struct rte_timer *tim, uint64_t ticks,
 }
 VERSION_SYMBOL(rte_timer_reset, _v20, 2.0);
 
-int
+int __vsym
 rte_timer_reset_v1905(struct rte_timer *tim, uint64_t ticks,
 		      enum rte_timer_type type, unsigned int tim_lcore,
 		      rte_timer_cb_t fct, void *arg)
@@ -657,14 +657,14 @@ __rte_timer_stop(struct rte_timer *tim, int local_is_locked,
 }
 
 /* Stop the timer associated with the timer handle tim */
-int
+int __vsym
 rte_timer_stop_v20(struct rte_timer *tim)
 {
 	return __rte_timer_stop(tim, 0, &default_timer_data);
 }
 VERSION_SYMBOL(rte_timer_stop, _v20, 2.0);
 
-int
+int __vsym
 rte_timer_stop_v1905(struct rte_timer *tim)
 {
 	return rte_timer_alt_stop(default_data_id, tim);
@@ -817,14 +817,14 @@ __rte_timer_manage(struct rte_timer_data *timer_data)
 	priv_timer[lcore_id].running_tim = NULL;
 }
 
-void
+void __vsym
 rte_timer_manage_v20(void)
 {
 	__rte_timer_manage(&default_timer_data);
 }
 VERSION_SYMBOL(rte_timer_manage, _v20, 2.0);
 
-int
+int __vsym
 rte_timer_manage_v1905(void)
 {
 	struct rte_timer_data *timer_data;
@@ -1074,14 +1074,14 @@ __rte_timer_dump_stats(struct rte_timer_data *timer_data __rte_unused, FILE *f)
 #endif
 }
 
-void
+void __vsym
 rte_timer_dump_stats_v20(FILE *f)
 {
 	__rte_timer_dump_stats(&default_timer_data, f);
 }
 VERSION_SYMBOL(rte_timer_dump_stats, _v20, 2.0);
 
-int
+int __vsym
 rte_timer_dump_stats_v1905(FILE *f)
 {
 	return rte_timer_alt_dump_stats(default_data_id, f);
mk/toolchain/clang/rte.toolchain-compat.mk +4 -0
index e6189b498..78f96c648 100644
@@ -20,3 +20,7 @@ CLANG_MINOR_VERSION := $(shell echo $(CLANG_VERSION) | cut -f2 -d.)
 ifeq ($(shell test $(CLANG_MAJOR_VERSION)$(CLANG_MINOR_VERSION) -lt 35 && echo 1), 1)
 	CC_SUPPORTS_Z := false
 endif
+
+ifeq ($(shell test $(CLANG_MAJOR_VERSION)$(CLANG_MINOR_VERSION) -lt 60 && echo 1), 1)
+	CONFIG_RTE_ENABLE_LTO=n
+endif
mk/toolchain/clang/rte.vars.mk +8 -0
index 3c49dc568..3b1fa05f9 100644
@@ -48,6 +48,14 @@ endif
 # process cpu flags
 include $(RTE_SDK)/mk/toolchain/$(RTE_TOOLCHAIN)/rte.toolchain-compat.mk
 
+ifeq ($(CONFIG_RTE_ENABLE_LTO),y)
+# 'fat-lto' is used since pmdinfogen needs to have 'this_pmd_nameX'
+# exported in symbol table and without this option only internal
+# representation is present.
+TOOLCHAIN_CFLAGS += -flto -ffat-lto-objects
+TOOLCHAIN_LDFLAGS += -flto
+endif
+
 # workaround clang bug with warning "missing field initializer" for "= {0}"
 WERROR_FLAGS += -Wno-missing-field-initializers
 
mk/toolchain/gcc/rte.toolchain-compat.mk +4 -0
index ea40a11c0..ad4fad83c 100644
@@ -88,6 +88,10 @@ else
 		MACHINE_CFLAGS := $(filter-out -march% -mtune% -msse%,$(MACHINE_CFLAGS))
 	endif
 
+	ifeq ($(shell test $(GCC_VERSION) -lt 45 && echo 1), 1)
+		CONFIG_RTE_ENABLE_LTO=n
+	endif
+
 	# Disable thunderx PMD for gcc < 4.7
 	ifeq ($(shell test $(GCC_VERSION) -lt 47 && echo 1), 1)
 		CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD=d
mk/toolchain/gcc/rte.vars.mk +12 -0
index b852fcfd7..9fc704193 100644
@@ -62,6 +62,18 @@ endif
 # process cpu flags
 include $(RTE_SDK)/mk/toolchain/$(RTE_TOOLCHAIN)/rte.toolchain-compat.mk
 
+ifeq ($(CONFIG_RTE_ENABLE_LTO),y)
+# 'fat-lto' is used since pmdinfogen needs to have 'this_pmd_nameX'
+# exported in symbol table and without this option only internal
+# representation is present.
+TOOLCHAIN_CFLAGS += -flto -ffat-lto-objects
+TOOLCHAIN_LDFLAGS += -flto
+# workaround for GCC bug 81440
+ifeq ($(shell test $(GCC_VERSION) -lt 80 && echo 1), 1)
+WERROR_FLAGS += -Wno-lto-type-mismatch
+endif
+endif
+
 # workaround GCC bug with warning "missing initializer" for "= {0}"
 ifeq ($(shell test $(GCC_VERSION) -lt 47 && echo 1), 1)
 WERROR_FLAGS += -Wno-missing-field-initializers
mk/toolchain/icc/rte.vars.mk +8 -0
index aa1422bf1..8aa87aa1e 100644
@@ -54,5 +54,13 @@ endif
 # process cpu flags
 include $(RTE_SDK)/mk/toolchain/$(RTE_TOOLCHAIN)/rte.toolchain-compat.mk
 
+ifeq ($(CONFIG_RTE_ENABLE_LTO),y)
+# 'fat-lto' is used since pmdinfogen needs to have 'this_pmd_nameX'
+# exported in symbol table and without this option only internal
+# representation is present.
+TOOLCHAIN_CFLAGS += -flto -ffat-lto-objects
+TOOLCHAIN_LDFLAGS += -flto
+endif
+
 export CC AS AR LD OBJCOPY OBJDUMP STRIP READELF
 export TOOLCHAIN_CFLAGS TOOLCHAIN_LDFLAGS TOOLCHAIN_ASFLAGS
#4 lore raw

Re: [dpdk-dev] [PATCH v2 01/10] build: add an option to enable LTO build

Quoted 2 lines · Ray Kinsella

this is cool, good work.
comments below.

[...]>> +CONFIG_RTE_ENABLE_LTO=n

Quoted 8 lines
Quoted 4 lines

+
#
# Compile to share library
#

Why would we make this optional in this way and expand the matrix of different ways to build DPDK. To ask another way, why wouldn't a user turn on GSO.

Compilation time is much longer. In a normal hack|fix/compile/repeat cycle with "compile" part being simple "make" the link time might be a bit annoying. So I imagine keeping LTO off for the most part of the dev cycle and then at the end when doing release/cleanup turn LTO on - to either get release build ready or to get some set of warnings that you address before yet another attempt to release build. By the way - this make config option is equivalent to meson 'b_lto' option, which by default is off, so we have similar behaviour in both build types.

Regards
Andrzej

PS. I assumed that you've meant "LTO" not "GSO" - if not, then please explain what you've meant.

#5 lore raw

Re: [dpdk-dev] [PATCH v2 01/10] build: add an option to enable LTO build

Quoted 1 line · Andrzej Ostruszka

On 9/18/19 3:32 PM, Ray Kinsella wrote:

...>

Quoted 6 lines

Compilation time is much longer. In a normal hack|fix/compile/repeat cycle with "compile" part being simple "make" the link time might be a bit annoying. So I imagine keeping LTO off for the most part of the dev cycle and then at the end when doing release/cleanup turn LTO on - to either get release build ready or to get some set of warnings that you address before yet another attempt to release build.

Well look, I would say a few things.

1. I see build times going down dramatically with Meson/Ninja in any case.
2. You don't want any false positivess - i.e. misleading the contributor into thinking their code builds fine, and then someone switches on LTO and suddenly there is a problem that was missed (and hopefully gets caught by CI) - and BTW I have seen these LTO specific issues in the past.

I strongly believe we will make all our lives easier, by having as few ways of building DPDK as possible.

Quoted 10 lines

By the way - this
make config option is equivalent to meson 'b_lto' option, which by default is off, so we have similar behaviour in both build types.

Regards
Andrzej

PS. I assumed that you've meant "LTO" not "GSO" - if not, then please explain what you've meant.

Your are correct, GSO was on my mind apologies.

#6 lore raw

Re: [dpdk-dev] [PATCH v2 01/10] build: add an option to enable LTO build

Quoted 12 lines · Ray Kinsella
Quoted 1 line · Andrzej Ostruszka

On 9/18/19 3:32 PM, Ray Kinsella wrote:

...>

Quoted 6 lines

Compilation time is much longer. In a normal hack|fix/compile/repeat cycle with "compile" part being simple "make" the link time might be a bit annoying. So I imagine keeping LTO off for the most part of the dev cycle and then at the end when doing release/cleanup turn LTO on - to either get release build ready or to get some set of warnings that you address before yet another attempt to release build.

Quoted 3 lines

Well look, I would say a few things.

1. I see build times going down dramatically with Meson/Ninja in any case.

In the general case yes, but mainly it helps if you have a large number of cores. For anyone building in a CI with only a couple of cores, meson+ninja isn't going to help much.

/Bruce

#7 lore raw

Re: [dpdk-dev] [PATCH v2 01/10] build: add an option to enable LTO build

Quoted 15 lines · Bruce Richardson
Quoted 14 lines · Ray Kinsella
Quoted 1 line · Andrzej Ostruszka

On 9/18/19 3:32 PM, Ray Kinsella wrote:

...>

Quoted 6 lines

Compilation time is much longer. In a normal hack|fix/compile/repeat cycle with "compile" part being simple "make" the link time might be a bit annoying. So I imagine keeping LTO off for the most part of the dev cycle and then at the end when doing release/cleanup turn LTO on - to either get release build ready or to get some set of warnings that you address before yet another attempt to release build.

Well look, I would say a few things.

Quoted 8 lines
Quoted 1 line

1. I see build times going down dramatically with Meson/Ninja in any case.

In the general case yes, but mainly it helps if you have a large number of cores. For anyone building in a CI with only a couple of cores, meson+ninja isn't going to help much.

/Bruce

Very true, and I completely acknowledge that LTO adds compilation time. I would just like to see it in or out, not another build time option.

Ray K

#8 lore raw

Re: [dpdk-dev] [PATCH v2 01/10] build: add an option to enable LTO build

20/09/2019 09:38, Ray Kinsella:

Quoted 23 lines · Bruce Richardson
Quoted 21 lines · Ray Kinsella
Quoted 12 lines · Andrzej Ostruszka
Quoted 1 line · Ray Kinsella

...>

Quoted 6 lines

Compilation time is much longer. In a normal hack|fix/compile/repeat cycle with "compile" part being simple "make" the link time might be a bit annoying. So I imagine keeping LTO off for the most part of the dev cycle and then at the end when doing release/cleanup turn LTO on - to either get release build ready or to get some set of warnings that you address before yet another attempt to release build.

Quoted 3 lines

Well look, I would say a few things.

1. I see build times going down dramatically with Meson/Ninja in any case.

In the general case yes, but mainly it helps if you have a large number of cores. For anyone building in a CI with only a couple of cores, meson+ninja isn't going to help much.

Quoted 5 lines
Quoted 2 lines

/Bruce

Very true, and I completely acknowledge that LTO adds compilation time. I would just like to see it in or out, not another build time option.

Please can we get some numbers to understand how longer it is?

#9 lore raw

Re: [dpdk-dev] [PATCH v2 01/10] build: add an option to enable LTO build

Quoted 24 lines · Thomas Monjalon

20/09/2019 09:38, Ray Kinsella:

Quoted 23 lines · Bruce Richardson
Quoted 21 lines · Ray Kinsella
Quoted 12 lines · Andrzej Ostruszka
Quoted 1 line · Ray Kinsella

...>

Quoted 6 lines

Compilation time is much longer. In a normal hack|fix/compile/repeat cycle with "compile" part being simple "make" the link time might be a bit annoying. So I imagine keeping LTO off for the most part of the dev cycle and then at the end when doing release/cleanup turn LTO on - to either get release build ready or to get some set of warnings that you address before yet another attempt to release build.

Quoted 3 lines

Well look, I would say a few things.

1. I see build times going down dramatically with Meson/Ninja in any case.

In the general case yes, but mainly it helps if you have a large number of cores. For anyone building in a CI with only a couple of cores, meson+ninja isn't going to help much.

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

/Bruce

Very true, and I completely acknowledge that LTO adds compilation time. I would just like to see it in or out, not another build time option.

Please can we get some numbers to understand how longer it is?

I will note here, that it will always be a function of the number of object files involved. We should measure it now to understand ...

#10 lore raw

Re: [dpdk-dev] [PATCH v2 01/10] build: add an option to enable LTO build

Quoted 30 lines · Ray Kinsella
Quoted 27 lines · Thomas Monjalon

20/09/2019 09:38, Ray Kinsella:

Quoted 26 lines · Bruce Richardson
Quoted 23 lines · Ray Kinsella
Quoted 12 lines · Andrzej Ostruszka
Quoted 1 line · Ray Kinsella

...>

Quoted 6 lines

Compilation time is much longer. In a normal hack|fix/compile/repeat cycle with "compile" part being simple "make" the link time might be a bit annoying. So I imagine keeping LTO off for the most part of the dev cycle and then at the end when doing release/cleanup turn LTO on - to either get release build ready or to get some set of warnings that you address before yet another attempt to release build.

Quoted 3 lines

Well look, I would say a few things.

1. I see build times going down dramatically with Meson/Ninja in any case.

In the general case yes, but mainly it helps if you have a large number of cores. For anyone building in a CI with only a couple of cores, meson+ninja isn't going to help much.

/Bruce

Quoted 10 lines
Quoted 5 lines
Quoted 2 lines

Very true, and I completely acknowledge that LTO adds compilation time. I would just like to see it in or out, not another build time option.

Please can we get some numbers to understand how longer it is?

I will note here, that it will always be a function of the number of object files involved. We should measure it now to understand ...

On my system the non-LTO from-scratch build takes about one minute. With LTO enabled, it takes about five minutes.

In addition, it makes our application *run* slower as well.

This reinforces past experience I have with LTO - it's more cool than useful, unless you care much about the resulting code size.

#11 lore raw

Re: [dpdk-dev] [PATCH v2 01/10] build: add an option to enable LTO build

In reply to Thomas Monjalon

On 9/23/19 9:23 AM, Thomas Monjalon wrote:
[...]

Quoted 1 line

Please can we get some numbers to understand how longer it is?

Below numbers are for make based (make -j8) clean build on my system:

non-LTO
real: 144.56s, user:451.81s, sys:48.46s, CPU:346%

LTO
real: 607.20s, user:2141.71s, sys:88.36s, CPU:367%

So it is similar ~5x increase as Mattias has reported. Have not measured it, but the lion share of that increase is due to linking of 'test' apps.

I would vote for leaving LTO as an option - although I must admit I did not get what Ray meant by saying:

20/09/2019 09:38, Ray Kinsella:
[...]

Quoted 1 line

I would just like to see it in or out, not another build time option.

If "in or out" means "either accept the patches with LTO on and no config option or reject them" then I disagree. Even if run time improvements are questionable I find the additional link time warnings beneficial and would like to have an easy way to turn them on when doing final touches before pushing out.

Regards
Andrzej

#12 lore raw

Re: [dpdk-dev] [PATCH v2 01/10] build: add an option to enable LTO build

Quoted 9 lines · Andrzej Ostruszka

On 9/23/19 9:23 AM, Thomas Monjalon wrote:
[...]

Quoted 1 line

Please can we get some numbers to understand how longer it is?

Below numbers are for make based (make -j8) clean build on my system:

non-LTO
real: 144.56s, user:451.81s, sys:48.46s, CPU:346%

Quoted 7 lines

LTO
real: 607.20s, user:2141.71s, sys:88.36s, CPU:367%

So it is similar ~5x increase as Mattias has reported. Have not measured it, but the lion share of that increase is due to linking of 'test' apps.

Interesting. Do we want to explicitly not use lto for the test app?

Quoted 5 lines

I would vote for leaving LTO as an option - although I must admit I did not get what Ray meant by saying:

20/09/2019 09:38, Ray Kinsella:
[...]

Quoted 7 lines
Quoted 1 line

I would just like to see it in or out, not another build time option.

If "in or out" means "either accept the patches with LTO on and no config option or reject them" then I disagree. Even if run time improvements are questionable I find the additional link time warnings beneficial and would like to have an easy way to turn them on when doing final touches before pushing out.

Looking at it from the meson build view-point, support for lto is built into the tool itself, so the support can't be removed as such - it will either result in a working build or not. Therefore, I'm for taking a patch to support lto in meson, whatever about supporting it through make etc.

/Bruce

#13 lore raw

Re: [dpdk-dev] [PATCH v2 01/10] build: add an option to enable LTO build

Quoted 1 line · Bruce Richardson

On Mon, Sep 23, 2019 at 02:03:35PM +0200, Andrzej Ostruszka wrote:

[...]

Quoted 6 lines
Quoted 4 lines

So it is similar ~5x increase as Mattias has reported. Have not measured it, but the lion share of that increase is due to linking of 'test' apps.

Interesting. Do we want to explicitly not use lto for the test app?

It is the linking of these apps where LTO really kicks in. During compilation of objects compiler just additionally generates internal representation (in extra sections of ELF object). Linking these objects into library does not do much - so the actual optimization is done when producing final executable.

It might be so that when using dedicated user app the penalty would be much smaller since the amount of code used would be much smaller - but FWIW all the warnings that I was fixing were produced by the linking of app/test* binaries.

Regards
Andrzej

#14 lore raw

Re: [dpdk-dev] [PATCH v2 01/10] build: add an option to enable LTO build

Quoted 16 lines · Andrzej Ostruszka
Quoted 1 line · Bruce Richardson

On Mon, Sep 23, 2019 at 02:03:35PM +0200, Andrzej Ostruszka wrote:

[...]

Quoted 6 lines
Quoted 4 lines

So it is similar ~5x increase as Mattias has reported. Have not measured it, but the lion share of that increase is due to linking of 'test' apps.

Interesting. Do we want to explicitly not use lto for the test app?

It is the linking of these apps where LTO really kicks in. During compilation of objects compiler just additionally generates internal representation (in extra sections of ELF object). Linking these objects into library does not do much - so the actual optimization is done when producing final executable.

Quoted 5 lines

It might be so that when using dedicated user app the penalty would be much smaller since the amount of code used would be much smaller - but FWIW all the warnings that I was fixing were produced by the linking of app/test* binaries.

I wasn't suggesting removing it for testpmd test-acl etc., just for the autotest "test" binary since it's the one with the most .o files.

However, testing on my system with the meson build, I'm getting lots of link errors [See below]. Any suggestions?

/Bruce

/usr/bin/ld: /tmp/dpdk-testpmd.hncbtE.ltrans93.ltrans.o: in function `ena_stop':
<artificial>:(.text+0x9ed6): undefined reference to `rte_timer_stop'
/usr/bin/ld: /tmp/dpdk-testpmd.hncbtE.ltrans93.ltrans.o: in function `ena_start':
<artificial>:(.text+0xae23): undefined reference to `rte_timer_reset'
/usr/bin/ld: /tmp/dpdk-testpmd.hncbtE.ltrans93.ltrans.o: in function `eth_ena_dev_init.cold':
<artificial>:(.text.unlikely+0x224a): undefined reference to rte_timer_subsystem_init' /usr/bin/ld: /tmp/dpdk-testpmd.hncbtE.ltrans119.ltrans.o: in function rte_distributor_flush_v1705':
<artificial>:(.text+0xa229): undefined reference to `rte_distributor_process'
/usr/bin/ld: <artificial>:(.text+0xa256): undefined reference to rte_distributor_process' /usr/bin/ld: /tmp/dpdk-testpmd.hncbtE.ltrans119.ltrans.o: in function rte_distributor_get_pkt_v1705':
<artificial>:(.text+0xd24b): undefined reference to `rte_distributor_request_pkt'
/usr/bin/ld: <artificial>:(.text+0xd259): undefined reference to rte_distributor_poll_pkt' /usr/bin/ld: /tmp/dpdk-testpmd.hncbtE.ltrans122.ltrans.o: in function rte_table_lpm_ipv6_lookup.lto_priv.0':
<artificial>:(.text+0x810c): undefined reference to `rte_lpm6_lookup'
/usr/bin/ld: /tmp/dpdk-testpmd.hncbtE.ltrans122.ltrans.o: in function `rte_table_lpm_ipv6_entry_delete.lto_priv.0':
<artificial>:(.text+0x81bb): undefined reference to `rte_lpm6_is_rule_present'
/usr/bin/ld: /tmp/dpdk-testpmd.hncbtE.ltrans122.ltrans.o: in function `rte_table_lpm_ipv6_entry_add.lto_priv.0':
<artificial>:(.text+0x8e13): undefined reference to `rte_lpm6_is_rule_present'
/usr/bin/ld: <artificial>:(.text+0x8e79): undefined reference to `rte_lpm6_add'
/usr/bin/ld: /tmp/dpdk-testpmd.hncbtE.ltrans122.ltrans.o: in function `rte_table_lpm_entry_delete.lto_priv.0':
<artificial>:(.text+0x9b15): undefined reference to `rte_lpm_is_rule_present'
/usr/bin/ld: <artificial>:(.text+0x9b4c): undefined reference to `rte_lpm_delete'
/usr/bin/ld: /tmp/dpdk-testpmd.hncbtE.ltrans122.ltrans.o: in function `rte_table_lpm_entry_add.lto_priv.0':
<artificial>:(.text+0x9c06): undefined reference to `rte_lpm_is_rule_present'
/usr/bin/ld: <artificial>:(.text+0x9c73): undefined reference to `rte_lpm_add'
/usr/bin/ld: /tmp/dpdk-testpmd.hncbtE.ltrans122.ltrans.o: in function `rte_table_lpm_free.lto_priv.0':
<artificial>:(.text+0x9d12): undefined reference to `rte_lpm_free'
/usr/bin/ld: /tmp/dpdk-testpmd.hncbtE.ltrans122.ltrans.o: in function `rte_table_lpm_create.lto_priv.0':
<artificial>:(.text+0x9e01): undefined reference to `rte_lpm_create'
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

#15 lore raw

Re: [dpdk-dev] [PATCH v2 01/10] build: add an option to enable LTO build

On 9/23/19 6:13 PM, Bruce Richardson wrote:
[...]

Quoted 7 lines

However, testing on my system with the meson build, I'm getting lots of link errors [See below]. Any suggestions?

/Bruce

/usr/bin/ld: /tmp/dpdk-testpmd.hncbtE.ltrans93.ltrans.o: in function `ena_stop':
<artificial>:(.text+0x9ed6): undefined reference to `rte_timer_stop'

[...]

What is 'default_library'? It should be 'shared' as mentioned in the docs. The problem is that RTE_BUILD_SHARED_LIB is statically defined in rte_config.h used by meson build. This results in broken static libraries (those that are using versioned symbols - like timer/lpm/distributor) - since the MAP_STATIC_SYMBOL macro defining the default alias is empty. With LTO compiler (or rather linker) is quite aggressive in removing stuff that it thinks is not needed.

Regards
Andrzej

PS. IMHO this SHARED_LIB define should be removed from the rte_config.h and meson.build should be updated to detect 'default_library' and add it as needed. Don't know exactly how meson behaves if 'default_library' is 'both' - the docs say that it reuses objects from static build, so we might have to work around it for LTO & 'both'.

#16 lore raw

Re: [dpdk-dev] [PATCH v2 01/10] build: add an option to enable LTO build

Quoted 19 lines · Andrzej Ostruszka

On 9/23/19 6:13 PM, Bruce Richardson wrote:
[...]

Quoted 7 lines

However, testing on my system with the meson build, I'm getting lots of link errors [See below]. Any suggestions?

/Bruce

/usr/bin/ld: /tmp/dpdk-testpmd.hncbtE.ltrans93.ltrans.o: in function `ena_stop':
<artificial>:(.text+0x9ed6): undefined reference to `rte_timer_stop'

[...]

What is 'default_library'? It should be 'shared' as mentioned in the docs. The problem is that RTE_BUILD_SHARED_LIB is statically defined in rte_config.h used by meson build. This results in broken static libraries (those that are using versioned symbols - like timer/lpm/distributor) - since the MAP_STATIC_SYMBOL macro defining the default alias is empty. With LTO compiler (or rather linker) is quite aggressive in removing stuff that it thinks is not needed.

Quoted 8 lines

Regards
Andrzej

PS. IMHO this SHARED_LIB define should be removed from the rte_config.h and meson.build should be updated to detect 'default_library' and add it as needed. Don't know exactly how meson behaves if 'default_library' is 'both' - the docs say that it reuses objects from static build, so we might have to work around it for LTO & 'both'.

That proposal won't work either as we build both static and shared libraries in all cases - the default_library value only affects whether the test apps are linked against the .a or .so files.

The real issue seems to be that the compat.h header has different compilation paths for static and shared libraries, which means that any C file including it can't have a .o file that can be used for both a .a and a .so simultaneously. Having it default to the shared library path seems to work fine thus far, but with LTO it seems broken as you say. Adding Neil as the original file author of this in case he has any suggestions here. I'd really rather not have to go back to building .a's and .so's separately.

Regards,
/Bruce

#17 lore raw

Re: [dpdk-dev] [PATCH v2 01/10] build: add an option to enable LTO build

Quoted 1 line · Bruce Richardson

On Tue, Sep 24, 2019 at 08:46:25AM +0200, Andrzej Ostruszka wrote:

[...]

Quoted 10 lines
Quoted 5 lines

PS. IMHO this SHARED_LIB define should be removed from the rte_config.h and meson.build should be updated to detect 'default_library' and add it as needed. Don't know exactly how meson behaves if 'default_library' is 'both' - the docs say that it reuses objects from static build, so we might have to work around it for LTO & 'both'.

That proposal won't work either as we build both static and shared libraries in all cases - the default_library value only affects whether the test apps are linked against the .a or .so files.

Quoted 7 lines

The real issue seems to be that the compat.h header has different compilation paths for static and shared libraries, which means that any C file including it can't have a .o file that can be used for both a .a and a .so simultaneously. Having it default to the shared library path seems to work fine thus far, but with LTO it seems broken as you say. Adding Neil as the original file author of this in case he has any suggestions here. I'd really rather not have to go back to building .a's and .so's separately.

I thought about this already a bit and I see two ways out.

The one which depends on meson's 'default_library' - for "static"/"shared" the case would be easy (along the lines I've mentioned). The "both" would be tricky if we want to rely on meson built-in behaviour.

The other is that we use 'default_library' as we do today (as an indicator which library should be used when linking apps). In that case all is good apart from building static libs out of object files. That would require additional step. That is each object, for which source uses BIND_DEFAULT_SYMBOL/MAP_STATIC_SYMBOL, would need to be processed with "objcopy --add-symbol" adding correct alias before being used to create static lib.

Just my 2 cents.

Regards
Andrzej

#18 lore raw

Re: [dpdk-dev] [PATCH v2 01/10] build: add an option to enable LTO build

Quoted 22 lines · Andrzej Ostruszka
Quoted 1 line · Bruce Richardson

On Tue, Sep 24, 2019 at 08:46:25AM +0200, Andrzej Ostruszka wrote:

[...]

Quoted 10 lines
Quoted 5 lines

PS. IMHO this SHARED_LIB define should be removed from the rte_config.h and meson.build should be updated to detect 'default_library' and add it as needed. Don't know exactly how meson behaves if 'default_library' is 'both' - the docs say that it reuses objects from static build, so we might have to work around it for LTO & 'both'.

That proposal won't work either as we build both static and shared libraries in all cases - the default_library value only affects whether the test apps are linked against the .a or .so files.

Quoted 7 lines

The real issue seems to be that the compat.h header has different compilation paths for static and shared libraries, which means that any C file including it can't have a .o file that can be used for both a .a and a .so simultaneously. Having it default to the shared library path seems to work fine thus far, but with LTO it seems broken as you say. Adding Neil as the original file author of this in case he has any suggestions here. I'd really rather not have to go back to building .a's and .so's separately.

Quoted 7 lines

I thought about this already a bit and I see two ways out.

The one which depends on meson's 'default_library' - for "static"/"shared" the case would be easy (along the lines I've mentioned). The "both" would be tricky if we want to rely on meson built-in behaviour.

Ignoring the both case, which I think is just unsupported right now - it wasn't there when we added support originally for building DPDK - using the shared/static setting still won't solve the problem since it only applied to DPDK's own apps. The reason for building both static and shared libs is to give apps the flexibility to link against either as they prefer - using the pkg-config appropriately. It's perfectly valid to have testpmd linked statically in a DPDK build so that it can be run from the build directory, yet have other apps on the system linked dynamically.

Quoted 8 lines

The other is that we use 'default_library' as we do today (as an indicator which library should be used when linking apps). In that case all is good apart from building static libs out of object files. That would require additional step. That is each object, for which source uses BIND_DEFAULT_SYMBOL/MAP_STATIC_SYMBOL, would need to be processed with "objcopy --add-symbol" adding correct alias before being used to create static lib.

Interesting. It seems a bit awkward a solution, but may be doable.

Quoted 4 lines

Just my 2 cents.

Regards
Andrzej

#19 lore raw

Re: [dpdk-dev] [PATCH v2 01/10] build: add an option to enable LTO build

In reply to Bruce Richardson
Quoted 33 lines · Bruce Richardson
Quoted 19 lines · Andrzej Ostruszka

On 9/23/19 6:13 PM, Bruce Richardson wrote:
[...]

Quoted 7 lines

However, testing on my system with the meson build, I'm getting lots of link errors [See below]. Any suggestions?

/Bruce

/usr/bin/ld: /tmp/dpdk-testpmd.hncbtE.ltrans93.ltrans.o: in function `ena_stop':
<artificial>:(.text+0x9ed6): undefined reference to `rte_timer_stop'

[...]

What is 'default_library'? It should be 'shared' as mentioned in the docs. The problem is that RTE_BUILD_SHARED_LIB is statically defined in rte_config.h used by meson build. This results in broken static libraries (those that are using versioned symbols - like timer/lpm/distributor) - since the MAP_STATIC_SYMBOL macro defining the default alias is empty. With LTO compiler (or rather linker) is quite aggressive in removing stuff that it thinks is not needed.

Quoted 8 lines

Regards
Andrzej

PS. IMHO this SHARED_LIB define should be removed from the rte_config.h and meson.build should be updated to detect 'default_library' and add it as needed. Don't know exactly how meson behaves if 'default_library' is 'both' - the docs say that it reuses objects from static build, so we might have to work around it for LTO & 'both'.

That proposal won't work either as we build both static and shared libraries in all cases - the default_library value only affects whether the test apps are linked against the .a or .so files.

Quoted 8 lines

The real issue seems to be that the compat.h header has different compilation paths for static and shared libraries, which means that any C file including it can't have a .o file that can be used for both a .a and a .so simultaneously. Having it default to the shared library path seems to work fine thus far, but with LTO it seems broken as you say. Adding Neil as the original file author of this in case he has any suggestions here. I'd really rather not have to go back to building .a's and .so's separately.

The notion of using the same object file to link to a static archive and a dso seems somewhat suspect to me in general. I say that because I don't see a way for the linker to know/prove at link time that the options used to compile an object for target (a) will be the same as those used to compile the same object for target (b). In this particular case, you've identified an issue in compilation changes that triggers off the building of dso's vs static archives, but I could envision a scenario in which you might try to build targets for BSD and Linux in parallel, or even for different machines (i.e. build for a least common denominator x8664 target, and a highly optimized recent x8664 processor with all the ISA extensions enabled). We don't do that currently now of course, but we could, and the only way we could do so would be to rebuild all the objects with the compilation flags for each separately.

That said, if the goal is to just overcome this particular situation, it might (strong might), be sufficient to simply augment the MAP_STATIC_SYMBOL macro in the CONFIG_RTE_BUILD_SHARED_LIB=n case to append the 'used' attribute. Ostensibly, LTO would be smart enough then to not eliminate the symbol? Just a thought.

But I think we need to take care here. While its fine solve this particular situation, I think the notion of reusing objects for multiple link targets has the potential to uncover many issues of this class, which won't be as solveable without having to just rebuild objects from scratch.

Neil

Quoted 3 lines

Regards,
/Bruce

#20 lore raw

Re: [dpdk-dev] [PATCH v2 01/10] build: add an option to enable LTO build

Quoted 7 lines · Neil Horman
Quoted 6 lines · Bruce Richardson
Quoted 5 lines · Andrzej Ostruszka

On 9/23/19 6:13 PM, Bruce Richardson wrote:
[...]

Quoted 3 lines

However, testing on my system with the meson build, I'm getting lots of link errors [See below]. Any suggestions?

Quoted 8 lines
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines

/Bruce

/usr/bin/ld: /tmp/dpdk-testpmd.hncbtE.ltrans93.ltrans.o: in function `ena_stop':
<artificial>:(.text+0x9ed6): undefined reference to `rte_timer_stop'

[...]

The notion of using the same object file to link to a static archive and a dso seems somewhat suspect to me in general.

Well I would say there are two different things going on here, one doesn't exclude the other.

Quoted 4 lines

I say that because I don't see a way
for the linker to know/prove at link time that the options used to compile an object for target (a) will be the same as those used to compile the same object for target (b).

Yes and no. So I might build an object differently for say a different platform, but I am not sure I would necessarily build it differently for static versus dynamic (linking is a different story of course).

(lto is also a different story, as when you build objects with lto on, as I remember you end up with guile not bytecode)

Quoted 8 lines

In this particular case, you've identified an issue in
compilation changes that triggers off the building of dso's vs static archives, but I could envision a scenario in which you might try to build targets for BSD and Linux in parallel, or even for different machines (i.e. build for a least common denominator x8664 target, and a highly optimized recent x8664 processor with all the ISA extensions enabled). We don't do that currently now of course, but we could, and the only way we could do so would be to rebuild all the objects with the compilation flags for each separately.

Ok - but there is nothing in the above in this that precludes all of these object variants being used in both static and dynamic builds, it's all down to how they are integrated - FD.io VPP does this out of the box as it happens.

Quoted 6 lines

That said, if the goal is to just overcome this particular situation, it might (strong might), be sufficient to simply augment the MAP_STATIC_SYMBOL macro in the CONFIG_RTE_BUILD_SHARED_LIB=n case to append the 'used' attribute. Ostensibly, LTO would be smart enough then to not eliminate the symbol? Just a thought.

+1, that would be a simple solution.

Quoted 11 lines

But I think we need to take care here. While its fine solve this particular situation, I think the notion of reusing objects for multiple link targets has the potential to uncover many issues of this class, which won't be as solveable without having to just rebuild objects from scratch.

Neil

Quoted 3 lines

Regards,
/Bruce

#21 lore raw

Re: [dpdk-dev] [PATCH v2 01/10] build: add an option to enable LTO build

In reply to Neil Horman
Quoted 3 lines · Neil Horman
Quoted 2 lines · Bruce Richardson
Quoted 1 line · Andrzej Ostruszka

On 9/23/19 6:13 PM, Bruce Richardson wrote:

[...]

Quoted 10 lines
Quoted 8 lines

The real issue seems to be that the compat.h header has different compilation paths for static and shared libraries, which means that any C file including it can't have a .o file that can be used for both a .a and a .so simultaneously. Having it default to the shared library path seems to work fine thus far, but with LTO it seems broken as you say. Adding Neil as the original file author of this in case he has any suggestions here. I'd really rather not have to go back to building .a's and .so's separately.

The notion of using the same object file to link to a static archive and a dso seems somewhat suspect to me in general.

I'd think so too ... but there might be something fishy with gcc here. More on this below.

[...]

Quoted 5 lines

That said, if the goal is to just overcome this particular situation, it might (strong might), be sufficient to simply augment the MAP_STATIC_SYMBOL macro in the CONFIG_RTE_BUILD_SHARED_LIB=n case to append the 'used' attribute. Ostensibly, LTO would be smart enough then to not eliminate the symbol? Just a thought.

Just to clarify things here is the current status in a nutshell.
1. The make based build work with LTO (both static and shared). It is using CONFIG_*_SHARED_LIB option and thus different paths of rte_compat.h are used.

2. The meson build is not using config and has "SHARED" fixed to "y" in config/rte_config.h. This works and produces:
- shared library that is working when linking both w/ and w/o LTO
- static library that is only working when linking w/o LTO - when linking with LTO then it complains about missing symbols for which different symbol versions are defined.

Augmenting MAP_STATIC_SYMBOL won't help since it is not used at all in meson build.

I've played around with couple ideas. Some of them might sound stupid for you - but I'll report them anyway

Modifying symbol tables
-----------------------
I thought that since problems are only with versioned symbols then I'll try to modify the symbols tables archives so they look like in "static make" case e.g. (for just one symbol):

$ objcopy --strip-symbol=rte_timer_subsystem_init@DPDK_2.0 \
	--redefine-sym \
rte_timer_subsystem_init@@DPDK_19.05=rte_timer_subsystem_init \
	librte_timer.a

After this the symbol table looks like in fully static case but this doesn't work so I'm pretty sure that when using LTO linker does not check the symbol table at all and just looks in "lto" sections.

Adding static macro
-------------------
I've added for the shared case macro:

#define MAP_STATIC_SYMBOL(f, p) f \
	__attribute__((alias(RTE_STR(p)),weak))

with the idea that maybe during linking against *.a versioned symbols are not taken into account and if I add weak non-versioned symbol then it will be used when linking against *.a and when linking against *.so the strong versioned one will be used. This thinking is in contrast with the fact that meson build works w/o LTO where only versioned symbols are present in those problematic libs :) - and it doesn't work. Linker reports multiple definitions.

So adding these two together it seems to me that:
a) gcc is using only internal "lto" sections when linking with LTO
b) gcc is storing those versioned symbols in those "lto" sections
c) when linking w/ LTO against archives is not capable to use those versioned symbols in "lto" sections
d) when linking w/ LTO against shared libraries is capable to use versioned symbols from "lto" sections
e) when linking w/o LTO against archive is capable to use versioned symbols from global symbol table (in contrast with point 'c' above)

I'd appreciate some input from those who know gcc internals. In the meantime I'll try to come up with minimal example and follow up on gcc related lists/groups.

Regards
Andrzej

#22 lore raw

Re: [dpdk-dev] [PATCH v2 01/10] build: add an option to enable LTO build

Quoted 81 lines · Andrzej Ostruszka
Quoted 3 lines · Neil Horman
Quoted 2 lines · Bruce Richardson
Quoted 1 line · Andrzej Ostruszka

On 9/23/19 6:13 PM, Bruce Richardson wrote:

[...]

Quoted 10 lines
Quoted 8 lines

The real issue seems to be that the compat.h header has different compilation paths for static and shared libraries, which means that any C file including it can't have a .o file that can be used for both a .a and a .so simultaneously. Having it default to the shared library path seems to work fine thus far, but with LTO it seems broken as you say. Adding Neil as the original file author of this in case he has any suggestions here. I'd really rather not have to go back to building .a's and .so's separately.

The notion of using the same object file to link to a static archive and a dso seems somewhat suspect to me in general.

I'd think so too ... but there might be something fishy with gcc here. More on this below.

[...]

Quoted 5 lines

That said, if the goal is to just overcome this particular situation, it might (strong might), be sufficient to simply augment the MAP_STATIC_SYMBOL macro in the CONFIG_RTE_BUILD_SHARED_LIB=n case to append the 'used' attribute. Ostensibly, LTO would be smart enough then to not eliminate the symbol? Just a thought.

Just to clarify things here is the current status in a nutshell.
1. The make based build work with LTO (both static and shared). It is using CONFIG_*_SHARED_LIB option and thus different paths of rte_compat.h are used.

2. The meson build is not using config and has "SHARED" fixed to "y" in config/rte_config.h. This works and produces:
- shared library that is working when linking both w/ and w/o LTO
- static library that is only working when linking w/o LTO - when linking with LTO then it complains about missing symbols for which different symbol versions are defined.

Augmenting MAP_STATIC_SYMBOL won't help since it is not used at all in meson build.

I've played around with couple ideas. Some of them might sound stupid for you - but I'll report them anyway

Modifying symbol tables
-----------------------
I thought that since problems are only with versioned symbols then I'll try to modify the symbols tables archives so they look like in "static make" case e.g. (for just one symbol):

$ objcopy --strip-symbol=rte_timer_subsystem_init@DPDK_2.0 \
	--redefine-sym \
rte_timer_subsystem_init@@DPDK_19.05=rte_timer_subsystem_init \
	librte_timer.a

After this the symbol table looks like in fully static case but this doesn't work so I'm pretty sure that when using LTO linker does not check the symbol table at all and just looks in "lto" sections.

Adding static macro
-------------------
I've added for the shared case macro:

#define MAP_STATIC_SYMBOL(f, p) f \
	__attribute__((alias(RTE_STR(p)),weak))

with the idea that maybe during linking against *.a versioned symbols are not taken into account and if I add weak non-versioned symbol then it will be used when linking against *.a and when linking against *.so the strong versioned one will be used. This thinking is in contrast with the fact that meson build works w/o LTO where only versioned symbols are present in those problematic libs :) - and it doesn't work. Linker reports multiple definitions.

So adding these two together it seems to me that:
a) gcc is using only internal "lto" sections when linking with LTO
b) gcc is storing those versioned symbols in those "lto" sections
c) when linking w/ LTO against archives is not capable to use those versioned symbols in "lto" sections
d) when linking w/ LTO against shared libraries is capable to use versioned symbols from "lto" sections
e) when linking w/o LTO against archive is capable to use versioned symbols from global symbol table (in contrast with point 'c' above)

Quoted 4 lines

I'd appreciate some input from those who know gcc internals. In the meantime I'll try to come up with minimal example and follow up on gcc related lists/groups.

Please have a look over the patchset I just posted, as one possible solution to this. http://patches.dpdk.org/project/dpdk/list/?series=6594

For the general case of compiling DPDK, a given block of C code is going to result in the same object file whether compiled for static or shared (so long as -fPIC is passed to the compile), so running two compiles for each and every C file would be wasted effort. However, for those files with function versioning, I think any proper solution has to involve compiling twice with different macros, so that's what the patchset seeks to achieve.

/Bruce

#23 lore raw

Re: [dpdk-dev] [PATCH v2 01/10] build: add an option to enable LTO build

In reply to Andrzej Ostruszka
Quoted 6 lines

If "in or out" means "either accept the patches with LTO on and no config option or reject them" then I disagree. Even if run time improvements are questionable I find the additional link time warnings beneficial and would like to have an easy way to turn them on when doing final touches before pushing out.

My rational for my "in or out" suggestion, is always to have as few rarely used build options as possible. To reduce the number of things the community has to support.

Ray K

#24 lore raw

Re: [dpdk-dev] [PATCH v2 01/10] build: add an option to enable LTO build

In reply to Ray Kinsella

18/09/2019 15:32, Ray Kinsella:

Quoted 8 lines

this is cool, good work.
comments below.

Quoted 4 lines · Andrzej Ostruszka
lib/librte_distributor/rte_distributor.c +0 -0
@@ -32,7 +32,7 @@ EAL_REGISTER_TAILQ(rte_dist_burst_tailq)
 
Quoted 8 lines

lib/librte_distributor/rte_distributor.c EAL_REGISTER_TAILQ(rte_dist_burst_tailq) · line 35

Quoted 4 lines
lib/librte_distributor/rte_distributor.c +1 -1
 /**** Burst Packet APIs called by workers ****/
 
-void
+void __vsym

all these additional __vsym annotations looks like they belong in a seperate patch, as they are fixing a bug and are not directly related to adding LTO the build system.

Andrzej, you did not reply to this question.
This is a real blocker for merging this series.

Should __vsym addition be in a separate patch? Should we document its use in rte_function_versioning.h and versioning.rst?

Please explain, thanks.

#25 lore raw

Re: [dpdk-dev] [PATCH v2 01/10] build: add an option to enable LTO build

Quoted 18 lines · Thomas Monjalon

18/09/2019 15:32, Ray Kinsella:

Quoted 8 lines

this is cool, good work.
comments below.

Quoted 4 lines · Andrzej Ostruszka
lib/librte_distributor/rte_distributor.c +0 -0
@@ -32,7 +32,7 @@ EAL_REGISTER_TAILQ(rte_dist_burst_tailq)
 
Quoted 8 lines

lib/librte_distributor/rte_distributor.c EAL_REGISTER_TAILQ(rte_dist_burst_tailq) · line 35

Quoted 4 lines
lib/librte_distributor/rte_distributor.c +1 -1
 /**** Burst Packet APIs called by workers ****/
 
-void
+void __vsym

all these additional __vsym annotations looks like they belong in a seperate patch, as they are fixing a bug and are not directly related to adding LTO the build system.

Quoted 2 lines

Andrzej, you did not reply to this question.
This is a real blocker for merging this series.

Thomas, thank you for the reminder. Somehow that comment has escaped me
- although I've read it then.

Quoted 1 line

Should __vsym addition be in a separate patch?

I'm fine both ways. You could argue that:
- it is a bug since '__vsym' clearly annotates the function as being used as a particular version of a symbol and as such it was missing
- or as a part of enablement for LTO since without it compiler/linker should not be removing given function and '__vsym' really is just a "attribute(used)" to tell optimizing compiler/linker that this function should not be removed.

Since you raised that question I'm guessing that you prefer it to be in a separate patch - so, unless you object now, I'm going to split it in the next version and have it as a first patch.

Quoted 2 lines

Should we document its use in rte_function_versioning.h and versioning.rst?

Yes, I think so. I'll add that.

Again, thank you for the reminder and comments.

Regards
Andrzej

#26 lore raw

Re: [dpdk-dev] [PATCH v2 01/10] build: add an option to enable LTO build

28/10/2019 09:36, Andrzej Ostruszka:

Quoted 39 lines · Thomas Monjalon
Quoted 18 lines

18/09/2019 15:32, Ray Kinsella:

Quoted 8 lines

this is cool, good work.
comments below.

Quoted 4 lines · Andrzej Ostruszka
lib/librte_distributor/rte_distributor.c +0 -0
@@ -32,7 +32,7 @@ EAL_REGISTER_TAILQ(rte_dist_burst_tailq)
 
Quoted 8 lines

lib/librte_distributor/rte_distributor.c EAL_REGISTER_TAILQ(rte_dist_burst_tailq) · line 35

Quoted 4 lines
lib/librte_distributor/rte_distributor.c +1 -1
 /**** Burst Packet APIs called by workers ****/
 
-void
+void __vsym

all these additional __vsym annotations looks like they belong in a seperate patch, as they are fixing a bug and are not directly related to adding LTO the build system.

Quoted 2 lines

Andrzej, you did not reply to this question.
This is a real blocker for merging this series.

Thomas, thank you for the reminder. Somehow that comment has escaped me
- although I've read it then.

Quoted 1 line

Should __vsym addition be in a separate patch?

I'm fine both ways. You could argue that:
- it is a bug since '__vsym' clearly annotates the function as being used as a particular version of a symbol and as such it was missing
- or as a part of enablement for LTO since without it compiler/linker should not be removing given function and '__vsym' really is just a "attribute(used)" to tell optimizing compiler/linker that this function should not be removed.

Since you raised that question I'm guessing that you prefer it to be in a separate patch - so, unless you object now, I'm going to split it in the next version and have it as a first patch.

Quoted 6 lines
Quoted 2 lines

Should we document its use in rte_function_versioning.h and versioning.rst?

Yes, I think so. I'll add that.

Again, thank you for the reminder and comments.

Thank you, a separate patch with doc update is very welcome.

#27 lore raw

Re: [dpdk-dev] [PATCH v2 01/10] build: add an option to enable LTO build

In reply to Andrzej Ostruszka
Quoted 1 line · Andrzej Ostruszka

On 10/27/19 12:31 PM, Thomas Monjalon wrote:

[...]

Quoted 4 lines
Quoted 2 lines

Should we document its use in rte_function_versioning.h and versioning.rst?

Yes, I think so. I'll add that.

One quick notice. There is a slight mismatch between documentation of VERSION_SYMBOL/BIND_DEFAULT_SYMBOL and their implementation i.e. the docs claim that the underscore is added by these macros (b_e) while it is currently being supplied at the macro invocations.

I'll try to update docs to match the header but if you'd like to change it the other way* please let me know.

Regards
Andrzej

[*] Change the implementation of these macros to use:

    ... RTE_STR(b) "_" RTE_STR(e)

and remove underscore from all invocations of these macros.

#28 lore raw

Re: [dpdk-dev] [PATCH v2 01/10] build: add an option to enable LTO build

28/10/2019 13:12, Andrzej Ostruszka:

Quoted 16 lines · Andrzej Ostruszka
Quoted 1 line · Thomas Monjalon

[...]

Quoted 4 lines
Quoted 2 lines

Should we document its use in rte_function_versioning.h and versioning.rst?

Yes, I think so. I'll add that.

One quick notice. There is a slight mismatch between documentation of VERSION_SYMBOL/BIND_DEFAULT_SYMBOL and their implementation i.e. the docs claim that the underscore is added by these macros (b_e) while it is currently being supplied at the macro invocations.

I'll try to update docs to match the header but if you'd like to change it the other way* please let me know.

Quoted 6 lines

Regards
Andrzej

[*] Change the implementation of these macros to use:

    ... RTE_STR(b) "_" RTE_STR(e)

and remove underscore from all invocations of these macros.

Please do the doc fix in a separate patch, Cc'ing Neil Horman who can decide what is best.
Thanks

#29 lore raw

PATCH v2 2/10 eventdev: fix possible use of uninitialized var

In reply to Andrzej Ostruszka

Fix the logic for the case of event queue allowing all schedule types.

---
 lib/librte_eventdev/rte_event_timer_adapter.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
lib/librte_eventdev/rte_event_timer_adapter.c +4 -4
index 5ce399eca..161e21a68 100644
@@ -706,11 +706,11 @@ check_destination_event_queue(struct rte_event_timer *evtim,
 				       RTE_EVENT_QUEUE_ATTR_SCHEDULE_TYPE,
 				       &sched_type);
 
-	if ((ret < 0 && ret != -EOVERFLOW) ||
-	    evtim->ev.sched_type != sched_type)
-		return -1;
+	if ((ret == 0 && evtim->ev.sched_type == sched_type) ||
+	    ret == -EOVERFLOW)
+		return 0;
 
-	return 0;
+	return -1;
 }
 
 static int
Signature · 1 line
2.17.1
Reviewed-by on v2 2/10
#30 lore raw

Re: [dpdk-dev] [PATCH v2 02/10] eventdev: fix possible use of uninitialized var

Quoted 4 lines · Andrzej Ostruszka

Fix the logic for the case of event queue allowing all schedule types.

This fix looks good to me.

# Please fix check-git-log.sh errors.
Missing 'Fixes' tag:
eventdev: fix possible use of uninitialized var

# If it is reported by LTO, please add the compiler warning output in git commit.

With the above fix:

Quoted 27 lines
---
 lib/librte_eventdev/rte_event_timer_adapter.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
lib/librte_eventdev/rte_event_timer_adapter.c +4 -5
index 5ce399eca..161e21a68 100644
@@ -706,11 +706,11 @@ check_destination_event_queue(struct rte_event_timer *evtim,
                                       RTE_EVENT_QUEUE_ATTR_SCHEDULE_TYPE,
                                       &sched_type);
 
-       if ((ret < 0 && ret != -EOVERFLOW) ||
-           evtim->ev.sched_type != sched_type)
-               return -1;
+       if ((ret == 0 && evtim->ev.sched_type == sched_type) ||
+           ret == -EOVERFLOW)
+               return 0;
 
-       return 0;
+       return -1;
 }
 
 static int
--

2.17.1

#31 lore raw

PATCH v2 3/10 app/eventdev: fix maybe-uninitialized warnings for LTO build

In reply to Andrzej Ostruszka

During LTO build compiler reports some 'false positive' warnings about variables being possibly used uninitialized. This patch silences these warnings.

---
 app/test-eventdev/test_perf_common.c     | 2 +-
 app/test-eventdev/test_pipeline_common.c | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)
app/test-eventdev/test_perf_common.c +1 -1
index aa925a7ef..a974685cb 100644
@@ -439,7 +439,7 @@ perf_event_timer_adapter_setup(struct test_perf *t)
 
 		if (!(adapter_info.caps &
 				RTE_EVENT_TIMER_ADAPTER_CAP_INTERNAL_PORT)) {
-			uint32_t service_id;
+			uint32_t service_id = -1U;
 
 			rte_event_timer_adapter_service_id_get(wl,
 					&service_id);
app/test-eventdev/test_pipeline_common.c +2 -2
index 16c49b860..813d0cf44 100644
@@ -306,7 +306,7 @@ pipeline_event_rx_adapter_setup(struct evt_options *opt, uint8_t stride,
 		}
 
 		if (!(cap & RTE_EVENT_ETH_RX_ADAPTER_CAP_INTERNAL_PORT)) {
-			uint32_t service_id;
+			uint32_t service_id = -1U;
 
 			rte_event_eth_rx_adapter_service_id_get(prod,
 					&service_id);
@@ -358,7 +358,7 @@ pipeline_event_tx_adapter_setup(struct evt_options *opt,
 		}
 
 		if (!(cap & RTE_EVENT_ETH_TX_ADAPTER_CAP_INTERNAL_PORT)) {
-			uint32_t service_id;
+			uint32_t service_id = -1U;
 
 			rte_event_eth_tx_adapter_service_id_get(consm,
 					&service_id);
Signature · 1 line
2.17.1
Reviewed-by on v2 3/10
#32 lore raw

Re: [dpdk-dev] [PATCH v2 03/10] app/eventdev: fix maybe-uninitialized warnings for LTO build

Quoted 6 lines · Andrzej Ostruszka

During LTO build compiler reports some 'false positive' warnings about variables being possibly used uninitialized. This patch silences these warnings.

# Please fix check-git-log.sh errors.
Missing 'Fixes' tag:
app/eventdev: fix maybe-uninitialized warnings for LTO build

# Please add the compiler 'false positive' warning output in git commit.

# Since it is a slow path change, IMO there is no harm in introducing this change to make compiler happy.

With the above fix:

Quoted 43 lines
---
 app/test-eventdev/test_perf_common.c     | 2 +-
 app/test-eventdev/test_pipeline_common.c | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)
app/test-eventdev/test_perf_common.c +1 -1
index aa925a7ef..a974685cb 100644
@@ -439,7 +439,7 @@ perf_event_timer_adapter_setup(struct test_perf *t)
 
                if (!(adapter_info.caps &
                                RTE_EVENT_TIMER_ADAPTER_CAP_INTERNAL_PORT)) {
-                       uint32_t service_id;
+                       uint32_t service_id = -1U;
 
                        rte_event_timer_adapter_service_id_get(wl,
                                        &service_id);
app/test-eventdev/test_pipeline_common.c +2 -3
index 16c49b860..813d0cf44 100644
@@ -306,7 +306,7 @@ pipeline_event_rx_adapter_setup(struct evt_options *opt, uint8_t stride,
                }
 
                if (!(cap & RTE_EVENT_ETH_RX_ADAPTER_CAP_INTERNAL_PORT)) {
-                       uint32_t service_id;
+                       uint32_t service_id = -1U;
 
                        rte_event_eth_rx_adapter_service_id_get(prod,
                                        &service_id);
@@ -358,7 +358,7 @@ pipeline_event_tx_adapter_setup(struct evt_options *opt,
                }
 
                if (!(cap & RTE_EVENT_ETH_TX_ADAPTER_CAP_INTERNAL_PORT)) {
-                       uint32_t service_id;
+                       uint32_t service_id = -1U;
 
                        rte_event_eth_tx_adapter_service_id_get(consm,
                                        &service_id);
--

2.17.1

#33 lore raw

PATCH v2 4/10 event/octeontx2: fix maybe-uninitialized warnings for LTO build

In reply to Andrzej Ostruszka

During LTO build compiler reports some 'false positive' warnings about variables being possibly used uninitialized. This patch silences these warnings.

---
 drivers/event/octeontx2/otx2_tim_worker.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/event/octeontx2/otx2_tim_worker.h +1 -1
index b193e2cab..50db6543c 100644
@@ -337,7 +337,7 @@ tim_add_entry_brst(struct otx2_tim_ring * const tim_ring,
 		   const struct otx2_tim_ent *ents,
 		   const uint16_t nb_timers, const uint8_t flags)
 {
-	struct otx2_tim_ent *chunk;
+	struct otx2_tim_ent *chunk = NULL;
 	struct otx2_tim_bkt *bkt;
 	uint16_t chunk_remainder;
 	uint16_t index = 0;
Signature · 1 line
2.17.1
#34 lore raw

PATCH v2 5/10 app/test: fix maybe-uninitialized warnings for LTO build

In reply to Andrzej Ostruszka

During LTO build compiler reports some 'false positive' warnings about variables being possibly used uninitialized. This patch silences these warnings.

---
 app/test/test_hash_readwrite.c     |  2 +-
 app/test/test_link_bonding_mode4.c | 10 ++++++++--
 app/test/test_memzone.c            |  3 ++-
 3 files changed, 11 insertions(+), 4 deletions(-)
app/test/test_hash_readwrite.c +1 -1
index 4376b099b..615767fb6 100644
@@ -298,7 +298,7 @@ test_rw_reader(void *arg)
 
 	begin = rte_rdtsc_precise();
 	for (i = 0; i < read_cnt; i++) {
-		void *data;
+		void *data = arg;
 		rte_hash_lookup_data(tbl_rw_test_param.h,
 				tbl_rw_test_param.keys + i,
 				&data);
app/test/test_link_bonding_mode4.c +8 -2
index bbb4e9cce..5f7df1b5e 100644
@@ -224,7 +224,7 @@ configure_ethdev(uint16_t port_id, uint8_t start)
 static int
 add_slave(struct slave_conf *slave, uint8_t start)
 {
-	struct rte_ether_addr addr, addr_check;
+	struct rte_ether_addr addr, addr_check = { { 0 } };
 
 	/* Some sanity check */
 	RTE_VERIFY(test_params.slave_ports <= slave &&
@@ -578,7 +578,13 @@ bond_get_update_timeout_ms(void)
 {
 	struct rte_eth_bond_8023ad_conf conf;
 
-	rte_eth_bond_8023ad_conf_get(test_params.bonded_port_id, &conf);
+	if (rte_eth_bond_8023ad_conf_get(test_params.bonded_port_id, &conf) < 0) {
+		RTE_LOG(DEBUG, EAL, "Failed to get bonding configuration: "
+				    "%s at %d\n", __func__, __LINE__);
+		RTE_TEST_TRACE_FAILURE(__FILE__, __LINE__, __func__);
+		return 0;
+	}
+
 	return conf.update_timeout_ms;
 }
 
app/test/test_memzone.c +2 -1
index 7501b63c5..c284dcb44 100644
@@ -476,7 +476,8 @@ find_max_block_free_size(unsigned int align, unsigned int socket_id)
 	struct rte_malloc_socket_stats stats;
 	size_t len, overhead;
 
-	rte_malloc_get_socket_stats(socket_id, &stats);
+	if (rte_malloc_get_socket_stats(socket_id, &stats) < 0)
+		return 0;
 
 	len = stats.greatest_free_size;
 	overhead = MALLOC_ELEM_OVERHEAD;
Signature · 1 line
2.17.1
#35 lore raw

PATCH v2 6/10 net/dpaa2: fix possible use of uninitialized vars

In reply to Andrzej Ostruszka

This patch fixes 'maybe-uninitialized' warnings reported by compiler when using LTO.

---
 drivers/net/dpaa2/base/dpaa2_hw_dpni.c | 1 +
 drivers/net/dpaa2/mc/dpkg.c            | 2 +-
 drivers/net/dpaa2/mc/dpni.c            | 9 ++++++---
 3 files changed, 8 insertions(+), 4 deletions(-)
drivers/net/dpaa2/base/dpaa2_hw_dpni.c +1 -0
index 56e2e56a3..2f6534a31 100644
@@ -51,6 +51,7 @@ rte_pmd_dpaa2_set_custom_hash(uint16_t port_id,
 	kg_cfg.extracts[0].type = DPKG_EXTRACT_FROM_DATA;
 	kg_cfg.extracts[0].extract.from_data.offset = offset;
 	kg_cfg.extracts[0].extract.from_data.size = size;
+	kg_cfg.extracts[0].num_of_byte_masks = 0;
 	kg_cfg.num_extracts = 1;
 
 	ret = dpkg_prepare_key_cfg(&kg_cfg, p_params);
drivers/net/dpaa2/mc/dpkg.c +1 -1
index 80f94f40e..7aa63ea12 100644
@@ -63,7 +63,7 @@ dpkg_prepare_key_cfg(const struct dpkg_profile_cfg *cfg, uint8_t *key_cfg_buf)
 		dpkg_set_field(extr->extract_type, EXTRACT_TYPE,
 			       cfg->extracts[i].type);
 
-		for (j = 0; j < DPKG_NUM_OF_MASKS; j++) {
+		for (j = 0; j < extr->num_of_byte_masks; j++) {
 			extr->masks[j].mask = cfg->extracts[i].masks[j].mask;
 			extr->masks[j].offset =
 				cfg->extracts[i].masks[j].offset;
drivers/net/dpaa2/mc/dpni.c +6 -3
index 362cd476f..b74a1a317 100644
@@ -1803,10 +1803,13 @@ int dpni_set_congestion_notification(struct fsl_mc_io *mc_io,
 	cmd_params->qtype = qtype;
 	cmd_params->tc = tc_id;
 	cmd_params->congestion_point = cfg->cg_point;
-	cmd_params->cgid = (uint8_t)cfg->cgid;
-	cmd_params->dest_id = cpu_to_le32(cfg->dest_cfg.dest_id);
+	if (cfg->cg_point == DPNI_CP_CONGESTION_GROUP)
+		cmd_params->cgid = (uint8_t)cfg->cgid;
+	if (cfg->dest_cfg.dest_type != DPNI_DEST_NONE) {
+		cmd_params->dest_id = cpu_to_le32(cfg->dest_cfg.dest_id);
+		cmd_params->dest_priority = cfg->dest_cfg.priority;
+	}
 	cmd_params->notification_mode = cpu_to_le16(cfg->notification_mode);
-	cmd_params->dest_priority = cfg->dest_cfg.priority;
 	cmd_params->message_iova = cpu_to_le64(cfg->message_iova);
 	cmd_params->message_ctx = cpu_to_le64(cfg->message_ctx);
 	cmd_params->threshold_entry = cpu_to_le32(cfg->threshold_entry);
Signature · 1 line
2.17.1
#36 lore raw

PATCH v2 7/10 net/e1000: fix maybe-uninitialized warnings for LTO build

In reply to Andrzej Ostruszka

During LTO build compiler reports some 'false positive' warnings about variables being possibly used uninitialized. This patch silences these warnings.

---
 drivers/net/e1000/base/e1000_82543.c   | 2 +-
 drivers/net/e1000/base/e1000_ich8lan.c | 2 +-
 drivers/net/e1000/base/e1000_phy.c     | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)
drivers/net/e1000/base/e1000_82543.c +1 -1
index 810899eb6..dfde2a8b9 100644
@@ -1027,7 +1027,7 @@ STATIC s32 e1000_setup_copper_link_82543(struct e1000_hw *hw)
 {
 	u32 ctrl;
 	s32 ret_val;
-	bool link;
+	bool link = true;
 
 	DEBUGFUNC("e1000_setup_copper_link_82543");
 
drivers/net/e1000/base/e1000_ich8lan.c +1 -1
index accc6ea01..5241cf698 100644
@@ -5533,7 +5533,7 @@ void e1000_igp3_phy_powerdown_workaround_ich8lan(struct e1000_hw *hw)
 void e1000_gig_downshift_workaround_ich8lan(struct e1000_hw *hw)
 {
 	s32 ret_val;
-	u16 reg_data;
+	u16 reg_data = 0;
 
 	DEBUGFUNC("e1000_gig_downshift_workaround_ich8lan");
 
drivers/net/e1000/base/e1000_phy.c +1 -1
index 7d08f836f..956c06747 100644
@@ -1664,7 +1664,7 @@ s32 e1000_copper_link_autoneg(struct e1000_hw *hw)
 s32 e1000_setup_copper_link_generic(struct e1000_hw *hw)
 {
 	s32 ret_val;
-	bool link;
+	bool link = true;
 
 	DEBUGFUNC("e1000_setup_copper_link_generic");
 
Signature · 1 line
2.17.1
#37 lore raw

PATCH v2 8/10 net/i40e: fix maybe-uninitialized warnings for LTO build

In reply to Andrzej Ostruszka

During LTO build compiler reports some 'false positive' warnings about variables being possibly used uninitialized. This patch silences these warnings.

---
 drivers/net/i40e/i40e_ethdev.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/i40e/i40e_ethdev.c +1 -1
index 4e40b7ab5..525a6b69b 100644
@@ -8376,7 +8376,7 @@ static int
 i40e_add_vxlan_port(struct i40e_pf *pf, uint16_t port, int udp_type)
 {
 	int  idx, ret;
-	uint8_t filter_idx;
+	uint8_t filter_idx = 0;
 	struct i40e_hw *hw = I40E_PF_TO_HW(pf);
 
 	idx = i40e_get_vxlan_port_idx(pf, port);
Signature · 1 line
2.17.1
#38 lore raw

PATCH v2 9/10 net/ifc: fix maybe-uninitialized warnings for LTO build

In reply to Andrzej Ostruszka

During LTO build compiler reports some 'false positive' warnings about variables being possibly used uninitialized. This patch silences these warnings.

---
 drivers/net/ifc/ifcvf_vdpa.c | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)
drivers/net/ifc/ifcvf_vdpa.c +9 -5
index 8de9ef199..4cdff8712 100644
@@ -304,8 +304,8 @@ vdpa_ifcvf_stop(struct ifcvf_internal *internal)
 	struct ifcvf_hw *hw = &internal->hw;
 	uint32_t i;
 	int vid;
-	uint64_t features;
-	uint64_t log_base, log_size;
+	uint64_t features = 0;
+	uint64_t log_base = 0, log_size = 0;
 	uint64_t len;
 
 	vid = internal->vid;
@@ -348,6 +348,8 @@ vdpa_enable_vfio_intr(struct ifcvf_internal *internal, bool m_rx)
 	struct rte_vhost_vring vring;
 	int fd;
 
+	vring.callfd = -1;
+
 	nr_vring = rte_vhost_get_vring_num(internal->vid);
 
 	irq_set = (struct vfio_irq_set *)irq_set_buf;
@@ -442,6 +444,7 @@ notify_relay(void *arg)
 	}
 	internal->epfd = epfd;
 
+	vring.kickfd = -1;
 	for (qid = 0; qid < q_num; qid++) {
 		ev.events = EPOLLIN | EPOLLPRI;
 		rte_vhost_get_vhost_vring(internal->vid, qid, &vring);
@@ -577,7 +580,7 @@ m_ifcvf_start(struct ifcvf_internal *internal)
 	struct ifcvf_hw *hw = &internal->hw;
 	uint32_t i, nr_vring;
 	int vid, ret;
-	struct rte_vhost_vring vq;
+	struct rte_vhost_vring vq = { 0 };
 	void *vring_buf;
 	uint64_t m_vring_iova = IFCVF_MEDIATED_VRING;
 	uint64_t size;
@@ -721,6 +724,7 @@ vring_relay(void *arg)
 	}
 	internal->epfd = epfd;
 
+	vring.kickfd = -1;
 	for (qid = 0; qid < q_num; qid++) {
 		ev.events = EPOLLIN | EPOLLPRI;
 		rte_vhost_get_vhost_vring(vid, qid, &vring);
@@ -930,11 +934,11 @@ ifcvf_dev_close(int vid)
 static int
 ifcvf_set_features(int vid)
 {
-	uint64_t features;
+	uint64_t features = 0;
 	int did;
 	struct internal_list *list;
 	struct ifcvf_internal *internal;
-	uint64_t log_base, log_size;
+	uint64_t log_base = 0, log_size = 0;
 
 	did = rte_vhost_get_vdpa_device_id(vid);
 	list = find_internal_resource_by_did(did);
Signature · 1 line
2.17.1
#39 lore raw

PATCH v2 10/10 net/qede: fix maybe-uninitialized warnings for LTO build

In reply to Andrzej Ostruszka

During LTO build compiler reports some 'false positive' warnings about variables being possibly used uninitialized. This patch silences these warnings.

---
 drivers/net/qede/base/ecore_mcp.c | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)
drivers/net/qede/base/ecore_mcp.c +7 -6
index 6c6560688..666c0fe12 100644
@@ -2278,7 +2278,7 @@ enum _ecore_status_t ecore_mcp_trans_speed_mask(struct ecore_hwfn *p_hwfn,
 						struct ecore_ptt *p_ptt,
 						u32 *p_speed_mask)
 {
-	u32 transceiver_type, transceiver_state;
+	u32 transceiver_type = ETH_TRANSCEIVER_TYPE_NONE, transceiver_state;
 
 	ecore_mcp_get_transceiver_data(p_hwfn, p_ptt, &transceiver_state,
 				       &transceiver_type);
@@ -3163,7 +3163,8 @@ enum _ecore_status_t ecore_mcp_nvm_put_file_begin(struct ecore_dev *p_dev,
 enum _ecore_status_t ecore_mcp_nvm_write(struct ecore_dev *p_dev, u32 cmd,
 					 u32 addr, u8 *p_buf, u32 len)
 {
-	u32 buf_idx, buf_size, nvm_cmd, nvm_offset, resp, param;
+	u32 buf_idx, buf_size, nvm_cmd, nvm_offset;
+	u32 resp = FW_MSG_CODE_ERROR, param;
 	struct ecore_hwfn *p_hwfn = ECORE_LEADING_HWFN(p_dev);
 	enum _ecore_status_t rc = ECORE_INVAL;
 	struct ecore_ptt *p_ptt;
@@ -3370,7 +3371,7 @@ enum _ecore_status_t ecore_mcp_gpio_read(struct ecore_hwfn *p_hwfn,
 					 u16 gpio, u32 *gpio_val)
 {
 	enum _ecore_status_t rc = ECORE_SUCCESS;
-	u32 drv_mb_param = 0, rsp;
+	u32 drv_mb_param = 0, rsp = 0;
 
 	drv_mb_param = (gpio << DRV_MB_PARAM_GPIO_NUMBER_OFFSET);
 
@@ -3391,7 +3392,7 @@ enum _ecore_status_t ecore_mcp_gpio_write(struct ecore_hwfn *p_hwfn,
 					  u16 gpio, u16 gpio_val)
 {
 	enum _ecore_status_t rc = ECORE_SUCCESS;
-	u32 drv_mb_param = 0, param, rsp;
+	u32 drv_mb_param = 0, param, rsp = 0;
 
 	drv_mb_param = (gpio << DRV_MB_PARAM_GPIO_NUMBER_OFFSET) |
 		(gpio_val << DRV_MB_PARAM_GPIO_VALUE_OFFSET);
@@ -3481,7 +3482,7 @@ enum _ecore_status_t ecore_mcp_bist_clock_test(struct ecore_hwfn *p_hwfn,
 enum _ecore_status_t ecore_mcp_bist_nvm_test_get_num_images(
 	struct ecore_hwfn *p_hwfn, struct ecore_ptt *p_ptt, u32 *num_images)
 {
-	u32 drv_mb_param = 0, rsp;
+	u32 drv_mb_param = 0, rsp = 0;
 	enum _ecore_status_t rc = ECORE_SUCCESS;
 
 	drv_mb_param = (DRV_MB_PARAM_BIST_NVM_TEST_NUM_IMAGES <<
@@ -3865,7 +3866,7 @@ enum _ecore_status_t
 __ecore_mcp_resc_lock(struct ecore_hwfn *p_hwfn, struct ecore_ptt *p_ptt,
 		      struct ecore_resc_lock_params *p_params)
 {
-	u32 param = 0, mcp_resp, mcp_param;
+	u32 param = 0, mcp_resp = 0, mcp_param = 0;
 	u8 opcode;
 	enum _ecore_status_t rc;
 
Signature · 1 line
2.17.1

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

#42 lore raw

Re: [dpdk-dev] [PATCH v3 01/10] build: add an option to enable LTO build

Quoted 78 lines · Andrzej Ostruszka

This patch adds an option to enable link time optimization. In addition to LTO option itself (-flto) fat-lto-objects are being used. This is because during the build pmdinfogen scans the generated ELF objects to find this_pmd_name* symbol in symbol table. Without fat-lto-objects gcc produces ELF only with extra symbols for internal use during linking.

---
 .travis.yml                                  |  7 ++++
 config/common_base                           |  5 +++
 config/meson.build                           | 15 ++++++++
 doc/guides/prog_guide/lto.rst                | 36 ++++++++++++++++++++
 doc/guides/rel_notes/release_19_11.rst       |  8 +++++
 lib/librte_distributor/rte_distributor.c     | 18 +++++-----
 lib/librte_distributor/rte_distributor_v20.c | 18 +++++-----
 lib/librte_lpm/rte_lpm.c                     | 28 +++++++--------
 lib/librte_lpm/rte_lpm6.c                    | 16 ++++-----
 lib/librte_timer/rte_timer.c                 | 20 +++++------
 mk/toolchain/gcc/rte.toolchain-compat.mk     |  4 +++
 mk/toolchain/gcc/rte.vars.mk                 | 12 +++++++
 mk/toolchain/icc/rte.vars.mk                 |  8 +++++
 13 files changed, 145 insertions(+), 50 deletions(-)
 create mode 100644 doc/guides/prog_guide/lto.rst
.travis.yml +7 -0
index 781f9f666..70d221852 100644
@@ -31,6 +31,7 @@ env:
   - DEF_LIB="static" OPTS="-Denable_kmods=false"
   - DEF_LIB="shared" OPTS="-Denable_kmods=false"
   - DEF_LIB="shared" RUN_TESTS=1 BUILD_DOCS=1
+  - DEF_LIB="shared" OPTS="-Db_lto=true"
 
 matrix:
   include:
@@ -100,6 +101,12 @@ matrix:
       apt:
         packages:
           - *extra_packages
+  - env: DEF_LIB="shared" OPTS="-Db_lto=true" EXTRA_PACKAGES=1
+    compiler: gcc
+    addons:
+      apt:
+        packages:
+          - *extra_packages
 
 
 script: ./.ci/${TRAVIS_OS_NAME}-build.sh
config/common_base +5 -0
index 8ef75c203..73a55fdec 100644
@@ -49,6 +49,11 @@ CONFIG_RTE_FORCE_INTRINSICS=n
 #
 CONFIG_RTE_ARCH_STRICT_ALIGN=n
 
+#
+# Enable link time optimization
+#
+CONFIG_RTE_ENABLE_LTO=n
+
 #
 # Compile to share library
 #
config/meson.build +5 -0
index 2bafea530..97bbc323b 100644
@@ -196,3 +196,18 @@ add_project_arguments('-D_GNU_SOURCE', language: 'c')
 if is_freebsd
 	add_project_arguments('-D__BSD_VISIBLE', language: 'c')
 endif
+
+if get_option('b_lto')
+	if cc.has_argument('-ffat-lto-objects')
+		add_project_arguments('-ffat-lto-objects', language: 'c')
+	else

config/meson.build add_project_arguments('-D_GNU_SOURCE', language: 'c') · line 213

Quoted 10 lines
config/meson.build +10 -0
+		error('compiler does not support fat LTO objects - please turn LTO off')
+	endif
+	if cc.get_id() == 'gcc'
+		# workaround for bug 81440
+		if cc.version().version_compare('<8.0')
+			add_project_arguments('-Wno-lto-type-mismatch', language: 'c')
+			add_project_link_arguments('-Wno-lto-type-mismatch', language: 'c')
+		endif
+	endif
+endif

Just wondering on this what would be the impact with blindly disabling this warning? Would we miss any common issues accidentally?

If we keep the current conditional code, suggest we merge the two checks for gcc and gcc version together into one if statement so we can reduce indentation level by one.

/Bruce

#43 lore raw

Re: [dpdk-dev] [PATCH v3 01/10] build: add an option to enable LTO build

Thank you Bruce for the comment. The original patch set did not manage to get to the dev list (I've sent it from my Marvell account and it stuck at moderation). So I'll take your "indent" comment and will send another version which will hopefully get through.

On 10/21/19 2:59 PM, Bruce Richardson wrote:
[...]

Quoted 18 lines
Quoted 18 lines
config/meson.build +10 -0
index 2bafea530..97bbc323b 100644
@@ -196,3 +196,18 @@ add_project_arguments('-D_GNU_SOURCE', language: 'c')
 if is_freebsd
 	add_project_arguments('-D__BSD_VISIBLE', language: 'c')
 endif
+
+if get_option('b_lto')
+	if cc.has_argument('-ffat-lto-objects')
+		add_project_arguments('-ffat-lto-objects', language: 'c')
+	else
+		error('compiler does not support fat LTO objects - please turn LTO off')
+	endif
+	if cc.get_id() == 'gcc'
+		# workaround for bug 81440
+		if cc.version().version_compare('<8.0')
Quoted 8 lines

config/meson.build add_project_arguments('-D_GNU_SOURCE', language: 'c') · line 213

Quoted 5 lines
config/meson.build +5 -0
+			add_project_arguments('-Wno-lto-type-mismatch', language: 'c')
+			add_project_link_arguments('-Wno-lto-type-mismatch', language: 'c')
+		endif
+	endif
+endif

Just wondering on this what would be the impact with blindly disabling this warning? Would we miss any common issues accidentally?

This warning is for the cases where declarations (of e.g. global variable) have different types in different compilation units - these type of things can only be caught with LTO. I don't think we need be worrying about that in DPDK - the problem is a bug in gcc that produces these warnings for the case of structs with flexible arrays at the end (which are fairly common in DPDK).

Quoted 3 lines

If we keep the current conditional code, suggest we merge the two checks for gcc and gcc version together into one if statement so we can reduce indentation level by one.

Agree. Will send another version - which hopefully will get through to the dev list.

Regards
Andrzej

#44 lore raw

v4 cover Add an option to use LTO for DPDK build

In reply to Andrzej Ostruszka

This patch series adds an option to make use of link time optimization (if compiler has support for it). It is split as follows:
- 1st patch (build) is the enablement
- remaining patches are fixes for the warnings produced by the compiler and they are split by directory/subsystem so their maintainers can easily find and verify the changes - please note that there are two groups:
* errors (or possible errors) - with title "fix possible use ..."
* false positives - warnings that _I_ think are not valid and the changes are made only to silence the compiler.

v4 Changes:
-----------
- merge nested conditionals in config/meson.build into one

v3 Changes:
-----------
- removed support for clang (only gcc and icc remain):
- it turned out that 'fat-lto-objects' is not really supported - it is accepted as a flag but ignored e.g. clang v6.0:
clang: error: optimization flag '-ffat-lto-objects' is not supported [-Werror,-Wignored-optimization-argument]
this was probably masked previously by playing around with meson and testing for the availability of this flag and dropping LTO if not supported.
- improved commit messages:
- added 'Fixes' tags where appropriate
- included snippets of compiler warnings
- changed commit names for the commits cleaning up 'false positive' compiler warnings - these are not really 'fixes' for the code, just workarounds for compiler shortcomings and check-git-log.sh was complaining about missing 'Fixes' tag.

v2 Changes:
-----------
- fixed building of shared libraries:
- gcc does not scan top level assembler statements so it missed that function implementations for specific versions were being used and was removing them
- fixed meson.build config files:
- moved from 'enable_lto' project option to built-in 'b_lto'
- documented that 'default_library' should be 'shared':
with 'default_library=static' (the default) the meson build with LTO is broken - this is because, SHARED_LIB is fixed in rte_config.h used by meson which leads to no alias for default version in generated static libraries (MAP_STATIC_SYMBOL() is empty)
- app/test: added log for failed bonding "config get"

Andrzej Ostruszka (10):
build: add an option to enable LTO build
eventdev: fix possible use of uninitialized var app/eventdev: clean LTO build warnings (maybe-uninitialized) event/octeontx2: clean LTO build warnings (maybe-uninitialized) app/test: clean LTO build warnings (maybe-uninitialized) net/dpaa2: fix possible use of uninitialized vars net/e1000: clean LTO build warnings (maybe-uninitialized) net/i40e: clean LTO build warnings (maybe-uninitialized) net/ifc: clean LTO build warnings (maybe-uninitialized) net/qede: clean LTO build warnings (maybe-uninitialized)

 .travis.yml                                   |  7 ++++
 app/test-eventdev/test_perf_common.c          |  2 +-
 app/test-eventdev/test_pipeline_common.c      |  4 +--
 app/test/test_hash_readwrite.c                |  2 +-
 app/test/test_link_bonding_mode4.c            | 10 ++++--
 app/test/test_memzone.c                       |  3 +-
 config/common_base                            |  5 +++
 config/meson.build                            | 13 +++++++
 doc/guides/prog_guide/lto.rst                 | 36 +++++++++++++++++++
 doc/guides/rel_notes/release_19_11.rst        |  8 +++++
 drivers/event/octeontx2/otx2_tim_worker.h     |  2 +-
 drivers/net/dpaa2/base/dpaa2_hw_dpni.c        |  1 +
 drivers/net/dpaa2/mc/dpkg.c                   |  2 +-
 drivers/net/dpaa2/mc/dpni.c                   |  9 +++--
 drivers/net/e1000/base/e1000_82543.c          |  2 +-
 drivers/net/e1000/base/e1000_ich8lan.c        |  2 +-
 drivers/net/e1000/base/e1000_phy.c            |  2 +-
 drivers/net/i40e/i40e_ethdev.c                |  2 +-
 drivers/net/ifc/ifcvf_vdpa.c                  | 14 +++++---
 drivers/net/qede/base/ecore_mcp.c             | 13 +++----
 lib/librte_distributor/rte_distributor.c      | 18 +++++-----
 lib/librte_distributor/rte_distributor_v20.c  | 18 +++++-----
 lib/librte_eventdev/rte_event_timer_adapter.c |  8 ++---
 lib/librte_lpm/rte_lpm.c                      | 28 +++++++--------
 lib/librte_lpm/rte_lpm6.c                     | 16 ++++-----
 lib/librte_timer/rte_timer.c                  | 20 +++++------
 mk/toolchain/gcc/rte.toolchain-compat.mk      |  4 +++
 mk/toolchain/gcc/rte.vars.mk                  | 12 +++++++
 mk/toolchain/icc/rte.vars.mk                  |  8 +++++
 29 files changed, 190 insertions(+), 81 deletions(-)
 create mode 100644 doc/guides/prog_guide/lto.rst
Signature · 1 line
2.17.1
#45 lore raw

PATCH v4 1/10 build: add an option to enable LTO build

This patch adds an option to enable link time optimization. In addition to LTO option itself (-flto) fat-lto-objects are being used. This is because during the build pmdinfogen scans the generated ELF objects to find this_pmd_name* symbol in symbol table. Without fat-lto-objects gcc produces ELF only with extra symbols for internal use during linking.

---
 .travis.yml                                  |  7 ++++
 config/common_base                           |  5 +++
 config/meson.build                           | 13 +++++++
 doc/guides/prog_guide/lto.rst                | 36 ++++++++++++++++++++
 doc/guides/rel_notes/release_19_11.rst       |  8 +++++
 lib/librte_distributor/rte_distributor.c     | 18 +++++-----
 lib/librte_distributor/rte_distributor_v20.c | 18 +++++-----
 lib/librte_lpm/rte_lpm.c                     | 28 +++++++--------
 lib/librte_lpm/rte_lpm6.c                    | 16 ++++-----
 lib/librte_timer/rte_timer.c                 | 20 +++++------
 mk/toolchain/gcc/rte.toolchain-compat.mk     |  4 +++
 mk/toolchain/gcc/rte.vars.mk                 | 12 +++++++
 mk/toolchain/icc/rte.vars.mk                 |  8 +++++
 13 files changed, 143 insertions(+), 50 deletions(-)
 create mode 100644 doc/guides/prog_guide/lto.rst
.travis.yml +7 -0
index 781f9f666..70d221852 100644
@@ -31,6 +31,7 @@ env:
   - DEF_LIB="static" OPTS="-Denable_kmods=false"
   - DEF_LIB="shared" OPTS="-Denable_kmods=false"
   - DEF_LIB="shared" RUN_TESTS=1 BUILD_DOCS=1
+  - DEF_LIB="shared" OPTS="-Db_lto=true"
 
 matrix:
   include:
@@ -100,6 +101,12 @@ matrix:
       apt:
         packages:
           - *extra_packages
+  - env: DEF_LIB="shared" OPTS="-Db_lto=true" EXTRA_PACKAGES=1
+    compiler: gcc
+    addons:
+      apt:
+        packages:
+          - *extra_packages
 
 
 script: ./.ci/${TRAVIS_OS_NAME}-build.sh
config/common_base +5 -0
index 8ef75c203..73a55fdec 100644
@@ -49,6 +49,11 @@ CONFIG_RTE_FORCE_INTRINSICS=n
 #
 CONFIG_RTE_ARCH_STRICT_ALIGN=n
 
+#
+# Enable link time optimization
+#
+CONFIG_RTE_ENABLE_LTO=n
+
 #
 # Compile to share library
 #
config/meson.build +13 -0
index 2bafea530..1a5093118 100644
@@ -196,3 +196,16 @@ add_project_arguments('-D_GNU_SOURCE', language: 'c')
 if is_freebsd
 	add_project_arguments('-D__BSD_VISIBLE', language: 'c')
 endif
+
+if get_option('b_lto')
+	if cc.has_argument('-ffat-lto-objects')
+		add_project_arguments('-ffat-lto-objects', language: 'c')
+	else
+		error('compiler does not support fat LTO objects - please turn LTO off')
+	endif
+	# workaround for gcc bug 81440
+	if cc.get_id() == 'gcc' and cc.version().version_compare('<8.0')
+		add_project_arguments('-Wno-lto-type-mismatch', language: 'c')
+		add_project_link_arguments('-Wno-lto-type-mismatch', language: 'c')
+	endif
+endif
doc/guides/prog_guide/lto.rst new file +36 -0
new file mode 100644
index 000000000..b6daabc86
@@ -0,0 +1,36 @@
+Link Time Optimization
+======================
+
+The DPDK framework supports compilation with link time optimization
+turned on.  This depends obviously on the capabilities of the compiler
+to do "whole program" optimization at link time and is available only
+for compilers that support that feature (gcc and icc).  To be more
+specific compiler have to support creation of ELF objects containing
+both normal code and internal representation (fat-lto-objects).  This is
+required since during build some code is generated by parsing produced
+ELF objects (pmdinfogen).
+
+The amount of performance gain that one can get from LTO depends on the
+compiler and the code that is being compiled.  However LTO is also
+useful for additional code analysis done by the compiler.  In particular
+due to interprocedural analysis compiler can produce additional warnings
+about variables that might be used uninitialized.  Some of these
+warnings might be "false positives" though and you might need to
+explicitly initialize variable in order to silence the compiler.
+
+Link time optimization can be enabled for whole DPDK framework by
+setting:
+
+.. code-block:: console
+    CONFIG_ENABLE_LTO=y
+
+in config file for the case of make based build and by:
+
+.. code-block:: console
+    meson build -Db_lto=true
+    ninja -C build
+
+for the case of meson based build.
+
+Please note that turning LTO on causes considerable extension of
+compilation time.
doc/guides/rel_notes/release_19_11.rst +8 -0
index 8490d897c..97b4f4083 100644
@@ -56,6 +56,14 @@ New Features
      Also, make sure to start the actual text at the margin.
      =========================================================
 
+**Added build support for Link Time Optimization.**
+
+ LTO is an optimization technique used by the compiler to perform whole
+ program analysis and optimization at link time.  In order to do that
+ compilers store their internal representation of the source code that
+ the linker uses at the final stage of compilation process.
+
+ See :doc:`../prog_guide/lto` for more information:
 
 Removed Items
 -------------
lib/librte_distributor/rte_distributor.c +9 -9
index 21eb1fb0a..848250f4a 100644
@@ -32,7 +32,7 @@ EAL_REGISTER_TAILQ(rte_dist_burst_tailq)
 
 /**** Burst Packet APIs called by workers ****/
 
-void
+void __vsym
 rte_distributor_request_pkt_v1705(struct rte_distributor *d,
 		unsigned int worker_id, struct rte_mbuf **oldpkt,
 		unsigned int count)
@@ -84,7 +84,7 @@ MAP_STATIC_SYMBOL(void rte_distributor_request_pkt(struct rte_distributor *d,
 		unsigned int count),
 		rte_distributor_request_pkt_v1705);
 
-int
+int __vsym
 rte_distributor_poll_pkt_v1705(struct rte_distributor *d,
 		unsigned int worker_id, struct rte_mbuf **pkts)
 {
@@ -124,7 +124,7 @@ MAP_STATIC_SYMBOL(int rte_distributor_poll_pkt(struct rte_distributor *d,
 		unsigned int worker_id, struct rte_mbuf **pkts),
 		rte_distributor_poll_pkt_v1705);
 
-int
+int __vsym
 rte_distributor_get_pkt_v1705(struct rte_distributor *d,
 		unsigned int worker_id, struct rte_mbuf **pkts,
 		struct rte_mbuf **oldpkt, unsigned int return_count)
@@ -159,7 +159,7 @@ MAP_STATIC_SYMBOL(int rte_distributor_get_pkt(struct rte_distributor *d,
 		struct rte_mbuf **oldpkt, unsigned int return_count),
 		rte_distributor_get_pkt_v1705);
 
-int
+int __vsym
 rte_distributor_return_pkt_v1705(struct rte_distributor *d,
 		unsigned int worker_id, struct rte_mbuf **oldpkt, int num)
 {
@@ -335,7 +335,7 @@ release(struct rte_distributor *d, unsigned int wkr)
 
 
 /* process a set of packets to distribute them to workers */
-int
+int __vsym
 rte_distributor_process_v1705(struct rte_distributor *d,
 		struct rte_mbuf **mbufs, unsigned int num_mbufs)
 {
@@ -476,7 +476,7 @@ MAP_STATIC_SYMBOL(int rte_distributor_process(struct rte_distributor *d,
 		rte_distributor_process_v1705);
 
 /* return to the caller, packets returned from workers */
-int
+int __vsym
 rte_distributor_returned_pkts_v1705(struct rte_distributor *d,
 		struct rte_mbuf **mbufs, unsigned int max_mbufs)
 {
@@ -526,7 +526,7 @@ total_outstanding(const struct rte_distributor *d)
  * Flush the distributor, so that there are no outstanding packets in flight or
  * queued up.
  */
-int
+int __vsym
 rte_distributor_flush_v1705(struct rte_distributor *d)
 {
 	unsigned int flushed;
@@ -561,7 +561,7 @@ MAP_STATIC_SYMBOL(int rte_distributor_flush(struct rte_distributor *d),
 		rte_distributor_flush_v1705);
 
 /* clears the internal returns array in the distributor */
-void
+void __vsym
 rte_distributor_clear_returns_v1705(struct rte_distributor *d)
 {
 	unsigned int wkr;
@@ -581,7 +581,7 @@ MAP_STATIC_SYMBOL(void rte_distributor_clear_returns(struct rte_distributor *d),
 		rte_distributor_clear_returns_v1705);
 
 /* creates a distributor instance */
-struct rte_distributor *
+struct rte_distributor * __vsym
 rte_distributor_create_v1705(const char *name,
 		unsigned int socket_id,
 		unsigned int num_workers,
lib/librte_distributor/rte_distributor_v20.c +9 -9
index cdc0969a8..31c766421 100644
@@ -27,7 +27,7 @@ EAL_REGISTER_TAILQ(rte_distributor_tailq)
 
 /**** APIs called by workers ****/
 
-void
+void __vsym
 rte_distributor_request_pkt_v20(struct rte_distributor_v20 *d,
 		unsigned worker_id, struct rte_mbuf *oldpkt)
 {
@@ -40,7 +40,7 @@ rte_distributor_request_pkt_v20(struct rte_distributor_v20 *d,
 }
 VERSION_SYMBOL(rte_distributor_request_pkt, _v20, 2.0);
 
-struct rte_mbuf *
+struct rte_mbuf * __vsym
 rte_distributor_poll_pkt_v20(struct rte_distributor_v20 *d,
 		unsigned worker_id)
 {
@@ -54,7 +54,7 @@ rte_distributor_poll_pkt_v20(struct rte_distributor_v20 *d,
 }
 VERSION_SYMBOL(rte_distributor_poll_pkt, _v20, 2.0);
 
-struct rte_mbuf *
+struct rte_mbuf * __vsym
 rte_distributor_get_pkt_v20(struct rte_distributor_v20 *d,
 		unsigned worker_id, struct rte_mbuf *oldpkt)
 {
@@ -66,7 +66,7 @@ rte_distributor_get_pkt_v20(struct rte_distributor_v20 *d,
 }
 VERSION_SYMBOL(rte_distributor_get_pkt, _v20, 2.0);
 
-int
+int __vsym
 rte_distributor_return_pkt_v20(struct rte_distributor_v20 *d,
 		unsigned worker_id, struct rte_mbuf *oldpkt)
 {
@@ -191,7 +191,7 @@ process_returns(struct rte_distributor_v20 *d)
 }
 
 /* process a set of packets to distribute them to workers */
-int
+int __vsym
 rte_distributor_process_v20(struct rte_distributor_v20 *d,
 		struct rte_mbuf **mbufs, unsigned num_mbufs)
 {
@@ -296,7 +296,7 @@ rte_distributor_process_v20(struct rte_distributor_v20 *d,
 VERSION_SYMBOL(rte_distributor_process, _v20, 2.0);
 
 /* return to the caller, packets returned from workers */
-int
+int __vsym
 rte_distributor_returned_pkts_v20(struct rte_distributor_v20 *d,
 		struct rte_mbuf **mbufs, unsigned max_mbufs)
 {
@@ -334,7 +334,7 @@ total_outstanding(const struct rte_distributor_v20 *d)
 
 /* flush the distributor, so that there are no outstanding packets in flight or
  * queued up. */
-int
+int __vsym
 rte_distributor_flush_v20(struct rte_distributor_v20 *d)
 {
 	const unsigned flushed = total_outstanding(d);
@@ -347,7 +347,7 @@ rte_distributor_flush_v20(struct rte_distributor_v20 *d)
 VERSION_SYMBOL(rte_distributor_flush, _v20, 2.0);
 
 /* clears the internal returns array in the distributor */
-void
+void __vsym
 rte_distributor_clear_returns_v20(struct rte_distributor_v20 *d)
 {
 	d->returns.start = d->returns.count = 0;
@@ -358,7 +358,7 @@ rte_distributor_clear_returns_v20(struct rte_distributor_v20 *d)
 VERSION_SYMBOL(rte_distributor_clear_returns, _v20, 2.0);
 
 /* creates a distributor instance */
-struct rte_distributor_v20 *
+struct rte_distributor_v20 * __vsym
 rte_distributor_create_v20(const char *name,
 		unsigned socket_id,
 		unsigned num_workers)
lib/librte_lpm/rte_lpm.c +14 -14
index 3a929a1b1..a2fba8d61 100644
@@ -89,7 +89,7 @@ depth_to_range(uint8_t depth)
 /*
  * Find an existing lpm table and return a pointer to it.
  */
-struct rte_lpm_v20 *
+struct rte_lpm_v20 * __vsym
 rte_lpm_find_existing_v20(const char *name)
 {
 	struct rte_lpm_v20 *l = NULL;
@@ -115,7 +115,7 @@ rte_lpm_find_existing_v20(const char *name)
 }
 VERSION_SYMBOL(rte_lpm_find_existing, _v20, 2.0);
 
-struct rte_lpm *
+struct rte_lpm * __vsym
 rte_lpm_find_existing_v1604(const char *name)
 {
 	struct rte_lpm *l = NULL;
@@ -146,7 +146,7 @@ MAP_STATIC_SYMBOL(struct rte_lpm *rte_lpm_find_existing(const char *name),
 /*
  * Allocates memory for LPM object
  */
-struct rte_lpm_v20 *
+struct rte_lpm_v20 * __vsym
 rte_lpm_create_v20(const char *name, int socket_id, int max_rules,
 		__rte_unused int flags)
 {
@@ -219,7 +219,7 @@ rte_lpm_create_v20(const char *name, int socket_id, int max_rules,
 }
 VERSION_SYMBOL(rte_lpm_create, _v20, 2.0);
 
-struct rte_lpm *
+struct rte_lpm * __vsym
 rte_lpm_create_v1604(const char *name, int socket_id,
 		const struct rte_lpm_config *config)
 {
@@ -328,7 +328,7 @@ MAP_STATIC_SYMBOL(
 /*
  * Deallocates memory for given LPM table.
  */
-void
+void __vsym
 rte_lpm_free_v20(struct rte_lpm_v20 *lpm)
 {
 	struct rte_lpm_list *lpm_list;
@@ -357,7 +357,7 @@ rte_lpm_free_v20(struct rte_lpm_v20 *lpm)
 }
 VERSION_SYMBOL(rte_lpm_free, _v20, 2.0);
 
-void
+void __vsym
 rte_lpm_free_v1604(struct rte_lpm *lpm)
 {
 	struct rte_lpm_list *lpm_list;
@@ -1176,7 +1176,7 @@ add_depth_big_v1604(struct rte_lpm *lpm, uint32_t ip_masked, uint8_t depth,
 /*
  * Add a route
  */
-int
+int __vsym
 rte_lpm_add_v20(struct rte_lpm_v20 *lpm, uint32_t ip, uint8_t depth,
 		uint8_t next_hop)
 {
@@ -1217,7 +1217,7 @@ rte_lpm_add_v20(struct rte_lpm_v20 *lpm, uint32_t ip, uint8_t depth,
 }
 VERSION_SYMBOL(rte_lpm_add, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm_add_v1604(struct rte_lpm *lpm, uint32_t ip, uint8_t depth,
 		uint32_t next_hop)
 {
@@ -1263,7 +1263,7 @@ MAP_STATIC_SYMBOL(int rte_lpm_add(struct rte_lpm *lpm, uint32_t ip,
 /*
  * Look for a rule in the high-level rules table
  */
-int
+int __vsym
 rte_lpm_is_rule_present_v20(struct rte_lpm_v20 *lpm, uint32_t ip, uint8_t depth,
 uint8_t *next_hop)
 {
@@ -1290,7 +1290,7 @@ uint8_t *next_hop)
 }
 VERSION_SYMBOL(rte_lpm_is_rule_present, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm_is_rule_present_v1604(struct rte_lpm *lpm, uint32_t ip, uint8_t depth,
 uint32_t *next_hop)
 {
@@ -1843,7 +1843,7 @@ delete_depth_big_v1604(struct rte_lpm *lpm, uint32_t ip_masked,
 /*
  * Deletes a rule
  */
-int
+int __vsym
 rte_lpm_delete_v20(struct rte_lpm_v20 *lpm, uint32_t ip, uint8_t depth)
 {
 	int32_t rule_to_delete_index, sub_rule_index;
@@ -1897,7 +1897,7 @@ rte_lpm_delete_v20(struct rte_lpm_v20 *lpm, uint32_t ip, uint8_t depth)
 }
 VERSION_SYMBOL(rte_lpm_delete, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm_delete_v1604(struct rte_lpm *lpm, uint32_t ip, uint8_t depth)
 {
 	int32_t rule_to_delete_index, sub_rule_index;
@@ -1956,7 +1956,7 @@ MAP_STATIC_SYMBOL(int rte_lpm_delete(struct rte_lpm *lpm, uint32_t ip,
 /*
  * Delete all rules from the LPM table.
  */
-void
+void __vsym
 rte_lpm_delete_all_v20(struct rte_lpm_v20 *lpm)
 {
 	/* Zero rule information. */
@@ -1973,7 +1973,7 @@ rte_lpm_delete_all_v20(struct rte_lpm_v20 *lpm)
 }
 VERSION_SYMBOL(rte_lpm_delete_all, _v20, 2.0);
 
-void
+void __vsym
 rte_lpm_delete_all_v1604(struct rte_lpm *lpm)
 {
 	/* Zero rule information. */
lib/librte_lpm/rte_lpm6.c +8 -8
index 9b8aeb972..49a7fea1d 100644
@@ -811,7 +811,7 @@ add_step(struct rte_lpm6 *lpm, struct rte_lpm6_tbl_entry *tbl,
 /*
  * Add a route
  */
-int
+int __vsym
 rte_lpm6_add_v20(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
 		uint8_t next_hop)
 {
@@ -861,7 +861,7 @@ simulate_add(struct rte_lpm6 *lpm, const uint8_t *masked_ip, uint8_t depth)
 	return 0;
 }
 
-int
+int __vsym
 rte_lpm6_add_v1705(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
 		uint32_t next_hop)
 {
@@ -954,7 +954,7 @@ lookup_step(const struct rte_lpm6 *lpm, const struct rte_lpm6_tbl_entry *tbl,
 /*
  * Looks up an IP
  */
-int
+int __vsym
 rte_lpm6_lookup_v20(const struct rte_lpm6 *lpm, uint8_t *ip, uint8_t *next_hop)
 {
 	uint32_t next_hop32 = 0;
@@ -972,7 +972,7 @@ rte_lpm6_lookup_v20(const struct rte_lpm6 *lpm, uint8_t *ip, uint8_t *next_hop)
 }
 VERSION_SYMBOL(rte_lpm6_lookup, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm6_lookup_v1705(const struct rte_lpm6 *lpm, uint8_t *ip,
 		uint32_t *next_hop)
 {
@@ -1007,7 +1007,7 @@ MAP_STATIC_SYMBOL(int rte_lpm6_lookup(const struct rte_lpm6 *lpm, uint8_t *ip,
 /*
  * Looks up a group of IP addresses
  */
-int
+int __vsym
 rte_lpm6_lookup_bulk_func_v20(const struct rte_lpm6 *lpm,
 		uint8_t ips[][RTE_LPM6_IPV6_ADDR_SIZE],
 		int16_t * next_hops, unsigned n)
@@ -1048,7 +1048,7 @@ rte_lpm6_lookup_bulk_func_v20(const struct rte_lpm6 *lpm,
 }
 VERSION_SYMBOL(rte_lpm6_lookup_bulk_func, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm6_lookup_bulk_func_v1705(const struct rte_lpm6 *lpm,
 		uint8_t ips[][RTE_LPM6_IPV6_ADDR_SIZE],
 		int32_t *next_hops, unsigned int n)
@@ -1098,7 +1098,7 @@ MAP_STATIC_SYMBOL(int rte_lpm6_lookup_bulk_func(const struct rte_lpm6 *lpm,
 /*
  * Look for a rule in the high-level rules table
  */
-int
+int __vsym
 rte_lpm6_is_rule_present_v20(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
 		uint8_t *next_hop)
 {
@@ -1118,7 +1118,7 @@ rte_lpm6_is_rule_present_v20(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
 }
 VERSION_SYMBOL(rte_lpm6_is_rule_present, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm6_is_rule_present_v1705(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
 		uint32_t *next_hop)
 {
lib/librte_timer/rte_timer.c +10 -10
index bdcf05d06..e560ace06 100644
@@ -131,7 +131,7 @@ rte_timer_data_dealloc(uint32_t id)
 	return 0;
 }
 
-void
+void __vsym
 rte_timer_subsystem_init_v20(void)
 {
 	unsigned lcore_id;
@@ -153,7 +153,7 @@ VERSION_SYMBOL(rte_timer_subsystem_init, _v20, 2.0);
  * secondary processes should be empty, the zeroth entry can be shared by
  * multiple processes.
  */
-int
+int __vsym
 rte_timer_subsystem_init_v1905(void)
 {
 	const struct rte_memzone *mz;
@@ -551,7 +551,7 @@ __rte_timer_reset(struct rte_timer *tim, uint64_t expire,
 }
 
 /* Reset and start the timer associated with the timer handle tim */
-int
+int __vsym
 rte_timer_reset_v20(struct rte_timer *tim, uint64_t ticks,
 		    enum rte_timer_type type, unsigned int tim_lcore,
 		    rte_timer_cb_t fct, void *arg)
@@ -574,7 +574,7 @@ rte_timer_reset_v20(struct rte_timer *tim, uint64_t ticks,
 }
 VERSION_SYMBOL(rte_timer_reset, _v20, 2.0);
 
-int
+int __vsym
 rte_timer_reset_v1905(struct rte_timer *tim, uint64_t ticks,
 		      enum rte_timer_type type, unsigned int tim_lcore,
 		      rte_timer_cb_t fct, void *arg)
@@ -657,14 +657,14 @@ __rte_timer_stop(struct rte_timer *tim, int local_is_locked,
 }
 
 /* Stop the timer associated with the timer handle tim */
-int
+int __vsym
 rte_timer_stop_v20(struct rte_timer *tim)
 {
 	return __rte_timer_stop(tim, 0, &default_timer_data);
 }
 VERSION_SYMBOL(rte_timer_stop, _v20, 2.0);
 
-int
+int __vsym
 rte_timer_stop_v1905(struct rte_timer *tim)
 {
 	return rte_timer_alt_stop(default_data_id, tim);
@@ -817,14 +817,14 @@ __rte_timer_manage(struct rte_timer_data *timer_data)
 	priv_timer[lcore_id].running_tim = NULL;
 }
 
-void
+void __vsym
 rte_timer_manage_v20(void)
 {
 	__rte_timer_manage(&default_timer_data);
 }
 VERSION_SYMBOL(rte_timer_manage, _v20, 2.0);
 
-int
+int __vsym
 rte_timer_manage_v1905(void)
 {
 	struct rte_timer_data *timer_data;
@@ -1074,14 +1074,14 @@ __rte_timer_dump_stats(struct rte_timer_data *timer_data __rte_unused, FILE *f)
 #endif
 }
 
-void
+void __vsym
 rte_timer_dump_stats_v20(FILE *f)
 {
 	__rte_timer_dump_stats(&default_timer_data, f);
 }
 VERSION_SYMBOL(rte_timer_dump_stats, _v20, 2.0);
 
-int
+int __vsym
 rte_timer_dump_stats_v1905(FILE *f)
 {
 	return rte_timer_alt_dump_stats(default_data_id, f);
mk/toolchain/gcc/rte.toolchain-compat.mk +4 -0
index ea40a11c0..ad4fad83c 100644
@@ -88,6 +88,10 @@ else
 		MACHINE_CFLAGS := $(filter-out -march% -mtune% -msse%,$(MACHINE_CFLAGS))
 	endif
 
+	ifeq ($(shell test $(GCC_VERSION) -lt 45 && echo 1), 1)
+		CONFIG_RTE_ENABLE_LTO=n
+	endif
+
 	# Disable thunderx PMD for gcc < 4.7
 	ifeq ($(shell test $(GCC_VERSION) -lt 47 && echo 1), 1)
 		CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD=d
mk/toolchain/gcc/rte.vars.mk +12 -0
index b852fcfd7..9fc704193 100644
@@ -62,6 +62,18 @@ endif
 # process cpu flags
 include $(RTE_SDK)/mk/toolchain/$(RTE_TOOLCHAIN)/rte.toolchain-compat.mk
 
+ifeq ($(CONFIG_RTE_ENABLE_LTO),y)
+# 'fat-lto' is used since pmdinfogen needs to have 'this_pmd_nameX'
+# exported in symbol table and without this option only internal
+# representation is present.
+TOOLCHAIN_CFLAGS += -flto -ffat-lto-objects
+TOOLCHAIN_LDFLAGS += -flto
+# workaround for GCC bug 81440
+ifeq ($(shell test $(GCC_VERSION) -lt 80 && echo 1), 1)
+WERROR_FLAGS += -Wno-lto-type-mismatch
+endif
+endif
+
 # workaround GCC bug with warning "missing initializer" for "= {0}"
 ifeq ($(shell test $(GCC_VERSION) -lt 47 && echo 1), 1)
 WERROR_FLAGS += -Wno-missing-field-initializers
mk/toolchain/icc/rte.vars.mk +8 -0
index aa1422bf1..8aa87aa1e 100644
@@ -54,5 +54,13 @@ endif
 # process cpu flags
 include $(RTE_SDK)/mk/toolchain/$(RTE_TOOLCHAIN)/rte.toolchain-compat.mk
 
+ifeq ($(CONFIG_RTE_ENABLE_LTO),y)
+# 'fat-lto' is used since pmdinfogen needs to have 'this_pmd_nameX'
+# exported in symbol table and without this option only internal
+# representation is present.
+TOOLCHAIN_CFLAGS += -flto -ffat-lto-objects
+TOOLCHAIN_LDFLAGS += -flto
+endif
+
 export CC AS AR LD OBJCOPY OBJDUMP STRIP READELF
 export TOOLCHAIN_CFLAGS TOOLCHAIN_LDFLAGS TOOLCHAIN_ASFLAGS
Signature · 1 line
2.17.1
Acked-by on v4 1/10
#46 lore raw

Re: [dpdk-dev] [PATCH v4 01/10] build: add an option to enable LTO build

Quoted 76 lines · Andrzej Ostruszka

This patch adds an option to enable link time optimization. In addition to LTO option itself (-flto) fat-lto-objects are being used. This is because during the build pmdinfogen scans the generated ELF objects to find this_pmd_name* symbol in symbol table. Without fat-lto-objects gcc produces ELF only with extra symbols for internal use during linking.

---
 .travis.yml                                  |  7 ++++
 config/common_base                           |  5 +++
 config/meson.build                           | 13 +++++++
 doc/guides/prog_guide/lto.rst                | 36 ++++++++++++++++++++
 doc/guides/rel_notes/release_19_11.rst       |  8 +++++
 lib/librte_distributor/rte_distributor.c     | 18 +++++-----
 lib/librte_distributor/rte_distributor_v20.c | 18 +++++-----
 lib/librte_lpm/rte_lpm.c                     | 28 +++++++--------
 lib/librte_lpm/rte_lpm6.c                    | 16 ++++-----
 lib/librte_timer/rte_timer.c                 | 20 +++++------
 mk/toolchain/gcc/rte.toolchain-compat.mk     |  4 +++
 mk/toolchain/gcc/rte.vars.mk                 | 12 +++++++
 mk/toolchain/icc/rte.vars.mk                 |  8 +++++
 13 files changed, 143 insertions(+), 50 deletions(-)
 create mode 100644 doc/guides/prog_guide/lto.rst
.travis.yml +7 -0
index 781f9f666..70d221852 100644
@@ -31,6 +31,7 @@ env:
   - DEF_LIB="static" OPTS="-Denable_kmods=false"
   - DEF_LIB="shared" OPTS="-Denable_kmods=false"
   - DEF_LIB="shared" RUN_TESTS=1 BUILD_DOCS=1
+  - DEF_LIB="shared" OPTS="-Db_lto=true"
 
 matrix:
   include:
@@ -100,6 +101,12 @@ matrix:
       apt:
         packages:
           - *extra_packages
+  - env: DEF_LIB="shared" OPTS="-Db_lto=true" EXTRA_PACKAGES=1
+    compiler: gcc
+    addons:
+      apt:
+        packages:
+          - *extra_packages
 
 
 script: ./.ci/${TRAVIS_OS_NAME}-build.sh
config/common_base +5 -0
index 8ef75c203..73a55fdec 100644
@@ -49,6 +49,11 @@ CONFIG_RTE_FORCE_INTRINSICS=n
 #
 CONFIG_RTE_ARCH_STRICT_ALIGN=n
 
+#
+# Enable link time optimization
+#
+CONFIG_RTE_ENABLE_LTO=n
+
 #
 # Compile to share library
 #
config/meson.build +3 -0
index 2bafea530..1a5093118 100644
@@ -196,3 +196,16 @@ add_project_arguments('-D_GNU_SOURCE', language: 'c')
 if is_freebsd
 	add_project_arguments('-D__BSD_VISIBLE', language: 'c')
 endif
+
+if get_option('b_lto')
+	if cc.has_argument('-ffat-lto-objects')

config/meson.build add_project_arguments('-D_GNU_SOURCE', language: 'c') · line 211

Quoted 10 lines
config/meson.build +10 -0
+		add_project_arguments('-ffat-lto-objects', language: 'c')
+	else
+		error('compiler does not support fat LTO objects - please turn LTO off')
+	endif
+	# workaround for gcc bug 81440
+	if cc.get_id() == 'gcc' and cc.version().version_compare('<8.0')
+		add_project_arguments('-Wno-lto-type-mismatch', language: 'c')
+		add_project_link_arguments('-Wno-lto-type-mismatch', language: 'c')
+	endif
+endif

This LGTM

Quoted 4 lines
doc/guides/prog_guide/lto.rst new file +0 -1
new file mode 100644
index 000000000..b6daabc86
--- /dev/null

<snip>

mk/toolchain/gcc/rte.toolchain-compat.mk else · line 94

Quoted 12 lines
mk/toolchain/gcc/rte.toolchain-compat.mk +4 -0
index ea40a11c0..ad4fad83c 100644
@@ -88,6 +88,10 @@ else
 		MACHINE_CFLAGS := $(filter-out -march% -mtune% -msse%,$(MACHINE_CFLAGS))
 	endif
 
+	ifeq ($(shell test $(GCC_VERSION) -lt 45 && echo 1), 1)
+		CONFIG_RTE_ENABLE_LTO=n
+	endif
+

I don't think we support GCC versions that far back any more, but this is fairly harmless, so probably ok to keep.

#47 lore raw

Re: [dpdk-dev] [PATCH v4 01/10] build: add an option to enable LTO build

In reply to Andrzej Ostruszka

Adding John and Marko for doc review.

22/10/2019 13:54, Andrzej Ostruszka:

doc/guides/prog_guide/lto.rst

Quoted 2 lines
doc/guides/prog_guide/lto.rst new file +0 -0

This new file is not included in the prog guide index.

doc/guides/prog_guide/lto.rst

Quoted 1 line
doc/guides/prog_guide/lto.rst new file +0 -0
@@ -0,0 +1,36 @@

You are missing the mandatory SPDX and Copyright header.

doc/guides/prog_guide/lto.rst line 4

Quoted 4 lines
doc/guides/prog_guide/lto.rst new file +4 -0
+Link Time Optimization
+======================
+
+The DPDK framework supports compilation with link time optimization

Instead of "The DPDK framework", I prefer just "DPDK".

doc/guides/prog_guide/lto.rst line 5

Quoted 1 line
doc/guides/prog_guide/lto.rst new file +1 -0
+turned on.  This depends obviously on the capabilities of the compiler

Not sure what you are talking about.
If it is about fat objects, it is not obvious.

doc/guides/prog_guide/lto.rst line 7

Quoted 2 lines
doc/guides/prog_guide/lto.rst new file +2 -0
+to do "whole program" optimization at link time and is available only
+for compilers that support that feature (gcc and icc).  To be more

Please start a new sentence at the beginning of a line in the RsT file. It is easier to review and update.

doc/guides/prog_guide/lto.rst line 8

Quoted 1 line
doc/guides/prog_guide/lto.rst new file +1 -0
+specific compiler have to support creation of ELF objects containing

A comma is missing after "specific".

doc/guides/prog_guide/lto.rst line 19

Quoted 11 lines
doc/guides/prog_guide/lto.rst new file +11 -0
+both normal code and internal representation (fat-lto-objects).  This is
+required since during build some code is generated by parsing produced
+ELF objects (pmdinfogen).
+
+The amount of performance gain that one can get from LTO depends on the
+compiler and the code that is being compiled.  However LTO is also
+useful for additional code analysis done by the compiler.  In particular
+due to interprocedural analysis compiler can produce additional warnings
+about variables that might be used uninitialized.  Some of these
+warnings might be "false positives" though and you might need to
+explicitly initialize variable in order to silence the compiler.

Any word about build speed degradation?

doc/guides/prog_guide/lto.rst line 31

Quoted 12 lines
doc/guides/prog_guide/lto.rst new file +12 -0
+
+Link time optimization can be enabled for whole DPDK framework by
+setting:
+
+.. code-block:: console
+    CONFIG_ENABLE_LTO=y
+
+in config file for the case of make based build and by:
+
+.. code-block:: console
+    meson build -Db_lto=true
+    ninja -C build

No need to add the ninja step here.

doc/guides/prog_guide/lto.rst line 33

Quoted 1 line
doc/guides/prog_guide/lto.rst new file +1 -0
+for the case of meson based build.

Better to describe the case (make or meson) before the code-block.

doc/guides/prog_guide/lto.rst line 36

Quoted 2 lines
doc/guides/prog_guide/lto.rst new file +2 -0
+Please note that turning LTO on causes considerable extension of
+compilation time.

"compilation time" is not accurate.
When referring to compilation + linking, it is better to use the words "build time".

Quoted 4 lines
doc/guides/rel_notes/release_19_11.rst +0 -0
@@ -56,6 +56,14 @@ New Features
      Also, make sure to start the actual text at the margin.

doc/guides/rel_notes/release_19_11.rst New Features · line 66

Quoted 10 lines
doc/guides/rel_notes/release_19_11.rst +8 -0
      =========================================================
 
+**Added build support for Link Time Optimization.**
+
+ LTO is an optimization technique used by the compiler to perform whole
+ program analysis and optimization at link time.  In order to do that
+ compilers store their internal representation of the source code that
+ the linker uses at the final stage of compilation process.
+
+ See :doc:`../prog_guide/lto` for more information:

Please rebase and add a blank line when adding a paragraph.

lib/librte_distributor/rte_distributor.c

Quoted 8 lines
doc/guides/rel_notes/release_19_11.rst +0 -0
 Removed Items
 -------------
 
lib/librte_distributor/rte_distributor.c +1 -1
-void
+void __vsym

I replied in v2 about this change. Better to not talk about it here.

I wanted to merge this series in 19.11-rc1 but I think it is better to move its merge in 19.11-rc2.
Thanks for the work.

#48 lore raw

Re: [dpdk-dev] [PATCH v4 01/10] build: add an option to enable LTO build

Thank you Thomas for the comments. Please see below for replies.

Quoted 7 lines · Thomas Monjalon

Adding John and Marko for doc review.

22/10/2019 13:54, Andrzej Ostruszka:

doc/guides/prog_guide/lto.rst

Quoted 2 lines
doc/guides/prog_guide/lto.rst new file +0 -0

This new file is not included in the prog guide index.

Will add it (between "writing efficient code" and "profile app").

Quoted 3 lines

doc/guides/prog_guide/lto.rst

Quoted 1 line
doc/guides/prog_guide/lto.rst new file +0 -0
@@ -0,0 +1,36 @@

You are missing the mandatory SPDX and Copyright header.

Will add.

Quoted 6 lines

doc/guides/prog_guide/lto.rst line 4

Quoted 4 lines
doc/guides/prog_guide/lto.rst new file +4 -0
+Link Time Optimization
+======================
+
+The DPDK framework supports compilation with link time optimization

Instead of "The DPDK framework", I prefer just "DPDK".

OK

Quoted 6 lines

doc/guides/prog_guide/lto.rst line 5

Quoted 1 line
doc/guides/prog_guide/lto.rst new file +1 -0
+turned on.  This depends obviously on the capabilities of the compiler

Not sure what you are talking about.
If it is about fat objects, it is not obvious.

Quoted 1 line
doc/guides/prog_guide/lto.rst new file +1 -0
+to do "whole program" optimization at link time and is available only

That was not a reference to some formally defined "capabilities" just an "obvious" statement that in order to benefit from LTO one needs a compiler "capable of" performing it. I will rephrase this paragraph as:

-8<-------------
The DPDK supports compilation with link time optimization turned on. This depends obviously on the ability of the compiler to do "whole program" optimization at link time and is available only for compilers that support that feature (gcc and icc).
To be more specific, compiler (in addition to performing LTO) have to support creation of ELF objects containing both normal code and internal representation (fat-lto-objects). This is required since during build some code is generated by parsing produced ELF objects (pmdinfogen). -8<-------------

Will it suffice? If not, then please suggest wording to be used.

Quoted 4 lines

doc/guides/prog_guide/lto.rst line 7

Quoted 1 line
doc/guides/prog_guide/lto.rst new file +1 -0
+for compilers that support that feature (gcc and icc).  To be more

Please start a new sentence at the beginning of a line in the RsT file. It is easier to review and update.

OK

Quoted 3 lines

doc/guides/prog_guide/lto.rst line 8

Quoted 1 line
doc/guides/prog_guide/lto.rst new file +1 -0
+specific compiler have to support creation of ELF objects containing

A comma is missing after "specific".

OK

Quoted 5 lines
Quoted 5 lines
doc/guides/prog_guide/lto.rst new file +5 -0
+both normal code and internal representation (fat-lto-objects).  This is
+required since during build some code is generated by parsing produced
+ELF objects (pmdinfogen).
+
+The amount of performance gain that one can get from LTO depends on the
Quoted 8 lines

doc/guides/prog_guide/lto.rst line 19

Quoted 6 lines
doc/guides/prog_guide/lto.rst new file +6 -0
+compiler and the code that is being compiled.  However LTO is also
+useful for additional code analysis done by the compiler.  In particular
+due to interprocedural analysis compiler can produce additional warnings
+about variables that might be used uninitialized.  Some of these
+warnings might be "false positives" though and you might need to
+explicitly initialize variable in order to silence the compiler.

Any word about build speed degradation?

It is mentioned at the end of this file - I will move it here.

Quoted 5 lines
Quoted 5 lines
doc/guides/prog_guide/lto.rst new file +5 -0
+Link time optimization can be enabled for whole DPDK framework by
+setting:
+
+.. code-block:: console
+    CONFIG_ENABLE_LTO=y
Quoted 8 lines

doc/guides/prog_guide/lto.rst line 31

Quoted 6 lines
doc/guides/prog_guide/lto.rst new file +6 -0
+
+in config file for the case of make based build and by:
+
+.. code-block:: console
+    meson build -Db_lto=true
+    ninja -C build

No need to add the ninja step here.

OK

Quoted 3 lines

doc/guides/prog_guide/lto.rst line 33

Quoted 1 line
doc/guides/prog_guide/lto.rst new file +1 -0
+for the case of meson based build.

Better to describe the case (make or meson) before the code-block.

OK

Quoted 6 lines

doc/guides/prog_guide/lto.rst line 36

Quoted 2 lines
doc/guides/prog_guide/lto.rst new file +2 -0
+Please note that turning LTO on causes considerable extension of
+compilation time.

"compilation time" is not accurate.
When referring to compilation + linking, it is better to use the words "build time".

OK

Quoted 6 lines
Quoted 6 lines
doc/guides/rel_notes/release_19_11.rst +0 -0
@@ -56,6 +56,14 @@ New Features
      Also, make sure to start the actual text at the margin.
      =========================================================
 
Quoted 10 lines

doc/guides/rel_notes/release_19_11.rst New Features · line 66

Quoted 8 lines
doc/guides/rel_notes/release_19_11.rst +8 -0
+**Added build support for Link Time Optimization.**
+
+ LTO is an optimization technique used by the compiler to perform whole
+ program analysis and optimization at link time.  In order to do that
+ compilers store their internal representation of the source code that
+ the linker uses at the final stage of compilation process.
+
+ See :doc:`../prog_guide/lto` for more information:

Please rebase and add a blank line when adding a paragraph.

OK

[...]

Quoted 2 lines

I wanted to merge this series in 19.11-rc1 but I think it is better to move its merge in 19.11-rc2.

OK. I'll send the next version today.

Thank you again for the comments.

Regards
Andrzej

#49 lore raw

Re: [dpdk-dev] [PATCH v4 01/10] build: add an option to enable LTO build

28/10/2019 11:47, Andrzej Ostruszka:

Quoted 23 lines · Thomas Monjalon
Quoted 6 lines
Quoted 1 line

+turned on. This depends obviously on the capabilities of the compiler

Not sure what you are talking about.
If it is about fat objects, it is not obvious.

Quoted 1 line

+to do "whole program" optimization at link time and is available only

That was not a reference to some formally defined "capabilities" just an "obvious" statement that in order to benefit from LTO one needs a compiler "capable of" performing it. I will rephrase this paragraph as:

-8<-------------
The DPDK supports compilation with link time optimization turned on. This depends obviously on the ability of the compiler to do "whole program" optimization at link time and is available only for compilers that support that feature (gcc and icc).
To be more specific, compiler (in addition to performing LTO) have to support creation of ELF objects containing both normal code and internal representation (fat-lto-objects). This is required since during build some code is generated by parsing produced ELF objects (pmdinfogen). -8<-------------

Quoted 1 line

Will it suffice? If not, then please suggest wording to be used.

There is a confusion here:
"compilers that support that feature (gcc and icc)" In my understanding, clang supports LTO but is not mentioned because of the lack of fat object support.
I think you should mention gcc and icc only after explaining the second constraint (fat objects).

Quoted 6 lines
Quoted 4 lines
Quoted 1 line

+for compilers that support that feature (gcc and icc). To be more

Please start a new sentence at the beginning of a line in the RsT file. It is easier to review and update.

OK

Please consider such comment to apply to the whole file :)

[...]

Quoted 3 lines

OK. I'll send the next version today.

Thank you again for the comments.

Thank you

#50 lore raw

PATCH v4 2/10 eventdev: fix possible use of uninitialized var

In reply to Andrzej Ostruszka

Fix the logic for the case of event queue allowing all schedule types.

Compiler warning pointing to this error (with LTO enabled):
error: ‘sched_type’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
if ((ret < 0 && ret != -EOVERFLOW) ||

Fixes 6750b21bd6af ("eventdev: add default software timer adapter")
---
 lib/librte_eventdev/rte_event_timer_adapter.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
lib/librte_eventdev/rte_event_timer_adapter.c +4 -4
index 5ce399eca..161e21a68 100644
@@ -706,11 +706,11 @@ check_destination_event_queue(struct rte_event_timer *evtim,
 				       RTE_EVENT_QUEUE_ATTR_SCHEDULE_TYPE,
 				       &sched_type);
 
-	if ((ret < 0 && ret != -EOVERFLOW) ||
-	    evtim->ev.sched_type != sched_type)
-		return -1;
+	if ((ret == 0 && evtim->ev.sched_type == sched_type) ||
+	    ret == -EOVERFLOW)
+		return 0;
 
-	return 0;
+	return -1;
 }
 
 static int
Signature · 1 line
2.17.1
#51 lore raw

PATCH v4 3/10 app/eventdev: clean LTO build warnings (maybe-uninitialized)

In reply to Andrzej Ostruszka

During LTO build compiler reports some 'false positive' warnings about variables being possibly used uninitialized. This patch silences these warnings.

Exemplary compiler warning to suppress (with LTO enabled):
error: ‘service_id’ may be used uninitialized in this function [-Werror=maybe-uninitialized]

    ret = evt_service_setup(service_id);
---
 app/test-eventdev/test_perf_common.c     | 2 +-
 app/test-eventdev/test_pipeline_common.c | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)
app/test-eventdev/test_perf_common.c +1 -1
index aa925a7ef..a974685cb 100644
@@ -439,7 +439,7 @@ perf_event_timer_adapter_setup(struct test_perf *t)
 
 		if (!(adapter_info.caps &
 				RTE_EVENT_TIMER_ADAPTER_CAP_INTERNAL_PORT)) {
-			uint32_t service_id;
+			uint32_t service_id = -1U;
 
 			rte_event_timer_adapter_service_id_get(wl,
 					&service_id);
app/test-eventdev/test_pipeline_common.c +2 -2
index 16c49b860..813d0cf44 100644
@@ -306,7 +306,7 @@ pipeline_event_rx_adapter_setup(struct evt_options *opt, uint8_t stride,
 		}
 
 		if (!(cap & RTE_EVENT_ETH_RX_ADAPTER_CAP_INTERNAL_PORT)) {
-			uint32_t service_id;
+			uint32_t service_id = -1U;
 
 			rte_event_eth_rx_adapter_service_id_get(prod,
 					&service_id);
@@ -358,7 +358,7 @@ pipeline_event_tx_adapter_setup(struct evt_options *opt,
 		}
 
 		if (!(cap & RTE_EVENT_ETH_TX_ADAPTER_CAP_INTERNAL_PORT)) {
-			uint32_t service_id;
+			uint32_t service_id = -1U;
 
 			rte_event_eth_tx_adapter_service_id_get(consm,
 					&service_id);
Signature · 1 line
2.17.1
#52 lore raw

PATCH v4 4/10 event/octeontx2: clean LTO build warnings (maybe-uninitialized)

In reply to Andrzej Ostruszka

During LTO build compiler reports some 'false positive' warnings about variables being possibly used uninitialized. This patch silences these warnings.

Exemplary compiler warning to suppress (with LTO enabled):
error: ‘chunk’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
bkt->current_chunk = (uintptr_t)chunk;

---
 drivers/event/octeontx2/otx2_tim_worker.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/event/octeontx2/otx2_tim_worker.h +1 -1
index b193e2cab..50db6543c 100644
@@ -337,7 +337,7 @@ tim_add_entry_brst(struct otx2_tim_ring * const tim_ring,
 		   const struct otx2_tim_ent *ents,
 		   const uint16_t nb_timers, const uint8_t flags)
 {
-	struct otx2_tim_ent *chunk;
+	struct otx2_tim_ent *chunk = NULL;
 	struct otx2_tim_bkt *bkt;
 	uint16_t chunk_remainder;
 	uint16_t index = 0;
Signature · 1 line
2.17.1
#53 lore raw

PATCH v4 5/10 app/test: clean LTO build warnings (maybe-uninitialized)

In reply to Andrzej Ostruszka

During LTO build compiler reports some 'false positive' warnings about variables being possibly used uninitialized. This patch silences these warnings.

Exemplary compiler warning to suppress (with LTO enabled):
error: ‘stats.greatest_free_size’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
return len - overhead;

---
 app/test/test_hash_readwrite.c     |  2 +-
 app/test/test_link_bonding_mode4.c | 10 ++++++++--
 app/test/test_memzone.c            |  3 ++-
 3 files changed, 11 insertions(+), 4 deletions(-)
app/test/test_hash_readwrite.c +1 -1
index 4376b099b..615767fb6 100644
@@ -298,7 +298,7 @@ test_rw_reader(void *arg)
 
 	begin = rte_rdtsc_precise();
 	for (i = 0; i < read_cnt; i++) {
-		void *data;
+		void *data = arg;
 		rte_hash_lookup_data(tbl_rw_test_param.h,
 				tbl_rw_test_param.keys + i,
 				&data);
app/test/test_link_bonding_mode4.c +8 -2
index bbb4e9cce..5f7df1b5e 100644
@@ -224,7 +224,7 @@ configure_ethdev(uint16_t port_id, uint8_t start)
 static int
 add_slave(struct slave_conf *slave, uint8_t start)
 {
-	struct rte_ether_addr addr, addr_check;
+	struct rte_ether_addr addr, addr_check = { { 0 } };
 
 	/* Some sanity check */
 	RTE_VERIFY(test_params.slave_ports <= slave &&
@@ -578,7 +578,13 @@ bond_get_update_timeout_ms(void)
 {
 	struct rte_eth_bond_8023ad_conf conf;
 
-	rte_eth_bond_8023ad_conf_get(test_params.bonded_port_id, &conf);
+	if (rte_eth_bond_8023ad_conf_get(test_params.bonded_port_id, &conf) < 0) {
+		RTE_LOG(DEBUG, EAL, "Failed to get bonding configuration: "
+				    "%s at %d\n", __func__, __LINE__);
+		RTE_TEST_TRACE_FAILURE(__FILE__, __LINE__, __func__);
+		return 0;
+	}
+
 	return conf.update_timeout_ms;
 }
 
app/test/test_memzone.c +2 -1
index 7501b63c5..c284dcb44 100644
@@ -476,7 +476,8 @@ find_max_block_free_size(unsigned int align, unsigned int socket_id)
 	struct rte_malloc_socket_stats stats;
 	size_t len, overhead;
 
-	rte_malloc_get_socket_stats(socket_id, &stats);
+	if (rte_malloc_get_socket_stats(socket_id, &stats) < 0)
+		return 0;
 
 	len = stats.greatest_free_size;
 	overhead = MALLOC_ELEM_OVERHEAD;
Signature · 1 line
2.17.1
#54 lore raw

PATCH v4 6/10 net/dpaa2: fix possible use of uninitialized vars

In reply to Andrzej Ostruszka

This patch fixes 'maybe-uninitialized' warnings reported by compiler when using LTO.

Compiler warning pointing to this error (with LTO enabled):
error: ‘kg_cfg.extracts[0].masks[0].mask’ may be used uninitialized in this function [-Werror=maybe-uninitialized]

    extr->masks[j].mask = cfg->extracts[i].masks[j].mask;
Fixes 16bbc98a3e63 ("bus/fslmc: update MC to 10.3.x")
---
 drivers/net/dpaa2/base/dpaa2_hw_dpni.c | 1 +
 drivers/net/dpaa2/mc/dpkg.c            | 2 +-
 drivers/net/dpaa2/mc/dpni.c            | 9 ++++++---
 3 files changed, 8 insertions(+), 4 deletions(-)
drivers/net/dpaa2/base/dpaa2_hw_dpni.c +1 -0
index 56e2e56a3..2f6534a31 100644
@@ -51,6 +51,7 @@ rte_pmd_dpaa2_set_custom_hash(uint16_t port_id,
 	kg_cfg.extracts[0].type = DPKG_EXTRACT_FROM_DATA;
 	kg_cfg.extracts[0].extract.from_data.offset = offset;
 	kg_cfg.extracts[0].extract.from_data.size = size;
+	kg_cfg.extracts[0].num_of_byte_masks = 0;
 	kg_cfg.num_extracts = 1;
 
 	ret = dpkg_prepare_key_cfg(&kg_cfg, p_params);
drivers/net/dpaa2/mc/dpkg.c +1 -1
index 80f94f40e..7aa63ea12 100644
@@ -63,7 +63,7 @@ dpkg_prepare_key_cfg(const struct dpkg_profile_cfg *cfg, uint8_t *key_cfg_buf)
 		dpkg_set_field(extr->extract_type, EXTRACT_TYPE,
 			       cfg->extracts[i].type);
 
-		for (j = 0; j < DPKG_NUM_OF_MASKS; j++) {
+		for (j = 0; j < extr->num_of_byte_masks; j++) {
 			extr->masks[j].mask = cfg->extracts[i].masks[j].mask;
 			extr->masks[j].offset =
 				cfg->extracts[i].masks[j].offset;
drivers/net/dpaa2/mc/dpni.c +6 -3
index 362cd476f..b74a1a317 100644
@@ -1803,10 +1803,13 @@ int dpni_set_congestion_notification(struct fsl_mc_io *mc_io,
 	cmd_params->qtype = qtype;
 	cmd_params->tc = tc_id;
 	cmd_params->congestion_point = cfg->cg_point;
-	cmd_params->cgid = (uint8_t)cfg->cgid;
-	cmd_params->dest_id = cpu_to_le32(cfg->dest_cfg.dest_id);
+	if (cfg->cg_point == DPNI_CP_CONGESTION_GROUP)
+		cmd_params->cgid = (uint8_t)cfg->cgid;
+	if (cfg->dest_cfg.dest_type != DPNI_DEST_NONE) {
+		cmd_params->dest_id = cpu_to_le32(cfg->dest_cfg.dest_id);
+		cmd_params->dest_priority = cfg->dest_cfg.priority;
+	}
 	cmd_params->notification_mode = cpu_to_le16(cfg->notification_mode);
-	cmd_params->dest_priority = cfg->dest_cfg.priority;
 	cmd_params->message_iova = cpu_to_le64(cfg->message_iova);
 	cmd_params->message_ctx = cpu_to_le64(cfg->message_ctx);
 	cmd_params->threshold_entry = cpu_to_le32(cfg->threshold_entry);
Signature · 1 line
2.17.1
#55 lore raw

PATCH v4 7/10 net/e1000: clean LTO build warnings (maybe-uninitialized)

In reply to Andrzej Ostruszka

During LTO build compiler reports some 'false positive' warnings about variables being possibly used uninitialized. This patch silences these warnings.

Exemplary compiler warning to suppress (with LTO enabled):
error: ‘link’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
if (link) {

---
 drivers/net/e1000/base/e1000_82543.c   | 2 +-
 drivers/net/e1000/base/e1000_ich8lan.c | 2 +-
 drivers/net/e1000/base/e1000_phy.c     | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)
drivers/net/e1000/base/e1000_82543.c +1 -1
index 810899eb6..dfde2a8b9 100644
@@ -1027,7 +1027,7 @@ STATIC s32 e1000_setup_copper_link_82543(struct e1000_hw *hw)
 {
 	u32 ctrl;
 	s32 ret_val;
-	bool link;
+	bool link = true;
 
 	DEBUGFUNC("e1000_setup_copper_link_82543");
 
drivers/net/e1000/base/e1000_ich8lan.c +1 -1
index accc6ea01..5241cf698 100644
@@ -5533,7 +5533,7 @@ void e1000_igp3_phy_powerdown_workaround_ich8lan(struct e1000_hw *hw)
 void e1000_gig_downshift_workaround_ich8lan(struct e1000_hw *hw)
 {
 	s32 ret_val;
-	u16 reg_data;
+	u16 reg_data = 0;
 
 	DEBUGFUNC("e1000_gig_downshift_workaround_ich8lan");
 
drivers/net/e1000/base/e1000_phy.c +1 -1
index 7d08f836f..956c06747 100644
@@ -1664,7 +1664,7 @@ s32 e1000_copper_link_autoneg(struct e1000_hw *hw)
 s32 e1000_setup_copper_link_generic(struct e1000_hw *hw)
 {
 	s32 ret_val;
-	bool link;
+	bool link = true;
 
 	DEBUGFUNC("e1000_setup_copper_link_generic");
 
Signature · 1 line
2.17.1
#56 lore raw

PATCH v4 8/10 net/i40e: clean LTO build warnings (maybe-uninitialized)

In reply to Andrzej Ostruszka

During LTO build compiler reports some 'false positive' warnings about variables being possibly used uninitialized. This patch silences these warnings.

Exemplary compiler warning to suppress (with LTO enabled):
error: ‘filter_idx’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
PMD_DRV_LOG(INFO, "Added port %d with AQ command with index %d",

---
 drivers/net/i40e/i40e_ethdev.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/i40e/i40e_ethdev.c +1 -1
index 4e40b7ab5..525a6b69b 100644
@@ -8376,7 +8376,7 @@ static int
 i40e_add_vxlan_port(struct i40e_pf *pf, uint16_t port, int udp_type)
 {
 	int  idx, ret;
-	uint8_t filter_idx;
+	uint8_t filter_idx = 0;
 	struct i40e_hw *hw = I40E_PF_TO_HW(pf);
 
 	idx = i40e_get_vxlan_port_idx(pf, port);
Signature · 1 line
2.17.1
#57 lore raw

PATCH v4 9/10 net/ifc: clean LTO build warnings (maybe-uninitialized)

In reply to Andrzej Ostruszka

During LTO build compiler reports some 'false positive' warnings about variables being possibly used uninitialized. This patch silences these warnings.

Exemplary compiler warning to suppress (with LTO enabled):
error: ‘features’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
if (RTE_VHOST_NEED_LOG(features)) {

---
 drivers/net/ifc/ifcvf_vdpa.c | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)
drivers/net/ifc/ifcvf_vdpa.c +9 -5
index 8de9ef199..4cdff8712 100644
@@ -304,8 +304,8 @@ vdpa_ifcvf_stop(struct ifcvf_internal *internal)
 	struct ifcvf_hw *hw = &internal->hw;
 	uint32_t i;
 	int vid;
-	uint64_t features;
-	uint64_t log_base, log_size;
+	uint64_t features = 0;
+	uint64_t log_base = 0, log_size = 0;
 	uint64_t len;
 
 	vid = internal->vid;
@@ -348,6 +348,8 @@ vdpa_enable_vfio_intr(struct ifcvf_internal *internal, bool m_rx)
 	struct rte_vhost_vring vring;
 	int fd;
 
+	vring.callfd = -1;
+
 	nr_vring = rte_vhost_get_vring_num(internal->vid);
 
 	irq_set = (struct vfio_irq_set *)irq_set_buf;
@@ -442,6 +444,7 @@ notify_relay(void *arg)
 	}
 	internal->epfd = epfd;
 
+	vring.kickfd = -1;
 	for (qid = 0; qid < q_num; qid++) {
 		ev.events = EPOLLIN | EPOLLPRI;
 		rte_vhost_get_vhost_vring(internal->vid, qid, &vring);
@@ -577,7 +580,7 @@ m_ifcvf_start(struct ifcvf_internal *internal)
 	struct ifcvf_hw *hw = &internal->hw;
 	uint32_t i, nr_vring;
 	int vid, ret;
-	struct rte_vhost_vring vq;
+	struct rte_vhost_vring vq = { 0 };
 	void *vring_buf;
 	uint64_t m_vring_iova = IFCVF_MEDIATED_VRING;
 	uint64_t size;
@@ -721,6 +724,7 @@ vring_relay(void *arg)
 	}
 	internal->epfd = epfd;
 
+	vring.kickfd = -1;
 	for (qid = 0; qid < q_num; qid++) {
 		ev.events = EPOLLIN | EPOLLPRI;
 		rte_vhost_get_vhost_vring(vid, qid, &vring);
@@ -930,11 +934,11 @@ ifcvf_dev_close(int vid)
 static int
 ifcvf_set_features(int vid)
 {
-	uint64_t features;
+	uint64_t features = 0;
 	int did;
 	struct internal_list *list;
 	struct ifcvf_internal *internal;
-	uint64_t log_base, log_size;
+	uint64_t log_base = 0, log_size = 0;
 
 	did = rte_vhost_get_vdpa_device_id(vid);
 	list = find_internal_resource_by_did(did);
Signature · 1 line
2.17.1
#58 lore raw

PATCH v4 10/10 net/qede: clean LTO build warnings (maybe-uninitialized)

In reply to Andrzej Ostruszka

During LTO build compiler reports some 'false positive' warnings about variables being possibly used uninitialized. This patch silences these warnings.

Exemplary compiler warning to suppress (with LTO enabled):
error: ‘transceiver_type’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
switch (transceiver_type) {

---
 drivers/net/qede/base/ecore_mcp.c | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)
drivers/net/qede/base/ecore_mcp.c +7 -6
index 6c6560688..666c0fe12 100644
@@ -2278,7 +2278,7 @@ enum _ecore_status_t ecore_mcp_trans_speed_mask(struct ecore_hwfn *p_hwfn,
 						struct ecore_ptt *p_ptt,
 						u32 *p_speed_mask)
 {
-	u32 transceiver_type, transceiver_state;
+	u32 transceiver_type = ETH_TRANSCEIVER_TYPE_NONE, transceiver_state;
 
 	ecore_mcp_get_transceiver_data(p_hwfn, p_ptt, &transceiver_state,
 				       &transceiver_type);
@@ -3163,7 +3163,8 @@ enum _ecore_status_t ecore_mcp_nvm_put_file_begin(struct ecore_dev *p_dev,
 enum _ecore_status_t ecore_mcp_nvm_write(struct ecore_dev *p_dev, u32 cmd,
 					 u32 addr, u8 *p_buf, u32 len)
 {
-	u32 buf_idx, buf_size, nvm_cmd, nvm_offset, resp, param;
+	u32 buf_idx, buf_size, nvm_cmd, nvm_offset;
+	u32 resp = FW_MSG_CODE_ERROR, param;
 	struct ecore_hwfn *p_hwfn = ECORE_LEADING_HWFN(p_dev);
 	enum _ecore_status_t rc = ECORE_INVAL;
 	struct ecore_ptt *p_ptt;
@@ -3370,7 +3371,7 @@ enum _ecore_status_t ecore_mcp_gpio_read(struct ecore_hwfn *p_hwfn,
 					 u16 gpio, u32 *gpio_val)
 {
 	enum _ecore_status_t rc = ECORE_SUCCESS;
-	u32 drv_mb_param = 0, rsp;
+	u32 drv_mb_param = 0, rsp = 0;
 
 	drv_mb_param = (gpio << DRV_MB_PARAM_GPIO_NUMBER_OFFSET);
 
@@ -3391,7 +3392,7 @@ enum _ecore_status_t ecore_mcp_gpio_write(struct ecore_hwfn *p_hwfn,
 					  u16 gpio, u16 gpio_val)
 {
 	enum _ecore_status_t rc = ECORE_SUCCESS;
-	u32 drv_mb_param = 0, param, rsp;
+	u32 drv_mb_param = 0, param, rsp = 0;
 
 	drv_mb_param = (gpio << DRV_MB_PARAM_GPIO_NUMBER_OFFSET) |
 		(gpio_val << DRV_MB_PARAM_GPIO_VALUE_OFFSET);
@@ -3481,7 +3482,7 @@ enum _ecore_status_t ecore_mcp_bist_clock_test(struct ecore_hwfn *p_hwfn,
 enum _ecore_status_t ecore_mcp_bist_nvm_test_get_num_images(
 	struct ecore_hwfn *p_hwfn, struct ecore_ptt *p_ptt, u32 *num_images)
 {
-	u32 drv_mb_param = 0, rsp;
+	u32 drv_mb_param = 0, rsp = 0;
 	enum _ecore_status_t rc = ECORE_SUCCESS;
 
 	drv_mb_param = (DRV_MB_PARAM_BIST_NVM_TEST_NUM_IMAGES <<
@@ -3865,7 +3866,7 @@ enum _ecore_status_t
 __ecore_mcp_resc_lock(struct ecore_hwfn *p_hwfn, struct ecore_ptt *p_ptt,
 		      struct ecore_resc_lock_params *p_params)
 {
-	u32 param = 0, mcp_resp, mcp_param;
+	u32 param = 0, mcp_resp = 0, mcp_param = 0;
 	u8 opcode;
 	enum _ecore_status_t rc;
 
Signature · 1 line
2.17.1
#59 lore raw
In reply to Andrzej Ostruszka
Quoted 48 lines · Andrzej Ostruszka

This patch series adds an option to make use of link time optimization (if compiler has support for it). It is split as follows:
- 1st patch (build) is the enablement
- remaining patches are fixes for the warnings produced by the compiler and they are split by directory/subsystem so their maintainers can easily find and verify the changes - please note that there are two groups:
* errors (or possible errors) - with title "fix possible use ..."
* false positives - warnings that _I_ think are not valid and the changes are made only to silence the compiler.

v4 Changes:
-----------
- merge nested conditionals in config/meson.build into one

v3 Changes:
-----------
- removed support for clang (only gcc and icc remain):
- it turned out that 'fat-lto-objects' is not really supported - it is accepted as a flag but ignored e.g. clang v6.0:
clang: error: optimization flag '-ffat-lto-objects' is not supported [-Werror,-Wignored-optimization-argument]
this was probably masked previously by playing around with meson and testing for the availability of this flag and dropping LTO if not supported.
- improved commit messages:
- added 'Fixes' tags where appropriate
- included snippets of compiler warnings
- changed commit names for the commits cleaning up 'false positive' compiler warnings - these are not really 'fixes' for the code, just workarounds for compiler shortcomings and check-git-log.sh was complaining about missing 'Fixes' tag.

v2 Changes:
-----------
- fixed building of shared libraries:
- gcc does not scan top level assembler statements so it missed that function implementations for specific versions were being used and was removing them
- fixed meson.build config files:
- moved from 'enable_lto' project option to built-in 'b_lto'
- documented that 'default_library' should be 'shared':
with 'default_library=static' (the default) the meson build with LTO is broken - this is because, SHARED_LIB is fixed in rte_config.h used by meson which leads to no alias for default version in generated static libraries (MAP_STATIC_SYMBOL() is empty)
- app/test: added log for failed bonding "config get"

Quoted 12 lines

Andrzej Ostruszka (10):
build: add an option to enable LTO build
eventdev: fix possible use of uninitialized var app/eventdev: clean LTO build warnings (maybe-uninitialized) event/octeontx2: clean LTO build warnings (maybe-uninitialized) app/test: clean LTO build warnings (maybe-uninitialized) net/dpaa2: fix possible use of uninitialized vars net/e1000: clean LTO build warnings (maybe-uninitialized) net/i40e: clean LTO build warnings (maybe-uninitialized) net/ifc: clean LTO build warnings (maybe-uninitialized) net/qede: clean LTO build warnings (maybe-uninitialized)

Does this have a dependency on set 6726 [1] ?

Otherwise, this looks something useful to have support for in DPDK, so

[1] http://patches.dpdk.org/project/dpdk/list/?series=6726

#60 lore raw
Quoted 2 lines · Bruce Richardson
Quoted 1 line · Andrzej Ostruszka

This patch series adds an option to make use of link time optimization

[...]

Quoted 1 line

Does this have a dependency on set 6726 [1] ?

[...]

Quoted 1 line

[1] http://patches.dpdk.org/project/dpdk/list/?series=672

For the meson part it does. I should have stated that explicitly since I've cleaned the docs (no longer required to have "default_libary=shared").

Regards
Andrzej

#61 lore raw

v5 cover Add an option to use LTO for DPDK build

In reply to Andrzej Ostruszka

This patch series adds an option to make use of link time optimization (if compiler has support for it). It is split as follows:
- 1st patch fixes missing __vsym annotations (needed for LTO)
- 2nd is the LTO enablement
- remaining patches are fixes for the warnings produced by the compiler and they are split by directory/subsystem so their maintainers can easily find and verify the changes - please note that there are two groups:
* errors (or possible errors) - with title "fix possible use ..."
* false positives - warnings that _I_ think are not valid and the changes are made only to silence the compiler.

v5 Changes:
-----------
- rebased to the v19.11 (one additional warning silenced in net/qede)
- initial patch has been split into two:
- one adding '__vsym' annotations and clarifying versioning docs
- another adding the actual LTO enablement
- addressed various documentation flaws pointed during review

v4 Changes:
-----------
- merge nested conditionals in config/meson.build into one

v3 Changes:
-----------
- removed support for clang (only gcc and icc remain):
- it turned out that 'fat-lto-objects' is not really supported - it is accepted as a flag but ignored e.g. clang v6.0:
clang: error: optimization flag '-ffat-lto-objects' is not supported [-Werror,-Wignored-optimization-argument]
this was probably masked previously by playing around with meson and testing for the availability of this flag and dropping LTO if not supported.
- improved commit messages:
- added 'Fixes' tags where appropriate
- included snippets of compiler warnings
- changed commit names for the commits cleaning up 'false positive' compiler warnings - these are not really 'fixes' for the code, just workarounds for compiler shortcomings and check-git-log.sh was complaining about missing 'Fixes' tag.

v2 Changes:
-----------
- fixed building of shared libraries:
- gcc does not scan top level assembler statements so it missed that function implementations for specific versions were being used and was removing them
- fixed meson.build config files:
- moved from 'enable_lto' project option to built-in 'b_lto'
- documented that 'default_library' should be 'shared':
with 'default_library=static' (the default) the meson build with LTO is broken - this is because, SHARED_LIB is fixed in rte_config.h used by meson which leads to no alias for default version in generated static libraries (MAP_STATIC_SYMBOL() is empty)
- app/test: added log for failed bonding "config get"

Andrzej Ostruszka (11):
build: annotate versioned symbols with __vsym macro
build: add an option to enable LTO build
eventdev: fix possible use of uninitialized var app/eventdev: clean LTO build warnings (maybe-uninitialized) event/octeontx2: clean LTO build warnings (maybe-uninitialized) app/test: clean LTO build warnings (maybe-uninitialized) net/dpaa2: fix possible use of uninitialized vars net/e1000: clean LTO build warnings (maybe-uninitialized) net/i40e: clean LTO build warnings (maybe-uninitialized) net/ifc: clean LTO build warnings (maybe-uninitialized) net/qede: clean LTO build warnings (maybe-uninitialized)

 .travis.yml                                   |  7 ++++
 app/test-eventdev/test_perf_common.c          |  2 +-
 app/test-eventdev/test_pipeline_common.c      |  4 +-
 app/test/test_hash_readwrite.c                |  2 +-
 app/test/test_link_bonding_mode4.c            |  8 +++-
 app/test/test_memzone.c                       |  3 +-
 config/common_base                            |  5 +++
 config/meson.build                            | 13 ++++++
 doc/guides/contributing/versioning.rst        | 18 ++++++---
 doc/guides/prog_guide/index.rst               |  1 +
 doc/guides/prog_guide/lto.rst                 | 40 +++++++++++++++++++
 doc/guides/rel_notes/release_19_11.rst        |  9 +++++
 drivers/event/octeontx2/otx2_tim_worker.h     |  2 +-
 drivers/net/dpaa2/base/dpaa2_hw_dpni.c        |  1 +
 drivers/net/dpaa2/mc/dpkg.c                   |  2 +-
 drivers/net/dpaa2/mc/dpni.c                   |  9 +++--
 drivers/net/e1000/base/e1000_82543.c          |  2 +-
 drivers/net/e1000/base/e1000_ich8lan.c        |  2 +-
 drivers/net/e1000/base/e1000_phy.c            |  2 +-
 drivers/net/i40e/i40e_ethdev.c                |  2 +-
 drivers/net/ifc/ifcvf_vdpa.c                  | 14 ++++---
 drivers/net/qede/base/ecore_mcp.c             | 15 +++----
 lib/librte_distributor/rte_distributor.c      | 18 ++++-----
 lib/librte_distributor/rte_distributor_v20.c  | 18 ++++-----
 .../common/include/rte_function_versioning.h  | 11 ++++-
 lib/librte_eventdev/rte_event_timer_adapter.c |  8 ++--
 lib/librte_lpm/rte_lpm.c                      | 28 ++++++-------
 lib/librte_lpm/rte_lpm6.c                     | 16 ++++----
 lib/librte_timer/rte_timer.c                  | 20 +++++-----
 mk/toolchain/gcc/rte.toolchain-compat.mk      |  4 ++
 mk/toolchain/gcc/rte.vars.mk                  | 12 ++++++
 mk/toolchain/icc/rte.vars.mk                  |  8 ++++
 32 files changed, 217 insertions(+), 89 deletions(-)
 create mode 100644 doc/guides/prog_guide/lto.rst
Signature · 1 line
2.17.1
#62 lore raw

PATCH v5 1/11 build: annotate versioned symbols with __vsym macro

Every implementation of a particular version of given symbol needs to be marked in its declaration as such (using __vsym macro). This patch fixes this and also clarifies the documentation about that.

---
 doc/guides/contributing/versioning.rst        | 18 ++++++++----
 lib/librte_distributor/rte_distributor.c      | 18 ++++++------
 lib/librte_distributor/rte_distributor_v20.c  | 18 ++++++------
 .../common/include/rte_function_versioning.h  | 11 ++++++--
 lib/librte_lpm/rte_lpm.c                      | 28 +++++++++----------
 lib/librte_lpm/rte_lpm6.c                     | 16 +++++------
 lib/librte_timer/rte_timer.c                  | 20 ++++++-------
 7 files changed, 71 insertions(+), 58 deletions(-)
doc/guides/contributing/versioning.rst +12 -6
index 64984c54e..fcd2d50f1 100644
@@ -215,16 +215,20 @@ library so that older binaries need not be immediately recompiled.
 The macros exported are:
 
 * ``VERSION_SYMBOL(b, e, n)``: Creates a symbol version table entry binding
-  versioned symbol ``b@DPDK_n`` to the internal function ``b_e``.
+  versioned symbol ``b@DPDK_n`` to the internal function ``be``.
 
 * ``BIND_DEFAULT_SYMBOL(b, e, n)``: Creates a symbol version entry instructing
   the linker to bind references to symbol ``b`` to the internal symbol
-  ``b_e``.
+  ``be``.
 
 * ``MAP_STATIC_SYMBOL(f, p)``: Declare the prototype ``f``, and map it to the
   fully qualified function ``p``, so that if a symbol becomes versioned, it
   can still be mapped back to the public symbol name.
 
+* ``__vsym``:  Annotation to be used in a declaration of the internal symbol
+  ``be`` to signal that it is being used as an implementation of a particular
+  version of symbol ``b``.
+
 Examples of ABI Macro use
 ^^^^^^^^^^^^^^^^^^^^^^^^^
 
@@ -345,8 +349,9 @@ with the public symbol name
 
 .. code-block:: c
 
-  struct rte_acl_ctx *
+ -struct rte_acl_ctx *
  -rte_acl_create(const struct rte_acl_param *param)
+ +struct rte_acl_ctx * __vsym
  +rte_acl_create_v20(const struct rte_acl_param *param)
  {
         size_t sz;
@@ -354,7 +359,8 @@ with the public symbol name
         ...
 
 Note that the base name of the symbol was kept intact, as this is conducive to
-the macros used for versioning symbols.  That is our next step, mapping this new
+the macros used for versioning symbols and we have annotated the function as an
+implementation of versioned symbol.  That is our next step, mapping this new
 symbol name to the initial symbol name at version node 2.0.  Immediately after
 the function, we add this line of code
 
@@ -374,7 +380,7 @@ name, with a different suffix, and  implement it appropriately
 
 .. code-block:: c
 
-   struct rte_acl_ctx *
+   struct rte_acl_ctx * __vsym
    rte_acl_create_v21(const struct rte_acl_param *param, int debug);
    {
         struct rte_acl_ctx *ctx = rte_acl_create_v20(param);
@@ -423,7 +429,7 @@ defined, we add this
 
 .. code-block:: c
 
-   struct rte_acl_ctx *
+   struct rte_acl_ctx * __vsym
    rte_acl_create_v21(const struct rte_acl_param *param, int debug)
    {
         ...
lib/librte_distributor/rte_distributor.c +9 -9
index 7df97df92..2cc32ddba 100644
@@ -32,7 +32,7 @@ EAL_REGISTER_TAILQ(rte_dist_burst_tailq)
 
 /**** Burst Packet APIs called by workers ****/
 
-void
+void __vsym
 rte_distributor_request_pkt_v1705(struct rte_distributor *d,
 		unsigned int worker_id, struct rte_mbuf **oldpkt,
 		unsigned int count)
@@ -89,7 +89,7 @@ MAP_STATIC_SYMBOL(void rte_distributor_request_pkt(struct rte_distributor *d,
 		unsigned int count),
 		rte_distributor_request_pkt_v1705);
 
-int
+int __vsym
 rte_distributor_poll_pkt_v1705(struct rte_distributor *d,
 		unsigned int worker_id, struct rte_mbuf **pkts)
 {
@@ -134,7 +134,7 @@ MAP_STATIC_SYMBOL(int rte_distributor_poll_pkt(struct rte_distributor *d,
 		unsigned int worker_id, struct rte_mbuf **pkts),
 		rte_distributor_poll_pkt_v1705);
 
-int
+int __vsym
 rte_distributor_get_pkt_v1705(struct rte_distributor *d,
 		unsigned int worker_id, struct rte_mbuf **pkts,
 		struct rte_mbuf **oldpkt, unsigned int return_count)
@@ -169,7 +169,7 @@ MAP_STATIC_SYMBOL(int rte_distributor_get_pkt(struct rte_distributor *d,
 		struct rte_mbuf **oldpkt, unsigned int return_count),
 		rte_distributor_get_pkt_v1705);
 
-int
+int __vsym
 rte_distributor_return_pkt_v1705(struct rte_distributor *d,
 		unsigned int worker_id, struct rte_mbuf **oldpkt, int num)
 {
@@ -359,7 +359,7 @@ release(struct rte_distributor *d, unsigned int wkr)
 
 
 /* process a set of packets to distribute them to workers */
-int
+int __vsym
 rte_distributor_process_v1705(struct rte_distributor *d,
 		struct rte_mbuf **mbufs, unsigned int num_mbufs)
 {
@@ -506,7 +506,7 @@ MAP_STATIC_SYMBOL(int rte_distributor_process(struct rte_distributor *d,
 		rte_distributor_process_v1705);
 
 /* return to the caller, packets returned from workers */
-int
+int __vsym
 rte_distributor_returned_pkts_v1705(struct rte_distributor *d,
 		struct rte_mbuf **mbufs, unsigned int max_mbufs)
 {
@@ -556,7 +556,7 @@ total_outstanding(const struct rte_distributor *d)
  * Flush the distributor, so that there are no outstanding packets in flight or
  * queued up.
  */
-int
+int __vsym
 rte_distributor_flush_v1705(struct rte_distributor *d)
 {
 	unsigned int flushed;
@@ -591,7 +591,7 @@ MAP_STATIC_SYMBOL(int rte_distributor_flush(struct rte_distributor *d),
 		rte_distributor_flush_v1705);
 
 /* clears the internal returns array in the distributor */
-void
+void __vsym
 rte_distributor_clear_returns_v1705(struct rte_distributor *d)
 {
 	unsigned int wkr;
@@ -613,7 +613,7 @@ MAP_STATIC_SYMBOL(void rte_distributor_clear_returns(struct rte_distributor *d),
 		rte_distributor_clear_returns_v1705);
 
 /* creates a distributor instance */
-struct rte_distributor *
+struct rte_distributor * __vsym
 rte_distributor_create_v1705(const char *name,
 		unsigned int socket_id,
 		unsigned int num_workers,
lib/librte_distributor/rte_distributor_v20.c +9 -9
index db6c49258..7a6fddf55 100644
@@ -27,7 +27,7 @@ EAL_REGISTER_TAILQ(rte_distributor_tailq)
 
 /**** APIs called by workers ****/
 
-void
+void __vsym
 rte_distributor_request_pkt_v20(struct rte_distributor_v20 *d,
 		unsigned worker_id, struct rte_mbuf *oldpkt)
 {
@@ -43,7 +43,7 @@ rte_distributor_request_pkt_v20(struct rte_distributor_v20 *d,
 }
 VERSION_SYMBOL(rte_distributor_request_pkt, _v20, 2.0);
 
-struct rte_mbuf *
+struct rte_mbuf * __vsym
 rte_distributor_poll_pkt_v20(struct rte_distributor_v20 *d,
 		unsigned worker_id)
 {
@@ -59,7 +59,7 @@ rte_distributor_poll_pkt_v20(struct rte_distributor_v20 *d,
 }
 VERSION_SYMBOL(rte_distributor_poll_pkt, _v20, 2.0);
 
-struct rte_mbuf *
+struct rte_mbuf * __vsym
 rte_distributor_get_pkt_v20(struct rte_distributor_v20 *d,
 		unsigned worker_id, struct rte_mbuf *oldpkt)
 {
@@ -71,7 +71,7 @@ rte_distributor_get_pkt_v20(struct rte_distributor_v20 *d,
 }
 VERSION_SYMBOL(rte_distributor_get_pkt, _v20, 2.0);
 
-int
+int __vsym
 rte_distributor_return_pkt_v20(struct rte_distributor_v20 *d,
 		unsigned worker_id, struct rte_mbuf *oldpkt)
 {
@@ -204,7 +204,7 @@ process_returns(struct rte_distributor_v20 *d)
 }
 
 /* process a set of packets to distribute them to workers */
-int
+int __vsym
 rte_distributor_process_v20(struct rte_distributor_v20 *d,
 		struct rte_mbuf **mbufs, unsigned num_mbufs)
 {
@@ -321,7 +321,7 @@ rte_distributor_process_v20(struct rte_distributor_v20 *d,
 VERSION_SYMBOL(rte_distributor_process, _v20, 2.0);
 
 /* return to the caller, packets returned from workers */
-int
+int __vsym
 rte_distributor_returned_pkts_v20(struct rte_distributor_v20 *d,
 		struct rte_mbuf **mbufs, unsigned max_mbufs)
 {
@@ -359,7 +359,7 @@ total_outstanding(const struct rte_distributor_v20 *d)
 
 /* flush the distributor, so that there are no outstanding packets in flight or
  * queued up. */
-int
+int __vsym
 rte_distributor_flush_v20(struct rte_distributor_v20 *d)
 {
 	const unsigned flushed = total_outstanding(d);
@@ -372,7 +372,7 @@ rte_distributor_flush_v20(struct rte_distributor_v20 *d)
 VERSION_SYMBOL(rte_distributor_flush, _v20, 2.0);
 
 /* clears the internal returns array in the distributor */
-void
+void __vsym
 rte_distributor_clear_returns_v20(struct rte_distributor_v20 *d)
 {
 	d->returns.start = d->returns.count = 0;
@@ -383,7 +383,7 @@ rte_distributor_clear_returns_v20(struct rte_distributor_v20 *d)
 VERSION_SYMBOL(rte_distributor_clear_returns, _v20, 2.0);
 
 /* creates a distributor instance */
-struct rte_distributor_v20 *
+struct rte_distributor_v20 * __vsym
 rte_distributor_create_v20(const char *name,
 		unsigned socket_id,
 		unsigned num_workers)
lib/librte_eal/common/include/rte_function_versioning.h +9 -2
index 55e88ffae..c924351d5 100644
@@ -42,16 +42,23 @@
 /*
  * VERSION_SYMBOL
  * Creates a symbol version table entry binding symbol <b>@DPDK_<n> to the internal
- * function name <b>_<e>
+ * function name <b><e>
  */
 #define VERSION_SYMBOL(b, e, n) __asm__(".symver " RTE_STR(b) RTE_STR(e) ", " RTE_STR(b) "@DPDK_" RTE_STR(n))
 
 /*
  * BIND_DEFAULT_SYMBOL
  * Creates a symbol version entry instructing the linker to bind references to
- * symbol <b> to the internal symbol <b>_<e>
+ * symbol <b> to the internal symbol <b><e>
  */
 #define BIND_DEFAULT_SYMBOL(b, e, n) __asm__(".symver " RTE_STR(b) RTE_STR(e) ", " RTE_STR(b) "@@DPDK_" RTE_STR(n))
+
+/*
+ * __vsym
+ * Annotation to be used in declaration of the internal symbol <b><e> to signal
+ * that it is being used as an implementation of a particular version of symbol
+ * <b>.
+ */
 #define __vsym __attribute__((used))
 
 /*
lib/librte_lpm/rte_lpm.c +14 -14
index c96395e26..106916dc8 100644
@@ -90,7 +90,7 @@ depth_to_range(uint8_t depth)
 /*
  * Find an existing lpm table and return a pointer to it.
  */
-struct rte_lpm_v20 *
+struct rte_lpm_v20 * __vsym
 rte_lpm_find_existing_v20(const char *name)
 {
 	struct rte_lpm_v20 *l = NULL;
@@ -116,7 +116,7 @@ rte_lpm_find_existing_v20(const char *name)
 }
 VERSION_SYMBOL(rte_lpm_find_existing, _v20, 2.0);
 
-struct rte_lpm *
+struct rte_lpm * __vsym
 rte_lpm_find_existing_v1604(const char *name)
 {
 	struct rte_lpm *l = NULL;
@@ -147,7 +147,7 @@ MAP_STATIC_SYMBOL(struct rte_lpm *rte_lpm_find_existing(const char *name),
 /*
  * Allocates memory for LPM object
  */
-struct rte_lpm_v20 *
+struct rte_lpm_v20 * __vsym
 rte_lpm_create_v20(const char *name, int socket_id, int max_rules,
 		__rte_unused int flags)
 {
@@ -220,7 +220,7 @@ rte_lpm_create_v20(const char *name, int socket_id, int max_rules,
 }
 VERSION_SYMBOL(rte_lpm_create, _v20, 2.0);
 
-struct rte_lpm *
+struct rte_lpm * __vsym
 rte_lpm_create_v1604(const char *name, int socket_id,
 		const struct rte_lpm_config *config)
 {
@@ -329,7 +329,7 @@ MAP_STATIC_SYMBOL(
 /*
  * Deallocates memory for given LPM table.
  */
-void
+void __vsym
 rte_lpm_free_v20(struct rte_lpm_v20 *lpm)
 {
 	struct rte_lpm_list *lpm_list;
@@ -358,7 +358,7 @@ rte_lpm_free_v20(struct rte_lpm_v20 *lpm)
 }
 VERSION_SYMBOL(rte_lpm_free, _v20, 2.0);
 
-void
+void __vsym
 rte_lpm_free_v1604(struct rte_lpm *lpm)
 {
 	struct rte_lpm_list *lpm_list;
@@ -1177,7 +1177,7 @@ add_depth_big_v1604(struct rte_lpm *lpm, uint32_t ip_masked, uint8_t depth,
 /*
  * Add a route
  */
-int
+int __vsym
 rte_lpm_add_v20(struct rte_lpm_v20 *lpm, uint32_t ip, uint8_t depth,
 		uint8_t next_hop)
 {
@@ -1218,7 +1218,7 @@ rte_lpm_add_v20(struct rte_lpm_v20 *lpm, uint32_t ip, uint8_t depth,
 }
 VERSION_SYMBOL(rte_lpm_add, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm_add_v1604(struct rte_lpm *lpm, uint32_t ip, uint8_t depth,
 		uint32_t next_hop)
 {
@@ -1264,7 +1264,7 @@ MAP_STATIC_SYMBOL(int rte_lpm_add(struct rte_lpm *lpm, uint32_t ip,
 /*
  * Look for a rule in the high-level rules table
  */
-int
+int __vsym
 rte_lpm_is_rule_present_v20(struct rte_lpm_v20 *lpm, uint32_t ip, uint8_t depth,
 uint8_t *next_hop)
 {
@@ -1291,7 +1291,7 @@ uint8_t *next_hop)
 }
 VERSION_SYMBOL(rte_lpm_is_rule_present, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm_is_rule_present_v1604(struct rte_lpm *lpm, uint32_t ip, uint8_t depth,
 uint32_t *next_hop)
 {
@@ -1844,7 +1844,7 @@ delete_depth_big_v1604(struct rte_lpm *lpm, uint32_t ip_masked,
 /*
  * Deletes a rule
  */
-int
+int __vsym
 rte_lpm_delete_v20(struct rte_lpm_v20 *lpm, uint32_t ip, uint8_t depth)
 {
 	int32_t rule_to_delete_index, sub_rule_index;
@@ -1898,7 +1898,7 @@ rte_lpm_delete_v20(struct rte_lpm_v20 *lpm, uint32_t ip, uint8_t depth)
 }
 VERSION_SYMBOL(rte_lpm_delete, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm_delete_v1604(struct rte_lpm *lpm, uint32_t ip, uint8_t depth)
 {
 	int32_t rule_to_delete_index, sub_rule_index;
@@ -1957,7 +1957,7 @@ MAP_STATIC_SYMBOL(int rte_lpm_delete(struct rte_lpm *lpm, uint32_t ip,
 /*
  * Delete all rules from the LPM table.
  */
-void
+void __vsym
 rte_lpm_delete_all_v20(struct rte_lpm_v20 *lpm)
 {
 	/* Zero rule information. */
@@ -1974,7 +1974,7 @@ rte_lpm_delete_all_v20(struct rte_lpm_v20 *lpm)
 }
 VERSION_SYMBOL(rte_lpm_delete_all, _v20, 2.0);
 
-void
+void __vsym
 rte_lpm_delete_all_v1604(struct rte_lpm *lpm)
 {
 	/* Zero rule information. */
lib/librte_lpm/rte_lpm6.c +8 -8
index e20f82460..0d161dc32 100644
@@ -812,7 +812,7 @@ add_step(struct rte_lpm6 *lpm, struct rte_lpm6_tbl_entry *tbl,
 /*
  * Add a route
  */
-int
+int __vsym
 rte_lpm6_add_v20(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
 		uint8_t next_hop)
 {
@@ -862,7 +862,7 @@ simulate_add(struct rte_lpm6 *lpm, const uint8_t *masked_ip, uint8_t depth)
 	return 0;
 }
 
-int
+int __vsym
 rte_lpm6_add_v1705(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
 		uint32_t next_hop)
 {
@@ -955,7 +955,7 @@ lookup_step(const struct rte_lpm6 *lpm, const struct rte_lpm6_tbl_entry *tbl,
 /*
  * Looks up an IP
  */
-int
+int __vsym
 rte_lpm6_lookup_v20(const struct rte_lpm6 *lpm, uint8_t *ip, uint8_t *next_hop)
 {
 	uint32_t next_hop32 = 0;
@@ -973,7 +973,7 @@ rte_lpm6_lookup_v20(const struct rte_lpm6 *lpm, uint8_t *ip, uint8_t *next_hop)
 }
 VERSION_SYMBOL(rte_lpm6_lookup, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm6_lookup_v1705(const struct rte_lpm6 *lpm, uint8_t *ip,
 		uint32_t *next_hop)
 {
@@ -1008,7 +1008,7 @@ MAP_STATIC_SYMBOL(int rte_lpm6_lookup(const struct rte_lpm6 *lpm, uint8_t *ip,
 /*
  * Looks up a group of IP addresses
  */
-int
+int __vsym
 rte_lpm6_lookup_bulk_func_v20(const struct rte_lpm6 *lpm,
 		uint8_t ips[][RTE_LPM6_IPV6_ADDR_SIZE],
 		int16_t * next_hops, unsigned n)
@@ -1049,7 +1049,7 @@ rte_lpm6_lookup_bulk_func_v20(const struct rte_lpm6 *lpm,
 }
 VERSION_SYMBOL(rte_lpm6_lookup_bulk_func, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm6_lookup_bulk_func_v1705(const struct rte_lpm6 *lpm,
 		uint8_t ips[][RTE_LPM6_IPV6_ADDR_SIZE],
 		int32_t *next_hops, unsigned int n)
@@ -1099,7 +1099,7 @@ MAP_STATIC_SYMBOL(int rte_lpm6_lookup_bulk_func(const struct rte_lpm6 *lpm,
 /*
  * Look for a rule in the high-level rules table
  */
-int
+int __vsym
 rte_lpm6_is_rule_present_v20(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
 		uint8_t *next_hop)
 {
@@ -1119,7 +1119,7 @@ rte_lpm6_is_rule_present_v20(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
 }
 VERSION_SYMBOL(rte_lpm6_is_rule_present, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm6_is_rule_present_v1705(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
 		uint32_t *next_hop)
 {
lib/librte_timer/rte_timer.c +10 -10
index 3834c9473..381a9f43f 100644
@@ -131,7 +131,7 @@ rte_timer_data_dealloc(uint32_t id)
 	return 0;
 }
 
-void
+void __vsym
 rte_timer_subsystem_init_v20(void)
 {
 	unsigned lcore_id;
@@ -153,7 +153,7 @@ VERSION_SYMBOL(rte_timer_subsystem_init, _v20, 2.0);
  * secondary processes should be empty, the zeroth entry can be shared by
  * multiple processes.
  */
-int
+int __vsym
 rte_timer_subsystem_init_v1905(void)
 {
 	const struct rte_memzone *mz;
@@ -551,7 +551,7 @@ __rte_timer_reset(struct rte_timer *tim, uint64_t expire,
 }
 
 /* Reset and start the timer associated with the timer handle tim */
-int
+int __vsym
 rte_timer_reset_v20(struct rte_timer *tim, uint64_t ticks,
 		    enum rte_timer_type type, unsigned int tim_lcore,
 		    rte_timer_cb_t fct, void *arg)
@@ -574,7 +574,7 @@ rte_timer_reset_v20(struct rte_timer *tim, uint64_t ticks,
 }
 VERSION_SYMBOL(rte_timer_reset, _v20, 2.0);
 
-int
+int __vsym
 rte_timer_reset_v1905(struct rte_timer *tim, uint64_t ticks,
 		      enum rte_timer_type type, unsigned int tim_lcore,
 		      rte_timer_cb_t fct, void *arg)
@@ -657,14 +657,14 @@ __rte_timer_stop(struct rte_timer *tim, int local_is_locked,
 }
 
 /* Stop the timer associated with the timer handle tim */
-int
+int __vsym
 rte_timer_stop_v20(struct rte_timer *tim)
 {
 	return __rte_timer_stop(tim, 0, &default_timer_data);
 }
 VERSION_SYMBOL(rte_timer_stop, _v20, 2.0);
 
-int
+int __vsym
 rte_timer_stop_v1905(struct rte_timer *tim)
 {
 	return rte_timer_alt_stop(default_data_id, tim);
@@ -817,14 +817,14 @@ __rte_timer_manage(struct rte_timer_data *timer_data)
 	priv_timer[lcore_id].running_tim = NULL;
 }
 
-void
+void __vsym
 rte_timer_manage_v20(void)
 {
 	__rte_timer_manage(&default_timer_data);
 }
 VERSION_SYMBOL(rte_timer_manage, _v20, 2.0);
 
-int
+int __vsym
 rte_timer_manage_v1905(void)
 {
 	struct rte_timer_data *timer_data;
@@ -1074,14 +1074,14 @@ __rte_timer_dump_stats(struct rte_timer_data *timer_data __rte_unused, FILE *f)
 #endif
 }
 
-void
+void __vsym
 rte_timer_dump_stats_v20(FILE *f)
 {
 	__rte_timer_dump_stats(&default_timer_data, f);
 }
 VERSION_SYMBOL(rte_timer_dump_stats, _v20, 2.0);
 
-int
+int __vsym
 rte_timer_dump_stats_v1905(FILE *f)
 {
 	return rte_timer_alt_dump_stats(default_data_id, f);
Signature · 1 line
2.17.1
Acked-by on v5 1/11
#63 lore raw

Re: [dpdk-dev] [PATCH v5 01/11] build: annotate versioned symbols with __vsym macro

Quoted 582 lines · Andrzej Ostruszka

Every implementation of a particular version of given symbol needs to be marked in its declaration as such (using __vsym macro). This patch fixes this and also clarifies the documentation about that.

---
 doc/guides/contributing/versioning.rst        | 18 ++++++++----
 lib/librte_distributor/rte_distributor.c      | 18 ++++++------
 lib/librte_distributor/rte_distributor_v20.c  | 18 ++++++------
 .../common/include/rte_function_versioning.h  | 11 ++++++--
 lib/librte_lpm/rte_lpm.c                      | 28 +++++++++----------
 lib/librte_lpm/rte_lpm6.c                     | 16 +++++------
 lib/librte_timer/rte_timer.c                  | 20 ++++++-------
 7 files changed, 71 insertions(+), 58 deletions(-)
doc/guides/contributing/versioning.rst +12 -6
index 64984c54e..fcd2d50f1 100644
@@ -215,16 +215,20 @@ library so that older binaries need not be immediately recompiled.
 The macros exported are:
 
 * ``VERSION_SYMBOL(b, e, n)``: Creates a symbol version table entry binding
-  versioned symbol ``b@DPDK_n`` to the internal function ``b_e``.
+  versioned symbol ``b@DPDK_n`` to the internal function ``be``.
 
 * ``BIND_DEFAULT_SYMBOL(b, e, n)``: Creates a symbol version entry instructing
   the linker to bind references to symbol ``b`` to the internal symbol
-  ``b_e``.
+  ``be``.
 
 * ``MAP_STATIC_SYMBOL(f, p)``: Declare the prototype ``f``, and map it to the
   fully qualified function ``p``, so that if a symbol becomes versioned, it
   can still be mapped back to the public symbol name.
 
+* ``__vsym``:  Annotation to be used in a declaration of the internal symbol
+  ``be`` to signal that it is being used as an implementation of a particular
+  version of symbol ``b``.
+
 Examples of ABI Macro use
 ^^^^^^^^^^^^^^^^^^^^^^^^^
 
@@ -345,8 +349,9 @@ with the public symbol name
 
 .. code-block:: c
 
-  struct rte_acl_ctx *
+ -struct rte_acl_ctx *
  -rte_acl_create(const struct rte_acl_param *param)
+ +struct rte_acl_ctx * __vsym
  +rte_acl_create_v20(const struct rte_acl_param *param)
  {
         size_t sz;
@@ -354,7 +359,8 @@ with the public symbol name
...
 
 Note that the base name of the symbol was kept intact, as this is conducive to
-the macros used for versioning symbols.  That is our next step, mapping this new
+the macros used for versioning symbols and we have annotated the function as an
+implementation of versioned symbol.  That is our next step, mapping this new
 symbol name to the initial symbol name at version node 2.0.  Immediately after
 the function, we add this line of code
@@ -374,7 +380,7 @@ name, with a different suffix, and  implement it appropriately
 
 .. code-block:: c
 
-   struct rte_acl_ctx *
+   struct rte_acl_ctx * __vsym
    rte_acl_create_v21(const struct rte_acl_param *param, int debug);
    {
         struct rte_acl_ctx *ctx = rte_acl_create_v20(param);
@@ -423,7 +429,7 @@ defined, we add this
 
 .. code-block:: c
 
-   struct rte_acl_ctx *
+   struct rte_acl_ctx * __vsym
    rte_acl_create_v21(const struct rte_acl_param *param, int debug)
    {
...
lib/librte_distributor/rte_distributor.c +9 -9
index 7df97df92..2cc32ddba 100644
@@ -32,7 +32,7 @@ EAL_REGISTER_TAILQ(rte_dist_burst_tailq)
 
 /**** Burst Packet APIs called by workers ****/
 
-void
+void __vsym
 rte_distributor_request_pkt_v1705(struct rte_distributor *d,
 		unsigned int worker_id, struct rte_mbuf **oldpkt,
 		unsigned int count)
@@ -89,7 +89,7 @@ MAP_STATIC_SYMBOL(void rte_distributor_request_pkt(struct rte_distributor *d,
 		unsigned int count),
 		rte_distributor_request_pkt_v1705);
 
-int
+int __vsym
 rte_distributor_poll_pkt_v1705(struct rte_distributor *d,
 		unsigned int worker_id, struct rte_mbuf **pkts)
 {
@@ -134,7 +134,7 @@ MAP_STATIC_SYMBOL(int rte_distributor_poll_pkt(struct rte_distributor *d,
 		unsigned int worker_id, struct rte_mbuf **pkts),
 		rte_distributor_poll_pkt_v1705);
 
-int
+int __vsym
 rte_distributor_get_pkt_v1705(struct rte_distributor *d,
 		unsigned int worker_id, struct rte_mbuf **pkts,
 		struct rte_mbuf **oldpkt, unsigned int return_count)
@@ -169,7 +169,7 @@ MAP_STATIC_SYMBOL(int rte_distributor_get_pkt(struct rte_distributor *d,
 		struct rte_mbuf **oldpkt, unsigned int return_count),
 		rte_distributor_get_pkt_v1705);
 
-int
+int __vsym
 rte_distributor_return_pkt_v1705(struct rte_distributor *d,
 		unsigned int worker_id, struct rte_mbuf **oldpkt, int num)
 {
@@ -359,7 +359,7 @@ release(struct rte_distributor *d, unsigned int wkr)
 
 
 /* process a set of packets to distribute them to workers */
-int
+int __vsym
 rte_distributor_process_v1705(struct rte_distributor *d,
 		struct rte_mbuf **mbufs, unsigned int num_mbufs)
 {
@@ -506,7 +506,7 @@ MAP_STATIC_SYMBOL(int rte_distributor_process(struct rte_distributor *d,
 		rte_distributor_process_v1705);
 
 /* return to the caller, packets returned from workers */
-int
+int __vsym
 rte_distributor_returned_pkts_v1705(struct rte_distributor *d,
 		struct rte_mbuf **mbufs, unsigned int max_mbufs)
 {
@@ -556,7 +556,7 @@ total_outstanding(const struct rte_distributor *d)
  * Flush the distributor, so that there are no outstanding packets in flight or
  * queued up.
  */
-int
+int __vsym
 rte_distributor_flush_v1705(struct rte_distributor *d)
 {
 	unsigned int flushed;
@@ -591,7 +591,7 @@ MAP_STATIC_SYMBOL(int rte_distributor_flush(struct rte_distributor *d),
 		rte_distributor_flush_v1705);
 
 /* clears the internal returns array in the distributor */
-void
+void __vsym
 rte_distributor_clear_returns_v1705(struct rte_distributor *d)
 {
 	unsigned int wkr;
@@ -613,7 +613,7 @@ MAP_STATIC_SYMBOL(void rte_distributor_clear_returns(struct rte_distributor *d),
 		rte_distributor_clear_returns_v1705);
 
 /* creates a distributor instance */
-struct rte_distributor *
+struct rte_distributor * __vsym
 rte_distributor_create_v1705(const char *name,
 		unsigned int socket_id,
 		unsigned int num_workers,
lib/librte_distributor/rte_distributor_v20.c +9 -9
index db6c49258..7a6fddf55 100644
@@ -27,7 +27,7 @@ EAL_REGISTER_TAILQ(rte_distributor_tailq)
 
 /**** APIs called by workers ****/
 
-void
+void __vsym
 rte_distributor_request_pkt_v20(struct rte_distributor_v20 *d,
 		unsigned worker_id, struct rte_mbuf *oldpkt)
 {
@@ -43,7 +43,7 @@ rte_distributor_request_pkt_v20(struct rte_distributor_v20 *d,
 }
 VERSION_SYMBOL(rte_distributor_request_pkt, _v20, 2.0);
 
-struct rte_mbuf *
+struct rte_mbuf * __vsym
 rte_distributor_poll_pkt_v20(struct rte_distributor_v20 *d,
 		unsigned worker_id)
 {
@@ -59,7 +59,7 @@ rte_distributor_poll_pkt_v20(struct rte_distributor_v20 *d,
 }
 VERSION_SYMBOL(rte_distributor_poll_pkt, _v20, 2.0);
 
-struct rte_mbuf *
+struct rte_mbuf * __vsym
 rte_distributor_get_pkt_v20(struct rte_distributor_v20 *d,
 		unsigned worker_id, struct rte_mbuf *oldpkt)
 {
@@ -71,7 +71,7 @@ rte_distributor_get_pkt_v20(struct rte_distributor_v20 *d,
 }
 VERSION_SYMBOL(rte_distributor_get_pkt, _v20, 2.0);
 
-int
+int __vsym
 rte_distributor_return_pkt_v20(struct rte_distributor_v20 *d,
 		unsigned worker_id, struct rte_mbuf *oldpkt)
 {
@@ -204,7 +204,7 @@ process_returns(struct rte_distributor_v20 *d)
 }
 
 /* process a set of packets to distribute them to workers */
-int
+int __vsym
 rte_distributor_process_v20(struct rte_distributor_v20 *d,
 		struct rte_mbuf **mbufs, unsigned num_mbufs)
 {
@@ -321,7 +321,7 @@ rte_distributor_process_v20(struct rte_distributor_v20 *d,
 VERSION_SYMBOL(rte_distributor_process, _v20, 2.0);
 
 /* return to the caller, packets returned from workers */
-int
+int __vsym
 rte_distributor_returned_pkts_v20(struct rte_distributor_v20 *d,
 		struct rte_mbuf **mbufs, unsigned max_mbufs)
 {
@@ -359,7 +359,7 @@ total_outstanding(const struct rte_distributor_v20 *d)
 
 /* flush the distributor, so that there are no outstanding packets in flight or
  * queued up. */
-int
+int __vsym
 rte_distributor_flush_v20(struct rte_distributor_v20 *d)
 {
 	const unsigned flushed = total_outstanding(d);
@@ -372,7 +372,7 @@ rte_distributor_flush_v20(struct rte_distributor_v20 *d)
 VERSION_SYMBOL(rte_distributor_flush, _v20, 2.0);
 
 /* clears the internal returns array in the distributor */
-void
+void __vsym
 rte_distributor_clear_returns_v20(struct rte_distributor_v20 *d)
 {
 	d->returns.start = d->returns.count = 0;
@@ -383,7 +383,7 @@ rte_distributor_clear_returns_v20(struct rte_distributor_v20 *d)
 VERSION_SYMBOL(rte_distributor_clear_returns, _v20, 2.0);
 
 /* creates a distributor instance */
-struct rte_distributor_v20 *
+struct rte_distributor_v20 * __vsym
 rte_distributor_create_v20(const char *name,
 		unsigned socket_id,
 		unsigned num_workers)
lib/librte_eal/common/include/rte_function_versioning.h +9 -2
index 55e88ffae..c924351d5 100644
@@ -42,16 +42,23 @@
 /*
  * VERSION_SYMBOL
  * Creates a symbol version table entry binding symbol <b>@DPDK_<n> to the internal
- * function name <b>_<e>
+ * function name <b><e>
  */
 #define VERSION_SYMBOL(b, e, n) __asm__(".symver " RTE_STR(b) RTE_STR(e) ", " RTE_STR(b) "@DPDK_" RTE_STR(n))
 
 /*
  * BIND_DEFAULT_SYMBOL
  * Creates a symbol version entry instructing the linker to bind references to
- * symbol <b> to the internal symbol <b>_<e>
+ * symbol <b> to the internal symbol <b><e>
  */
 #define BIND_DEFAULT_SYMBOL(b, e, n) __asm__(".symver " RTE_STR(b) RTE_STR(e) ", " RTE_STR(b) "@@DPDK_" RTE_STR(n))
+
+/*
+ * __vsym
+ * Annotation to be used in declaration of the internal symbol <b><e> to signal
+ * that it is being used as an implementation of a particular version of symbol
+ * <b>.
+ */
 #define __vsym __attribute__((used))
 
 /*
lib/librte_lpm/rte_lpm.c +14 -14
index c96395e26..106916dc8 100644
@@ -90,7 +90,7 @@ depth_to_range(uint8_t depth)
 /*
  * Find an existing lpm table and return a pointer to it.
  */
-struct rte_lpm_v20 *
+struct rte_lpm_v20 * __vsym
 rte_lpm_find_existing_v20(const char *name)
 {
 	struct rte_lpm_v20 *l = NULL;
@@ -116,7 +116,7 @@ rte_lpm_find_existing_v20(const char *name)
 }
 VERSION_SYMBOL(rte_lpm_find_existing, _v20, 2.0);
 
-struct rte_lpm *
+struct rte_lpm * __vsym
 rte_lpm_find_existing_v1604(const char *name)
 {
 	struct rte_lpm *l = NULL;
@@ -147,7 +147,7 @@ MAP_STATIC_SYMBOL(struct rte_lpm *rte_lpm_find_existing(const char *name),
 /*
  * Allocates memory for LPM object
  */
-struct rte_lpm_v20 *
+struct rte_lpm_v20 * __vsym
 rte_lpm_create_v20(const char *name, int socket_id, int max_rules,
 		__rte_unused int flags)
 {
@@ -220,7 +220,7 @@ rte_lpm_create_v20(const char *name, int socket_id, int max_rules,
 }
 VERSION_SYMBOL(rte_lpm_create, _v20, 2.0);
 
-struct rte_lpm *
+struct rte_lpm * __vsym
 rte_lpm_create_v1604(const char *name, int socket_id,
 		const struct rte_lpm_config *config)
 {
@@ -329,7 +329,7 @@ MAP_STATIC_SYMBOL(
 /*
  * Deallocates memory for given LPM table.
  */
-void
+void __vsym
 rte_lpm_free_v20(struct rte_lpm_v20 *lpm)
 {
 	struct rte_lpm_list *lpm_list;
@@ -358,7 +358,7 @@ rte_lpm_free_v20(struct rte_lpm_v20 *lpm)
 }
 VERSION_SYMBOL(rte_lpm_free, _v20, 2.0);
 
-void
+void __vsym
 rte_lpm_free_v1604(struct rte_lpm *lpm)
 {
 	struct rte_lpm_list *lpm_list;
@@ -1177,7 +1177,7 @@ add_depth_big_v1604(struct rte_lpm *lpm, uint32_t ip_masked, uint8_t depth,
 /*
  * Add a route
  */
-int
+int __vsym
 rte_lpm_add_v20(struct rte_lpm_v20 *lpm, uint32_t ip, uint8_t depth,
 		uint8_t next_hop)
 {
@@ -1218,7 +1218,7 @@ rte_lpm_add_v20(struct rte_lpm_v20 *lpm, uint32_t ip, uint8_t depth,
 }
 VERSION_SYMBOL(rte_lpm_add, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm_add_v1604(struct rte_lpm *lpm, uint32_t ip, uint8_t depth,
 		uint32_t next_hop)
 {
@@ -1264,7 +1264,7 @@ MAP_STATIC_SYMBOL(int rte_lpm_add(struct rte_lpm *lpm, uint32_t ip,
 /*
  * Look for a rule in the high-level rules table
  */
-int
+int __vsym
 rte_lpm_is_rule_present_v20(struct rte_lpm_v20 *lpm, uint32_t ip, uint8_t depth,
 uint8_t *next_hop)
 {
@@ -1291,7 +1291,7 @@ uint8_t *next_hop)
 }
 VERSION_SYMBOL(rte_lpm_is_rule_present, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm_is_rule_present_v1604(struct rte_lpm *lpm, uint32_t ip, uint8_t depth,
 uint32_t *next_hop)
 {
@@ -1844,7 +1844,7 @@ delete_depth_big_v1604(struct rte_lpm *lpm, uint32_t ip_masked,
 /*
  * Deletes a rule
  */
-int
+int __vsym
 rte_lpm_delete_v20(struct rte_lpm_v20 *lpm, uint32_t ip, uint8_t depth)
 {
 	int32_t rule_to_delete_index, sub_rule_index;
@@ -1898,7 +1898,7 @@ rte_lpm_delete_v20(struct rte_lpm_v20 *lpm, uint32_t ip, uint8_t depth)
 }
 VERSION_SYMBOL(rte_lpm_delete, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm_delete_v1604(struct rte_lpm *lpm, uint32_t ip, uint8_t depth)
 {
 	int32_t rule_to_delete_index, sub_rule_index;
@@ -1957,7 +1957,7 @@ MAP_STATIC_SYMBOL(int rte_lpm_delete(struct rte_lpm *lpm, uint32_t ip,
 /*
  * Delete all rules from the LPM table.
  */
-void
+void __vsym
 rte_lpm_delete_all_v20(struct rte_lpm_v20 *lpm)
 {
 	/* Zero rule information. */
@@ -1974,7 +1974,7 @@ rte_lpm_delete_all_v20(struct rte_lpm_v20 *lpm)
 }
 VERSION_SYMBOL(rte_lpm_delete_all, _v20, 2.0);
 
-void
+void __vsym
 rte_lpm_delete_all_v1604(struct rte_lpm *lpm)
 {
 	/* Zero rule information. */
lib/librte_lpm/rte_lpm6.c +8 -8
index e20f82460..0d161dc32 100644
@@ -812,7 +812,7 @@ add_step(struct rte_lpm6 *lpm, struct rte_lpm6_tbl_entry *tbl,
 /*
  * Add a route
  */
-int
+int __vsym
 rte_lpm6_add_v20(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
 		uint8_t next_hop)
 {
@@ -862,7 +862,7 @@ simulate_add(struct rte_lpm6 *lpm, const uint8_t *masked_ip, uint8_t depth)
 	return 0;
 }
 
-int
+int __vsym
 rte_lpm6_add_v1705(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
 		uint32_t next_hop)
 {
@@ -955,7 +955,7 @@ lookup_step(const struct rte_lpm6 *lpm, const struct rte_lpm6_tbl_entry *tbl,
 /*
  * Looks up an IP
  */
-int
+int __vsym
 rte_lpm6_lookup_v20(const struct rte_lpm6 *lpm, uint8_t *ip, uint8_t *next_hop)
 {
 	uint32_t next_hop32 = 0;
@@ -973,7 +973,7 @@ rte_lpm6_lookup_v20(const struct rte_lpm6 *lpm, uint8_t *ip, uint8_t *next_hop)
 }
 VERSION_SYMBOL(rte_lpm6_lookup, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm6_lookup_v1705(const struct rte_lpm6 *lpm, uint8_t *ip,
 		uint32_t *next_hop)
 {
@@ -1008,7 +1008,7 @@ MAP_STATIC_SYMBOL(int rte_lpm6_lookup(const struct rte_lpm6 *lpm, uint8_t *ip,
 /*
  * Looks up a group of IP addresses
  */
-int
+int __vsym
 rte_lpm6_lookup_bulk_func_v20(const struct rte_lpm6 *lpm,
 		uint8_t ips[][RTE_LPM6_IPV6_ADDR_SIZE],
 		int16_t * next_hops, unsigned n)
@@ -1049,7 +1049,7 @@ rte_lpm6_lookup_bulk_func_v20(const struct rte_lpm6 *lpm,
 }
 VERSION_SYMBOL(rte_lpm6_lookup_bulk_func, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm6_lookup_bulk_func_v1705(const struct rte_lpm6 *lpm,
 		uint8_t ips[][RTE_LPM6_IPV6_ADDR_SIZE],
 		int32_t *next_hops, unsigned int n)
@@ -1099,7 +1099,7 @@ MAP_STATIC_SYMBOL(int rte_lpm6_lookup_bulk_func(const struct rte_lpm6 *lpm,
 /*
  * Look for a rule in the high-level rules table
  */
-int
+int __vsym
 rte_lpm6_is_rule_present_v20(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
 		uint8_t *next_hop)
 {
@@ -1119,7 +1119,7 @@ rte_lpm6_is_rule_present_v20(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
 }
 VERSION_SYMBOL(rte_lpm6_is_rule_present, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm6_is_rule_present_v1705(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
 		uint32_t *next_hop)
 {
lib/librte_timer/rte_timer.c +10 -10
index 3834c9473..381a9f43f 100644
@@ -131,7 +131,7 @@ rte_timer_data_dealloc(uint32_t id)
 	return 0;
 }
 
-void
+void __vsym
 rte_timer_subsystem_init_v20(void)
 {
 	unsigned lcore_id;
@@ -153,7 +153,7 @@ VERSION_SYMBOL(rte_timer_subsystem_init, _v20, 2.0);
  * secondary processes should be empty, the zeroth entry can be shared by
  * multiple processes.
  */
-int
+int __vsym
 rte_timer_subsystem_init_v1905(void)
 {
 	const struct rte_memzone *mz;
@@ -551,7 +551,7 @@ __rte_timer_reset(struct rte_timer *tim, uint64_t expire,
 }
 
 /* Reset and start the timer associated with the timer handle tim */
-int
+int __vsym
 rte_timer_reset_v20(struct rte_timer *tim, uint64_t ticks,
 		    enum rte_timer_type type, unsigned int tim_lcore,
 		    rte_timer_cb_t fct, void *arg)
@@ -574,7 +574,7 @@ rte_timer_reset_v20(struct rte_timer *tim, uint64_t ticks,
 }
 VERSION_SYMBOL(rte_timer_reset, _v20, 2.0);
 
-int
+int __vsym
 rte_timer_reset_v1905(struct rte_timer *tim, uint64_t ticks,
 		      enum rte_timer_type type, unsigned int tim_lcore,
 		      rte_timer_cb_t fct, void *arg)
@@ -657,14 +657,14 @@ __rte_timer_stop(struct rte_timer *tim, int local_is_locked,
 }
 
 /* Stop the timer associated with the timer handle tim */
-int
+int __vsym
 rte_timer_stop_v20(struct rte_timer *tim)
 {
 	return __rte_timer_stop(tim, 0, &default_timer_data);
 }
 VERSION_SYMBOL(rte_timer_stop, _v20, 2.0);
 
-int
+int __vsym
 rte_timer_stop_v1905(struct rte_timer *tim)
 {
 	return rte_timer_alt_stop(default_data_id, tim);
@@ -817,14 +817,14 @@ __rte_timer_manage(struct rte_timer_data *timer_data)
 	priv_timer[lcore_id].running_tim = NULL;
 }
 
-void
+void __vsym
 rte_timer_manage_v20(void)
 {
 	__rte_timer_manage(&default_timer_data);
 }
 VERSION_SYMBOL(rte_timer_manage, _v20, 2.0);
 
-int
+int __vsym
 rte_timer_manage_v1905(void)
 {
 	struct rte_timer_data *timer_data;
@@ -1074,14 +1074,14 @@ __rte_timer_dump_stats(struct rte_timer_data *timer_data __rte_unused, FILE *f)
 #endif
 }
 
-void
+void __vsym
 rte_timer_dump_stats_v20(FILE *f)
 {
 	__rte_timer_dump_stats(&default_timer_data, f);
 }
 VERSION_SYMBOL(rte_timer_dump_stats, _v20, 2.0);
 
-int
+int __vsym
 rte_timer_dump_stats_v1905(FILE *f)
 {
 	return rte_timer_alt_dump_stats(default_data_id, f);
Signature · 1 line
2.17.1
#64 lore raw

PATCH v5 2/11 build: add an option to enable LTO build

In reply to Andrzej Ostruszka

This patch adds an option to enable link time optimization. In addition to LTO option itself (-flto) fat-lto-objects are being used. This is because during the build pmdinfogen scans the generated ELF objects to find this_pmd_name* symbol in symbol table. Without fat-lto-objects gcc produces ELF only with extra symbols for internal use during linking.

---
 .travis.yml                              |  7 +++++
 config/common_base                       |  5 +++
 config/meson.build                       | 13 ++++++++
 doc/guides/prog_guide/index.rst          |  1 +
 doc/guides/prog_guide/lto.rst            | 40 ++++++++++++++++++++++++
 doc/guides/rel_notes/release_19_11.rst   |  9 ++++++
 mk/toolchain/gcc/rte.toolchain-compat.mk |  4 +++
 mk/toolchain/gcc/rte.vars.mk             | 12 +++++++
 mk/toolchain/icc/rte.vars.mk             |  8 +++++
 9 files changed, 99 insertions(+)
 create mode 100644 doc/guides/prog_guide/lto.rst
.travis.yml +7 -0
index 3d6ef2959..3cd746dba 100644
@@ -34,6 +34,7 @@ env:
   - DEF_LIB="static" OPTS="-Denable_kmods=false"
   - DEF_LIB="shared" OPTS="-Denable_kmods=false"
   - DEF_LIB="shared" RUN_TESTS=1
+  - DEF_LIB="shared" OPTS="-Db_lto=true"
 
 matrix:
   include:
@@ -105,6 +106,12 @@ matrix:
       apt:
         packages:
           - *extra_packages
+  - env: DEF_LIB="shared" OPTS="-Db_lto=true" EXTRA_PACKAGES=1
+    compiler: gcc
+    addons:
+      apt:
+        packages:
+          - *extra_packages
 
 
 script: ./.ci/${TRAVIS_OS_NAME}-build.sh
config/common_base +5 -0
index b2be3d96a..0d1207166 100644
@@ -49,6 +49,11 @@ CONFIG_RTE_FORCE_INTRINSICS=n
 #
 CONFIG_RTE_ARCH_STRICT_ALIGN=n
 
+#
+# Enable link time optimization
+#
+CONFIG_RTE_ENABLE_LTO=n
+
 #
 # Compile to share library
 #
config/meson.build +13 -0
index e1ebdad26..2b1cb92e7 100644
@@ -225,3 +225,16 @@ add_project_arguments('-D_GNU_SOURCE', language: 'c')
 if is_freebsd
 	add_project_arguments('-D__BSD_VISIBLE', language: 'c')
 endif
+
+if get_option('b_lto')
+	if cc.has_argument('-ffat-lto-objects')
+		add_project_arguments('-ffat-lto-objects', language: 'c')
+	else
+		error('compiler does not support fat LTO objects - please turn LTO off')
+	endif
+	# workaround for gcc bug 81440
+	if cc.get_id() == 'gcc' and cc.version().version_compare('<8.0')
+		add_project_arguments('-Wno-lto-type-mismatch', language: 'c')
+		add_project_link_arguments('-Wno-lto-type-mismatch', language: 'c')
+	endif
+endif
doc/guides/prog_guide/index.rst +1 -0
index 692409af8..dc4851c57 100644
@@ -65,5 +65,6 @@ Programmer's Guide
     ext_app_lib_make_help
     perf_opt_guidelines
     writing_efficient_code
+    lto
     profile_app
     glossary
doc/guides/prog_guide/lto.rst new file +40 -0
new file mode 100644
index 000000000..50aecc9e5
@@ -0,0 +1,40 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2019 Marvell International Ltd.
+
+Link Time Optimization
+======================
+
+The DPDK supports compilation with link time optimization turned on.
+This depends obviously on the ability of the compiler to do "whole
+program" optimization at link time and is available only for compilers
+that support that feature.
+To be more specific, compiler (in addition to performing LTO) have to
+support creation of ELF objects containing both normal code and internal
+representation (called fat-lto-objects in gcc and icc).
+This is required since during build some code is generated by parsing
+produced ELF objects (pmdinfogen).
+
+The amount of performance gain that one can get from LTO depends on the
+compiler and the code that is being compiled.
+However LTO is also useful for additional code analysis done by the
+compiler.
+In particular due to interprocedural analysis compiler can produce
+additional warnings about variables that might be used uninitialized.
+Some of these warnings might be "false positives" though and you might
+need to explicitly initialize variable in order to silence the compiler.
+
+Please note that turning LTO on causes considerable extension of
+build time.
+
+When using make based build, link time optimization can be enabled for
+the whole DPDK by setting:
+
+.. code-block:: console
+    CONFIG_ENABLE_LTO=y
+
+in config file.
+For the meson based build it can be enabled by setting meson built-in
+'b_lto' option:
+
+.. code-block:: console
+    meson build -Db_lto=true
doc/guides/rel_notes/release_19_11.rst +9 -0
index ae8e7b2f0..b11abe2fe 100644
@@ -231,6 +231,15 @@ New Features
   * Added a console command to testpmd app, ``show port (port_id) ptypes`` which
     gives ability to print port supported ptypes in different protocol layers.
 
+* **Added build support for Link Time Optimization.**
+
+  LTO is an optimization technique used by the compiler to perform whole
+  program analysis and optimization at link time.  In order to do that
+  compilers store their internal representation of the source code that
+  the linker uses at the final stage of compilation process.
+
+  See :doc:`../prog_guide/lto` for more information:
+
 
 Removed Items
 -------------
mk/toolchain/gcc/rte.toolchain-compat.mk +4 -0
index ea40a11c0..ad4fad83c 100644
@@ -88,6 +88,10 @@ else
 		MACHINE_CFLAGS := $(filter-out -march% -mtune% -msse%,$(MACHINE_CFLAGS))
 	endif
 
+	ifeq ($(shell test $(GCC_VERSION) -lt 45 && echo 1), 1)
+		CONFIG_RTE_ENABLE_LTO=n
+	endif
+
 	# Disable thunderx PMD for gcc < 4.7
 	ifeq ($(shell test $(GCC_VERSION) -lt 47 && echo 1), 1)
 		CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD=d
mk/toolchain/gcc/rte.vars.mk +12 -0
index b852fcfd7..9fc704193 100644
@@ -62,6 +62,18 @@ endif
 # process cpu flags
 include $(RTE_SDK)/mk/toolchain/$(RTE_TOOLCHAIN)/rte.toolchain-compat.mk
 
+ifeq ($(CONFIG_RTE_ENABLE_LTO),y)
+# 'fat-lto' is used since pmdinfogen needs to have 'this_pmd_nameX'
+# exported in symbol table and without this option only internal
+# representation is present.
+TOOLCHAIN_CFLAGS += -flto -ffat-lto-objects
+TOOLCHAIN_LDFLAGS += -flto
+# workaround for GCC bug 81440
+ifeq ($(shell test $(GCC_VERSION) -lt 80 && echo 1), 1)
+WERROR_FLAGS += -Wno-lto-type-mismatch
+endif
+endif
+
 # workaround GCC bug with warning "missing initializer" for "= {0}"
 ifeq ($(shell test $(GCC_VERSION) -lt 47 && echo 1), 1)
 WERROR_FLAGS += -Wno-missing-field-initializers
mk/toolchain/icc/rte.vars.mk +8 -0
index aa1422bf1..8aa87aa1e 100644
@@ -54,5 +54,13 @@ endif
 # process cpu flags
 include $(RTE_SDK)/mk/toolchain/$(RTE_TOOLCHAIN)/rte.toolchain-compat.mk
 
+ifeq ($(CONFIG_RTE_ENABLE_LTO),y)
+# 'fat-lto' is used since pmdinfogen needs to have 'this_pmd_nameX'
+# exported in symbol table and without this option only internal
+# representation is present.
+TOOLCHAIN_CFLAGS += -flto -ffat-lto-objects
+TOOLCHAIN_LDFLAGS += -flto
+endif
+
 export CC AS AR LD OBJCOPY OBJDUMP STRIP READELF
 export TOOLCHAIN_CFLAGS TOOLCHAIN_LDFLAGS TOOLCHAIN_ASFLAGS
Signature · 1 line
2.17.1
#65 lore raw

PATCH v5 3/11 eventdev: fix possible use of uninitialized var

In reply to Andrzej Ostruszka

Fix the logic for the case of event queue allowing all schedule types.

Compiler warning pointing to this error (with LTO enabled):
error: ‘sched_type’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
if ((ret < 0 && ret != -EOVERFLOW) ||

Fixes 6750b21bd6af ("eventdev: add default software timer adapter")
---
 lib/librte_eventdev/rte_event_timer_adapter.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
lib/librte_eventdev/rte_event_timer_adapter.c +4 -4
index 5ce399eca..161e21a68 100644
@@ -706,11 +706,11 @@ check_destination_event_queue(struct rte_event_timer *evtim,
 				       RTE_EVENT_QUEUE_ATTR_SCHEDULE_TYPE,
 				       &sched_type);
 
-	if ((ret < 0 && ret != -EOVERFLOW) ||
-	    evtim->ev.sched_type != sched_type)
-		return -1;
+	if ((ret == 0 && evtim->ev.sched_type == sched_type) ||
+	    ret == -EOVERFLOW)
+		return 0;
 
-	return 0;
+	return -1;
 }
 
 static int
Signature · 1 line
2.17.1
#66 lore raw

PATCH v5 4/11 app/eventdev: clean LTO build warnings (maybe-uninitialized)

In reply to Andrzej Ostruszka

During LTO build compiler reports some 'false positive' warnings about variables being possibly used uninitialized. This patch silences these warnings.

Exemplary compiler warning to suppress (with LTO enabled):
error: ‘service_id’ may be used uninitialized in this function [-Werror=maybe-uninitialized]

    ret = evt_service_setup(service_id);
---
 app/test-eventdev/test_perf_common.c     | 2 +-
 app/test-eventdev/test_pipeline_common.c | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)
app/test-eventdev/test_perf_common.c +1 -1
index e24519179..e7cf75a7d 100644
@@ -439,7 +439,7 @@ perf_event_timer_adapter_setup(struct test_perf *t)
 
 		if (!(adapter_info.caps &
 				RTE_EVENT_TIMER_ADAPTER_CAP_INTERNAL_PORT)) {
-			uint32_t service_id;
+			uint32_t service_id = -1U;
 
 			rte_event_timer_adapter_service_id_get(wl,
 					&service_id);
app/test-eventdev/test_pipeline_common.c +2 -2
index 386aca299..160461fb2 100644
@@ -326,7 +326,7 @@ pipeline_event_rx_adapter_setup(struct evt_options *opt, uint8_t stride,
 		}
 
 		if (!(cap & RTE_EVENT_ETH_RX_ADAPTER_CAP_INTERNAL_PORT)) {
-			uint32_t service_id;
+			uint32_t service_id = -1U;
 
 			rte_event_eth_rx_adapter_service_id_get(prod,
 					&service_id);
@@ -378,7 +378,7 @@ pipeline_event_tx_adapter_setup(struct evt_options *opt,
 		}
 
 		if (!(cap & RTE_EVENT_ETH_TX_ADAPTER_CAP_INTERNAL_PORT)) {
-			uint32_t service_id;
+			uint32_t service_id = -1U;
 
 			rte_event_eth_tx_adapter_service_id_get(consm,
 					&service_id);
Signature · 1 line
2.17.1
#67 lore raw

PATCH v5 5/11 event/octeontx2: clean LTO build warnings (maybe-uninitialized)

In reply to Andrzej Ostruszka

During LTO build compiler reports some 'false positive' warnings about variables being possibly used uninitialized. This patch silences these warnings.

Exemplary compiler warning to suppress (with LTO enabled):
error: ‘chunk’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
bkt->current_chunk = (uintptr_t)chunk;

---
 drivers/event/octeontx2/otx2_tim_worker.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/event/octeontx2/otx2_tim_worker.h +1 -1
index b193e2cab..50db6543c 100644
@@ -337,7 +337,7 @@ tim_add_entry_brst(struct otx2_tim_ring * const tim_ring,
 		   const struct otx2_tim_ent *ents,
 		   const uint16_t nb_timers, const uint8_t flags)
 {
-	struct otx2_tim_ent *chunk;
+	struct otx2_tim_ent *chunk = NULL;
 	struct otx2_tim_bkt *bkt;
 	uint16_t chunk_remainder;
 	uint16_t index = 0;
Signature · 1 line
2.17.1
#68 lore raw

PATCH v5 6/11 app/test: clean LTO build warnings (maybe-uninitialized)

In reply to Andrzej Ostruszka

During LTO build compiler reports some 'false positive' warnings about variables being possibly used uninitialized. This patch silences these warnings.

Exemplary compiler warning to suppress (with LTO enabled):
error: ‘stats.greatest_free_size’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
return len - overhead;

---
 app/test/test_hash_readwrite.c     | 2 +-
 app/test/test_link_bonding_mode4.c | 8 +++++++-
 app/test/test_memzone.c            | 3 ++-
 3 files changed, 10 insertions(+), 3 deletions(-)
app/test/test_hash_readwrite.c +1 -1
index 4376b099b..615767fb6 100644
@@ -298,7 +298,7 @@ test_rw_reader(void *arg)
 
 	begin = rte_rdtsc_precise();
 	for (i = 0; i < read_cnt; i++) {
-		void *data;
+		void *data = arg;
 		rte_hash_lookup_data(tbl_rw_test_param.h,
 				tbl_rw_test_param.keys + i,
 				&data);
app/test/test_link_bonding_mode4.c +7 -1
index ff54d7b91..cf12f026d 100644
@@ -585,7 +585,13 @@ bond_get_update_timeout_ms(void)
 {
 	struct rte_eth_bond_8023ad_conf conf;
 
-	rte_eth_bond_8023ad_conf_get(test_params.bonded_port_id, &conf);
+	if (rte_eth_bond_8023ad_conf_get(test_params.bonded_port_id, &conf) < 0) {
+		RTE_LOG(DEBUG, EAL, "Failed to get bonding configuration: "
+				    "%s at %d\n", __func__, __LINE__);
+		RTE_TEST_TRACE_FAILURE(__FILE__, __LINE__, __func__);
+		return 0;
+	}
+
 	return conf.update_timeout_ms;
 }
 
app/test/test_memzone.c +2 -1
index 4d8744455..0343b0326 100644
@@ -475,7 +475,8 @@ find_max_block_free_size(unsigned int align, unsigned int socket_id)
 	struct rte_malloc_socket_stats stats;
 	size_t len, overhead;
 
-	rte_malloc_get_socket_stats(socket_id, &stats);
+	if (rte_malloc_get_socket_stats(socket_id, &stats) < 0)
+		return 0;
 
 	len = stats.greatest_free_size;
 	overhead = MALLOC_ELEM_OVERHEAD;
Signature · 1 line
2.17.1
#69 lore raw

PATCH v5 7/11 net/dpaa2: fix possible use of uninitialized vars

In reply to Andrzej Ostruszka

This patch fixes 'maybe-uninitialized' warnings reported by compiler when using LTO.

Compiler warning pointing to this error (with LTO enabled):
error: ‘kg_cfg.extracts[0].masks[0].mask’ may be used uninitialized in this function [-Werror=maybe-uninitialized]

    extr->masks[j].mask = cfg->extracts[i].masks[j].mask;
Fixes 16bbc98a3e63 ("bus/fslmc: update MC to 10.3.x")
---
 drivers/net/dpaa2/base/dpaa2_hw_dpni.c | 1 +
 drivers/net/dpaa2/mc/dpkg.c            | 2 +-
 drivers/net/dpaa2/mc/dpni.c            | 9 ++++++---
 3 files changed, 8 insertions(+), 4 deletions(-)
drivers/net/dpaa2/base/dpaa2_hw_dpni.c +1 -0
index 16555d7b0..47a8bda6a 100644
@@ -51,6 +51,7 @@ rte_pmd_dpaa2_set_custom_hash(uint16_t port_id,
 	kg_cfg.extracts[0].type = DPKG_EXTRACT_FROM_DATA;
 	kg_cfg.extracts[0].extract.from_data.offset = offset;
 	kg_cfg.extracts[0].extract.from_data.size = size;
+	kg_cfg.extracts[0].num_of_byte_masks = 0;
 	kg_cfg.num_extracts = 1;
 
 	ret = dpkg_prepare_key_cfg(&kg_cfg, p_params);
drivers/net/dpaa2/mc/dpkg.c +1 -1
index 80f94f40e..7aa63ea12 100644
@@ -63,7 +63,7 @@ dpkg_prepare_key_cfg(const struct dpkg_profile_cfg *cfg, uint8_t *key_cfg_buf)
 		dpkg_set_field(extr->extract_type, EXTRACT_TYPE,
 			       cfg->extracts[i].type);
 
-		for (j = 0; j < DPKG_NUM_OF_MASKS; j++) {
+		for (j = 0; j < extr->num_of_byte_masks; j++) {
 			extr->masks[j].mask = cfg->extracts[i].masks[j].mask;
 			extr->masks[j].offset =
 				cfg->extracts[i].masks[j].offset;
drivers/net/dpaa2/mc/dpni.c +6 -3
index 0950ee007..89a64b0ab 100644
@@ -1839,10 +1839,13 @@ int dpni_set_congestion_notification(struct fsl_mc_io *mc_io,
 	cmd_params->qtype = qtype;
 	cmd_params->tc = tc_id;
 	cmd_params->congestion_point = cfg->cg_point;
-	cmd_params->cgid = (uint8_t)cfg->cgid;
-	cmd_params->dest_id = cpu_to_le32(cfg->dest_cfg.dest_id);
+	if (cfg->cg_point == DPNI_CP_CONGESTION_GROUP)
+		cmd_params->cgid = (uint8_t)cfg->cgid;
+	if (cfg->dest_cfg.dest_type != DPNI_DEST_NONE) {
+		cmd_params->dest_id = cpu_to_le32(cfg->dest_cfg.dest_id);
+		cmd_params->dest_priority = cfg->dest_cfg.priority;
+	}
 	cmd_params->notification_mode = cpu_to_le16(cfg->notification_mode);
-	cmd_params->dest_priority = cfg->dest_cfg.priority;
 	cmd_params->message_iova = cpu_to_le64(cfg->message_iova);
 	cmd_params->message_ctx = cpu_to_le64(cfg->message_ctx);
 	cmd_params->threshold_entry = cpu_to_le32(cfg->threshold_entry);
Signature · 1 line
2.17.1
#70 lore raw

PATCH v5 8/11 net/e1000: clean LTO build warnings (maybe-uninitialized)

In reply to Andrzej Ostruszka

During LTO build compiler reports some 'false positive' warnings about variables being possibly used uninitialized. This patch silences these warnings.

Exemplary compiler warning to suppress (with LTO enabled):
error: ‘link’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
if (link) {

---
 drivers/net/e1000/base/e1000_82543.c   | 2 +-
 drivers/net/e1000/base/e1000_ich8lan.c | 2 +-
 drivers/net/e1000/base/e1000_phy.c     | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)
drivers/net/e1000/base/e1000_82543.c +1 -1
index 810899eb6..dfde2a8b9 100644
@@ -1027,7 +1027,7 @@ STATIC s32 e1000_setup_copper_link_82543(struct e1000_hw *hw)
 {
 	u32 ctrl;
 	s32 ret_val;
-	bool link;
+	bool link = true;
 
 	DEBUGFUNC("e1000_setup_copper_link_82543");
 
drivers/net/e1000/base/e1000_ich8lan.c +1 -1
index accc6ea01..5241cf698 100644
@@ -5533,7 +5533,7 @@ void e1000_igp3_phy_powerdown_workaround_ich8lan(struct e1000_hw *hw)
 void e1000_gig_downshift_workaround_ich8lan(struct e1000_hw *hw)
 {
 	s32 ret_val;
-	u16 reg_data;
+	u16 reg_data = 0;
 
 	DEBUGFUNC("e1000_gig_downshift_workaround_ich8lan");
 
drivers/net/e1000/base/e1000_phy.c +1 -1
index 7d08f836f..956c06747 100644
@@ -1664,7 +1664,7 @@ s32 e1000_copper_link_autoneg(struct e1000_hw *hw)
 s32 e1000_setup_copper_link_generic(struct e1000_hw *hw)
 {
 	s32 ret_val;
-	bool link;
+	bool link = true;
 
 	DEBUGFUNC("e1000_setup_copper_link_generic");
 
Signature · 1 line
2.17.1
#71 lore raw

PATCH v5 9/11 net/i40e: clean LTO build warnings (maybe-uninitialized)

In reply to Andrzej Ostruszka

During LTO build compiler reports some 'false positive' warnings about variables being possibly used uninitialized. This patch silences these warnings.

Exemplary compiler warning to suppress (with LTO enabled):
error: ‘filter_idx’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
PMD_DRV_LOG(INFO, "Added port %d with AQ command with index %d",

---
 drivers/net/i40e/i40e_ethdev.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/i40e/i40e_ethdev.c +1 -1
index 77a46832c..91d529b2f 100644
@@ -8480,7 +8480,7 @@ static int
 i40e_add_vxlan_port(struct i40e_pf *pf, uint16_t port, int udp_type)
 {
 	int  idx, ret;
-	uint8_t filter_idx;
+	uint8_t filter_idx = 0;
 	struct i40e_hw *hw = I40E_PF_TO_HW(pf);
 
 	idx = i40e_get_vxlan_port_idx(pf, port);
Signature · 1 line
2.17.1
#72 lore raw

PATCH v5 10/11 net/ifc: clean LTO build warnings (maybe-uninitialized)

In reply to Andrzej Ostruszka

During LTO build compiler reports some 'false positive' warnings about variables being possibly used uninitialized. This patch silences these warnings.

Exemplary compiler warning to suppress (with LTO enabled):
error: ‘features’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
if (RTE_VHOST_NEED_LOG(features)) {

---
 drivers/net/ifc/ifcvf_vdpa.c | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)
drivers/net/ifc/ifcvf_vdpa.c +9 -5
index 8de9ef199..4cdff8712 100644
@@ -304,8 +304,8 @@ vdpa_ifcvf_stop(struct ifcvf_internal *internal)
 	struct ifcvf_hw *hw = &internal->hw;
 	uint32_t i;
 	int vid;
-	uint64_t features;
-	uint64_t log_base, log_size;
+	uint64_t features = 0;
+	uint64_t log_base = 0, log_size = 0;
 	uint64_t len;
 
 	vid = internal->vid;
@@ -348,6 +348,8 @@ vdpa_enable_vfio_intr(struct ifcvf_internal *internal, bool m_rx)
 	struct rte_vhost_vring vring;
 	int fd;
 
+	vring.callfd = -1;
+
 	nr_vring = rte_vhost_get_vring_num(internal->vid);
 
 	irq_set = (struct vfio_irq_set *)irq_set_buf;
@@ -442,6 +444,7 @@ notify_relay(void *arg)
 	}
 	internal->epfd = epfd;
 
+	vring.kickfd = -1;
 	for (qid = 0; qid < q_num; qid++) {
 		ev.events = EPOLLIN | EPOLLPRI;
 		rte_vhost_get_vhost_vring(internal->vid, qid, &vring);
@@ -577,7 +580,7 @@ m_ifcvf_start(struct ifcvf_internal *internal)
 	struct ifcvf_hw *hw = &internal->hw;
 	uint32_t i, nr_vring;
 	int vid, ret;
-	struct rte_vhost_vring vq;
+	struct rte_vhost_vring vq = { 0 };
 	void *vring_buf;
 	uint64_t m_vring_iova = IFCVF_MEDIATED_VRING;
 	uint64_t size;
@@ -721,6 +724,7 @@ vring_relay(void *arg)
 	}
 	internal->epfd = epfd;
 
+	vring.kickfd = -1;
 	for (qid = 0; qid < q_num; qid++) {
 		ev.events = EPOLLIN | EPOLLPRI;
 		rte_vhost_get_vhost_vring(vid, qid, &vring);
@@ -930,11 +934,11 @@ ifcvf_dev_close(int vid)
 static int
 ifcvf_set_features(int vid)
 {
-	uint64_t features;
+	uint64_t features = 0;
 	int did;
 	struct internal_list *list;
 	struct ifcvf_internal *internal;
-	uint64_t log_base, log_size;
+	uint64_t log_base = 0, log_size = 0;
 
 	did = rte_vhost_get_vdpa_device_id(vid);
 	list = find_internal_resource_by_did(did);
Signature · 1 line
2.17.1
#73 lore raw

PATCH v5 11/11 net/qede: clean LTO build warnings (maybe-uninitialized)

In reply to Andrzej Ostruszka

During LTO build compiler reports some 'false positive' warnings about variables being possibly used uninitialized. This patch silences these warnings.

Exemplary compiler warning to suppress (with LTO enabled):
error: ‘transceiver_type’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
switch (transceiver_type) {

---
 drivers/net/qede/base/ecore_mcp.c | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)
drivers/net/qede/base/ecore_mcp.c +8 -7
index a5aa07438..7518765a0 100644
@@ -2328,7 +2328,7 @@ enum _ecore_status_t ecore_mcp_trans_speed_mask(struct ecore_hwfn *p_hwfn,
 						struct ecore_ptt *p_ptt,
 						u32 *p_speed_mask)
 {
-	u32 transceiver_type, transceiver_state;
+	u32 transceiver_type = ETH_TRANSCEIVER_TYPE_NONE, transceiver_state;
 
 	ecore_mcp_get_transceiver_data(p_hwfn, p_ptt, &transceiver_state,
 				       &transceiver_type);
@@ -3223,7 +3223,8 @@ enum _ecore_status_t ecore_mcp_nvm_put_file_begin(struct ecore_dev *p_dev,
 enum _ecore_status_t ecore_mcp_nvm_write(struct ecore_dev *p_dev, u32 cmd,
 					 u32 addr, u8 *p_buf, u32 len)
 {
-	u32 buf_idx, buf_size, nvm_cmd, nvm_offset, resp, param;
+	u32 buf_idx, buf_size, nvm_cmd, nvm_offset;
+	u32 resp = FW_MSG_CODE_ERROR, param;
 	struct ecore_hwfn *p_hwfn = ECORE_LEADING_HWFN(p_dev);
 	enum _ecore_status_t rc = ECORE_INVAL;
 	struct ecore_ptt *p_ptt;
@@ -3322,7 +3323,7 @@ enum _ecore_status_t ecore_mcp_nvm_set_secure_mode(struct ecore_dev *p_dev,
 {
 	struct ecore_hwfn *p_hwfn = ECORE_LEADING_HWFN(p_dev);
 	struct ecore_ptt *p_ptt;
-	u32 resp, param;
+	u32 resp = 0, param;
 	enum _ecore_status_t rc;
 
 	p_ptt = ecore_ptt_acquire(p_hwfn);
@@ -3430,7 +3431,7 @@ enum _ecore_status_t ecore_mcp_gpio_read(struct ecore_hwfn *p_hwfn,
 					 u16 gpio, u32 *gpio_val)
 {
 	enum _ecore_status_t rc = ECORE_SUCCESS;
-	u32 drv_mb_param = 0, rsp;
+	u32 drv_mb_param = 0, rsp = 0;
 
 	drv_mb_param = (gpio << DRV_MB_PARAM_GPIO_NUMBER_OFFSET);
 
@@ -3451,7 +3452,7 @@ enum _ecore_status_t ecore_mcp_gpio_write(struct ecore_hwfn *p_hwfn,
 					  u16 gpio, u16 gpio_val)
 {
 	enum _ecore_status_t rc = ECORE_SUCCESS;
-	u32 drv_mb_param = 0, param, rsp;
+	u32 drv_mb_param = 0, param, rsp = 0;
 
 	drv_mb_param = (gpio << DRV_MB_PARAM_GPIO_NUMBER_OFFSET) |
 		(gpio_val << DRV_MB_PARAM_GPIO_VALUE_OFFSET);
@@ -3541,7 +3542,7 @@ enum _ecore_status_t ecore_mcp_bist_clock_test(struct ecore_hwfn *p_hwfn,
 enum _ecore_status_t ecore_mcp_bist_nvm_test_get_num_images(
 	struct ecore_hwfn *p_hwfn, struct ecore_ptt *p_ptt, u32 *num_images)
 {
-	u32 drv_mb_param = 0, rsp;
+	u32 drv_mb_param = 0, rsp = 0;
 	enum _ecore_status_t rc = ECORE_SUCCESS;
 
 	drv_mb_param = (DRV_MB_PARAM_BIST_NVM_TEST_NUM_IMAGES <<
@@ -3925,7 +3926,7 @@ enum _ecore_status_t
 __ecore_mcp_resc_lock(struct ecore_hwfn *p_hwfn, struct ecore_ptt *p_ptt,
 		      struct ecore_resc_lock_params *p_params)
 {
-	u32 param = 0, mcp_resp, mcp_param;
+	u32 param = 0, mcp_resp = 0, mcp_param = 0;
 	u8 opcode;
 	enum _ecore_status_t rc;
 
Signature · 1 line
2.17.1
#74 lore raw

v6 cover Add an option to use LTO for DPDK build

In reply to Andrzej Ostruszka

This patch series adds an option to make use of link time optimization (if compiler has support for it). It is split as follows:
- 1st patch is just a minor doc fix for versioning macros
- 2nd patch fixes missing __vsym annotations (needed for LTO)
- 3rd is the LTO enablement
- remaining patches are fixes for the warnings produced by the compiler and they are split by directory/subsystem so their maintainers can easily find and verify the changes - please note that there are two groups:
* errors (or possible errors) - with title "fix possible use ..."
* false positives - warnings that _I_ think are not valid and the changes are made only to silence the compiler.

v6 Changes:
-----------
- yet another split of the patch: this time from __vsym one minor doc fix to versioning macros has been moved to another patch
- net/ifc - fixed initialization of vring in m_ifcvf_start():
after rebase definition rte_vhost_vring has changed and now simple '{ 0 }' produces proper warning - however instead of '{ { 0 } }' I went with memset() because I could not get gcc to be quiet about "missing initializer" (IMHO "{{0}}" is a correct initialization but gcc required me to spell out initializers to all members which looked ugly and would require update on further changes to vring)
- I've been suggested (offlist) to mark the patches that were previously "reviewed/acked" so I've added some "Reviewed|Acked-by" tags (if the ack was sent to a patch that was later split the tag is placed in both patches).

v5 Changes:
-----------
- rebased to the v19.11 (one additional warning silenced in net/qede)
- initial patch has been split into two:
- one adding '__vsym' annotations and clarifying versioning docs
- another adding the actual LTO enablement
- addressed various documentation flaws pointed during review

v4 Changes:
-----------
- merge nested conditionals in config/meson.build into one

v3 Changes:
-----------
- removed support for clang (only gcc and icc remain):
- it turned out that 'fat-lto-objects' is not really supported - it is accepted as a flag but ignored e.g. clang v6.0:
clang: error: optimization flag '-ffat-lto-objects' is not supported [-Werror,-Wignored-optimization-argument]
this was probably masked previously by playing around with meson and testing for the availability of this flag and dropping LTO if not supported.
- improved commit messages:
- added 'Fixes' tags where appropriate
- included snippets of compiler warnings
- changed commit names for the commits cleaning up 'false positive' compiler warnings - these are not really 'fixes' for the code, just workarounds for compiler shortcomings and check-git-log.sh was complaining about missing 'Fixes' tag.

v2 Changes:
-----------
- fixed building of shared libraries:
- gcc does not scan top level assembler statements so it missed that function implementations for specific versions were being used and was removing them
- fixed meson.build config files:
- moved from 'enable_lto' project option to built-in 'b_lto'
- documented that 'default_library' should be 'shared':
with 'default_library=static' (the default) the meson build with LTO is broken - this is because, SHARED_LIB is fixed in rte_config.h used by meson which leads to no alias for default version in generated static libraries (MAP_STATIC_SYMBOL() is empty)
- app/test: added log for failed bonding "config get"

Andrzej Ostruszka (12):
doc: fix description of versioning macros
build: annotate versioned symbols with __vsym macro
build: add an option to enable LTO build
eventdev: fix possible use of uninitialized var app/eventdev: clean LTO build warnings (maybe-uninitialized) event/octeontx2: clean LTO build warnings (maybe-uninitialized) app/test: clean LTO build warnings (maybe-uninitialized) net/dpaa2: fix possible use of uninitialized vars net/e1000: clean LTO build warnings (maybe-uninitialized) net/i40e: clean LTO build warnings (maybe-uninitialized) net/ifc: clean LTO build warnings (maybe-uninitialized) net/qede: clean LTO build warnings (maybe-uninitialized)

 .travis.yml                                   |  7 ++++
 app/test-eventdev/test_perf_common.c          |  2 +-
 app/test-eventdev/test_pipeline_common.c      |  4 +-
 app/test/test_hash_readwrite.c                |  2 +-
 app/test/test_link_bonding_mode4.c            |  8 +++-
 app/test/test_memzone.c                       |  3 +-
 config/common_base                            |  5 +++
 config/meson.build                            | 13 ++++++
 doc/guides/contributing/versioning.rst        | 18 ++++++---
 doc/guides/prog_guide/index.rst               |  1 +
 doc/guides/prog_guide/lto.rst                 | 40 +++++++++++++++++++
 doc/guides/rel_notes/release_19_11.rst        |  9 +++++
 drivers/event/octeontx2/otx2_tim_worker.h     |  2 +-
 drivers/net/dpaa2/base/dpaa2_hw_dpni.c        |  1 +
 drivers/net/dpaa2/mc/dpkg.c                   |  2 +-
 drivers/net/dpaa2/mc/dpni.c                   |  9 +++--
 drivers/net/e1000/base/e1000_82543.c          |  2 +-
 drivers/net/e1000/base/e1000_ich8lan.c        |  2 +-
 drivers/net/e1000/base/e1000_phy.c            |  2 +-
 drivers/net/i40e/i40e_ethdev.c                |  2 +-
 drivers/net/ifc/ifcvf_vdpa.c                  | 14 +++++--
 drivers/net/qede/base/ecore_mcp.c             | 15 +++----
 lib/librte_distributor/rte_distributor.c      | 18 ++++-----
 lib/librte_distributor/rte_distributor_v20.c  | 18 ++++-----
 .../common/include/rte_function_versioning.h  | 11 ++++-
 lib/librte_eventdev/rte_event_timer_adapter.c |  8 ++--
 lib/librte_lpm/rte_lpm.c                      | 28 ++++++-------
 lib/librte_lpm/rte_lpm6.c                     | 16 ++++----
 lib/librte_timer/rte_timer.c                  | 20 +++++-----
 mk/toolchain/gcc/rte.toolchain-compat.mk      |  4 ++
 mk/toolchain/gcc/rte.vars.mk                  | 12 ++++++
 mk/toolchain/icc/rte.vars.mk                  |  8 ++++
 32 files changed, 218 insertions(+), 88 deletions(-)
 create mode 100644 doc/guides/prog_guide/lto.rst
Signature · 1 line
2.17.1
#75 lore raw

PATCH v6 1/12 doc: fix description of versioning macros

This patch fixes documentation of versioning macros so that they are aligned with their implementation (no underscore is added by macros).

---
 doc/guides/contributing/versioning.rst                  | 4 ++--
 lib/librte_eal/common/include/rte_function_versioning.h | 4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)
doc/guides/contributing/versioning.rst +2 -2
index 64984c54e..8a38928c0 100644
@@ -215,11 +215,11 @@ library so that older binaries need not be immediately recompiled.
 The macros exported are:
 
 * ``VERSION_SYMBOL(b, e, n)``: Creates a symbol version table entry binding
-  versioned symbol ``b@DPDK_n`` to the internal function ``b_e``.
+  versioned symbol ``b@DPDK_n`` to the internal function ``be``.
 
 * ``BIND_DEFAULT_SYMBOL(b, e, n)``: Creates a symbol version entry instructing
   the linker to bind references to symbol ``b`` to the internal symbol
-  ``b_e``.
+  ``be``.
 
 * ``MAP_STATIC_SYMBOL(f, p)``: Declare the prototype ``f``, and map it to the
   fully qualified function ``p``, so that if a symbol becomes versioned, it
lib/librte_eal/common/include/rte_function_versioning.h +2 -2
index 55e88ffae..eae619d60 100644
@@ -42,14 +42,14 @@
 /*
  * VERSION_SYMBOL
  * Creates a symbol version table entry binding symbol <b>@DPDK_<n> to the internal
- * function name <b>_<e>
+ * function name <b><e>
  */
 #define VERSION_SYMBOL(b, e, n) __asm__(".symver " RTE_STR(b) RTE_STR(e) ", " RTE_STR(b) "@DPDK_" RTE_STR(n))
 
 /*
  * BIND_DEFAULT_SYMBOL
  * Creates a symbol version entry instructing the linker to bind references to
- * symbol <b> to the internal symbol <b>_<e>
+ * symbol <b> to the internal symbol <b><e>
  */
 #define BIND_DEFAULT_SYMBOL(b, e, n) __asm__(".symver " RTE_STR(b) RTE_STR(e) ", " RTE_STR(b) "@@DPDK_" RTE_STR(n))
 #define __vsym __attribute__((used))
Signature · 1 line
2.17.1
#76 lore raw

PATCH v6 2/12 build: annotate versioned symbols with __vsym macro

In reply to Andrzej Ostruszka

Every implementation of a particular version of given symbol needs to be marked in its declaration as such (using __vsym macro). This patch fixes this and also clarifies the documentation about that.

---
 doc/guides/contributing/versioning.rst        | 14 +++++++---
 lib/librte_distributor/rte_distributor.c      | 18 ++++++------
 lib/librte_distributor/rte_distributor_v20.c  | 18 ++++++------
 .../common/include/rte_function_versioning.h  |  7 +++++
 lib/librte_lpm/rte_lpm.c                      | 28 +++++++++----------
 lib/librte_lpm/rte_lpm6.c                     | 16 +++++------
 lib/librte_timer/rte_timer.c                  | 20 ++++++-------
 7 files changed, 67 insertions(+), 54 deletions(-)
doc/guides/contributing/versioning.rst +10 -4
index 8a38928c0..fcd2d50f1 100644
@@ -225,6 +225,10 @@ The macros exported are:
   fully qualified function ``p``, so that if a symbol becomes versioned, it
   can still be mapped back to the public symbol name.
 
+* ``__vsym``:  Annotation to be used in a declaration of the internal symbol
+  ``be`` to signal that it is being used as an implementation of a particular
+  version of symbol ``b``.
+
 Examples of ABI Macro use
 ^^^^^^^^^^^^^^^^^^^^^^^^^
 
@@ -345,8 +349,9 @@ with the public symbol name
 
 .. code-block:: c
 
-  struct rte_acl_ctx *
+ -struct rte_acl_ctx *
  -rte_acl_create(const struct rte_acl_param *param)
+ +struct rte_acl_ctx * __vsym
  +rte_acl_create_v20(const struct rte_acl_param *param)
  {
         size_t sz;
@@ -354,7 +359,8 @@ with the public symbol name
         ...
 
 Note that the base name of the symbol was kept intact, as this is conducive to
-the macros used for versioning symbols.  That is our next step, mapping this new
+the macros used for versioning symbols and we have annotated the function as an
+implementation of versioned symbol.  That is our next step, mapping this new
 symbol name to the initial symbol name at version node 2.0.  Immediately after
 the function, we add this line of code
 
@@ -374,7 +380,7 @@ name, with a different suffix, and  implement it appropriately
 
 .. code-block:: c
 
-   struct rte_acl_ctx *
+   struct rte_acl_ctx * __vsym
    rte_acl_create_v21(const struct rte_acl_param *param, int debug);
    {
         struct rte_acl_ctx *ctx = rte_acl_create_v20(param);
@@ -423,7 +429,7 @@ defined, we add this
 
 .. code-block:: c
 
-   struct rte_acl_ctx *
+   struct rte_acl_ctx * __vsym
    rte_acl_create_v21(const struct rte_acl_param *param, int debug)
    {
         ...
lib/librte_distributor/rte_distributor.c +9 -9
index 7df97df92..2cc32ddba 100644
@@ -32,7 +32,7 @@ EAL_REGISTER_TAILQ(rte_dist_burst_tailq)
 
 /**** Burst Packet APIs called by workers ****/
 
-void
+void __vsym
 rte_distributor_request_pkt_v1705(struct rte_distributor *d,
 		unsigned int worker_id, struct rte_mbuf **oldpkt,
 		unsigned int count)
@@ -89,7 +89,7 @@ MAP_STATIC_SYMBOL(void rte_distributor_request_pkt(struct rte_distributor *d,
 		unsigned int count),
 		rte_distributor_request_pkt_v1705);
 
-int
+int __vsym
 rte_distributor_poll_pkt_v1705(struct rte_distributor *d,
 		unsigned int worker_id, struct rte_mbuf **pkts)
 {
@@ -134,7 +134,7 @@ MAP_STATIC_SYMBOL(int rte_distributor_poll_pkt(struct rte_distributor *d,
 		unsigned int worker_id, struct rte_mbuf **pkts),
 		rte_distributor_poll_pkt_v1705);
 
-int
+int __vsym
 rte_distributor_get_pkt_v1705(struct rte_distributor *d,
 		unsigned int worker_id, struct rte_mbuf **pkts,
 		struct rte_mbuf **oldpkt, unsigned int return_count)
@@ -169,7 +169,7 @@ MAP_STATIC_SYMBOL(int rte_distributor_get_pkt(struct rte_distributor *d,
 		struct rte_mbuf **oldpkt, unsigned int return_count),
 		rte_distributor_get_pkt_v1705);
 
-int
+int __vsym
 rte_distributor_return_pkt_v1705(struct rte_distributor *d,
 		unsigned int worker_id, struct rte_mbuf **oldpkt, int num)
 {
@@ -359,7 +359,7 @@ release(struct rte_distributor *d, unsigned int wkr)
 
 
 /* process a set of packets to distribute them to workers */
-int
+int __vsym
 rte_distributor_process_v1705(struct rte_distributor *d,
 		struct rte_mbuf **mbufs, unsigned int num_mbufs)
 {
@@ -506,7 +506,7 @@ MAP_STATIC_SYMBOL(int rte_distributor_process(struct rte_distributor *d,
 		rte_distributor_process_v1705);
 
 /* return to the caller, packets returned from workers */
-int
+int __vsym
 rte_distributor_returned_pkts_v1705(struct rte_distributor *d,
 		struct rte_mbuf **mbufs, unsigned int max_mbufs)
 {
@@ -556,7 +556,7 @@ total_outstanding(const struct rte_distributor *d)
  * Flush the distributor, so that there are no outstanding packets in flight or
  * queued up.
  */
-int
+int __vsym
 rte_distributor_flush_v1705(struct rte_distributor *d)
 {
 	unsigned int flushed;
@@ -591,7 +591,7 @@ MAP_STATIC_SYMBOL(int rte_distributor_flush(struct rte_distributor *d),
 		rte_distributor_flush_v1705);
 
 /* clears the internal returns array in the distributor */
-void
+void __vsym
 rte_distributor_clear_returns_v1705(struct rte_distributor *d)
 {
 	unsigned int wkr;
@@ -613,7 +613,7 @@ MAP_STATIC_SYMBOL(void rte_distributor_clear_returns(struct rte_distributor *d),
 		rte_distributor_clear_returns_v1705);
 
 /* creates a distributor instance */
-struct rte_distributor *
+struct rte_distributor * __vsym
 rte_distributor_create_v1705(const char *name,
 		unsigned int socket_id,
 		unsigned int num_workers,
lib/librte_distributor/rte_distributor_v20.c +9 -9
index db6c49258..7a6fddf55 100644
@@ -27,7 +27,7 @@ EAL_REGISTER_TAILQ(rte_distributor_tailq)
 
 /**** APIs called by workers ****/
 
-void
+void __vsym
 rte_distributor_request_pkt_v20(struct rte_distributor_v20 *d,
 		unsigned worker_id, struct rte_mbuf *oldpkt)
 {
@@ -43,7 +43,7 @@ rte_distributor_request_pkt_v20(struct rte_distributor_v20 *d,
 }
 VERSION_SYMBOL(rte_distributor_request_pkt, _v20, 2.0);
 
-struct rte_mbuf *
+struct rte_mbuf * __vsym
 rte_distributor_poll_pkt_v20(struct rte_distributor_v20 *d,
 		unsigned worker_id)
 {
@@ -59,7 +59,7 @@ rte_distributor_poll_pkt_v20(struct rte_distributor_v20 *d,
 }
 VERSION_SYMBOL(rte_distributor_poll_pkt, _v20, 2.0);
 
-struct rte_mbuf *
+struct rte_mbuf * __vsym
 rte_distributor_get_pkt_v20(struct rte_distributor_v20 *d,
 		unsigned worker_id, struct rte_mbuf *oldpkt)
 {
@@ -71,7 +71,7 @@ rte_distributor_get_pkt_v20(struct rte_distributor_v20 *d,
 }
 VERSION_SYMBOL(rte_distributor_get_pkt, _v20, 2.0);
 
-int
+int __vsym
 rte_distributor_return_pkt_v20(struct rte_distributor_v20 *d,
 		unsigned worker_id, struct rte_mbuf *oldpkt)
 {
@@ -204,7 +204,7 @@ process_returns(struct rte_distributor_v20 *d)
 }
 
 /* process a set of packets to distribute them to workers */
-int
+int __vsym
 rte_distributor_process_v20(struct rte_distributor_v20 *d,
 		struct rte_mbuf **mbufs, unsigned num_mbufs)
 {
@@ -321,7 +321,7 @@ rte_distributor_process_v20(struct rte_distributor_v20 *d,
 VERSION_SYMBOL(rte_distributor_process, _v20, 2.0);
 
 /* return to the caller, packets returned from workers */
-int
+int __vsym
 rte_distributor_returned_pkts_v20(struct rte_distributor_v20 *d,
 		struct rte_mbuf **mbufs, unsigned max_mbufs)
 {
@@ -359,7 +359,7 @@ total_outstanding(const struct rte_distributor_v20 *d)
 
 /* flush the distributor, so that there are no outstanding packets in flight or
  * queued up. */
-int
+int __vsym
 rte_distributor_flush_v20(struct rte_distributor_v20 *d)
 {
 	const unsigned flushed = total_outstanding(d);
@@ -372,7 +372,7 @@ rte_distributor_flush_v20(struct rte_distributor_v20 *d)
 VERSION_SYMBOL(rte_distributor_flush, _v20, 2.0);
 
 /* clears the internal returns array in the distributor */
-void
+void __vsym
 rte_distributor_clear_returns_v20(struct rte_distributor_v20 *d)
 {
 	d->returns.start = d->returns.count = 0;
@@ -383,7 +383,7 @@ rte_distributor_clear_returns_v20(struct rte_distributor_v20 *d)
 VERSION_SYMBOL(rte_distributor_clear_returns, _v20, 2.0);
 
 /* creates a distributor instance */
-struct rte_distributor_v20 *
+struct rte_distributor_v20 * __vsym
 rte_distributor_create_v20(const char *name,
 		unsigned socket_id,
 		unsigned num_workers)
lib/librte_eal/common/include/rte_function_versioning.h +7 -0
index eae619d60..c924351d5 100644
@@ -52,6 +52,13 @@
  * symbol <b> to the internal symbol <b><e>
  */
 #define BIND_DEFAULT_SYMBOL(b, e, n) __asm__(".symver " RTE_STR(b) RTE_STR(e) ", " RTE_STR(b) "@@DPDK_" RTE_STR(n))
+
+/*
+ * __vsym
+ * Annotation to be used in declaration of the internal symbol <b><e> to signal
+ * that it is being used as an implementation of a particular version of symbol
+ * <b>.
+ */
 #define __vsym __attribute__((used))
 
 /*
lib/librte_lpm/rte_lpm.c +14 -14
index c96395e26..106916dc8 100644
@@ -90,7 +90,7 @@ depth_to_range(uint8_t depth)
 /*
  * Find an existing lpm table and return a pointer to it.
  */
-struct rte_lpm_v20 *
+struct rte_lpm_v20 * __vsym
 rte_lpm_find_existing_v20(const char *name)
 {
 	struct rte_lpm_v20 *l = NULL;
@@ -116,7 +116,7 @@ rte_lpm_find_existing_v20(const char *name)
 }
 VERSION_SYMBOL(rte_lpm_find_existing, _v20, 2.0);
 
-struct rte_lpm *
+struct rte_lpm * __vsym
 rte_lpm_find_existing_v1604(const char *name)
 {
 	struct rte_lpm *l = NULL;
@@ -147,7 +147,7 @@ MAP_STATIC_SYMBOL(struct rte_lpm *rte_lpm_find_existing(const char *name),
 /*
  * Allocates memory for LPM object
  */
-struct rte_lpm_v20 *
+struct rte_lpm_v20 * __vsym
 rte_lpm_create_v20(const char *name, int socket_id, int max_rules,
 		__rte_unused int flags)
 {
@@ -220,7 +220,7 @@ rte_lpm_create_v20(const char *name, int socket_id, int max_rules,
 }
 VERSION_SYMBOL(rte_lpm_create, _v20, 2.0);
 
-struct rte_lpm *
+struct rte_lpm * __vsym
 rte_lpm_create_v1604(const char *name, int socket_id,
 		const struct rte_lpm_config *config)
 {
@@ -329,7 +329,7 @@ MAP_STATIC_SYMBOL(
 /*
  * Deallocates memory for given LPM table.
  */
-void
+void __vsym
 rte_lpm_free_v20(struct rte_lpm_v20 *lpm)
 {
 	struct rte_lpm_list *lpm_list;
@@ -358,7 +358,7 @@ rte_lpm_free_v20(struct rte_lpm_v20 *lpm)
 }
 VERSION_SYMBOL(rte_lpm_free, _v20, 2.0);
 
-void
+void __vsym
 rte_lpm_free_v1604(struct rte_lpm *lpm)
 {
 	struct rte_lpm_list *lpm_list;
@@ -1177,7 +1177,7 @@ add_depth_big_v1604(struct rte_lpm *lpm, uint32_t ip_masked, uint8_t depth,
 /*
  * Add a route
  */
-int
+int __vsym
 rte_lpm_add_v20(struct rte_lpm_v20 *lpm, uint32_t ip, uint8_t depth,
 		uint8_t next_hop)
 {
@@ -1218,7 +1218,7 @@ rte_lpm_add_v20(struct rte_lpm_v20 *lpm, uint32_t ip, uint8_t depth,
 }
 VERSION_SYMBOL(rte_lpm_add, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm_add_v1604(struct rte_lpm *lpm, uint32_t ip, uint8_t depth,
 		uint32_t next_hop)
 {
@@ -1264,7 +1264,7 @@ MAP_STATIC_SYMBOL(int rte_lpm_add(struct rte_lpm *lpm, uint32_t ip,
 /*
  * Look for a rule in the high-level rules table
  */
-int
+int __vsym
 rte_lpm_is_rule_present_v20(struct rte_lpm_v20 *lpm, uint32_t ip, uint8_t depth,
 uint8_t *next_hop)
 {
@@ -1291,7 +1291,7 @@ uint8_t *next_hop)
 }
 VERSION_SYMBOL(rte_lpm_is_rule_present, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm_is_rule_present_v1604(struct rte_lpm *lpm, uint32_t ip, uint8_t depth,
 uint32_t *next_hop)
 {
@@ -1844,7 +1844,7 @@ delete_depth_big_v1604(struct rte_lpm *lpm, uint32_t ip_masked,
 /*
  * Deletes a rule
  */
-int
+int __vsym
 rte_lpm_delete_v20(struct rte_lpm_v20 *lpm, uint32_t ip, uint8_t depth)
 {
 	int32_t rule_to_delete_index, sub_rule_index;
@@ -1898,7 +1898,7 @@ rte_lpm_delete_v20(struct rte_lpm_v20 *lpm, uint32_t ip, uint8_t depth)
 }
 VERSION_SYMBOL(rte_lpm_delete, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm_delete_v1604(struct rte_lpm *lpm, uint32_t ip, uint8_t depth)
 {
 	int32_t rule_to_delete_index, sub_rule_index;
@@ -1957,7 +1957,7 @@ MAP_STATIC_SYMBOL(int rte_lpm_delete(struct rte_lpm *lpm, uint32_t ip,
 /*
  * Delete all rules from the LPM table.
  */
-void
+void __vsym
 rte_lpm_delete_all_v20(struct rte_lpm_v20 *lpm)
 {
 	/* Zero rule information. */
@@ -1974,7 +1974,7 @@ rte_lpm_delete_all_v20(struct rte_lpm_v20 *lpm)
 }
 VERSION_SYMBOL(rte_lpm_delete_all, _v20, 2.0);
 
-void
+void __vsym
 rte_lpm_delete_all_v1604(struct rte_lpm *lpm)
 {
 	/* Zero rule information. */
lib/librte_lpm/rte_lpm6.c +8 -8
index e20f82460..0d161dc32 100644
@@ -812,7 +812,7 @@ add_step(struct rte_lpm6 *lpm, struct rte_lpm6_tbl_entry *tbl,
 /*
  * Add a route
  */
-int
+int __vsym
 rte_lpm6_add_v20(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
 		uint8_t next_hop)
 {
@@ -862,7 +862,7 @@ simulate_add(struct rte_lpm6 *lpm, const uint8_t *masked_ip, uint8_t depth)
 	return 0;
 }
 
-int
+int __vsym
 rte_lpm6_add_v1705(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
 		uint32_t next_hop)
 {
@@ -955,7 +955,7 @@ lookup_step(const struct rte_lpm6 *lpm, const struct rte_lpm6_tbl_entry *tbl,
 /*
  * Looks up an IP
  */
-int
+int __vsym
 rte_lpm6_lookup_v20(const struct rte_lpm6 *lpm, uint8_t *ip, uint8_t *next_hop)
 {
 	uint32_t next_hop32 = 0;
@@ -973,7 +973,7 @@ rte_lpm6_lookup_v20(const struct rte_lpm6 *lpm, uint8_t *ip, uint8_t *next_hop)
 }
 VERSION_SYMBOL(rte_lpm6_lookup, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm6_lookup_v1705(const struct rte_lpm6 *lpm, uint8_t *ip,
 		uint32_t *next_hop)
 {
@@ -1008,7 +1008,7 @@ MAP_STATIC_SYMBOL(int rte_lpm6_lookup(const struct rte_lpm6 *lpm, uint8_t *ip,
 /*
  * Looks up a group of IP addresses
  */
-int
+int __vsym
 rte_lpm6_lookup_bulk_func_v20(const struct rte_lpm6 *lpm,
 		uint8_t ips[][RTE_LPM6_IPV6_ADDR_SIZE],
 		int16_t * next_hops, unsigned n)
@@ -1049,7 +1049,7 @@ rte_lpm6_lookup_bulk_func_v20(const struct rte_lpm6 *lpm,
 }
 VERSION_SYMBOL(rte_lpm6_lookup_bulk_func, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm6_lookup_bulk_func_v1705(const struct rte_lpm6 *lpm,
 		uint8_t ips[][RTE_LPM6_IPV6_ADDR_SIZE],
 		int32_t *next_hops, unsigned int n)
@@ -1099,7 +1099,7 @@ MAP_STATIC_SYMBOL(int rte_lpm6_lookup_bulk_func(const struct rte_lpm6 *lpm,
 /*
  * Look for a rule in the high-level rules table
  */
-int
+int __vsym
 rte_lpm6_is_rule_present_v20(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
 		uint8_t *next_hop)
 {
@@ -1119,7 +1119,7 @@ rte_lpm6_is_rule_present_v20(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
 }
 VERSION_SYMBOL(rte_lpm6_is_rule_present, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm6_is_rule_present_v1705(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
 		uint32_t *next_hop)
 {
lib/librte_timer/rte_timer.c +10 -10
index 3834c9473..381a9f43f 100644
@@ -131,7 +131,7 @@ rte_timer_data_dealloc(uint32_t id)
 	return 0;
 }
 
-void
+void __vsym
 rte_timer_subsystem_init_v20(void)
 {
 	unsigned lcore_id;
@@ -153,7 +153,7 @@ VERSION_SYMBOL(rte_timer_subsystem_init, _v20, 2.0);
  * secondary processes should be empty, the zeroth entry can be shared by
  * multiple processes.
  */
-int
+int __vsym
 rte_timer_subsystem_init_v1905(void)
 {
 	const struct rte_memzone *mz;
@@ -551,7 +551,7 @@ __rte_timer_reset(struct rte_timer *tim, uint64_t expire,
 }
 
 /* Reset and start the timer associated with the timer handle tim */
-int
+int __vsym
 rte_timer_reset_v20(struct rte_timer *tim, uint64_t ticks,
 		    enum rte_timer_type type, unsigned int tim_lcore,
 		    rte_timer_cb_t fct, void *arg)
@@ -574,7 +574,7 @@ rte_timer_reset_v20(struct rte_timer *tim, uint64_t ticks,
 }
 VERSION_SYMBOL(rte_timer_reset, _v20, 2.0);
 
-int
+int __vsym
 rte_timer_reset_v1905(struct rte_timer *tim, uint64_t ticks,
 		      enum rte_timer_type type, unsigned int tim_lcore,
 		      rte_timer_cb_t fct, void *arg)
@@ -657,14 +657,14 @@ __rte_timer_stop(struct rte_timer *tim, int local_is_locked,
 }
 
 /* Stop the timer associated with the timer handle tim */
-int
+int __vsym
 rte_timer_stop_v20(struct rte_timer *tim)
 {
 	return __rte_timer_stop(tim, 0, &default_timer_data);
 }
 VERSION_SYMBOL(rte_timer_stop, _v20, 2.0);
 
-int
+int __vsym
 rte_timer_stop_v1905(struct rte_timer *tim)
 {
 	return rte_timer_alt_stop(default_data_id, tim);
@@ -817,14 +817,14 @@ __rte_timer_manage(struct rte_timer_data *timer_data)
 	priv_timer[lcore_id].running_tim = NULL;
 }
 
-void
+void __vsym
 rte_timer_manage_v20(void)
 {
 	__rte_timer_manage(&default_timer_data);
 }
 VERSION_SYMBOL(rte_timer_manage, _v20, 2.0);
 
-int
+int __vsym
 rte_timer_manage_v1905(void)
 {
 	struct rte_timer_data *timer_data;
@@ -1074,14 +1074,14 @@ __rte_timer_dump_stats(struct rte_timer_data *timer_data __rte_unused, FILE *f)
 #endif
 }
 
-void
+void __vsym
 rte_timer_dump_stats_v20(FILE *f)
 {
 	__rte_timer_dump_stats(&default_timer_data, f);
 }
 VERSION_SYMBOL(rte_timer_dump_stats, _v20, 2.0);
 
-int
+int __vsym
 rte_timer_dump_stats_v1905(FILE *f)
 {
 	return rte_timer_alt_dump_stats(default_data_id, f);
Signature · 1 line
2.17.1
#77 lore raw

PATCH v6 3/12 build: add an option to enable LTO build

In reply to Andrzej Ostruszka

This patch adds an option to enable link time optimization. In addition to LTO option itself (-flto) fat-lto-objects are being used. This is because during the build pmdinfogen scans the generated ELF objects to find this_pmd_name* symbol in symbol table. Without fat-lto-objects gcc produces ELF only with extra symbols for internal use during linking.

---
 .travis.yml                              |  7 +++++
 config/common_base                       |  5 +++
 config/meson.build                       | 13 ++++++++
 doc/guides/prog_guide/index.rst          |  1 +
 doc/guides/prog_guide/lto.rst            | 40 ++++++++++++++++++++++++
 doc/guides/rel_notes/release_19_11.rst   |  9 ++++++
 mk/toolchain/gcc/rte.toolchain-compat.mk |  4 +++
 mk/toolchain/gcc/rte.vars.mk             | 12 +++++++
 mk/toolchain/icc/rte.vars.mk             |  8 +++++
 9 files changed, 99 insertions(+)
 create mode 100644 doc/guides/prog_guide/lto.rst
.travis.yml +7 -0
index 3d6ef2959..3cd746dba 100644
@@ -34,6 +34,7 @@ env:
   - DEF_LIB="static" OPTS="-Denable_kmods=false"
   - DEF_LIB="shared" OPTS="-Denable_kmods=false"
   - DEF_LIB="shared" RUN_TESTS=1
+  - DEF_LIB="shared" OPTS="-Db_lto=true"
 
 matrix:
   include:
@@ -105,6 +106,12 @@ matrix:
       apt:
         packages:
           - *extra_packages
+  - env: DEF_LIB="shared" OPTS="-Db_lto=true" EXTRA_PACKAGES=1
+    compiler: gcc
+    addons:
+      apt:
+        packages:
+          - *extra_packages
 
 
 script: ./.ci/${TRAVIS_OS_NAME}-build.sh
config/common_base +5 -0
index b2be3d96a..0d1207166 100644
@@ -49,6 +49,11 @@ CONFIG_RTE_FORCE_INTRINSICS=n
 #
 CONFIG_RTE_ARCH_STRICT_ALIGN=n
 
+#
+# Enable link time optimization
+#
+CONFIG_RTE_ENABLE_LTO=n
+
 #
 # Compile to share library
 #
config/meson.build +13 -0
index e1ebdad26..2b1cb92e7 100644
@@ -225,3 +225,16 @@ add_project_arguments('-D_GNU_SOURCE', language: 'c')
 if is_freebsd
 	add_project_arguments('-D__BSD_VISIBLE', language: 'c')
 endif
+
+if get_option('b_lto')
+	if cc.has_argument('-ffat-lto-objects')
+		add_project_arguments('-ffat-lto-objects', language: 'c')
+	else
+		error('compiler does not support fat LTO objects - please turn LTO off')
+	endif
+	# workaround for gcc bug 81440
+	if cc.get_id() == 'gcc' and cc.version().version_compare('<8.0')
+		add_project_arguments('-Wno-lto-type-mismatch', language: 'c')
+		add_project_link_arguments('-Wno-lto-type-mismatch', language: 'c')
+	endif
+endif
doc/guides/prog_guide/index.rst +1 -0
index 692409af8..dc4851c57 100644
@@ -65,5 +65,6 @@ Programmer's Guide
     ext_app_lib_make_help
     perf_opt_guidelines
     writing_efficient_code
+    lto
     profile_app
     glossary
doc/guides/prog_guide/lto.rst new file +40 -0
new file mode 100644
index 000000000..50aecc9e5
@@ -0,0 +1,40 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2019 Marvell International Ltd.
+
+Link Time Optimization
+======================
+
+The DPDK supports compilation with link time optimization turned on.
+This depends obviously on the ability of the compiler to do "whole
+program" optimization at link time and is available only for compilers
+that support that feature.
+To be more specific, compiler (in addition to performing LTO) have to
+support creation of ELF objects containing both normal code and internal
+representation (called fat-lto-objects in gcc and icc).
+This is required since during build some code is generated by parsing
+produced ELF objects (pmdinfogen).
+
+The amount of performance gain that one can get from LTO depends on the
+compiler and the code that is being compiled.
+However LTO is also useful for additional code analysis done by the
+compiler.
+In particular due to interprocedural analysis compiler can produce
+additional warnings about variables that might be used uninitialized.
+Some of these warnings might be "false positives" though and you might
+need to explicitly initialize variable in order to silence the compiler.
+
+Please note that turning LTO on causes considerable extension of
+build time.
+
+When using make based build, link time optimization can be enabled for
+the whole DPDK by setting:
+
+.. code-block:: console
+    CONFIG_ENABLE_LTO=y
+
+in config file.
+For the meson based build it can be enabled by setting meson built-in
+'b_lto' option:
+
+.. code-block:: console
+    meson build -Db_lto=true
doc/guides/rel_notes/release_19_11.rst +9 -0
index ae8e7b2f0..b11abe2fe 100644
@@ -231,6 +231,15 @@ New Features
   * Added a console command to testpmd app, ``show port (port_id) ptypes`` which
     gives ability to print port supported ptypes in different protocol layers.
 
+* **Added build support for Link Time Optimization.**
+
+  LTO is an optimization technique used by the compiler to perform whole
+  program analysis and optimization at link time.  In order to do that
+  compilers store their internal representation of the source code that
+  the linker uses at the final stage of compilation process.
+
+  See :doc:`../prog_guide/lto` for more information:
+
 
 Removed Items
 -------------
mk/toolchain/gcc/rte.toolchain-compat.mk +4 -0
index ea40a11c0..ad4fad83c 100644
@@ -88,6 +88,10 @@ else
 		MACHINE_CFLAGS := $(filter-out -march% -mtune% -msse%,$(MACHINE_CFLAGS))
 	endif
 
+	ifeq ($(shell test $(GCC_VERSION) -lt 45 && echo 1), 1)
+		CONFIG_RTE_ENABLE_LTO=n
+	endif
+
 	# Disable thunderx PMD for gcc < 4.7
 	ifeq ($(shell test $(GCC_VERSION) -lt 47 && echo 1), 1)
 		CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD=d
mk/toolchain/gcc/rte.vars.mk +12 -0
index b852fcfd7..9fc704193 100644
@@ -62,6 +62,18 @@ endif
 # process cpu flags
 include $(RTE_SDK)/mk/toolchain/$(RTE_TOOLCHAIN)/rte.toolchain-compat.mk
 
+ifeq ($(CONFIG_RTE_ENABLE_LTO),y)
+# 'fat-lto' is used since pmdinfogen needs to have 'this_pmd_nameX'
+# exported in symbol table and without this option only internal
+# representation is present.
+TOOLCHAIN_CFLAGS += -flto -ffat-lto-objects
+TOOLCHAIN_LDFLAGS += -flto
+# workaround for GCC bug 81440
+ifeq ($(shell test $(GCC_VERSION) -lt 80 && echo 1), 1)
+WERROR_FLAGS += -Wno-lto-type-mismatch
+endif
+endif
+
 # workaround GCC bug with warning "missing initializer" for "= {0}"
 ifeq ($(shell test $(GCC_VERSION) -lt 47 && echo 1), 1)
 WERROR_FLAGS += -Wno-missing-field-initializers
mk/toolchain/icc/rte.vars.mk +8 -0
index aa1422bf1..8aa87aa1e 100644
@@ -54,5 +54,13 @@ endif
 # process cpu flags
 include $(RTE_SDK)/mk/toolchain/$(RTE_TOOLCHAIN)/rte.toolchain-compat.mk
 
+ifeq ($(CONFIG_RTE_ENABLE_LTO),y)
+# 'fat-lto' is used since pmdinfogen needs to have 'this_pmd_nameX'
+# exported in symbol table and without this option only internal
+# representation is present.
+TOOLCHAIN_CFLAGS += -flto -ffat-lto-objects
+TOOLCHAIN_LDFLAGS += -flto
+endif
+
 export CC AS AR LD OBJCOPY OBJDUMP STRIP READELF
 export TOOLCHAIN_CFLAGS TOOLCHAIN_LDFLAGS TOOLCHAIN_ASFLAGS
Signature · 1 line
2.17.1
#78 lore raw

PATCH v6 4/12 eventdev: fix possible use of uninitialized var

In reply to Andrzej Ostruszka

Fix the logic for the case of event queue allowing all schedule types.

Compiler warning pointing to this error (with LTO enabled):
error: ‘sched_type’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
if ((ret < 0 && ret != -EOVERFLOW) ||

Fixes 6750b21bd6af ("eventdev: add default software timer adapter")
---
 lib/librte_eventdev/rte_event_timer_adapter.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
lib/librte_eventdev/rte_event_timer_adapter.c +4 -4
index 5ce399eca..161e21a68 100644
@@ -706,11 +706,11 @@ check_destination_event_queue(struct rte_event_timer *evtim,
 				       RTE_EVENT_QUEUE_ATTR_SCHEDULE_TYPE,
 				       &sched_type);
 
-	if ((ret < 0 && ret != -EOVERFLOW) ||
-	    evtim->ev.sched_type != sched_type)
-		return -1;
+	if ((ret == 0 && evtim->ev.sched_type == sched_type) ||
+	    ret == -EOVERFLOW)
+		return 0;
 
-	return 0;
+	return -1;
 }
 
 static int
Signature · 1 line
2.17.1
#79 lore raw

PATCH v6 5/12 app/eventdev: clean LTO build warnings (maybe-uninitialized)

In reply to Andrzej Ostruszka

During LTO build compiler reports some 'false positive' warnings about variables being possibly used uninitialized. This patch silences these warnings.

Exemplary compiler warning to suppress (with LTO enabled):
error: ‘service_id’ may be used uninitialized in this function [-Werror=maybe-uninitialized]

    ret = evt_service_setup(service_id);
---
 app/test-eventdev/test_perf_common.c     | 2 +-
 app/test-eventdev/test_pipeline_common.c | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)
app/test-eventdev/test_perf_common.c +1 -1
index e24519179..e7cf75a7d 100644
@@ -439,7 +439,7 @@ perf_event_timer_adapter_setup(struct test_perf *t)
 
 		if (!(adapter_info.caps &
 				RTE_EVENT_TIMER_ADAPTER_CAP_INTERNAL_PORT)) {
-			uint32_t service_id;
+			uint32_t service_id = -1U;
 
 			rte_event_timer_adapter_service_id_get(wl,
 					&service_id);
app/test-eventdev/test_pipeline_common.c +2 -2
index 386aca299..160461fb2 100644
@@ -326,7 +326,7 @@ pipeline_event_rx_adapter_setup(struct evt_options *opt, uint8_t stride,
 		}
 
 		if (!(cap & RTE_EVENT_ETH_RX_ADAPTER_CAP_INTERNAL_PORT)) {
-			uint32_t service_id;
+			uint32_t service_id = -1U;
 
 			rte_event_eth_rx_adapter_service_id_get(prod,
 					&service_id);
@@ -378,7 +378,7 @@ pipeline_event_tx_adapter_setup(struct evt_options *opt,
 		}
 
 		if (!(cap & RTE_EVENT_ETH_TX_ADAPTER_CAP_INTERNAL_PORT)) {
-			uint32_t service_id;
+			uint32_t service_id = -1U;
 
 			rte_event_eth_tx_adapter_service_id_get(consm,
 					&service_id);
Signature · 1 line
2.17.1
#80 lore raw

PATCH v6 6/12 event/octeontx2: clean LTO build warnings (maybe-uninitialized)

In reply to Andrzej Ostruszka

During LTO build compiler reports some 'false positive' warnings about variables being possibly used uninitialized. This patch silences these warnings.

Exemplary compiler warning to suppress (with LTO enabled):
error: ‘chunk’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
bkt->current_chunk = (uintptr_t)chunk;

---
 drivers/event/octeontx2/otx2_tim_worker.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/event/octeontx2/otx2_tim_worker.h +1 -1
index b193e2cab..50db6543c 100644
@@ -337,7 +337,7 @@ tim_add_entry_brst(struct otx2_tim_ring * const tim_ring,
 		   const struct otx2_tim_ent *ents,
 		   const uint16_t nb_timers, const uint8_t flags)
 {
-	struct otx2_tim_ent *chunk;
+	struct otx2_tim_ent *chunk = NULL;
 	struct otx2_tim_bkt *bkt;
 	uint16_t chunk_remainder;
 	uint16_t index = 0;
Signature · 1 line
2.17.1
#81 lore raw

PATCH v6 7/12 app/test: clean LTO build warnings (maybe-uninitialized)

In reply to Andrzej Ostruszka

During LTO build compiler reports some 'false positive' warnings about variables being possibly used uninitialized. This patch silences these warnings.

Exemplary compiler warning to suppress (with LTO enabled):
error: ‘stats.greatest_free_size’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
return len - overhead;

---
 app/test/test_hash_readwrite.c     | 2 +-
 app/test/test_link_bonding_mode4.c | 8 +++++++-
 app/test/test_memzone.c            | 3 ++-
 3 files changed, 10 insertions(+), 3 deletions(-)
app/test/test_hash_readwrite.c +1 -1
index 4376b099b..615767fb6 100644
@@ -298,7 +298,7 @@ test_rw_reader(void *arg)
 
 	begin = rte_rdtsc_precise();
 	for (i = 0; i < read_cnt; i++) {
-		void *data;
+		void *data = arg;
 		rte_hash_lookup_data(tbl_rw_test_param.h,
 				tbl_rw_test_param.keys + i,
 				&data);
app/test/test_link_bonding_mode4.c +7 -1
index ff54d7b91..cf12f026d 100644
@@ -585,7 +585,13 @@ bond_get_update_timeout_ms(void)
 {
 	struct rte_eth_bond_8023ad_conf conf;
 
-	rte_eth_bond_8023ad_conf_get(test_params.bonded_port_id, &conf);
+	if (rte_eth_bond_8023ad_conf_get(test_params.bonded_port_id, &conf) < 0) {
+		RTE_LOG(DEBUG, EAL, "Failed to get bonding configuration: "
+				    "%s at %d\n", __func__, __LINE__);
+		RTE_TEST_TRACE_FAILURE(__FILE__, __LINE__, __func__);
+		return 0;
+	}
+
 	return conf.update_timeout_ms;
 }
 
app/test/test_memzone.c +2 -1
index 4d8744455..0343b0326 100644
@@ -475,7 +475,8 @@ find_max_block_free_size(unsigned int align, unsigned int socket_id)
 	struct rte_malloc_socket_stats stats;
 	size_t len, overhead;
 
-	rte_malloc_get_socket_stats(socket_id, &stats);
+	if (rte_malloc_get_socket_stats(socket_id, &stats) < 0)
+		return 0;
 
 	len = stats.greatest_free_size;
 	overhead = MALLOC_ELEM_OVERHEAD;
Signature · 1 line
2.17.1
#82 lore raw

Re: [dpdk-dev] [PATCH v6 07/12] app/test: clean LTO build warnings (maybe-uninitialized)

Quoted 29 lines

-----Original Message-----
From: Andrzej Ostruszka [mailto:[email protected]]
Sent: Tuesday, October 29, 2019 7:12 AM
To: [email protected]; Wang, Yipeng1 <[email protected]>; Gobriel, Sameh <[email protected]>; Richardson, Bruce <[email protected]>; Chas Williams <[email protected]>; Burakov, Anatoly <[email protected]>
Cc: [email protected]; [email protected]
Subject: [PATCH v6 07/12] app/test: clean LTO build warnings (maybe-uninitialized)

During LTO build compiler reports some 'false positive' warnings about variables being possibly used uninitialized. This patch silences these warnings.

Exemplary compiler warning to suppress (with LTO enabled):
error: ‘stats.greatest_free_size’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
return len - overhead;

---

app/test/test_hash_readwrite.c     | 2 +-
app/test/test_link_bonding_mode4.c | 8 +++++++-
app/test/test_memzone.c            | 3 ++-
3 files changed, 10 insertions(+), 3 deletions(-)
app/test/test_hash_readwrite.c +0 -0
index 4376b099b..615767fb6 100644

app/test/test_hash_readwrite.c test_rw_reader(void *arg) · line 301

Quoted 5 lines
app/test/test_hash_readwrite.c +1 -1
@@ -298,7 +298,7 @@ test_rw_reader(void *arg)
 
 	begin = rte_rdtsc_precise();
 	for (i = 0; i < read_cnt; i++) {
-		void *data;
+		void *data = arg;

[Wang, Yipeng]
Hi Andrzej, thanks for the fix! Maybe you could initialize the data to be "NULL"? I think it makes more sense to be NULL rather than arg.

#83 lore raw

Re: [dpdk-dev] [PATCH v6 07/12] app/test: clean LTO build warnings (maybe-uninitialized)

Yipeng

Thank you for your comment. See my reply below.

Quoted 1 line · On 11/1/19 6:15 PM, Wang, Yipeng1 wrote:
Quoted 1 line

-----Original Message-----

[...]

Quoted 6 lines
Quoted 6 lines
app/test/test_hash_readwrite.c +0 -0
index 4376b099b..615767fb6 100644
@@ -298,7 +298,7 @@ test_rw_reader(void *arg)
 
Quoted 8 lines

app/test/test_hash_readwrite.c test_rw_reader(void *arg) · line 301

Quoted 4 lines
app/test/test_hash_readwrite.c +1 -1
 	begin = rte_rdtsc_precise();
 	for (i = 0; i < read_cnt; i++) {
-		void *data;
+		void *data = arg;

[Wang, Yipeng]
Hi Andrzej, thanks for the fix! Maybe you could initialize the data to be "NULL"? I think it makes more sense to be NULL rather than arg.

That actually is not a good idea. The conditional test below does not check return value from lookup and expects the returned pointer to data to be equal to the iteration count (these pointers do not point at anything). For the first loop iteration "data = NULL" could miss the problem of the key not being in the hash (key not in the hash, return value not checked but the pointer "data" is NULL as we expected since i = 0).

What I used is a bit hackish - I assumed that arg pointer (which is pointing to the "read_cnt") will not be within [0-read_cnt] address range. If that is too hackish then it would be better to rework this function to actually test for return value of hash lookup. Let me know what you think.

Regards
Andrzej

#84 lore raw

Re: [dpdk-dev] [PATCH v6 07/12] app/test: clean LTO build warnings (maybe-uninitialized)

Quoted 38 lines

-----Original Message-----
From: Andrzej Ostruszka [mailto:[email protected]]
Sent: Monday, November 4, 2019 5:48 AM
To: Wang, Yipeng1 <[email protected]>; [email protected]; Gobriel, Sameh <[email protected]>; Richardson, Bruce <[email protected]>; Chas Williams <[email protected]>; Burakov, Anatoly <[email protected]>
Cc: [email protected]; [email protected]
Subject: Re: [dpdk-dev] [PATCH v6 07/12] app/test: clean LTO build warnings (maybe-uninitialized)

Yipeng

Thank you for your comment. See my reply below.

Quoted 1 line · On 11/1/19 6:15 PM, Wang, Yipeng1 wrote:
Quoted 1 line

-----Original Message-----

[...]

Quoted 6 lines
Quoted 6 lines
app/test/test_hash_readwrite.c +0 -0
index 4376b099b..615767fb6 100644
@@ -298,7 +298,7 @@ test_rw_reader(void *arg)
 
Quoted 8 lines

app/test/test_hash_readwrite.c test_rw_reader(void *arg) · line 301

Quoted 4 lines
app/test/test_hash_readwrite.c +1 -1
 	begin = rte_rdtsc_precise();
 	for (i = 0; i < read_cnt; i++) {
-		void *data;
+		void *data = arg;

[Wang, Yipeng]
Hi Andrzej, thanks for the fix! Maybe you could initialize the data to be "NULL"? I think it makes more sense to be NULL rather than arg.

That actually is not a good idea. The conditional test below does not check return value from lookup and expects the returned pointer to data to be equal to the iteration count (these pointers do not point at anything). For the first loop iteration "data = NULL" could miss the problem of the key not being in the hash (key not in the hash, return value not checked but the pointer "data" is NULL as we expected since i = 0).

Quoted 8 lines

What I used is a bit hackish - I assumed that arg pointer (which is pointing to the "read_cnt") will not be within [0-read_cnt] address range. If that is too hackish then it would be better to rework this function to actually test for return value of hash lookup. Let me know what you think.

Regards
Andrzej

[Wang, Yipeng] Thank you for the explanation. I think what you said makes sense.

#85 lore raw

PATCH v6 8/12 net/dpaa2: fix possible use of uninitialized vars

In reply to Andrzej Ostruszka

This patch fixes 'maybe-uninitialized' warnings reported by compiler when using LTO.

Compiler warning pointing to this error (with LTO enabled):
error: ‘kg_cfg.extracts[0].masks[0].mask’ may be used uninitialized in this function [-Werror=maybe-uninitialized]

    extr->masks[j].mask = cfg->extracts[i].masks[j].mask;
Fixes 16bbc98a3e63 ("bus/fslmc: update MC to 10.3.x")
---
 drivers/net/dpaa2/base/dpaa2_hw_dpni.c | 1 +
 drivers/net/dpaa2/mc/dpkg.c            | 2 +-
 drivers/net/dpaa2/mc/dpni.c            | 9 ++++++---
 3 files changed, 8 insertions(+), 4 deletions(-)
drivers/net/dpaa2/base/dpaa2_hw_dpni.c +1 -0
index 16555d7b0..47a8bda6a 100644
@@ -51,6 +51,7 @@ rte_pmd_dpaa2_set_custom_hash(uint16_t port_id,
 	kg_cfg.extracts[0].type = DPKG_EXTRACT_FROM_DATA;
 	kg_cfg.extracts[0].extract.from_data.offset = offset;
 	kg_cfg.extracts[0].extract.from_data.size = size;
+	kg_cfg.extracts[0].num_of_byte_masks = 0;
 	kg_cfg.num_extracts = 1;
 
 	ret = dpkg_prepare_key_cfg(&kg_cfg, p_params);
drivers/net/dpaa2/mc/dpkg.c +1 -1
index 80f94f40e..7aa63ea12 100644
@@ -63,7 +63,7 @@ dpkg_prepare_key_cfg(const struct dpkg_profile_cfg *cfg, uint8_t *key_cfg_buf)
 		dpkg_set_field(extr->extract_type, EXTRACT_TYPE,
 			       cfg->extracts[i].type);
 
-		for (j = 0; j < DPKG_NUM_OF_MASKS; j++) {
+		for (j = 0; j < extr->num_of_byte_masks; j++) {
 			extr->masks[j].mask = cfg->extracts[i].masks[j].mask;
 			extr->masks[j].offset =
 				cfg->extracts[i].masks[j].offset;
drivers/net/dpaa2/mc/dpni.c +6 -3
index 0950ee007..89a64b0ab 100644
@@ -1839,10 +1839,13 @@ int dpni_set_congestion_notification(struct fsl_mc_io *mc_io,
 	cmd_params->qtype = qtype;
 	cmd_params->tc = tc_id;
 	cmd_params->congestion_point = cfg->cg_point;
-	cmd_params->cgid = (uint8_t)cfg->cgid;
-	cmd_params->dest_id = cpu_to_le32(cfg->dest_cfg.dest_id);
+	if (cfg->cg_point == DPNI_CP_CONGESTION_GROUP)
+		cmd_params->cgid = (uint8_t)cfg->cgid;
+	if (cfg->dest_cfg.dest_type != DPNI_DEST_NONE) {
+		cmd_params->dest_id = cpu_to_le32(cfg->dest_cfg.dest_id);
+		cmd_params->dest_priority = cfg->dest_cfg.priority;
+	}
 	cmd_params->notification_mode = cpu_to_le16(cfg->notification_mode);
-	cmd_params->dest_priority = cfg->dest_cfg.priority;
 	cmd_params->message_iova = cpu_to_le64(cfg->message_iova);
 	cmd_params->message_ctx = cpu_to_le64(cfg->message_ctx);
 	cmd_params->threshold_entry = cpu_to_le32(cfg->threshold_entry);
Signature · 1 line
2.17.1
#86 lore raw

Re: [dpdk-dev] [PATCH v6 08/12] net/dpaa2: fix possible use of uninitialized vars

Hi Andrzej

Quoted 43 lines

-----Original Message-----
From: Andrzej Ostruszka <[email protected]>
Sent: Tuesday, October 29, 2019 7:42 PM
To: [email protected]; Hemant Agrawal <[email protected]>; Sachin Saxena <[email protected]>
Cc: [email protected]; [email protected]
Subject: [PATCH v6 08/12] net/dpaa2: fix possible use of uninitialized vars
Importance: High

This patch fixes 'maybe-uninitialized' warnings reported by compiler when using LTO.

Compiler warning pointing to this error (with LTO enabled):
error: ‘kg_cfg.extracts[0].masks[0].mask’ may be used uninitialized in this function [-Werror=maybe-uninitialized]

    extr->masks[j].mask = cfg->extracts[i].masks[j].mask;
Fixes 16bbc98a3e63 ("bus/fslmc: update MC to 10.3.x")
---
 drivers/net/dpaa2/base/dpaa2_hw_dpni.c | 1 +
 drivers/net/dpaa2/mc/dpkg.c            | 2 +-
 drivers/net/dpaa2/mc/dpni.c            | 9 ++++++---
 3 files changed, 8 insertions(+), 4 deletions(-)
drivers/net/dpaa2/base/dpaa2_hw_dpni.c +1 -0
index 16555d7b0..47a8bda6a 100644
@@ -51,6 +51,7 @@ rte_pmd_dpaa2_set_custom_hash(uint16_t port_id,
 	kg_cfg.extracts[0].type = DPKG_EXTRACT_FROM_DATA;
 	kg_cfg.extracts[0].extract.from_data.offset = offset;
 	kg_cfg.extracts[0].extract.from_data.size = size;
+	kg_cfg.extracts[0].num_of_byte_masks = 0;
 	kg_cfg.num_extracts = 1;
 
 	ret = dpkg_prepare_key_cfg(&kg_cfg, p_params); diff --git

a/drivers/net/dpaa2/mc/dpkg.c b/drivers/net/dpaa2/mc/dpkg.c index 80f94f40e..7aa63ea12 100644

drivers/net/dpaa2/mc/dpkg.c dpkg_prepare_key_cfg(const struct dpkg_profile_cfg · line 66

Quoted 4 lines
drivers/net/dpaa2/mc/dpkg.c +0 -0
@@ -63,7 +63,7 @@ dpkg_prepare_key_cfg(const struct dpkg_profile_cfg
 *cfg, uint8_t *key_cfg_buf)
 		dpkg_set_field(extr->extract_type, EXTRACT_TYPE,
 			       cfg->extracts[i].type);
 

[Hemant] Please add a check here to avoid array overrun
+ if (extr->num_of_byte_masks > DPKG_NUM_OF_MASKS)
+ return -EINVAL;
+

Quoted 14 lines
drivers/net/dpaa2/mc/dpkg.c +1 -1
-		for (j = 0; j < DPKG_NUM_OF_MASKS; j++) {
+		for (j = 0; j < extr->num_of_byte_masks; j++) {
 			extr->masks[j].mask = cfg-
 >extracts[i].masks[j].mask;
drivers/net/dpaa2/mc/dpkg.c +0 -0
 			extr->masks[j].offset =
 				cfg->extracts[i].masks[j].offset;
drivers/net/dpaa2/mc/dpni.c +0 -0
index 0950ee007..89a64b0ab 100644
@@ -1839,10 +1839,13 @@ int dpni_set_congestion_notification(struct
 fsl_mc_io *mc_io,
 	cmd_params->qtype = qtype;
 	cmd_params->tc = tc_id;

drivers/net/dpaa2/mc/dpni.c int dpni_set_congestion_notification(struct · line 1849

Quoted 10 lines
drivers/net/dpaa2/mc/dpni.c +6 -2
 	cmd_params->congestion_point = cfg->cg_point;
-	cmd_params->cgid = (uint8_t)cfg->cgid;
-	cmd_params->dest_id = cpu_to_le32(cfg->dest_cfg.dest_id);
+	if (cfg->cg_point == DPNI_CP_CONGESTION_GROUP)
+		cmd_params->cgid = (uint8_t)cfg->cgid;
+	if (cfg->dest_cfg.dest_type != DPNI_DEST_NONE) {
+		cmd_params->dest_id = cpu_to_le32(cfg-
 >dest_cfg.dest_id);
+		cmd_params->dest_priority = cfg->dest_cfg.priority;
+	}

[Hemant] What is the explicit error you got here?
In the calling function, we are resetting the structure to 0. Which should have avoided Any un-initialized var errors?
struct dpni_congestion_notification_cfg cong_notif_cfg = {0};

Quoted 9 lines
drivers/net/dpaa2/mc/dpni.c +0 -1
 	cmd_params->notification_mode = cpu_to_le16(cfg-
 >notification_mode);
-	cmd_params->dest_priority = cfg->dest_cfg.priority;
drivers/net/dpaa2/mc/dpni.c +0 -0
 	cmd_params->message_iova = cpu_to_le64(cfg->message_iova);
 	cmd_params->message_ctx = cpu_to_le64(cfg->message_ctx);
 	cmd_params->threshold_entry = cpu_to_le32(cfg-
Quoted 1 line

threshold_entry);

--
2.17.1

#87 lore raw

Re: [dpdk-dev] [PATCH v6 08/12] net/dpaa2: fix possible use of uninitialized vars

Quoted 1 line · Hemant Agrawal

Hi Andrzej

Hello Hemant,

Thank you for your comments.

Quoted 2 lines
Quoted 2 lines

-----Original Message-----
From: Andrzej Ostruszka <[email protected]>

[...]

Review

Quoted 13 lines

drivers/net/dpaa2/mc/dpkg.c dpkg_prepare_key_cfg(const struct dpkg_profile_cfg · line 66

Quoted 9 lines

a/drivers/net/dpaa2/mc/dpkg.c b/drivers/net/dpaa2/mc/dpkg.c index 80f94f40e..7aa63ea12 100644

drivers/net/dpaa2/mc/dpkg.c +0 -0
@@ -63,7 +63,7 @@ dpkg_prepare_key_cfg(const struct dpkg_profile_cfg
 *cfg, uint8_t *key_cfg_buf)
 		dpkg_set_field(extr->extract_type, EXTRACT_TYPE,
 			       cfg->extracts[i].type);
 

[Hemant] Please add a check here to avoid array overrun

(quoted diff) +3 -0
+		if (extr->num_of_byte_masks > DPKG_NUM_OF_MASKS)
+			return -EINVAL;
+

Will do, thank you.

Quoted 21 lines
Quoted 21 lines
drivers/net/dpaa2/mc/dpkg.c +1 -1
-		for (j = 0; j < DPKG_NUM_OF_MASKS; j++) {
+		for (j = 0; j < extr->num_of_byte_masks; j++) {
 			extr->masks[j].mask = cfg-
 > extracts[i].masks[j].mask;
drivers/net/dpaa2/mc/dpkg.c +0 -0
 			extr->masks[j].offset =
 				cfg->extracts[i].masks[j].offset;
drivers/net/dpaa2/mc/dpni.c +4 -2
index 0950ee007..89a64b0ab 100644
@@ -1839,10 +1839,13 @@ int dpni_set_congestion_notification(struct
 fsl_mc_io *mc_io,
 	cmd_params->qtype = qtype;
 	cmd_params->tc = tc_id;
 	cmd_params->congestion_point = cfg->cg_point;
-	cmd_params->cgid = (uint8_t)cfg->cgid;
-	cmd_params->dest_id = cpu_to_le32(cfg->dest_cfg.dest_id);
+	if (cfg->cg_point == DPNI_CP_CONGESTION_GROUP)
+		cmd_params->cgid = (uint8_t)cfg->cgid;
+	if (cfg->dest_cfg.dest_type != DPNI_DEST_NONE) {
+		cmd_params->dest_id = cpu_to_le32(cfg-
Quoted 7 lines

drivers/net/dpaa2/mc/dpni.c int dpni_set_congestion_notification(struct fsl_mc_io *mc_io, · line 1847

Quoted 3 lines
drivers/net/dpaa2/mc/dpni.c +2 -0
 > dest_cfg.dest_id);
+		cmd_params->dest_priority = cfg->dest_cfg.priority;
+	}

[Hemant] What is the explicit error you got here?
In the calling function, we are resetting the structure to 0. Which should have avoided Any un-initialized var errors?
struct dpni_congestion_notification_cfg cong_notif_cfg = {0};

Indeed there is no warning from compiler here at the current version. The original version was developed against 18.05 and this is what I used to make compiler happy. I've kept this change (even though compiler does not report anything here) since to me it looked like a good change (use the cgid only in case of "congestion group" and use dest_* only when not DEST_NONE).

You have obviously more authority here so I will remove these changes (unless you prefer them to stay - in which case please indicate so).

Again thank you for the comments.

Regards
Andrzej

#88 lore raw

PATCH v6 9/12 net/e1000: clean LTO build warnings (maybe-uninitialized)

In reply to Andrzej Ostruszka

During LTO build compiler reports some 'false positive' warnings about variables being possibly used uninitialized. This patch silences these warnings.

Exemplary compiler warning to suppress (with LTO enabled):
error: ‘link’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
if (link) {

---
 drivers/net/e1000/base/e1000_82543.c   | 2 +-
 drivers/net/e1000/base/e1000_ich8lan.c | 2 +-
 drivers/net/e1000/base/e1000_phy.c     | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)
drivers/net/e1000/base/e1000_82543.c +1 -1
index 810899eb6..dfde2a8b9 100644
@@ -1027,7 +1027,7 @@ STATIC s32 e1000_setup_copper_link_82543(struct e1000_hw *hw)
 {
 	u32 ctrl;
 	s32 ret_val;
-	bool link;
+	bool link = true;
 
 	DEBUGFUNC("e1000_setup_copper_link_82543");
 
drivers/net/e1000/base/e1000_ich8lan.c +1 -1
index accc6ea01..5241cf698 100644
@@ -5533,7 +5533,7 @@ void e1000_igp3_phy_powerdown_workaround_ich8lan(struct e1000_hw *hw)
 void e1000_gig_downshift_workaround_ich8lan(struct e1000_hw *hw)
 {
 	s32 ret_val;
-	u16 reg_data;
+	u16 reg_data = 0;
 
 	DEBUGFUNC("e1000_gig_downshift_workaround_ich8lan");
 
drivers/net/e1000/base/e1000_phy.c +1 -1
index 7d08f836f..956c06747 100644
@@ -1664,7 +1664,7 @@ s32 e1000_copper_link_autoneg(struct e1000_hw *hw)
 s32 e1000_setup_copper_link_generic(struct e1000_hw *hw)
 {
 	s32 ret_val;
-	bool link;
+	bool link = true;
 
 	DEBUGFUNC("e1000_setup_copper_link_generic");
 
Signature · 1 line
2.17.1
#89 lore raw

PATCH v6 10/12 net/i40e: clean LTO build warnings (maybe-uninitialized)

In reply to Andrzej Ostruszka

During LTO build compiler reports some 'false positive' warnings about variables being possibly used uninitialized. This patch silences these warnings.

Exemplary compiler warning to suppress (with LTO enabled):
error: ‘filter_idx’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
PMD_DRV_LOG(INFO, "Added port %d with AQ command with index %d",

---
 drivers/net/i40e/i40e_ethdev.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/i40e/i40e_ethdev.c +1 -1
index 77a46832c..91d529b2f 100644
@@ -8480,7 +8480,7 @@ static int
 i40e_add_vxlan_port(struct i40e_pf *pf, uint16_t port, int udp_type)
 {
 	int  idx, ret;
-	uint8_t filter_idx;
+	uint8_t filter_idx = 0;
 	struct i40e_hw *hw = I40E_PF_TO_HW(pf);
 
 	idx = i40e_get_vxlan_port_idx(pf, port);
Signature · 1 line
2.17.1
#90 lore raw

Re: [dpdk-dev] [PATCH v6 10/12] net/i40e: clean LTO build warnings (maybe-uninitialized)

Quoted 11 lines

-----Original Message-----
From: Andrzej Ostruszka [mailto:[email protected]]
Sent: Tuesday, October 29, 2019 10:12 PM
To: [email protected]; Xing, Beilei <[email protected]>; Zhang, Qi Z <[email protected]>
Cc: [email protected]; [email protected]
Subject: [PATCH v6 10/12] net/i40e: clean LTO build warnings (maybe- uninitialized)

Quoted 7 lines

During LTO build compiler reports some 'false positive' warnings about variables being possibly used uninitialized. This patch silences these warnings.

Exemplary compiler warning to suppress (with LTO enabled):
error: ‘filter_idx’ may be used uninitialized in this function [-Werror=maybe- uninitialized]
PMD_DRV_LOG(INFO, "Added port %d with AQ command with index %d",

Hi,

Thanks for the patch. Please also add fix line here and Cc stable.

Quoted 20 lines
---
 drivers/net/i40e/i40e_ethdev.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/i40e/i40e_ethdev.c +1 -2
index 77a46832c..91d529b2f 100644
@@ -8480,7 +8480,7 @@ static int
 i40e_add_vxlan_port(struct i40e_pf *pf, uint16_t port, int udp_type)  {
 	int  idx, ret;
-	uint8_t filter_idx;
+	uint8_t filter_idx = 0;
 	struct i40e_hw *hw = I40E_PF_TO_HW(pf);
 
 	idx = i40e_get_vxlan_port_idx(pf, port);
--

2.17.1

#91 lore raw

Re: [dpdk-dev] [PATCH v6 10/12] net/i40e: clean LTO build warnings (maybe-uninitialized)

Beilei

Thank you for the comment, please see below.

Quoted 2 lines · Beilei
Quoted 2 lines

-----Original Message-----
From: Andrzej Ostruszka [mailto:[email protected]]

[...]

Quoted 6 lines
Quoted 6 lines

Subject: [PATCH v6 10/12] net/i40e: clean LTO build warnings (maybe- uninitialized)

During LTO build compiler reports some 'false positive' warnings about variables being possibly used uninitialized. This patch silences these warnings.

Quoted 8 lines
Quoted 4 lines

Exemplary compiler warning to suppress (with LTO enabled):
error: ‘filter_idx’ may be used uninitialized in this function [-Werror=maybe- uninitialized]
PMD_DRV_LOG(INFO, "Added port %d with AQ command with index %d",

Hi,

Thanks for the patch. Please also add fix line here and Cc stable.

Are you saying that there is actually a problem in the code?

IMHO the code is fine and this patch just prevents gcc from issuing a false warning. What I think is going on is that gcc looks into i40e_aq_add_udp_tunnel() and sees that filter_idx is updated under condition "if (!status ..." and figures out that if status is positive then filter_idx could be used uninitialized because in the calling function we test "if (ret < 0)". However it fails to take into account the actual enum definition (only non-positive values) and the implementation of i40e_asq_send_command().

So, if this is only work-around for the gcc deficiency, then I would prefer to not add any fixline and to not pull that patch into stable. Please let me know what you think.

Regards
Andrzej

#92 lore raw

PATCH v6 11/12 net/ifc: clean LTO build warnings (maybe-uninitialized)

In reply to Andrzej Ostruszka

During LTO build compiler reports some 'false positive' warnings about variables being possibly used uninitialized. This patch silences these warnings.

Exemplary compiler warning to suppress (with LTO enabled):
error: ‘features’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
if (RTE_VHOST_NEED_LOG(features)) {

---
 drivers/net/ifc/ifcvf_vdpa.c | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)
drivers/net/ifc/ifcvf_vdpa.c +10 -4
index 8de9ef199..9c562def0 100644
@@ -5,6 +5,7 @@
 #include <unistd.h>
 #include <pthread.h>
 #include <fcntl.h>
+#include <string.h>
 #include <sys/ioctl.h>
 #include <sys/epoll.h>
 #include <linux/virtio_net.h>
@@ -304,8 +305,8 @@ vdpa_ifcvf_stop(struct ifcvf_internal *internal)
 	struct ifcvf_hw *hw = &internal->hw;
 	uint32_t i;
 	int vid;
-	uint64_t features;
-	uint64_t log_base, log_size;
+	uint64_t features = 0;
+	uint64_t log_base = 0, log_size = 0;
 	uint64_t len;
 
 	vid = internal->vid;
@@ -348,6 +349,8 @@ vdpa_enable_vfio_intr(struct ifcvf_internal *internal, bool m_rx)
 	struct rte_vhost_vring vring;
 	int fd;
 
+	vring.callfd = -1;
+
 	nr_vring = rte_vhost_get_vring_num(internal->vid);
 
 	irq_set = (struct vfio_irq_set *)irq_set_buf;
@@ -442,6 +445,7 @@ notify_relay(void *arg)
 	}
 	internal->epfd = epfd;
 
+	vring.kickfd = -1;
 	for (qid = 0; qid < q_num; qid++) {
 		ev.events = EPOLLIN | EPOLLPRI;
 		rte_vhost_get_vhost_vring(internal->vid, qid, &vring);
@@ -583,6 +587,7 @@ m_ifcvf_start(struct ifcvf_internal *internal)
 	uint64_t size;
 	uint64_t gpa;
 
+	memset(&vq, 0, sizeof(vq));
 	vid = internal->vid;
 	nr_vring = rte_vhost_get_vring_num(vid);
 	rte_vhost_get_negotiated_features(vid, &hw->req_features);
@@ -721,6 +726,7 @@ vring_relay(void *arg)
 	}
 	internal->epfd = epfd;
 
+	vring.kickfd = -1;
 	for (qid = 0; qid < q_num; qid++) {
 		ev.events = EPOLLIN | EPOLLPRI;
 		rte_vhost_get_vhost_vring(vid, qid, &vring);
@@ -930,11 +936,11 @@ ifcvf_dev_close(int vid)
 static int
 ifcvf_set_features(int vid)
 {
-	uint64_t features;
+	uint64_t features = 0;
 	int did;
 	struct internal_list *list;
 	struct ifcvf_internal *internal;
-	uint64_t log_base, log_size;
+	uint64_t log_base = 0, log_size = 0;
 
 	did = rte_vhost_get_vdpa_device_id(vid);
 	list = find_internal_resource_by_did(did);
Signature · 1 line
2.17.1
#93 lore raw

PATCH v6 12/12 net/qede: clean LTO build warnings (maybe-uninitialized)

In reply to Andrzej Ostruszka

During LTO build compiler reports some 'false positive' warnings about variables being possibly used uninitialized. This patch silences these warnings.

Exemplary compiler warning to suppress (with LTO enabled):
error: ‘transceiver_type’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
switch (transceiver_type) {

---
 drivers/net/qede/base/ecore_mcp.c | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)
drivers/net/qede/base/ecore_mcp.c +8 -7
index a5aa07438..7518765a0 100644
@@ -2328,7 +2328,7 @@ enum _ecore_status_t ecore_mcp_trans_speed_mask(struct ecore_hwfn *p_hwfn,
 						struct ecore_ptt *p_ptt,
 						u32 *p_speed_mask)
 {
-	u32 transceiver_type, transceiver_state;
+	u32 transceiver_type = ETH_TRANSCEIVER_TYPE_NONE, transceiver_state;
 
 	ecore_mcp_get_transceiver_data(p_hwfn, p_ptt, &transceiver_state,
 				       &transceiver_type);
@@ -3223,7 +3223,8 @@ enum _ecore_status_t ecore_mcp_nvm_put_file_begin(struct ecore_dev *p_dev,
 enum _ecore_status_t ecore_mcp_nvm_write(struct ecore_dev *p_dev, u32 cmd,
 					 u32 addr, u8 *p_buf, u32 len)
 {
-	u32 buf_idx, buf_size, nvm_cmd, nvm_offset, resp, param;
+	u32 buf_idx, buf_size, nvm_cmd, nvm_offset;
+	u32 resp = FW_MSG_CODE_ERROR, param;
 	struct ecore_hwfn *p_hwfn = ECORE_LEADING_HWFN(p_dev);
 	enum _ecore_status_t rc = ECORE_INVAL;
 	struct ecore_ptt *p_ptt;
@@ -3322,7 +3323,7 @@ enum _ecore_status_t ecore_mcp_nvm_set_secure_mode(struct ecore_dev *p_dev,
 {
 	struct ecore_hwfn *p_hwfn = ECORE_LEADING_HWFN(p_dev);
 	struct ecore_ptt *p_ptt;
-	u32 resp, param;
+	u32 resp = 0, param;
 	enum _ecore_status_t rc;
 
 	p_ptt = ecore_ptt_acquire(p_hwfn);
@@ -3430,7 +3431,7 @@ enum _ecore_status_t ecore_mcp_gpio_read(struct ecore_hwfn *p_hwfn,
 					 u16 gpio, u32 *gpio_val)
 {
 	enum _ecore_status_t rc = ECORE_SUCCESS;
-	u32 drv_mb_param = 0, rsp;
+	u32 drv_mb_param = 0, rsp = 0;
 
 	drv_mb_param = (gpio << DRV_MB_PARAM_GPIO_NUMBER_OFFSET);
 
@@ -3451,7 +3452,7 @@ enum _ecore_status_t ecore_mcp_gpio_write(struct ecore_hwfn *p_hwfn,
 					  u16 gpio, u16 gpio_val)
 {
 	enum _ecore_status_t rc = ECORE_SUCCESS;
-	u32 drv_mb_param = 0, param, rsp;
+	u32 drv_mb_param = 0, param, rsp = 0;
 
 	drv_mb_param = (gpio << DRV_MB_PARAM_GPIO_NUMBER_OFFSET) |
 		(gpio_val << DRV_MB_PARAM_GPIO_VALUE_OFFSET);
@@ -3541,7 +3542,7 @@ enum _ecore_status_t ecore_mcp_bist_clock_test(struct ecore_hwfn *p_hwfn,
 enum _ecore_status_t ecore_mcp_bist_nvm_test_get_num_images(
 	struct ecore_hwfn *p_hwfn, struct ecore_ptt *p_ptt, u32 *num_images)
 {
-	u32 drv_mb_param = 0, rsp;
+	u32 drv_mb_param = 0, rsp = 0;
 	enum _ecore_status_t rc = ECORE_SUCCESS;
 
 	drv_mb_param = (DRV_MB_PARAM_BIST_NVM_TEST_NUM_IMAGES <<
@@ -3925,7 +3926,7 @@ enum _ecore_status_t
 __ecore_mcp_resc_lock(struct ecore_hwfn *p_hwfn, struct ecore_ptt *p_ptt,
 		      struct ecore_resc_lock_params *p_params)
 {
-	u32 param = 0, mcp_resp, mcp_param;
+	u32 param = 0, mcp_resp = 0, mcp_param = 0;
 	u8 opcode;
 	enum _ecore_status_t rc;
 
Signature · 1 line
2.17.1
#94 lore raw
In reply to Andrzej Ostruszka
Quoted 2 lines · Andrzej Ostruszka

This patch series adds an option to make use of link time optimization (if compiler has support for it).

[...]

Quoted 1 line
 .travis.yml                                   |  7 ++++

[...]

Aaron, Michael, all

I'd probably need some assistance with Travis. This patchset added following changes:

.travis.yml +7 -0
index 3d6ef2959..3cd746dba 100644
@@ -34,6 +34,7 @@ env:
   - DEF_LIB="static" OPTS="-Denable_kmods=false"
   - DEF_LIB="shared" OPTS="-Denable_kmods=false"
   - DEF_LIB="shared" RUN_TESTS=1
+  - DEF_LIB="shared" OPTS="-Db_lto=true"
 
 matrix:
   include:
@@ -105,6 +106,12 @@ matrix:
       apt:
         packages:
           - *extra_packages
+  - env: DEF_LIB="shared" OPTS="-Db_lto=true" EXTRA_PACKAGES=1
+    compiler: gcc
+    addons:
+      apt:
+        packages:
+          - *extra_packages
 
 

however I have to admit I'm not familiar with Travis and the actual meaning of that configuration (that is it's actual mapping to builds). I'm getting following CI errors:

1. https://travis-ci.com/ovsrobot/dpdk/jobs/250599578

This is with clang - this patchset should not be run with clang as it is not supported. In build "matrix" there is 'gcc' specified but somehow clang build is attempted. Please advice me how to change this .travis.yml to stop that - should I remove entry in "env"?

2. https://travis-ci.com/ovsrobot/dpdk/jobs/250599577
https://travis-ci.com/ovsrobot/dpdk/jobs/250599591

This is with gcc and meson with shared library and I can't reproduce this. The are two problems reported, first a compiler warning which seems to be not connected at all with the changes:

--8<------------------------ /usr/include/x86_64-linux-gnu/bits/unistd.h:39:9: warning: call to ‘__read_chk_warn’ declared with attribute warning: read called with bigger length than size of the destination buffer

  return __read_chk (__fd, __buf, __nbytes, __bos0 (__buf));
         ^

In function ‘__read_alias’,
inlined from ‘eal_intr_process_interrupts’ at
../lib/librte_eal/linux/eal/eal_interrupts.c:911:17,
inlined from ‘eal_intr_handle_interrupts’ at
../lib/librte_eal/linux/eal/eal_interrupts.c:1030:7,
inlined from ‘eal_intr_thread_main’ at
../lib/librte_eal/linux/eal/eal_interrupts.c:1100:3:
--8<------------------------

and a second linker error:

--8<------------------------
FAILED: gcc -o lib/librte_lpm.so.2.1 'lib/76b5a35@@rte_lpm@sha/librte_lpm_rte_lpm.c.o' 'lib/76b5a35@@rte_lpm@sha/librte_lpm_rte_lpm6.c.o' -flto -Wl,--as-needed -Wl,--no-undefined -Wl,-O1 -shared -fPIC -Wl,--start-group -Wl,-soname,librte_lpm.so.2 -Wl,--no-as-needed -pthread -lm -ldl -lnuma -Wno-lto-type-mismatch lib/librte_eal.so.12.1 lib/librte_kvargs.so.1.1 lib/librte_hash.so.2.1 lib/librte_ring.so.2.1 -Wl,--version-script=/home/travis/build/ovsrobot/dpdk/lib/librte_lpm/rte_lpm_version.map /usr/lib/x86_64-linux-gnu/libbsd.so -Wl,--end-group
'-Wl,-rpath,$ORIGIN/' -Wl,-rpath-link,/home/travis/build/ovsrobot/dpdk/build/lib /tmp/ccZPEQKf.ltrans3.ltrans.o: In function `rte_lpm6_delete_bulk_func':
<artificial>:(.text+0xf01): undefined reference to `rte_lpm6_add' --8<------------------------

As I've said I can't reproduce none of these problems - are they some CI issues?

I'd appreciate some help with these.

Regards
Andrzej

#95 lore raw
Quoted 40 lines · Andrzej Ostruszka
Quoted 2 lines · Andrzej Ostruszka

This patch series adds an option to make use of link time optimization (if compiler has support for it).

[...]

Quoted 1 line
 .travis.yml                                   |  7 ++++

[...]

Aaron, Michael, all

I'd probably need some assistance with Travis. This patchset added following changes:

.travis.yml +7 -0
index 3d6ef2959..3cd746dba 100644
@@ -34,6 +34,7 @@ env:
   - DEF_LIB="static" OPTS="-Denable_kmods=false"
   - DEF_LIB="shared" OPTS="-Denable_kmods=false"
   - DEF_LIB="shared" RUN_TESTS=1
+  - DEF_LIB="shared" OPTS="-Db_lto=true"
 
 matrix:
   include:
@@ -105,6 +106,12 @@ matrix:
       apt:
         packages:
           - *extra_packages
+  - env: DEF_LIB="shared" OPTS="-Db_lto=true" EXTRA_PACKAGES=1
+    compiler: gcc
+    addons:
+      apt:
+        packages:
+          - *extra_packages
 
 

however I have to admit I'm not familiar with Travis and the actual meaning of that configuration (that is it's actual mapping to builds). I'm getting following CI errors:

Quoted 6 lines

1. https://travis-ci.com/ovsrobot/dpdk/jobs/250599578

This is with clang - this patchset should not be run with clang as it is not supported. In build "matrix" there is 'gcc' specified but somehow clang build is attempted. Please advice me how to change this .travis.yml to stop that - should I remove entry in "env"?

Yes, do not use the non-matrix defines.

IE: drop the hunk '@@ -34,6 +34,7 @@ env:'

The other hunk '@@ -105,6 +106,12 @@ matrix:' is sufficient. You might want a secondary copy that includes AARCH64=1 if fat LTO is supported under ARM64 (assuming there's a benefit to doing this build).

Quoted 6 lines

2. https://travis-ci.com/ovsrobot/dpdk/jobs/250599577
https://travis-ci.com/ovsrobot/dpdk/jobs/250599591

This is with gcc and meson with shared library and I can't reproduce this. The are two problems reported, first a compiler warning which seems to be not connected at all with the changes:

These jobs are running on ubuntu xenial, using gcc 5.4.0 compiler.

Are you sure that you've passed all of the requisite linker flags for this version of gcc?

Alternatively, you can make a package list that will include a version of the compiler you expect to support. But that should also be reflected in the documentation. I didn't see any compiler restrictions documented.

Quoted 33 lines

--8<------------------------ /usr/include/x86_64-linux-gnu/bits/unistd.h:39:9: warning: call to ‘__read_chk_warn’ declared with attribute warning: read called with bigger length than size of the destination buffer

  return __read_chk (__fd, __buf, __nbytes, __bos0 (__buf));
         ^

In function ‘__read_alias’,
inlined from ‘eal_intr_process_interrupts’ at
../lib/librte_eal/linux/eal/eal_interrupts.c:911:17,
inlined from ‘eal_intr_handle_interrupts’ at
../lib/librte_eal/linux/eal/eal_interrupts.c:1030:7,
inlined from ‘eal_intr_thread_main’ at
../lib/librte_eal/linux/eal/eal_interrupts.c:1100:3:
--8<------------------------

and a second linker error:

--8<------------------------
FAILED: gcc -o lib/librte_lpm.so.2.1 'lib/76b5a35@@rte_lpm@sha/librte_lpm_rte_lpm.c.o' 'lib/76b5a35@@rte_lpm@sha/librte_lpm_rte_lpm6.c.o' -flto -Wl,--as-needed

.travis.yml +0 -3
-Wl,--no-undefined -Wl,-O1 -shared -fPIC -Wl,--start-group
-Wl,-soname,librte_lpm.so.2 -Wl,--no-as-needed -pthread -lm -ldl -lnuma
-Wno-lto-type-mismatch lib/librte_eal.so.12.1 lib/librte_kvargs.so.1.1

lib/librte_hash.so.2.1 lib/librte_ring.so.2.1 -Wl,--version-script=/home/travis/build/ovsrobot/dpdk/lib/librte_lpm/rte_lpm_version.map /usr/lib/x86_64-linux-gnu/libbsd.so -Wl,--end-group
'-Wl,-rpath,$ORIGIN/' -Wl,-rpath-link,/home/travis/build/ovsrobot/dpdk/build/lib /tmp/ccZPEQKf.ltrans3.ltrans.o: In function `rte_lpm6_delete_bulk_func':
<artificial>:(.text+0xf01): undefined reference to `rte_lpm6_add' --8<------------------------

Quoted 2 lines

As I've said I can't reproduce none of these problems - are they some CI issues?

Can you try to reproduce this with a stock xenial image? If I get the chance, I will do the same.

Quoted 4 lines

I'd appreciate some help with these.

Regards
Andrzej

#96 lore raw

v7 cover Add an option to use LTO for DPDK build

In reply to Andrzej Ostruszka

This patch series adds an option to make use of link time optimization (if compiler has support for it). It is split as follows:
- 1st patch is just a minor doc fix for versioning macros
- 2nd patch fixes missing __vsym annotations (needed for LTO)
- 3rd is the LTO enablement
- remaining patches are fixes for the warnings produced by the compiler and they are split by directory/subsystem so their maintainers can easily find and verify the changes - please note that there are two groups:
* errors (or possible errors) - with title "fix possible use ..."
* false positives - warnings that _I_ think are not valid and the changes are made only to silence the compiler.

v7 Changes:
-----------
- dpaa2:
- reverted some changes in dpaa2 that were not required to silence gcc warning
- added extra check to avoid possible array overrun
- updated travis:
- removed entry in "env"
- use gcc-7 to make build with LTO:
this is because in reproduced CI environment only gcc-7 package from ubuntu-toolchain-r/test PPA could finish the build without any warnings
- change the gcc version to 7.4 in rte.toolchain-compat.mk

v6 Changes:
-----------
- yet another split of the patch: this time from __vsym one minor doc fix to versioning macros has been moved to another patch
- net/ifc - fixed initialization of vring in m_ifcvf_start():
after rebase definition rte_vhost_vring has changed and now simple '{ 0 }' produces proper warning - however instead of '{ { 0 } }' I went with memset() because I could not get gcc to be quiet about "missing initializer" (IMHO "{{0}}" is a correct initialization but gcc required me to spell out initializers to all members which looked ugly and would require update on further changes to vring)
- I've been suggested (offlist) to mark the patches that were previously "reviewed/acked" so I've added some "Reviewed|Acked-by" tags (if the ack was sent to a patch that was later split the tag is placed in both patches).

v5 Changes:
-----------
- rebased to the v19.11 (one additional warning silenced in net/qede)
- initial patch has been split into two:
- one adding '__vsym' annotations and clarifying versioning docs
- another adding the actual LTO enablement
- addressed various documentation flaws pointed during review

v4 Changes:
-----------
- merge nested conditionals in config/meson.build into one

v3 Changes:
-----------
- removed support for clang (only gcc and icc remain):
- it turned out that 'fat-lto-objects' is not really supported - it is accepted as a flag but ignored e.g. clang v6.0:
clang: error: optimization flag '-ffat-lto-objects' is not supported [-Werror,-Wignored-optimization-argument]
this was probably masked previously by playing around with meson and testing for the availability of this flag and dropping LTO if not supported.
- improved commit messages:
- added 'Fixes' tags where appropriate
- included snippets of compiler warnings
- changed commit names for the commits cleaning up 'false positive' compiler warnings - these are not really 'fixes' for the code, just workarounds for compiler shortcomings and check-git-log.sh was complaining about missing 'Fixes' tag.

v2 Changes:
-----------
- fixed building of shared libraries:
- gcc does not scan top level assembler statements so it missed that function implementations for specific versions were being used and was removing them
- fixed meson.build config files:
- moved from 'enable_lto' project option to built-in 'b_lto'
- documented that 'default_library' should be 'shared':
with 'default_library=static' (the default) the meson build with LTO is broken - this is because, SHARED_LIB is fixed in rte_config.h used by meson which leads to no alias for default version in generated static libraries (MAP_STATIC_SYMBOL() is empty)
- app/test: added log for failed bonding "config get"

Andrzej Ostruszka (12):
doc: fix description of versioning macros
build: annotate versioned symbols with __vsym macro
build: add an option to enable LTO build
eventdev: fix possible use of uninitialized var app/eventdev: clean LTO build warnings (maybe-uninitialized) event/octeontx2: clean LTO build warnings (maybe-uninitialized) app/test: clean LTO build warnings (maybe-uninitialized) net/dpaa2: fix possible use of uninitialized vars net/e1000: clean LTO build warnings (maybe-uninitialized) net/i40e: clean LTO build warnings (maybe-uninitialized) net/ifc: clean LTO build warnings (maybe-uninitialized) net/qede: clean LTO build warnings (maybe-uninitialized)

 .travis.yml                                   |  9 +++++
 app/test-eventdev/test_perf_common.c          |  2 +-
 app/test-eventdev/test_pipeline_common.c      |  4 +-
 app/test/test_hash_readwrite.c                |  2 +-
 app/test/test_link_bonding_mode4.c            |  8 +++-
 app/test/test_memzone.c                       |  3 +-
 config/common_base                            |  5 +++
 config/meson.build                            | 13 ++++++
 doc/guides/contributing/versioning.rst        | 18 ++++++---
 doc/guides/prog_guide/index.rst               |  1 +
 doc/guides/prog_guide/lto.rst                 | 40 +++++++++++++++++++
 doc/guides/rel_notes/release_19_11.rst        |  9 +++++
 drivers/event/octeontx2/otx2_tim_worker.h     |  2 +-
 drivers/net/dpaa2/base/dpaa2_hw_dpni.c        |  1 +
 drivers/net/dpaa2/mc/dpkg.c                   |  5 ++-
 drivers/net/e1000/base/e1000_82543.c          |  2 +-
 drivers/net/e1000/base/e1000_ich8lan.c        |  2 +-
 drivers/net/e1000/base/e1000_phy.c            |  2 +-
 drivers/net/i40e/i40e_ethdev.c                |  2 +-
 drivers/net/ifc/ifcvf_vdpa.c                  | 14 +++++--
 drivers/net/qede/base/ecore_mcp.c             | 15 +++----
 lib/librte_distributor/rte_distributor.c      | 18 ++++-----
 lib/librte_distributor/rte_distributor_v20.c  | 18 ++++-----
 .../common/include/rte_function_versioning.h  | 11 ++++-
 lib/librte_eventdev/rte_event_timer_adapter.c |  8 ++--
 lib/librte_lpm/rte_lpm.c                      | 28 ++++++-------
 lib/librte_lpm/rte_lpm6.c                     | 16 ++++----
 lib/librte_timer/rte_timer.c                  | 20 +++++-----
 mk/toolchain/gcc/rte.toolchain-compat.mk      |  4 ++
 mk/toolchain/gcc/rte.vars.mk                  | 12 ++++++
 mk/toolchain/icc/rte.vars.mk                  |  8 ++++
 31 files changed, 217 insertions(+), 85 deletions(-)
 create mode 100644 doc/guides/prog_guide/lto.rst
Signature · 1 line
2.17.1
#97 lore raw

PATCH v7 1/12 doc: fix description of versioning macros

This patch fixes documentation of versioning macros so that they are aligned with their implementation (no underscore is added by macros).

Fixes f1ef9794f9bd ("doc: add ABI guidelines")
---
 doc/guides/contributing/versioning.rst                  | 4 ++--
 lib/librte_eal/common/include/rte_function_versioning.h | 4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)
doc/guides/contributing/versioning.rst +2 -2
index 64984c54e..8a38928c0 100644
@@ -215,11 +215,11 @@ library so that older binaries need not be immediately recompiled.
 The macros exported are:
 
 * ``VERSION_SYMBOL(b, e, n)``: Creates a symbol version table entry binding
-  versioned symbol ``b@DPDK_n`` to the internal function ``b_e``.
+  versioned symbol ``b@DPDK_n`` to the internal function ``be``.
 
 * ``BIND_DEFAULT_SYMBOL(b, e, n)``: Creates a symbol version entry instructing
   the linker to bind references to symbol ``b`` to the internal symbol
-  ``b_e``.
+  ``be``.
 
 * ``MAP_STATIC_SYMBOL(f, p)``: Declare the prototype ``f``, and map it to the
   fully qualified function ``p``, so that if a symbol becomes versioned, it
lib/librte_eal/common/include/rte_function_versioning.h +2 -2
index 55e88ffae..eae619d60 100644
@@ -42,14 +42,14 @@
 /*
  * VERSION_SYMBOL
  * Creates a symbol version table entry binding symbol <b>@DPDK_<n> to the internal
- * function name <b>_<e>
+ * function name <b><e>
  */
 #define VERSION_SYMBOL(b, e, n) __asm__(".symver " RTE_STR(b) RTE_STR(e) ", " RTE_STR(b) "@DPDK_" RTE_STR(n))
 
 /*
  * BIND_DEFAULT_SYMBOL
  * Creates a symbol version entry instructing the linker to bind references to
- * symbol <b> to the internal symbol <b>_<e>
+ * symbol <b> to the internal symbol <b><e>
  */
 #define BIND_DEFAULT_SYMBOL(b, e, n) __asm__(".symver " RTE_STR(b) RTE_STR(e) ", " RTE_STR(b) "@@DPDK_" RTE_STR(n))
 #define __vsym __attribute__((used))
Signature · 1 line
2.17.1
#98 lore raw

PATCH v7 2/12 build: annotate versioned symbols with __vsym macro

In reply to Andrzej Ostruszka

Every implementation of a particular version of given symbol needs to be marked in its declaration as such (using __vsym macro). This patch fixes this and also clarifies the documentation about that.

---
 doc/guides/contributing/versioning.rst        | 14 +++++++---
 lib/librte_distributor/rte_distributor.c      | 18 ++++++------
 lib/librte_distributor/rte_distributor_v20.c  | 18 ++++++------
 .../common/include/rte_function_versioning.h  |  7 +++++
 lib/librte_lpm/rte_lpm.c                      | 28 +++++++++----------
 lib/librte_lpm/rte_lpm6.c                     | 16 +++++------
 lib/librte_timer/rte_timer.c                  | 20 ++++++-------
 7 files changed, 67 insertions(+), 54 deletions(-)
doc/guides/contributing/versioning.rst +10 -4
index 8a38928c0..fcd2d50f1 100644
@@ -225,6 +225,10 @@ The macros exported are:
   fully qualified function ``p``, so that if a symbol becomes versioned, it
   can still be mapped back to the public symbol name.
 
+* ``__vsym``:  Annotation to be used in a declaration of the internal symbol
+  ``be`` to signal that it is being used as an implementation of a particular
+  version of symbol ``b``.
+
 Examples of ABI Macro use
 ^^^^^^^^^^^^^^^^^^^^^^^^^
 
@@ -345,8 +349,9 @@ with the public symbol name
 
 .. code-block:: c
 
-  struct rte_acl_ctx *
+ -struct rte_acl_ctx *
  -rte_acl_create(const struct rte_acl_param *param)
+ +struct rte_acl_ctx * __vsym
  +rte_acl_create_v20(const struct rte_acl_param *param)
  {
         size_t sz;
@@ -354,7 +359,8 @@ with the public symbol name
         ...
 
 Note that the base name of the symbol was kept intact, as this is conducive to
-the macros used for versioning symbols.  That is our next step, mapping this new
+the macros used for versioning symbols and we have annotated the function as an
+implementation of versioned symbol.  That is our next step, mapping this new
 symbol name to the initial symbol name at version node 2.0.  Immediately after
 the function, we add this line of code
 
@@ -374,7 +380,7 @@ name, with a different suffix, and  implement it appropriately
 
 .. code-block:: c
 
-   struct rte_acl_ctx *
+   struct rte_acl_ctx * __vsym
    rte_acl_create_v21(const struct rte_acl_param *param, int debug);
    {
         struct rte_acl_ctx *ctx = rte_acl_create_v20(param);
@@ -423,7 +429,7 @@ defined, we add this
 
 .. code-block:: c
 
-   struct rte_acl_ctx *
+   struct rte_acl_ctx * __vsym
    rte_acl_create_v21(const struct rte_acl_param *param, int debug)
    {
         ...
lib/librte_distributor/rte_distributor.c +9 -9
index 7df97df92..2cc32ddba 100644
@@ -32,7 +32,7 @@ EAL_REGISTER_TAILQ(rte_dist_burst_tailq)
 
 /**** Burst Packet APIs called by workers ****/
 
-void
+void __vsym
 rte_distributor_request_pkt_v1705(struct rte_distributor *d,
 		unsigned int worker_id, struct rte_mbuf **oldpkt,
 		unsigned int count)
@@ -89,7 +89,7 @@ MAP_STATIC_SYMBOL(void rte_distributor_request_pkt(struct rte_distributor *d,
 		unsigned int count),
 		rte_distributor_request_pkt_v1705);
 
-int
+int __vsym
 rte_distributor_poll_pkt_v1705(struct rte_distributor *d,
 		unsigned int worker_id, struct rte_mbuf **pkts)
 {
@@ -134,7 +134,7 @@ MAP_STATIC_SYMBOL(int rte_distributor_poll_pkt(struct rte_distributor *d,
 		unsigned int worker_id, struct rte_mbuf **pkts),
 		rte_distributor_poll_pkt_v1705);
 
-int
+int __vsym
 rte_distributor_get_pkt_v1705(struct rte_distributor *d,
 		unsigned int worker_id, struct rte_mbuf **pkts,
 		struct rte_mbuf **oldpkt, unsigned int return_count)
@@ -169,7 +169,7 @@ MAP_STATIC_SYMBOL(int rte_distributor_get_pkt(struct rte_distributor *d,
 		struct rte_mbuf **oldpkt, unsigned int return_count),
 		rte_distributor_get_pkt_v1705);
 
-int
+int __vsym
 rte_distributor_return_pkt_v1705(struct rte_distributor *d,
 		unsigned int worker_id, struct rte_mbuf **oldpkt, int num)
 {
@@ -359,7 +359,7 @@ release(struct rte_distributor *d, unsigned int wkr)
 
 
 /* process a set of packets to distribute them to workers */
-int
+int __vsym
 rte_distributor_process_v1705(struct rte_distributor *d,
 		struct rte_mbuf **mbufs, unsigned int num_mbufs)
 {
@@ -506,7 +506,7 @@ MAP_STATIC_SYMBOL(int rte_distributor_process(struct rte_distributor *d,
 		rte_distributor_process_v1705);
 
 /* return to the caller, packets returned from workers */
-int
+int __vsym
 rte_distributor_returned_pkts_v1705(struct rte_distributor *d,
 		struct rte_mbuf **mbufs, unsigned int max_mbufs)
 {
@@ -556,7 +556,7 @@ total_outstanding(const struct rte_distributor *d)
  * Flush the distributor, so that there are no outstanding packets in flight or
  * queued up.
  */
-int
+int __vsym
 rte_distributor_flush_v1705(struct rte_distributor *d)
 {
 	unsigned int flushed;
@@ -591,7 +591,7 @@ MAP_STATIC_SYMBOL(int rte_distributor_flush(struct rte_distributor *d),
 		rte_distributor_flush_v1705);
 
 /* clears the internal returns array in the distributor */
-void
+void __vsym
 rte_distributor_clear_returns_v1705(struct rte_distributor *d)
 {
 	unsigned int wkr;
@@ -613,7 +613,7 @@ MAP_STATIC_SYMBOL(void rte_distributor_clear_returns(struct rte_distributor *d),
 		rte_distributor_clear_returns_v1705);
 
 /* creates a distributor instance */
-struct rte_distributor *
+struct rte_distributor * __vsym
 rte_distributor_create_v1705(const char *name,
 		unsigned int socket_id,
 		unsigned int num_workers,
lib/librte_distributor/rte_distributor_v20.c +9 -9
index db6c49258..7a6fddf55 100644
@@ -27,7 +27,7 @@ EAL_REGISTER_TAILQ(rte_distributor_tailq)
 
 /**** APIs called by workers ****/
 
-void
+void __vsym
 rte_distributor_request_pkt_v20(struct rte_distributor_v20 *d,
 		unsigned worker_id, struct rte_mbuf *oldpkt)
 {
@@ -43,7 +43,7 @@ rte_distributor_request_pkt_v20(struct rte_distributor_v20 *d,
 }
 VERSION_SYMBOL(rte_distributor_request_pkt, _v20, 2.0);
 
-struct rte_mbuf *
+struct rte_mbuf * __vsym
 rte_distributor_poll_pkt_v20(struct rte_distributor_v20 *d,
 		unsigned worker_id)
 {
@@ -59,7 +59,7 @@ rte_distributor_poll_pkt_v20(struct rte_distributor_v20 *d,
 }
 VERSION_SYMBOL(rte_distributor_poll_pkt, _v20, 2.0);
 
-struct rte_mbuf *
+struct rte_mbuf * __vsym
 rte_distributor_get_pkt_v20(struct rte_distributor_v20 *d,
 		unsigned worker_id, struct rte_mbuf *oldpkt)
 {
@@ -71,7 +71,7 @@ rte_distributor_get_pkt_v20(struct rte_distributor_v20 *d,
 }
 VERSION_SYMBOL(rte_distributor_get_pkt, _v20, 2.0);
 
-int
+int __vsym
 rte_distributor_return_pkt_v20(struct rte_distributor_v20 *d,
 		unsigned worker_id, struct rte_mbuf *oldpkt)
 {
@@ -204,7 +204,7 @@ process_returns(struct rte_distributor_v20 *d)
 }
 
 /* process a set of packets to distribute them to workers */
-int
+int __vsym
 rte_distributor_process_v20(struct rte_distributor_v20 *d,
 		struct rte_mbuf **mbufs, unsigned num_mbufs)
 {
@@ -321,7 +321,7 @@ rte_distributor_process_v20(struct rte_distributor_v20 *d,
 VERSION_SYMBOL(rte_distributor_process, _v20, 2.0);
 
 /* return to the caller, packets returned from workers */
-int
+int __vsym
 rte_distributor_returned_pkts_v20(struct rte_distributor_v20 *d,
 		struct rte_mbuf **mbufs, unsigned max_mbufs)
 {
@@ -359,7 +359,7 @@ total_outstanding(const struct rte_distributor_v20 *d)
 
 /* flush the distributor, so that there are no outstanding packets in flight or
  * queued up. */
-int
+int __vsym
 rte_distributor_flush_v20(struct rte_distributor_v20 *d)
 {
 	const unsigned flushed = total_outstanding(d);
@@ -372,7 +372,7 @@ rte_distributor_flush_v20(struct rte_distributor_v20 *d)
 VERSION_SYMBOL(rte_distributor_flush, _v20, 2.0);
 
 /* clears the internal returns array in the distributor */
-void
+void __vsym
 rte_distributor_clear_returns_v20(struct rte_distributor_v20 *d)
 {
 	d->returns.start = d->returns.count = 0;
@@ -383,7 +383,7 @@ rte_distributor_clear_returns_v20(struct rte_distributor_v20 *d)
 VERSION_SYMBOL(rte_distributor_clear_returns, _v20, 2.0);
 
 /* creates a distributor instance */
-struct rte_distributor_v20 *
+struct rte_distributor_v20 * __vsym
 rte_distributor_create_v20(const char *name,
 		unsigned socket_id,
 		unsigned num_workers)
lib/librte_eal/common/include/rte_function_versioning.h +7 -0
index eae619d60..c924351d5 100644
@@ -52,6 +52,13 @@
  * symbol <b> to the internal symbol <b><e>
  */
 #define BIND_DEFAULT_SYMBOL(b, e, n) __asm__(".symver " RTE_STR(b) RTE_STR(e) ", " RTE_STR(b) "@@DPDK_" RTE_STR(n))
+
+/*
+ * __vsym
+ * Annotation to be used in declaration of the internal symbol <b><e> to signal
+ * that it is being used as an implementation of a particular version of symbol
+ * <b>.
+ */
 #define __vsym __attribute__((used))
 
 /*
lib/librte_lpm/rte_lpm.c +14 -14
index c96395e26..106916dc8 100644
@@ -90,7 +90,7 @@ depth_to_range(uint8_t depth)
 /*
  * Find an existing lpm table and return a pointer to it.
  */
-struct rte_lpm_v20 *
+struct rte_lpm_v20 * __vsym
 rte_lpm_find_existing_v20(const char *name)
 {
 	struct rte_lpm_v20 *l = NULL;
@@ -116,7 +116,7 @@ rte_lpm_find_existing_v20(const char *name)
 }
 VERSION_SYMBOL(rte_lpm_find_existing, _v20, 2.0);
 
-struct rte_lpm *
+struct rte_lpm * __vsym
 rte_lpm_find_existing_v1604(const char *name)
 {
 	struct rte_lpm *l = NULL;
@@ -147,7 +147,7 @@ MAP_STATIC_SYMBOL(struct rte_lpm *rte_lpm_find_existing(const char *name),
 /*
  * Allocates memory for LPM object
  */
-struct rte_lpm_v20 *
+struct rte_lpm_v20 * __vsym
 rte_lpm_create_v20(const char *name, int socket_id, int max_rules,
 		__rte_unused int flags)
 {
@@ -220,7 +220,7 @@ rte_lpm_create_v20(const char *name, int socket_id, int max_rules,
 }
 VERSION_SYMBOL(rte_lpm_create, _v20, 2.0);
 
-struct rte_lpm *
+struct rte_lpm * __vsym
 rte_lpm_create_v1604(const char *name, int socket_id,
 		const struct rte_lpm_config *config)
 {
@@ -329,7 +329,7 @@ MAP_STATIC_SYMBOL(
 /*
  * Deallocates memory for given LPM table.
  */
-void
+void __vsym
 rte_lpm_free_v20(struct rte_lpm_v20 *lpm)
 {
 	struct rte_lpm_list *lpm_list;
@@ -358,7 +358,7 @@ rte_lpm_free_v20(struct rte_lpm_v20 *lpm)
 }
 VERSION_SYMBOL(rte_lpm_free, _v20, 2.0);
 
-void
+void __vsym
 rte_lpm_free_v1604(struct rte_lpm *lpm)
 {
 	struct rte_lpm_list *lpm_list;
@@ -1177,7 +1177,7 @@ add_depth_big_v1604(struct rte_lpm *lpm, uint32_t ip_masked, uint8_t depth,
 /*
  * Add a route
  */
-int
+int __vsym
 rte_lpm_add_v20(struct rte_lpm_v20 *lpm, uint32_t ip, uint8_t depth,
 		uint8_t next_hop)
 {
@@ -1218,7 +1218,7 @@ rte_lpm_add_v20(struct rte_lpm_v20 *lpm, uint32_t ip, uint8_t depth,
 }
 VERSION_SYMBOL(rte_lpm_add, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm_add_v1604(struct rte_lpm *lpm, uint32_t ip, uint8_t depth,
 		uint32_t next_hop)
 {
@@ -1264,7 +1264,7 @@ MAP_STATIC_SYMBOL(int rte_lpm_add(struct rte_lpm *lpm, uint32_t ip,
 /*
  * Look for a rule in the high-level rules table
  */
-int
+int __vsym
 rte_lpm_is_rule_present_v20(struct rte_lpm_v20 *lpm, uint32_t ip, uint8_t depth,
 uint8_t *next_hop)
 {
@@ -1291,7 +1291,7 @@ uint8_t *next_hop)
 }
 VERSION_SYMBOL(rte_lpm_is_rule_present, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm_is_rule_present_v1604(struct rte_lpm *lpm, uint32_t ip, uint8_t depth,
 uint32_t *next_hop)
 {
@@ -1844,7 +1844,7 @@ delete_depth_big_v1604(struct rte_lpm *lpm, uint32_t ip_masked,
 /*
  * Deletes a rule
  */
-int
+int __vsym
 rte_lpm_delete_v20(struct rte_lpm_v20 *lpm, uint32_t ip, uint8_t depth)
 {
 	int32_t rule_to_delete_index, sub_rule_index;
@@ -1898,7 +1898,7 @@ rte_lpm_delete_v20(struct rte_lpm_v20 *lpm, uint32_t ip, uint8_t depth)
 }
 VERSION_SYMBOL(rte_lpm_delete, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm_delete_v1604(struct rte_lpm *lpm, uint32_t ip, uint8_t depth)
 {
 	int32_t rule_to_delete_index, sub_rule_index;
@@ -1957,7 +1957,7 @@ MAP_STATIC_SYMBOL(int rte_lpm_delete(struct rte_lpm *lpm, uint32_t ip,
 /*
  * Delete all rules from the LPM table.
  */
-void
+void __vsym
 rte_lpm_delete_all_v20(struct rte_lpm_v20 *lpm)
 {
 	/* Zero rule information. */
@@ -1974,7 +1974,7 @@ rte_lpm_delete_all_v20(struct rte_lpm_v20 *lpm)
 }
 VERSION_SYMBOL(rte_lpm_delete_all, _v20, 2.0);
 
-void
+void __vsym
 rte_lpm_delete_all_v1604(struct rte_lpm *lpm)
 {
 	/* Zero rule information. */
lib/librte_lpm/rte_lpm6.c +8 -8
index e20f82460..0d161dc32 100644
@@ -812,7 +812,7 @@ add_step(struct rte_lpm6 *lpm, struct rte_lpm6_tbl_entry *tbl,
 /*
  * Add a route
  */
-int
+int __vsym
 rte_lpm6_add_v20(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
 		uint8_t next_hop)
 {
@@ -862,7 +862,7 @@ simulate_add(struct rte_lpm6 *lpm, const uint8_t *masked_ip, uint8_t depth)
 	return 0;
 }
 
-int
+int __vsym
 rte_lpm6_add_v1705(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
 		uint32_t next_hop)
 {
@@ -955,7 +955,7 @@ lookup_step(const struct rte_lpm6 *lpm, const struct rte_lpm6_tbl_entry *tbl,
 /*
  * Looks up an IP
  */
-int
+int __vsym
 rte_lpm6_lookup_v20(const struct rte_lpm6 *lpm, uint8_t *ip, uint8_t *next_hop)
 {
 	uint32_t next_hop32 = 0;
@@ -973,7 +973,7 @@ rte_lpm6_lookup_v20(const struct rte_lpm6 *lpm, uint8_t *ip, uint8_t *next_hop)
 }
 VERSION_SYMBOL(rte_lpm6_lookup, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm6_lookup_v1705(const struct rte_lpm6 *lpm, uint8_t *ip,
 		uint32_t *next_hop)
 {
@@ -1008,7 +1008,7 @@ MAP_STATIC_SYMBOL(int rte_lpm6_lookup(const struct rte_lpm6 *lpm, uint8_t *ip,
 /*
  * Looks up a group of IP addresses
  */
-int
+int __vsym
 rte_lpm6_lookup_bulk_func_v20(const struct rte_lpm6 *lpm,
 		uint8_t ips[][RTE_LPM6_IPV6_ADDR_SIZE],
 		int16_t * next_hops, unsigned n)
@@ -1049,7 +1049,7 @@ rte_lpm6_lookup_bulk_func_v20(const struct rte_lpm6 *lpm,
 }
 VERSION_SYMBOL(rte_lpm6_lookup_bulk_func, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm6_lookup_bulk_func_v1705(const struct rte_lpm6 *lpm,
 		uint8_t ips[][RTE_LPM6_IPV6_ADDR_SIZE],
 		int32_t *next_hops, unsigned int n)
@@ -1099,7 +1099,7 @@ MAP_STATIC_SYMBOL(int rte_lpm6_lookup_bulk_func(const struct rte_lpm6 *lpm,
 /*
  * Look for a rule in the high-level rules table
  */
-int
+int __vsym
 rte_lpm6_is_rule_present_v20(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
 		uint8_t *next_hop)
 {
@@ -1119,7 +1119,7 @@ rte_lpm6_is_rule_present_v20(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
 }
 VERSION_SYMBOL(rte_lpm6_is_rule_present, _v20, 2.0);
 
-int
+int __vsym
 rte_lpm6_is_rule_present_v1705(struct rte_lpm6 *lpm, uint8_t *ip, uint8_t depth,
 		uint32_t *next_hop)
 {
lib/librte_timer/rte_timer.c +10 -10
index 3834c9473..381a9f43f 100644
@@ -131,7 +131,7 @@ rte_timer_data_dealloc(uint32_t id)
 	return 0;
 }
 
-void
+void __vsym
 rte_timer_subsystem_init_v20(void)
 {
 	unsigned lcore_id;
@@ -153,7 +153,7 @@ VERSION_SYMBOL(rte_timer_subsystem_init, _v20, 2.0);
  * secondary processes should be empty, the zeroth entry can be shared by
  * multiple processes.
  */
-int
+int __vsym
 rte_timer_subsystem_init_v1905(void)
 {
 	const struct rte_memzone *mz;
@@ -551,7 +551,7 @@ __rte_timer_reset(struct rte_timer *tim, uint64_t expire,
 }
 
 /* Reset and start the timer associated with the timer handle tim */
-int
+int __vsym
 rte_timer_reset_v20(struct rte_timer *tim, uint64_t ticks,
 		    enum rte_timer_type type, unsigned int tim_lcore,
 		    rte_timer_cb_t fct, void *arg)
@@ -574,7 +574,7 @@ rte_timer_reset_v20(struct rte_timer *tim, uint64_t ticks,
 }
 VERSION_SYMBOL(rte_timer_reset, _v20, 2.0);
 
-int
+int __vsym
 rte_timer_reset_v1905(struct rte_timer *tim, uint64_t ticks,
 		      enum rte_timer_type type, unsigned int tim_lcore,
 		      rte_timer_cb_t fct, void *arg)
@@ -657,14 +657,14 @@ __rte_timer_stop(struct rte_timer *tim, int local_is_locked,
 }
 
 /* Stop the timer associated with the timer handle tim */
-int
+int __vsym
 rte_timer_stop_v20(struct rte_timer *tim)
 {
 	return __rte_timer_stop(tim, 0, &default_timer_data);
 }
 VERSION_SYMBOL(rte_timer_stop, _v20, 2.0);
 
-int
+int __vsym
 rte_timer_stop_v1905(struct rte_timer *tim)
 {
 	return rte_timer_alt_stop(default_data_id, tim);
@@ -817,14 +817,14 @@ __rte_timer_manage(struct rte_timer_data *timer_data)
 	priv_timer[lcore_id].running_tim = NULL;
 }
 
-void
+void __vsym
 rte_timer_manage_v20(void)
 {
 	__rte_timer_manage(&default_timer_data);
 }
 VERSION_SYMBOL(rte_timer_manage, _v20, 2.0);
 
-int
+int __vsym
 rte_timer_manage_v1905(void)
 {
 	struct rte_timer_data *timer_data;
@@ -1074,14 +1074,14 @@ __rte_timer_dump_stats(struct rte_timer_data *timer_data __rte_unused, FILE *f)
 #endif
 }
 
-void
+void __vsym
 rte_timer_dump_stats_v20(FILE *f)
 {
 	__rte_timer_dump_stats(&default_timer_data, f);
 }
 VERSION_SYMBOL(rte_timer_dump_stats, _v20, 2.0);
 
-int
+int __vsym
 rte_timer_dump_stats_v1905(FILE *f)
 {
 	return rte_timer_alt_dump_stats(default_data_id, f);
Signature · 1 line
2.17.1
#99 lore raw

PATCH v7 3/12 build: add an option to enable LTO build

In reply to Andrzej Ostruszka

This patch adds an option to enable link time optimization. In addition to LTO option itself (-flto) fat-lto-objects are being used. This is because during the build pmdinfogen scans the generated ELF objects to find this_pmd_name* symbol in symbol table. Without fat-lto-objects gcc produces ELF only with extra symbols for internal use during linking.

---
 .travis.yml                              |  9 ++++++
 config/common_base                       |  5 +++
 config/meson.build                       | 13 ++++++++
 doc/guides/prog_guide/index.rst          |  1 +
 doc/guides/prog_guide/lto.rst            | 40 ++++++++++++++++++++++++
 doc/guides/rel_notes/release_19_11.rst   |  9 ++++++
 mk/toolchain/gcc/rte.toolchain-compat.mk |  4 +++
 mk/toolchain/gcc/rte.vars.mk             | 12 +++++++
 mk/toolchain/icc/rte.vars.mk             |  8 +++++
 9 files changed, 101 insertions(+)
 create mode 100644 doc/guides/prog_guide/lto.rst
.travis.yml +9 -0
index 3d6ef2959..a1c31d407 100644
@@ -105,6 +105,15 @@ matrix:
       apt:
         packages:
           - *extra_packages
+  - env: DEF_LIB="static" OPTS="-Db_lto=true" EXTRA_PACKAGES=1 CC=gcc-7
+    compiler: gcc
+    addons:
+      apt:
+        sources:
+            - ubuntu-toolchain-r-test
+        packages:
+          - *extra_packages
+          - gcc-7
 
 
 script: ./.ci/${TRAVIS_OS_NAME}-build.sh
config/common_base +5 -0
index b2be3d96a..0d1207166 100644
@@ -49,6 +49,11 @@ CONFIG_RTE_FORCE_INTRINSICS=n
 #
 CONFIG_RTE_ARCH_STRICT_ALIGN=n
 
+#
+# Enable link time optimization
+#
+CONFIG_RTE_ENABLE_LTO=n
+
 #
 # Compile to share library
 #
config/meson.build +13 -0
index e1ebdad26..2b1cb92e7 100644
@@ -225,3 +225,16 @@ add_project_arguments('-D_GNU_SOURCE', language: 'c')
 if is_freebsd
 	add_project_arguments('-D__BSD_VISIBLE', language: 'c')
 endif
+
+if get_option('b_lto')
+	if cc.has_argument('-ffat-lto-objects')
+		add_project_arguments('-ffat-lto-objects', language: 'c')
+	else
+		error('compiler does not support fat LTO objects - please turn LTO off')
+	endif
+	# workaround for gcc bug 81440
+	if cc.get_id() == 'gcc' and cc.version().version_compare('<8.0')
+		add_project_arguments('-Wno-lto-type-mismatch', language: 'c')
+		add_project_link_arguments('-Wno-lto-type-mismatch', language: 'c')
+	endif
+endif
doc/guides/prog_guide/index.rst +1 -0
index 692409af8..dc4851c57 100644
@@ -65,5 +65,6 @@ Programmer's Guide
     ext_app_lib_make_help
     perf_opt_guidelines
     writing_efficient_code
+    lto
     profile_app
     glossary
doc/guides/prog_guide/lto.rst new file +40 -0
new file mode 100644
index 000000000..50aecc9e5
@@ -0,0 +1,40 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2019 Marvell International Ltd.
+
+Link Time Optimization
+======================
+
+The DPDK supports compilation with link time optimization turned on.
+This depends obviously on the ability of the compiler to do "whole
+program" optimization at link time and is available only for compilers
+that support that feature.
+To be more specific, compiler (in addition to performing LTO) have to
+support creation of ELF objects containing both normal code and internal
+representation (called fat-lto-objects in gcc and icc).
+This is required since during build some code is generated by parsing
+produced ELF objects (pmdinfogen).
+
+The amount of performance gain that one can get from LTO depends on the
+compiler and the code that is being compiled.
+However LTO is also useful for additional code analysis done by the
+compiler.
+In particular due to interprocedural analysis compiler can produce
+additional warnings about variables that might be used uninitialized.
+Some of these warnings might be "false positives" though and you might
+need to explicitly initialize variable in order to silence the compiler.
+
+Please note that turning LTO on causes considerable extension of
+build time.
+
+When using make based build, link time optimization can be enabled for
+the whole DPDK by setting:
+
+.. code-block:: console
+    CONFIG_ENABLE_LTO=y
+
+in config file.
+For the meson based build it can be enabled by setting meson built-in
+'b_lto' option:
+
+.. code-block:: console
+    meson build -Db_lto=true
doc/guides/rel_notes/release_19_11.rst +9 -0
index ae8e7b2f0..b11abe2fe 100644
@@ -231,6 +231,15 @@ New Features
   * Added a console command to testpmd app, ``show port (port_id) ptypes`` which
     gives ability to print port supported ptypes in different protocol layers.
 
+* **Added build support for Link Time Optimization.**
+
+  LTO is an optimization technique used by the compiler to perform whole
+  program analysis and optimization at link time.  In order to do that
+  compilers store their internal representation of the source code that
+  the linker uses at the final stage of compilation process.
+
+  See :doc:`../prog_guide/lto` for more information:
+
 
 Removed Items
 -------------
mk/toolchain/gcc/rte.toolchain-compat.mk +4 -0
index ea40a11c0..69a53e5d0 100644
@@ -88,6 +88,10 @@ else
 		MACHINE_CFLAGS := $(filter-out -march% -mtune% -msse%,$(MACHINE_CFLAGS))
 	endif
 
+	ifeq ($(shell test $(GCC_VERSION) -lt 74 && echo 1), 1)
+		CONFIG_RTE_ENABLE_LTO=n
+	endif
+
 	# Disable thunderx PMD for gcc < 4.7
 	ifeq ($(shell test $(GCC_VERSION) -lt 47 && echo 1), 1)
 		CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD=d
mk/toolchain/gcc/rte.vars.mk +12 -0
index b852fcfd7..9fc704193 100644
@@ -62,6 +62,18 @@ endif
 # process cpu flags
 include $(RTE_SDK)/mk/toolchain/$(RTE_TOOLCHAIN)/rte.toolchain-compat.mk
 
+ifeq ($(CONFIG_RTE_ENABLE_LTO),y)
+# 'fat-lto' is used since pmdinfogen needs to have 'this_pmd_nameX'
+# exported in symbol table and without this option only internal
+# representation is present.
+TOOLCHAIN_CFLAGS += -flto -ffat-lto-objects
+TOOLCHAIN_LDFLAGS += -flto
+# workaround for GCC bug 81440
+ifeq ($(shell test $(GCC_VERSION) -lt 80 && echo 1), 1)
+WERROR_FLAGS += -Wno-lto-type-mismatch
+endif
+endif
+
 # workaround GCC bug with warning "missing initializer" for "= {0}"
 ifeq ($(shell test $(GCC_VERSION) -lt 47 && echo 1), 1)
 WERROR_FLAGS += -Wno-missing-field-initializers
mk/toolchain/icc/rte.vars.mk +8 -0
index aa1422bf1..8aa87aa1e 100644
@@ -54,5 +54,13 @@ endif
 # process cpu flags
 include $(RTE_SDK)/mk/toolchain/$(RTE_TOOLCHAIN)/rte.toolchain-compat.mk
 
+ifeq ($(CONFIG_RTE_ENABLE_LTO),y)
+# 'fat-lto' is used since pmdinfogen needs to have 'this_pmd_nameX'
+# exported in symbol table and without this option only internal
+# representation is present.
+TOOLCHAIN_CFLAGS += -flto -ffat-lto-objects
+TOOLCHAIN_LDFLAGS += -flto
+endif
+
 export CC AS AR LD OBJCOPY OBJDUMP STRIP READELF
 export TOOLCHAIN_CFLAGS TOOLCHAIN_LDFLAGS TOOLCHAIN_ASFLAGS
Signature · 1 line
2.17.1
#100 lore raw

PATCH v7 4/12 eventdev: fix possible use of uninitialized var

In reply to Andrzej Ostruszka

Fix the logic for the case of event queue allowing all schedule types.

Compiler warning pointing to this error (with LTO enabled):
error: ‘sched_type’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
if ((ret < 0 && ret != -EOVERFLOW) ||

Fixes 6750b21bd6af ("eventdev: add default software timer adapter")
---
 lib/librte_eventdev/rte_event_timer_adapter.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
lib/librte_eventdev/rte_event_timer_adapter.c +4 -4
index 5ce399eca..161e21a68 100644
@@ -706,11 +706,11 @@ check_destination_event_queue(struct rte_event_timer *evtim,
 				       RTE_EVENT_QUEUE_ATTR_SCHEDULE_TYPE,
 				       &sched_type);
 
-	if ((ret < 0 && ret != -EOVERFLOW) ||
-	    evtim->ev.sched_type != sched_type)
-		return -1;
+	if ((ret == 0 && evtim->ev.sched_type == sched_type) ||
+	    ret == -EOVERFLOW)
+		return 0;
 
-	return 0;
+	return -1;
 }
 
 static int
Signature · 1 line
2.17.1
#101 lore raw

PATCH v7 5/12 app/eventdev: clean LTO build warnings (maybe-uninitialized)

In reply to Andrzej Ostruszka

During LTO build compiler reports some 'false positive' warnings about variables being possibly used uninitialized. This patch silences these warnings.

Exemplary compiler warning to suppress (with LTO enabled):
error: ‘service_id’ may be used uninitialized in this function [-Werror=maybe-uninitialized]

    ret = evt_service_setup(service_id);
---
 app/test-eventdev/test_perf_common.c     | 2 +-
 app/test-eventdev/test_pipeline_common.c | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)
app/test-eventdev/test_perf_common.c +1 -1
index e24519179..e7cf75a7d 100644
@@ -439,7 +439,7 @@ perf_event_timer_adapter_setup(struct test_perf *t)
 
 		if (!(adapter_info.caps &
 				RTE_EVENT_TIMER_ADAPTER_CAP_INTERNAL_PORT)) {
-			uint32_t service_id;
+			uint32_t service_id = -1U;
 
 			rte_event_timer_adapter_service_id_get(wl,
 					&service_id);
app/test-eventdev/test_pipeline_common.c +2 -2
index 386aca299..160461fb2 100644
@@ -326,7 +326,7 @@ pipeline_event_rx_adapter_setup(struct evt_options *opt, uint8_t stride,
 		}
 
 		if (!(cap & RTE_EVENT_ETH_RX_ADAPTER_CAP_INTERNAL_PORT)) {
-			uint32_t service_id;
+			uint32_t service_id = -1U;
 
 			rte_event_eth_rx_adapter_service_id_get(prod,
 					&service_id);
@@ -378,7 +378,7 @@ pipeline_event_tx_adapter_setup(struct evt_options *opt,
 		}
 
 		if (!(cap & RTE_EVENT_ETH_TX_ADAPTER_CAP_INTERNAL_PORT)) {
-			uint32_t service_id;
+			uint32_t service_id = -1U;
 
 			rte_event_eth_tx_adapter_service_id_get(consm,
 					&service_id);
Signature · 1 line
2.17.1
#102 lore raw

PATCH v7 6/12 event/octeontx2: clean LTO build warnings (maybe-uninitialized)

In reply to Andrzej Ostruszka

During LTO build compiler reports some 'false positive' warnings about variables being possibly used uninitialized. This patch silences these warnings.

Exemplary compiler warning to suppress (with LTO enabled):
error: ‘chunk’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
bkt->current_chunk = (uintptr_t)chunk;

---
 drivers/event/octeontx2/otx2_tim_worker.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/event/octeontx2/otx2_tim_worker.h +1 -1
index b193e2cab..50db6543c 100644
@@ -337,7 +337,7 @@ tim_add_entry_brst(struct otx2_tim_ring * const tim_ring,
 		   const struct otx2_tim_ent *ents,
 		   const uint16_t nb_timers, const uint8_t flags)
 {
-	struct otx2_tim_ent *chunk;
+	struct otx2_tim_ent *chunk = NULL;
 	struct otx2_tim_bkt *bkt;
 	uint16_t chunk_remainder;
 	uint16_t index = 0;
Signature · 1 line
2.17.1
#103 lore raw

PATCH v7 7/12 app/test: clean LTO build warnings (maybe-uninitialized)

In reply to Andrzej Ostruszka

During LTO build compiler reports some 'false positive' warnings about variables being possibly used uninitialized. This patch silences these warnings.

Exemplary compiler warning to suppress (with LTO enabled):
error: ‘stats.greatest_free_size’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
return len - overhead;

---
 app/test/test_hash_readwrite.c     | 2 +-
 app/test/test_link_bonding_mode4.c | 8 +++++++-
 app/test/test_memzone.c            | 3 ++-
 3 files changed, 10 insertions(+), 3 deletions(-)
app/test/test_hash_readwrite.c +1 -1
index 4376b099b..615767fb6 100644
@@ -298,7 +298,7 @@ test_rw_reader(void *arg)
 
 	begin = rte_rdtsc_precise();
 	for (i = 0; i < read_cnt; i++) {
-		void *data;
+		void *data = arg;
 		rte_hash_lookup_data(tbl_rw_test_param.h,
 				tbl_rw_test_param.keys + i,
 				&data);
app/test/test_link_bonding_mode4.c +7 -1
index ff54d7b91..cf12f026d 100644
@@ -585,7 +585,13 @@ bond_get_update_timeout_ms(void)
 {
 	struct rte_eth_bond_8023ad_conf conf;
 
-	rte_eth_bond_8023ad_conf_get(test_params.bonded_port_id, &conf);
+	if (rte_eth_bond_8023ad_conf_get(test_params.bonded_port_id, &conf) < 0) {
+		RTE_LOG(DEBUG, EAL, "Failed to get bonding configuration: "
+				    "%s at %d\n", __func__, __LINE__);
+		RTE_TEST_TRACE_FAILURE(__FILE__, __LINE__, __func__);
+		return 0;
+	}
+
 	return conf.update_timeout_ms;
 }
 
app/test/test_memzone.c +2 -1
index 4d8744455..0343b0326 100644
@@ -475,7 +475,8 @@ find_max_block_free_size(unsigned int align, unsigned int socket_id)
 	struct rte_malloc_socket_stats stats;
 	size_t len, overhead;
 
-	rte_malloc_get_socket_stats(socket_id, &stats);
+	if (rte_malloc_get_socket_stats(socket_id, &stats) < 0)
+		return 0;
 
 	len = stats.greatest_free_size;
 	overhead = MALLOC_ELEM_OVERHEAD;
Signature · 1 line
2.17.1
Acked-by on v7 7/12
#104 lore raw

Re: [dpdk-dev] [PATCH v7 07/12] app/test: clean LTO build warnings (maybe-uninitialized)

Quoted 71 lines

-----Original Message-----
From: Andrzej Ostruszka [mailto:[email protected]]
Sent: Thursday, November 7, 2019 7:03 AM
To: [email protected]; Wang, Yipeng1 <[email protected]>; Gobriel, Sameh <[email protected]>; Richardson, Bruce <[email protected]>; Chas Williams <[email protected]>; Burakov, Anatoly <[email protected]>
Cc: [email protected]; [email protected]
Subject: [PATCH v7 07/12] app/test: clean LTO build warnings (maybe-uninitialized)

During LTO build compiler reports some 'false positive' warnings about variables being possibly used uninitialized. This patch silences these warnings.

Exemplary compiler warning to suppress (with LTO enabled):
error: ‘stats.greatest_free_size’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
return len - overhead;

---

app/test/test_hash_readwrite.c     | 2 +-
app/test/test_link_bonding_mode4.c | 8 +++++++-
app/test/test_memzone.c            | 3 ++-
3 files changed, 10 insertions(+), 3 deletions(-)
app/test/test_hash_readwrite.c +1 -1
index 4376b099b..615767fb6 100644
@@ -298,7 +298,7 @@ test_rw_reader(void *arg)
 
 	begin = rte_rdtsc_precise();
 	for (i = 0; i < read_cnt; i++) {
-		void *data;
+		void *data = arg;
 		rte_hash_lookup_data(tbl_rw_test_param.h,
 				tbl_rw_test_param.keys + i,
 				&data);
app/test/test_link_bonding_mode4.c +7 -1
index ff54d7b91..cf12f026d 100644
@@ -585,7 +585,13 @@ bond_get_update_timeout_ms(void)
 {
 	struct rte_eth_bond_8023ad_conf conf;
 
-	rte_eth_bond_8023ad_conf_get(test_params.bonded_port_id, &conf);
+	if (rte_eth_bond_8023ad_conf_get(test_params.bonded_port_id, &conf) < 0) {
+		RTE_LOG(DEBUG, EAL, "Failed to get bonding configuration: "
+				    "%s at %d\n", __func__, __LINE__);
+		RTE_TEST_TRACE_FAILURE(__FILE__, __LINE__, __func__);
+		return 0;
+	}
+
 	return conf.update_timeout_ms;
 }
 
app/test/test_memzone.c +2 -2
index 4d8744455..0343b0326 100644
@@ -475,7 +475,8 @@ find_max_block_free_size(unsigned int align, unsigned int socket_id)
 	struct rte_malloc_socket_stats stats;
 	size_t len, overhead;
 
-	rte_malloc_get_socket_stats(socket_id, &stats);
+	if (rte_malloc_get_socket_stats(socket_id, &stats) < 0)
+		return 0;
 
 	len = stats.greatest_free_size;
 	overhead = MALLOC_ELEM_OVERHEAD;
--
Quoted 1 line

2.17.1

[Wang, Yipeng]
For the test_hash_readwrite.c file:

#105 lore raw

PATCH v7 8/12 net/dpaa2: fix possible use of uninitialized vars

In reply to Andrzej Ostruszka

This patch fixes 'maybe-uninitialized' warnings reported by compiler when using LTO.

Compiler warning pointing to this error (with LTO enabled):
error: ‘kg_cfg.extracts[0].masks[0].mask’ may be used uninitialized in this function [-Werror=maybe-uninitialized]

    extr->masks[j].mask = cfg->extracts[i].masks[j].mask;
Fixes 16bbc98a3e63 ("bus/fslmc: update MC to 10.3.x")
---
 drivers/net/dpaa2/base/dpaa2_hw_dpni.c | 1 +
 drivers/net/dpaa2/mc/dpkg.c            | 5 ++++-
 2 files changed, 5 insertions(+), 1 deletion(-)
drivers/net/dpaa2/base/dpaa2_hw_dpni.c +1 -0
index 16555d7b0..47a8bda6a 100644
@@ -51,6 +51,7 @@ rte_pmd_dpaa2_set_custom_hash(uint16_t port_id,
 	kg_cfg.extracts[0].type = DPKG_EXTRACT_FROM_DATA;
 	kg_cfg.extracts[0].extract.from_data.offset = offset;
 	kg_cfg.extracts[0].extract.from_data.size = size;
+	kg_cfg.extracts[0].num_of_byte_masks = 0;
 	kg_cfg.num_extracts = 1;
 
 	ret = dpkg_prepare_key_cfg(&kg_cfg, p_params);
drivers/net/dpaa2/mc/dpkg.c +4 -1
index 80f94f40e..1e171eedc 100644
@@ -63,7 +63,10 @@ dpkg_prepare_key_cfg(const struct dpkg_profile_cfg *cfg, uint8_t *key_cfg_buf)
 		dpkg_set_field(extr->extract_type, EXTRACT_TYPE,
 			       cfg->extracts[i].type);
 
-		for (j = 0; j < DPKG_NUM_OF_MASKS; j++) {
+		if (extr->num_of_byte_masks > DPKG_NUM_OF_MASKS)
+			return -EINVAL;
+
+		for (j = 0; j < extr->num_of_byte_masks; j++) {
 			extr->masks[j].mask = cfg->extracts[i].masks[j].mask;
 			extr->masks[j].offset =
 				cfg->extracts[i].masks[j].offset;
Signature · 1 line
2.17.1
#106 lore raw

PATCH v7 9/12 net/e1000: clean LTO build warnings (maybe-uninitialized)

In reply to Andrzej Ostruszka

During LTO build compiler reports some 'false positive' warnings about variables being possibly used uninitialized. This patch silences these warnings.

Exemplary compiler warning to suppress (with LTO enabled):
error: ‘link’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
if (link) {

---
 drivers/net/e1000/base/e1000_82543.c   | 2 +-
 drivers/net/e1000/base/e1000_ich8lan.c | 2 +-
 drivers/net/e1000/base/e1000_phy.c     | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)
drivers/net/e1000/base/e1000_82543.c +1 -1
index 810899eb6..dfde2a8b9 100644
@@ -1027,7 +1027,7 @@ STATIC s32 e1000_setup_copper_link_82543(struct e1000_hw *hw)
 {
 	u32 ctrl;
 	s32 ret_val;
-	bool link;
+	bool link = true;
 
 	DEBUGFUNC("e1000_setup_copper_link_82543");
 
drivers/net/e1000/base/e1000_ich8lan.c +1 -1
index accc6ea01..5241cf698 100644
@@ -5533,7 +5533,7 @@ void e1000_igp3_phy_powerdown_workaround_ich8lan(struct e1000_hw *hw)
 void e1000_gig_downshift_workaround_ich8lan(struct e1000_hw *hw)
 {
 	s32 ret_val;
-	u16 reg_data;
+	u16 reg_data = 0;
 
 	DEBUGFUNC("e1000_gig_downshift_workaround_ich8lan");
 
drivers/net/e1000/base/e1000_phy.c +1 -1
index 7d08f836f..956c06747 100644
@@ -1664,7 +1664,7 @@ s32 e1000_copper_link_autoneg(struct e1000_hw *hw)
 s32 e1000_setup_copper_link_generic(struct e1000_hw *hw)
 {
 	s32 ret_val;
-	bool link;
+	bool link = true;
 
 	DEBUGFUNC("e1000_setup_copper_link_generic");
 
Signature · 1 line
2.17.1
#107 lore raw

PATCH v7 10/12 net/i40e: clean LTO build warnings (maybe-uninitialized)

In reply to Andrzej Ostruszka

During LTO build compiler reports some 'false positive' warnings about variables being possibly used uninitialized. This patch silences these warnings.

Exemplary compiler warning to suppress (with LTO enabled):
error: ‘filter_idx’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
PMD_DRV_LOG(INFO, "Added port %d with AQ command with index %d",

---
 drivers/net/i40e/i40e_ethdev.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/i40e/i40e_ethdev.c +1 -1
index 77a46832c..91d529b2f 100644
@@ -8480,7 +8480,7 @@ static int
 i40e_add_vxlan_port(struct i40e_pf *pf, uint16_t port, int udp_type)
 {
 	int  idx, ret;
-	uint8_t filter_idx;
+	uint8_t filter_idx = 0;
 	struct i40e_hw *hw = I40E_PF_TO_HW(pf);
 
 	idx = i40e_get_vxlan_port_idx(pf, port);
Signature · 1 line
2.17.1
#108 lore raw

PATCH v7 11/12 net/ifc: clean LTO build warnings (maybe-uninitialized)

In reply to Andrzej Ostruszka

During LTO build compiler reports some 'false positive' warnings about variables being possibly used uninitialized. This patch silences these warnings.

Exemplary compiler warning to suppress (with LTO enabled):
error: ‘features’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
if (RTE_VHOST_NEED_LOG(features)) {

---
 drivers/net/ifc/ifcvf_vdpa.c | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)
drivers/net/ifc/ifcvf_vdpa.c +10 -4
index 8de9ef199..9c562def0 100644
@@ -5,6 +5,7 @@
 #include <unistd.h>
 #include <pthread.h>
 #include <fcntl.h>
+#include <string.h>
 #include <sys/ioctl.h>
 #include <sys/epoll.h>
 #include <linux/virtio_net.h>
@@ -304,8 +305,8 @@ vdpa_ifcvf_stop(struct ifcvf_internal *internal)
 	struct ifcvf_hw *hw = &internal->hw;
 	uint32_t i;
 	int vid;
-	uint64_t features;
-	uint64_t log_base, log_size;
+	uint64_t features = 0;
+	uint64_t log_base = 0, log_size = 0;
 	uint64_t len;
 
 	vid = internal->vid;
@@ -348,6 +349,8 @@ vdpa_enable_vfio_intr(struct ifcvf_internal *internal, bool m_rx)
 	struct rte_vhost_vring vring;
 	int fd;
 
+	vring.callfd = -1;
+
 	nr_vring = rte_vhost_get_vring_num(internal->vid);
 
 	irq_set = (struct vfio_irq_set *)irq_set_buf;
@@ -442,6 +445,7 @@ notify_relay(void *arg)
 	}
 	internal->epfd = epfd;
 
+	vring.kickfd = -1;
 	for (qid = 0; qid < q_num; qid++) {
 		ev.events = EPOLLIN | EPOLLPRI;
 		rte_vhost_get_vhost_vring(internal->vid, qid, &vring);
@@ -583,6 +587,7 @@ m_ifcvf_start(struct ifcvf_internal *internal)
 	uint64_t size;
 	uint64_t gpa;
 
+	memset(&vq, 0, sizeof(vq));
 	vid = internal->vid;
 	nr_vring = rte_vhost_get_vring_num(vid);
 	rte_vhost_get_negotiated_features(vid, &hw->req_features);
@@ -721,6 +726,7 @@ vring_relay(void *arg)
 	}
 	internal->epfd = epfd;
 
+	vring.kickfd = -1;
 	for (qid = 0; qid < q_num; qid++) {
 		ev.events = EPOLLIN | EPOLLPRI;
 		rte_vhost_get_vhost_vring(vid, qid, &vring);
@@ -930,11 +936,11 @@ ifcvf_dev_close(int vid)
 static int
 ifcvf_set_features(int vid)
 {
-	uint64_t features;
+	uint64_t features = 0;
 	int did;
 	struct internal_list *list;
 	struct ifcvf_internal *internal;
-	uint64_t log_base, log_size;
+	uint64_t log_base = 0, log_size = 0;
 
 	did = rte_vhost_get_vdpa_device_id(vid);
 	list = find_internal_resource_by_did(did);
Signature · 1 line
2.17.1
#109 lore raw

PATCH v7 12/12 net/qede: clean LTO build warnings (maybe-uninitialized)

In reply to Andrzej Ostruszka

During LTO build compiler reports some 'false positive' warnings about variables being possibly used uninitialized. This patch silences these warnings.

Exemplary compiler warning to suppress (with LTO enabled):
error: ‘transceiver_type’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
switch (transceiver_type) {

---
 drivers/net/qede/base/ecore_mcp.c | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)
drivers/net/qede/base/ecore_mcp.c +8 -7
index a5aa07438..7518765a0 100644
@@ -2328,7 +2328,7 @@ enum _ecore_status_t ecore_mcp_trans_speed_mask(struct ecore_hwfn *p_hwfn,
 						struct ecore_ptt *p_ptt,
 						u32 *p_speed_mask)
 {
-	u32 transceiver_type, transceiver_state;
+	u32 transceiver_type = ETH_TRANSCEIVER_TYPE_NONE, transceiver_state;
 
 	ecore_mcp_get_transceiver_data(p_hwfn, p_ptt, &transceiver_state,
 				       &transceiver_type);
@@ -3223,7 +3223,8 @@ enum _ecore_status_t ecore_mcp_nvm_put_file_begin(struct ecore_dev *p_dev,
 enum _ecore_status_t ecore_mcp_nvm_write(struct ecore_dev *p_dev, u32 cmd,
 					 u32 addr, u8 *p_buf, u32 len)
 {
-	u32 buf_idx, buf_size, nvm_cmd, nvm_offset, resp, param;
+	u32 buf_idx, buf_size, nvm_cmd, nvm_offset;
+	u32 resp = FW_MSG_CODE_ERROR, param;
 	struct ecore_hwfn *p_hwfn = ECORE_LEADING_HWFN(p_dev);
 	enum _ecore_status_t rc = ECORE_INVAL;
 	struct ecore_ptt *p_ptt;
@@ -3322,7 +3323,7 @@ enum _ecore_status_t ecore_mcp_nvm_set_secure_mode(struct ecore_dev *p_dev,
 {
 	struct ecore_hwfn *p_hwfn = ECORE_LEADING_HWFN(p_dev);
 	struct ecore_ptt *p_ptt;
-	u32 resp, param;
+	u32 resp = 0, param;
 	enum _ecore_status_t rc;
 
 	p_ptt = ecore_ptt_acquire(p_hwfn);
@@ -3430,7 +3431,7 @@ enum _ecore_status_t ecore_mcp_gpio_read(struct ecore_hwfn *p_hwfn,
 					 u16 gpio, u32 *gpio_val)
 {
 	enum _ecore_status_t rc = ECORE_SUCCESS;
-	u32 drv_mb_param = 0, rsp;
+	u32 drv_mb_param = 0, rsp = 0;
 
 	drv_mb_param = (gpio << DRV_MB_PARAM_GPIO_NUMBER_OFFSET);
 
@@ -3451,7 +3452,7 @@ enum _ecore_status_t ecore_mcp_gpio_write(struct ecore_hwfn *p_hwfn,
 					  u16 gpio, u16 gpio_val)
 {
 	enum _ecore_status_t rc = ECORE_SUCCESS;
-	u32 drv_mb_param = 0, param, rsp;
+	u32 drv_mb_param = 0, param, rsp = 0;
 
 	drv_mb_param = (gpio << DRV_MB_PARAM_GPIO_NUMBER_OFFSET) |
 		(gpio_val << DRV_MB_PARAM_GPIO_VALUE_OFFSET);
@@ -3541,7 +3542,7 @@ enum _ecore_status_t ecore_mcp_bist_clock_test(struct ecore_hwfn *p_hwfn,
 enum _ecore_status_t ecore_mcp_bist_nvm_test_get_num_images(
 	struct ecore_hwfn *p_hwfn, struct ecore_ptt *p_ptt, u32 *num_images)
 {
-	u32 drv_mb_param = 0, rsp;
+	u32 drv_mb_param = 0, rsp = 0;
 	enum _ecore_status_t rc = ECORE_SUCCESS;
 
 	drv_mb_param = (DRV_MB_PARAM_BIST_NVM_TEST_NUM_IMAGES <<
@@ -3925,7 +3926,7 @@ enum _ecore_status_t
 __ecore_mcp_resc_lock(struct ecore_hwfn *p_hwfn, struct ecore_ptt *p_ptt,
 		      struct ecore_resc_lock_params *p_params)
 {
-	u32 param = 0, mcp_resp, mcp_param;
+	u32 param = 0, mcp_resp = 0, mcp_param = 0;
 	u8 opcode;
 	enum _ecore_status_t rc;
 
Signature · 1 line
2.17.1
#110 lore raw
In reply to Andrzej Ostruszka

07/11/2019 16:03, Andrzej Ostruszka:

Quoted 4 lines

This patch series adds an option to make use of link time optimization (if compiler has support for it). It is split as follows:
- 1st patch is just a minor doc fix for versioning macros
- 2nd patch fixes missing __vsym annotations (needed for LTO)

Quoted 8 lines

- 3rd is the LTO enablement
- remaining patches are fixes for the warnings produced by the compiler and they are split by directory/subsystem so their maintainers can easily find and verify the changes - please note that there are two groups:
* errors (or possible errors) - with title "fix possible use ..."
* false positives - warnings that _I_ think are not valid and the changes are made only to silence the compiler.

Applied, thanks

#111 lore raw
In reply to Andrzej Ostruszka

On a relate topic.

Last time, tried build a DPDK application using -fwhole-program gcc gave lots of warnings because it decided not to inline rte_memcpy.

Perhaps this might impact LTO as well. Really rte_memcpy_func should not be inline. We already optimize for the constant size case where inline makes sense. After that not so much.