Improve function versioning meson support
Subsystems (2)
cover Improve function versioning meson support
Adding support for LTO has exposed some issues with how the functions versioning was supported by meson, which was always set to build both shared and static libraries.
For plain C code, so long as the -fPIC compiler flag was passed, the output is identical whether or not the code is to be included in a static library or a dynamic one. Unfortunately, when using function versioning that no longer held as different macros were used for the versioned functions depending on which type of build it was. This means that any files that use versioning need to be built twice, with different defines in each case.
While the trivial solution here is just to rebuild everything twice, that involves a lot of unnecessary work when building DPDK. A better option is to identify those files or components which need multiple builds and rebuild only those. To do this, we add a new meson.build setting for libraries "use_function_versioning" and when that is set, we rebuild all source files twice, initially for static library and then with -DRTE_BUILD_SHARED_LIB for the shared library.
If the flag is not set, then the static versioning setting only is used, which could lead to the build succeeding but later causing problems. To avoid that, we add a new define which must be set when the versioning header is included. This addition while solving 1 problem raises 2 other, more minor problems:
* what to do with make builds? since make only builds one library type, we can just always define the new value.
* what about files that include rte_compat.h for the macro for "experimental"? To solve this, we can split compat.h in two, since the versioning macro should be internal only to DPDK (as no public header should expose anything but the latest APIs), while the experimental macros are primarily for public use.
Bruce Richardson (2):
eal: split compat header file
build: support building ABI versioned files twice
config/common_base | 1 +
config/rte_config.h | 3 ---
doc/api/doxy-api-index.md | 3 ++-
doc/guides/contributing/coding_style.rst | 7 ++++++
doc/guides/contributing/versioning.rst | 4 ++--
lib/librte_distributor/meson.build | 1 +
lib/librte_distributor/rte_distributor.c | 2 +-
lib/librte_distributor/rte_distributor_v20.c | 2 +-
lib/librte_eal/common/Makefile | 1 +
...rte_compat.h => rte_function_versioning.h} | 23 ++++++-------------
lib/librte_lpm/meson.build | 1 +
lib/librte_lpm/rte_lpm.c | 1 +
lib/librte_lpm/rte_lpm.h | 1 -
lib/librte_lpm/rte_lpm6.c | 1 +
lib/librte_timer/meson.build | 1 +
lib/librte_timer/rte_timer.c | 2 +-
lib/meson.build | 16 ++++++++++---
17 files changed, 41 insertions(+), 29 deletions(-)
rename lib/librte_eal/common/include/{rte_compat.h => rte_function_versioning.h} (89%)Signature · 1 line
2.21.0
PATCH 1/2 eal: split compat header file
The compat.h header file provided macros for two purposes:
1. it provided the macros for marking functions as rte_experimental
2. it provided the macros for doing function versioning
Although these were in the same file, #1 is something that is for use by public header files, which #2 is for internal use only. Therefore, we can split these into two headers, keeping #1 in rte_compat.h and #2 in a new file rte_function_versioning.h. For "make" builds, since internal objects pick up the headers from the "include/" folder, we need to add the new header to the installation list, but for "meson" builds it does not need to be installed as it's not for public use.
The rework also serves to allow the use of the function versioning macros to files that actually need them, so the use of experimental functions does not need including of the versioning code.
---
doc/api/doxy-api-index.md | 3 ++-
doc/guides/contributing/versioning.rst | 4 ++--
lib/librte_distributor/rte_distributor.c | 2 +-
lib/librte_distributor/rte_distributor_v20.c | 2 +-
lib/librte_eal/common/Makefile | 1 +
...rte_compat.h => rte_function_versioning.h} | 19 +++----------------
lib/librte_lpm/rte_lpm.c | 1 +
lib/librte_lpm/rte_lpm.h | 1 -
lib/librte_lpm/rte_lpm6.c | 1 +
lib/librte_timer/rte_timer.c | 2 +-
10 files changed, 13 insertions(+), 23 deletions(-)
rename lib/librte_eal/common/include/{rte_compat.h => rte_function_versioning.h} (89%)doc/api/doxy-api-index.md +2 -1
@@ -171,5 +171,6 @@ The public API headers are grouped by topics: - **misc**: [EAL config] (@ref rte_eal.h), [common] (@ref rte_common.h), - [ABI compat] (@ref rte_compat.h), + [experimental APIs] (@ref rte_compat.h), + [ABI versioning] (@ref rte_function_versioning.h), [version] (@ref rte_version.h)
doc/guides/contributing/versioning.rst +2 -2
@@ -206,7 +206,7 @@ functionality or behavior. When that occurs, it is desirable to allow for backward compatibility for a time with older binaries that are dynamically linked to the DPDK. -To support backward compatibility the ``rte_compat.h`` +To support backward compatibility the ``rte_function_versioning.h`` header file provides macros to use when updating exported functions. These macros are used in conjunction with the ``rte_<library>_version.map`` file for a given library to allow multiple versions of a symbol to exist in a shared
@@ -362,7 +362,7 @@ the function, we add this line of code VERSION_SYMBOL(rte_acl_create, _v20, 2.0); -Remembering to also add the rte_compat.h header to the requisite c file where +Remembering to also add the rte_function_versioning.h header to the requisite c file where these changes are being made. The above macro instructs the linker to create a new symbol ``rte_acl_create@DPDK_2.0``, which matches the symbol created in older builds, but now points to the above newly named function. We have now mapped
lib/librte_distributor/rte_distributor.c +1 -1
@@ -8,7 +8,7 @@#include <rte_mbuf.h> #include <rte_memory.h> #include <rte_cycles.h> -#include <rte_compat.h> +#include <rte_function_versioning.h> #include <rte_memzone.h> #include <rte_errno.h> #include <rte_string_fns.h>
lib/librte_distributor/rte_distributor_v20.c +1 -1
@@ -9,7 +9,7 @@#include <rte_memory.h> #include <rte_memzone.h> #include <rte_errno.h> -#include <rte_compat.h> +#include <rte_function_versioning.h> #include <rte_string_fns.h> #include <rte_eal_memconfig.h> #include <rte_pause.h>
lib/librte_eal/common/Makefile +1 -0
@@ -4,6 +4,7 @@ include $(RTE_SDK)/mk/rte.vars.mk INC := rte_branch_prediction.h rte_common.h rte_compat.h +INC += rte_function_versioning.h INC += rte_debug.h rte_eal.h rte_eal_interrupts.h INC += rte_errno.h rte_launch.h rte_lcore.h INC += rte_log.h rte_memory.h rte_memzone.h
lib/librte_eal/common/include/rte_function_versioning.h renamed from lib/librte_eal/common/include/rte_compat.h +3 -16
@@ -3,8 +3,8 @@* All rights reserved. */ -#ifndef _RTE_COMPAT_H_ -#define _RTE_COMPAT_H_ +#ifndef _RTE_FUNCTION_VERSIONING_H_ +#define _RTE_FUNCTION_VERSIONING_H_#include <rte_common.h> #ifdef RTE_BUILD_SHARED_LIB
@@ -76,17 +76,4 @@*/ #endif -#ifndef ALLOW_EXPERIMENTAL_API - -#define __rte_experimental \ -__attribute__((deprecated("Symbol is not yet part of stable ABI"), \ -section(".text.experimental"))) - -#else - -#define __rte_experimental \ -__attribute__((section(".text.experimental"))) - -#endif - -#endif /* _RTE_COMPAT_H_ */ +#endif /* _RTE_FUNCTION_VERSIONING_H_ */
lib/librte_lpm/rte_lpm.c +1 -0
@@ -22,6 +22,7 @@#include <rte_rwlock.h> #include <rte_spinlock.h> #include <rte_tailq.h> +#include <rte_function_versioning.h> #include "rte_lpm.h"
lib/librte_lpm/rte_lpm.h +0 -1
@@ -20,7 +20,6 @@#include <rte_memory.h> #include <rte_common.h> #include <rte_vect.h> -#include <rte_compat.h> #ifdef __cplusplusextern "C" {
lib/librte_lpm/rte_lpm6.c +1 -0
@@ -25,6 +25,7 @@#include <assert.h> #include <rte_jhash.h> #include <rte_tailq.h> +#include <rte_function_versioning.h> #include "rte_lpm6.h"
lib/librte_timer/rte_timer.c +1 -1
@@ -25,8 +25,8 @@#include <rte_pause.h> #include <rte_memzone.h> #include <rte_malloc.h> -#include <rte_compat.h> #include <rte_errno.h> +#include <rte_function_versioning.h> #include "rte_timer.h"
Signature · 1 line
2.21.0
Re: [dpdk-dev] [PATCH 1/2] eal: split compat header file
Quoted 17 lines · Bruce Richardson
The
compat.hheader file provided macros for two purposes:
1. it provided the macros for marking functions asrte_experimental
2. it provided the macros for doing function versioningAlthough these were in the same file, #1 is something that is for use by public header files, which #2 is for internal use only. Therefore, we can split these into two headers, keeping #1 in
rte_compat.hand #2 in a new filerte_function_versioning.h. For "make" builds, since internal objects pick up the headers from the "include/" folder, we need to add the new header to the installation list, but for "meson" builds it does not need to be installed as it's not for public use.The rework also serves to allow the use of the function versioning macros to files that actually need them, so the use of experimental functions does not need including of the versioning code.
Signed-off-by Bruce Richardson <[email protected]>
Quoted 14 lines
---
doc/api/doxy-api-index.md | 3 ++-
doc/guides/contributing/versioning.rst | 4 ++--
lib/librte_distributor/rte_distributor.c | 2 +-
lib/librte_distributor/rte_distributor_v20.c | 2 +-
lib/librte_eal/common/Makefile | 1 +
...rte_compat.h => rte_function_versioning.h} | 19 +++----------------
lib/librte_lpm/rte_lpm.c | 1 +
lib/librte_lpm/rte_lpm.h | 1 -
lib/librte_lpm/rte_lpm6.c | 1 +
lib/librte_timer/rte_timer.c | 2 +-
10 files changed, 13 insertions(+), 23 deletions(-)
rename lib/librte_eal/common/include/{rte_compat.h => rte_function_versioning.h} (89%)Apologies, but one of the parts of the split file somehow missed making the patch. I'll do a v2 soon.
PATCH 2/2 build: support building ABI versioned files twice
In reply to Bruce RichardsonAny file with ABI versioned functions needs different macros for shared and static builds, so we need to accomodate that. Rather than building everything twice, we just flag to the build system which libraries need that handling, by setting use_function_versioning in the meson.build files.
To ensure we don't get silent errors at build time due to this meson flag being missed, we add an explicit error to the function versioning header file if a known C macro is not defined. Since "make" builds always only build one of shared or static libraries, this define can be always set, and so is added to the common_base file. For meson, the build flag - and therefore the C define - is set for the three libraries that need the function versioning: "distributor", "lpm" and "timer".
--- config/common_base | 1 + config/rte_config.h | 3 --- doc/guides/contributing/coding_style.rst | 7 +++++++ lib/librte_distributor/meson.build | 1 + .../common/include/rte_function_versioning.h | 4 ++++ lib/librte_lpm/meson.build | 1 + lib/librte_timer/meson.build | 1 + lib/meson.build | 16 +++++++++++++--- 8 files changed, 28 insertions(+), 6 deletions(-)
config/common_base +1 -0
@@ -111,6 +111,7 @@ CONFIG_RTE_MAX_VFIO_CONTAINERS=64 CONFIG_RTE_MALLOC_DEBUG=n CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES=n CONFIG_RTE_USE_LIBBSD=n +CONFIG_RTE_USE_FUNCTION_VERSIONING=y # # Recognize/ignore the AVX/AVX512 CPU flags for performance/power testing.
config/rte_config.h +0 -3
@@ -31,9 +31,6 @@/****** library defines ********/ -/* compat defines */ -#define RTE_BUILD_SHARED_LIB -/* EAL defines */ #define RTE_MAX_HEAPS 32#define RTE_MAX_MEMSEG_LISTS 128
doc/guides/contributing/coding_style.rst +7 -0
@@ -948,6 +948,13 @@ reason built. For missing dependencies this should be of the form ``'missing dependency, "libname"'``. +use_function_versioning + **Default Value = false**. + Specifies if the library in question has ABI versioned functions. If it + has, this value should be set to ensure that the C files are compiled + twice with suitable parameters for each of shared or static library + builds. + version **Default Value = 1**. Specifies the ABI version of the library, and is used as the major
lib/librte_distributor/meson.build +1 -0
@@ -9,3 +9,4 @@ else endif headers = files('rte_distributor.h') deps += ['mbuf'] +use_function_versioning = true
lib/librte_eal/common/include/rte_function_versioning.h +4 -0
@@ -7,6 +7,10 @@#define _RTE_FUNCTION_VERSIONING_H_#include <rte_common.h> +#ifndef RTE_USE_FUNCTION_VERSIONING +#error Use of function versioning disabled, is "use_function_versioning=true" in meson.build? +#endif +#ifdef RTE_BUILD_SHARED_LIB /*
lib/librte_lpm/meson.build +1 -0
@@ -8,3 +8,4 @@ headers = files('rte_lpm.h', 'rte_lpm6.h') # without worrying about which architecture we actually need headers += files('rte_lpm_altivec.h', 'rte_lpm_neon.h', 'rte_lpm_sse.h') deps += ['hash'] +use_function_versioning = true
lib/librte_timer/meson.build +1 -0
@@ -4,3 +4,4 @@ sources = files('rte_timer.c') headers = files('rte_timer.h') allow_experimental_apis = true +use_function_versioning = true
lib/meson.build +13 -3
@@ -47,6 +47,7 @@ foreach l:libraries name = l version = 1 allow_experimental_apis = false + use_function_versioning = false sources = [] headers = [] includes = []
@@ -96,6 +97,9 @@ foreach l:libraries if allow_experimental_apis cflags += '-DALLOW_EXPERIMENTAL_API' endif + if use_function_versioning + cflags += '-DRTE_USE_FUNCTION_VERSIONING' + endif if get_option('per_library_versions') lib_version = '@[email protected]'.format(version)
@@ -117,9 +121,15 @@ foreach l:libraries include_directories: includes, dependencies: static_deps) - # then use pre-build objects to build shared lib - sources = [] - objs += static_lib.extract_all_objects(recursive: false) + if not use_function_versioning + # use pre-build objects to build shared lib + sources = [] + objs += static_lib.extract_all_objects(recursive: false) + else + # for compat we need to rebuild with + # RTE_BUILD_SHARED_LIB defined + cflags += '-DRTE_BUILD_SHARED_LIB' + endif version_map = '@0@/@1@/rte_@2@_version.map'.format( meson.current_source_dir(), dir_name, name) implib = dir_name + '.dll.a'
Signature · 1 line
2.21.0
v2 cover Improve function versioning meson support
In reply to Bruce RichardsonAdding support for LTO has exposed some issues with how the functions versioning was supported by meson, which was always set to build both shared and static libraries.
For plain C code, so long as the -fPIC compiler flag was passed, the output is identical whether or not the code is to be included in a static library or a dynamic one. Unfortunately, when using function versioning that no longer held as different macros were used for the versioned functions depending on which type of build it was. This means that any files that use versioning need to be built twice, with different defines in each case.
While the trivial solution here is just to rebuild everything twice, that involves a lot of unnecessary work when building DPDK. A better option is to identify those files or components which need multiple builds and rebuild only those. To do this, we add a new meson.build setting for libraries "use_function_versioning" and when that is set, we rebuild all source files twice, initially for static library and then with -DRTE_BUILD_SHARED_LIB for the shared library.
If the flag is not set, then the static versioning setting only is used, which could lead to the build succeeding but later causing problems. To avoid that, we add a new define which must be set when the versioning header is included. This addition while solving 1 problem raises 2 other, more minor problems:
* what to do with make builds? since make only builds one library type, we can just always define the new value.
* what about files that include rte_compat.h for the macro for "experimental"? To solve this, we can split compat.h in two, since the versioning macro should be internal only to DPDK (as no public header should expose anything but the latest APIs), while the experimental macros are primarily for public use.
V2: added in file that missed a "git add" when doing V1
Bruce Richardson (2):
eal: split compat header file
build: support building ABI versioned files twice
config/common_base | 1 + config/rte_config.h | 3 - doc/api/doxy-api-index.md | 3 +- doc/guides/contributing/coding_style.rst | 7 ++ doc/guides/contributing/versioning.rst | 4 +- lib/librte_distributor/meson.build | 1 + lib/librte_distributor/rte_distributor.c | 2 +- lib/librte_distributor/rte_distributor_v20.c | 2 +- lib/librte_eal/common/Makefile | 1 + lib/librte_eal/common/include/rte_compat.h | 70 ---------------- .../common/include/rte_function_versioning.h | 83 +++++++++++++++++++ lib/librte_lpm/meson.build | 1 + lib/librte_lpm/rte_lpm.c | 1 + lib/librte_lpm/rte_lpm.h | 1 - lib/librte_lpm/rte_lpm6.c | 1 + lib/librte_timer/meson.build | 1 + lib/librte_timer/rte_timer.c | 2 +- lib/meson.build | 16 +++- 18 files changed, 117 insertions(+), 83 deletions(-) create mode 100644 lib/librte_eal/common/include/rte_function_versioning.h
Signature · 1 line
2.21.0
PATCH v2 1/2 eal: split compat header file
The compat.h header file provided macros for two purposes:
1. it provided the macros for marking functions as rte_experimental
2. it provided the macros for doing function versioning
Although these were in the same file, #1 is something that is for use by public header files, which #2 is for internal use only. Therefore, we can split these into two headers, keeping #1 in rte_compat.h and #2 in a new file rte_function_versioning.h. For "make" builds, since internal objects pick up the headers from the "include/" folder, we need to add the new header to the installation list, but for "meson" builds it does not need to be installed as it's not for public use.
The rework also serves to allow the use of the function versioning macros to files that actually need them, so the use of experimental functions does not need including of the versioning code.
---
V2: added in missing rte_compat.h file
--- doc/api/doxy-api-index.md | 3 +- doc/guides/contributing/versioning.rst | 4 +- lib/librte_distributor/rte_distributor.c | 2 +- lib/librte_distributor/rte_distributor_v20.c | 2 +- lib/librte_eal/common/Makefile | 1 + lib/librte_eal/common/include/rte_compat.h | 70 ---------------- .../common/include/rte_function_versioning.h | 79 +++++++++++++++++++ lib/librte_lpm/rte_lpm.c | 1 + lib/librte_lpm/rte_lpm.h | 1 - lib/librte_lpm/rte_lpm6.c | 1 + lib/librte_timer/rte_timer.c | 2 +- 11 files changed, 89 insertions(+), 77 deletions(-) create mode 100644 lib/librte_eal/common/include/rte_function_versioning.h
doc/api/doxy-api-index.md +2 -1
@@ -171,5 +171,6 @@ The public API headers are grouped by topics: - **misc**: [EAL config] (@ref rte_eal.h), [common] (@ref rte_common.h), - [ABI compat] (@ref rte_compat.h), + [experimental APIs] (@ref rte_compat.h), + [ABI versioning] (@ref rte_function_versioning.h), [version] (@ref rte_version.h)
doc/guides/contributing/versioning.rst +2 -2
@@ -206,7 +206,7 @@ functionality or behavior. When that occurs, it is desirable to allow for backward compatibility for a time with older binaries that are dynamically linked to the DPDK. -To support backward compatibility the ``rte_compat.h`` +To support backward compatibility the ``rte_function_versioning.h`` header file provides macros to use when updating exported functions. These macros are used in conjunction with the ``rte_<library>_version.map`` file for a given library to allow multiple versions of a symbol to exist in a shared
@@ -362,7 +362,7 @@ the function, we add this line of code VERSION_SYMBOL(rte_acl_create, _v20, 2.0); -Remembering to also add the rte_compat.h header to the requisite c file where +Remembering to also add the rte_function_versioning.h header to the requisite c file where these changes are being made. The above macro instructs the linker to create a new symbol ``rte_acl_create@DPDK_2.0``, which matches the symbol created in older builds, but now points to the above newly named function. We have now mapped
lib/librte_distributor/rte_distributor.c +1 -1
@@ -8,7 +8,7 @@#include <rte_mbuf.h> #include <rte_memory.h> #include <rte_cycles.h> -#include <rte_compat.h> +#include <rte_function_versioning.h> #include <rte_memzone.h> #include <rte_errno.h> #include <rte_string_fns.h>
lib/librte_distributor/rte_distributor_v20.c +1 -1
@@ -9,7 +9,7 @@#include <rte_memory.h> #include <rte_memzone.h> #include <rte_errno.h> -#include <rte_compat.h> +#include <rte_function_versioning.h> #include <rte_string_fns.h> #include <rte_eal_memconfig.h> #include <rte_pause.h>
lib/librte_eal/common/Makefile +1 -0
@@ -4,6 +4,7 @@ include $(RTE_SDK)/mk/rte.vars.mk INC := rte_branch_prediction.h rte_common.h rte_compat.h +INC += rte_function_versioning.h INC += rte_debug.h rte_eal.h rte_eal_interrupts.h INC += rte_errno.h rte_launch.h rte_lcore.h INC += rte_log.h rte_memory.h rte_memzone.h
lib/librte_eal/common/include/rte_compat.h +0 -70
76 lines not on this page · raw message
lib/librte_eal/common/include/rte_function_versioning.h new file +79 -0
79 lines not on this page · raw message
lib/librte_lpm/rte_lpm.c +1 -0
@@ -22,6 +22,7 @@#include <rte_rwlock.h> #include <rte_spinlock.h> #include <rte_tailq.h> +#include <rte_function_versioning.h> #include "rte_lpm.h"
lib/librte_lpm/rte_lpm.h +0 -1
@@ -20,7 +20,6 @@#include <rte_memory.h> #include <rte_common.h> #include <rte_vect.h> -#include <rte_compat.h> #ifdef __cplusplusextern "C" {
lib/librte_lpm/rte_lpm6.c +1 -0
@@ -25,6 +25,7 @@#include <assert.h> #include <rte_jhash.h> #include <rte_tailq.h> +#include <rte_function_versioning.h> #include "rte_lpm6.h"
lib/librte_timer/rte_timer.c +1 -1
@@ -25,8 +25,8 @@#include <rte_pause.h> #include <rte_memzone.h> #include <rte_malloc.h> -#include <rte_compat.h> #include <rte_errno.h> +#include <rte_function_versioning.h> #include "rte_timer.h"
Signature · 1 line
2.21.0
PATCH v2 2/2 build: support building ABI versioned files twice
In reply to Bruce RichardsonAny file with ABI versioned functions needs different macros for shared and static builds, so we need to accomodate that. Rather than building everything twice, we just flag to the build system which libraries need that handling, by setting use_function_versioning in the meson.build files.
To ensure we don't get silent errors at build time due to this meson flag being missed, we add an explicit error to the function versioning header file if a known C macro is not defined. Since "make" builds always only build one of shared or static libraries, this define can be always set, and so is added to the common_base file. For meson, the build flag - and therefore the C define - is set for the three libraries that need the function versioning: "distributor", "lpm" and "timer".
--- config/common_base | 1 + config/rte_config.h | 3 --- doc/guides/contributing/coding_style.rst | 7 +++++++ lib/librte_distributor/meson.build | 1 + .../common/include/rte_function_versioning.h | 4 ++++ lib/librte_lpm/meson.build | 1 + lib/librte_timer/meson.build | 1 + lib/meson.build | 16 +++++++++++++--- 8 files changed, 28 insertions(+), 6 deletions(-)
config/common_base +1 -0
@@ -111,6 +111,7 @@ CONFIG_RTE_MAX_VFIO_CONTAINERS=64 CONFIG_RTE_MALLOC_DEBUG=n CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES=n CONFIG_RTE_USE_LIBBSD=n +CONFIG_RTE_USE_FUNCTION_VERSIONING=y # # Recognize/ignore the AVX/AVX512 CPU flags for performance/power testing.
config/rte_config.h +0 -3
@@ -31,9 +31,6 @@/****** library defines ********/ -/* compat defines */ -#define RTE_BUILD_SHARED_LIB -/* EAL defines */ #define RTE_MAX_HEAPS 32#define RTE_MAX_MEMSEG_LISTS 128
doc/guides/contributing/coding_style.rst +7 -0
@@ -948,6 +948,13 @@ reason built. For missing dependencies this should be of the form ``'missing dependency, "libname"'``. +use_function_versioning + **Default Value = false**. + Specifies if the library in question has ABI versioned functions. If it + has, this value should be set to ensure that the C files are compiled + twice with suitable parameters for each of shared or static library + builds. + version **Default Value = 1**. Specifies the ABI version of the library, and is used as the major
lib/librte_distributor/meson.build +1 -0
@@ -9,3 +9,4 @@ else endif headers = files('rte_distributor.h') deps += ['mbuf'] +use_function_versioning = true
lib/librte_eal/common/include/rte_function_versioning.h +4 -0
@@ -7,6 +7,10 @@#define _RTE_FUNCTION_VERSIONING_H_#include <rte_common.h> +#ifndef RTE_USE_FUNCTION_VERSIONING +#error Use of function versioning disabled, is "use_function_versioning=true" in meson.build? +#endif +#ifdef RTE_BUILD_SHARED_LIB /*
lib/librte_lpm/meson.build +1 -0
@@ -8,3 +8,4 @@ headers = files('rte_lpm.h', 'rte_lpm6.h') # without worrying about which architecture we actually need headers += files('rte_lpm_altivec.h', 'rte_lpm_neon.h', 'rte_lpm_sse.h') deps += ['hash'] +use_function_versioning = true
lib/librte_timer/meson.build +1 -0
@@ -4,3 +4,4 @@ sources = files('rte_timer.c') headers = files('rte_timer.h') allow_experimental_apis = true +use_function_versioning = true
lib/meson.build +13 -3
@@ -47,6 +47,7 @@ foreach l:libraries name = l version = 1 allow_experimental_apis = false + use_function_versioning = false sources = [] headers = [] includes = []
@@ -96,6 +97,9 @@ foreach l:libraries if allow_experimental_apis cflags += '-DALLOW_EXPERIMENTAL_API' endif + if use_function_versioning + cflags += '-DRTE_USE_FUNCTION_VERSIONING' + endif if get_option('per_library_versions') lib_version = '@[email protected]'.format(version)
@@ -117,9 +121,15 @@ foreach l:libraries include_directories: includes, dependencies: static_deps) - # then use pre-build objects to build shared lib - sources = [] - objs += static_lib.extract_all_objects(recursive: false) + if not use_function_versioning + # use pre-build objects to build shared lib + sources = [] + objs += static_lib.extract_all_objects(recursive: false) + else + # for compat we need to rebuild with + # RTE_BUILD_SHARED_LIB defined + cflags += '-DRTE_BUILD_SHARED_LIB' + endif version_map = '@0@/@1@/rte_@2@_version.map'.format( meson.current_source_dir(), dir_name, name) implib = dir_name + '.dll.a'
Signature · 1 line
2.21.0
Re: [dpdk-dev] [PATCH v2 2/2] build: support building ABI versioned files twice
Thanks Bruce for the patch. I like the idea of splitting versioning out of rte_compat.h, but I have some comments.
On 9/27/19 10:59 PM, Bruce Richardson wrote:
[...]
config/common_base CONFIG_RTE_MAX_VFIO_CONTAINERS=64 · line 114
Quoted 7 lines
config/common_base +1
-0@@ -111,6 +111,7 @@ CONFIG_RTE_MAX_VFIO_CONTAINERS=64 CONFIG_RTE_MALLOC_DEBUG=n CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES=n CONFIG_RTE_USE_LIBBSD=n +CONFIG_RTE_USE_FUNCTION_VERSIONING=y
I'm not fond of this config option - it is not really an option to be changed by the user. I would prefer to just add flag to CFLAGS in mk/target/generic/rte.vars.mk.
config/rte_config.h line 36
Quoted 13 lines
config/common_base +0
-0# # Recognize/ignore the AVX/AVX512 CPU flags for performance/power testing.config/rte_config.h +0
-3@@ -31,9 +31,6 @@/****** library defines ********/ -/* compat defines */ -#define RTE_BUILD_SHARED_LIB -
So now everything builds "as static lib" (but with "-fPIC") apart from those libraries that use symbol versioning. I'm OK with that however I'd like to note that code might be using RTE_BUILD_SHARED_LIB and do different things e.g. app/test-bbdev/test_bbdev_perf.c. I know that was already the case - just wanted to say that aloud to make sure we are all aware of this :).
Quoted 5 lines
doc/guides/contributing/coding_style.rst +0
-0
doc/guides/contributing/coding_style.rst reason · line 957
Quoted 10 lines
doc/guides/contributing/coding_style.rst +7
-0@@ -948,6 +948,13 @@ reason built. For missing dependencies this should be of the form ``'missing dependency, "libname"'``. +use_function_versioning + **Default Value = false**. + Specifies if the library in question has ABI versioned functions. If it + has, this value should be set to ensure that the C files are compiled + twice with suitable parameters for each of shared or static library + builds. +
Maybe a different name for this option? In general an "ideal theoretical" solution would be for build system to figure out on its own that separate build is necessary automatically - but that might incur some performance penalty (additional grep'ing of sources or so). So I'm fine with this option however I'd like to rename it to actually indicate what it's effect is. Like 'separate_build' or 'split_build' or 'rebuild_objects' or ...
The intention of using of versioned symbols is already indicated by inclusion of the relevant header.
lib/librte_eal/common/include/rte_function_versioning.h line 12
Quoted 11 lines
lib/librte_eal/common/include/rte_function_versioning.h +3
-0@@ -7,6 +7,10 @@#define _RTE_FUNCTION_VERSIONING_H_#include <rte_common.h> +#ifndef RTE_USE_FUNCTION_VERSIONING +#error Use of function versioning disabled, is "use_function_versioning=true" in meson.build? +#endif
If you accept above suggestion then change this message to something
like: "Function versioning requires 'separate_build=true' in meson.build"
BTW it turned out that this need for separate build for versioned symbols is a result of long standing gcc bug:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=48200
I'll test this with clang and if this will work then maybe we could guard this #if with another check for 'gcc'.
Best regards
Andrzej
Re: [dpdk-dev] [PATCH v2 2/2] build: support building ABI versioned files twice
Quoted 5 lines · Andrzej Ostruszka
Thanks Bruce for the patch. I like the idea of splitting versioning out of
rte_compat.h, but I have some comments.On 9/27/19 10:59 PM, Bruce Richardson wrote:
[...]
Quoted 12 lines
config/common_base CONFIG_RTE_MAX_VFIO_CONTAINERS=64 · line 114
Quoted 7 lines
config/common_base +1
-0@@ -111,6 +111,7 @@ CONFIG_RTE_MAX_VFIO_CONTAINERS=64 CONFIG_RTE_MALLOC_DEBUG=n CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES=n CONFIG_RTE_USE_LIBBSD=n +CONFIG_RTE_USE_FUNCTION_VERSIONING=yI'm not fond of this config option - it is not really an option to be changed by the user. I would prefer to just add flag to CFLAGS in
mk/target/generic/rte.vars.mk.
Ok, that sounds reasonable enough.
Quoted 10 lines
Quoted 10 lines
config/common_base +0
-0# # Recognize/ignore the AVX/AVX512 CPU flags for performance/power testing.config/rte_config.h +0
-0@@ -31,9 +31,6 @@/****** library defines ********/
Quoted 10 lines
config/rte_config.h line 36
Quoted 3 lines
config/rte_config.h +0
-3-/* compat defines */ -#define RTE_BUILD_SHARED_LIB -So now everything builds "as static lib" (but with "-fPIC") apart from those libraries that use symbol versioning. I'm OK with that however I'd like to note that code might be using
RTE_BUILD_SHARED_LIBand do different things e.g.app/test-bbdev/test_bbdev_perf.c. I know that was already the case - just wanted to say that aloud to make sure we are all aware of this :).
Thanks for pointing this out, I wasn't aware of it! Doing a git grep this seems to be the only place in a C file (other than rte_config.h and rte_compat.h) where the SHARED_LIB flag is being checked. I'll need to follow up on that to see what the logic is there, because it seems strange to require such a check.
Quoted 9 lines
Quoted 9 lines
doc/guides/contributing/coding_style.rst +0
-0@@ -948,6 +948,13 @@ reason built. For missing dependencies this should be of the form ``'missing dependency, "libname"'``.
Quoted 12 lines
doc/guides/contributing/coding_style.rst reason · line 957
Quoted 7 lines
doc/guides/contributing/coding_style.rst +7
-0+use_function_versioning + **Default Value = false**. + Specifies if the library in question has ABI versioned functions. If it + has, this value should be set to ensure that the C files are compiled + twice with suitable parameters for each of shared or static library + builds. +Maybe a different name for this option? In general an "ideal theoretical" solution would be for build system to figure out on its own that separate build is necessary automatically - but that might incur some performance penalty (additional grep'ing of sources or so).
I was thinking about that, and how we can do it automatically. The trouble is that for correctness we would need to recheck every file after it had changed, and since the result of the check means that we have different build steps it would basically mean doing a full reconfiguration for every file change. That's not really practical, hence this proposed solution.
Quoted 7 lines
So I'm
fine with this option however I'd like to rename it to actually indicate what it's effect is. Like 'separate_build' or 'split_build' or 'rebuild_objects' or ...The intention of using of versioned symbols is already indicated by inclusion of the relevant header.
I actually feel the opposite. I'd rather have the name tied in to the fact that it's related to using function versioning - subject to what is found on investigating the #ifdef in the bbdev perf test. However, if you feel strongly about the name something else, I can probably compromise on it :-)
Quoted 9 lines
Quoted 9 lines
lib/librte_eal/common/include/rte_function_versioning.h +0
-0@@ -7,6 +7,10 @@#define _RTE_FUNCTION_VERSIONING_H_#include <rte_common.h>
Quoted 7 lines
lib/librte_eal/common/include/rte_function_versioning.h line 12
Quoted 3 lines
lib/librte_eal/common/include/rte_function_versioning.h +3
-0+#ifndef RTE_USE_FUNCTION_VERSIONING +#error Use of function versioning disabled, is "use_function_versioning=true" in meson.build? +#endif If you accept above suggestion then change this message to something
like: "Function versioning requires 'separate_build=true' in meson.build"
Agreed. This obviously needs to be kept in sync with whatever the build flag is.
Quoted 11 lines
BTW it turned out that this need for separate build for versioned symbols is a result of long standing gcc bug:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=48200I'll test this with clang and if this will work then maybe we could guard this #if with another check for 'gcc'.
Best regards
AndrzejTested-by Andrzej Ostruszka <[email protected]>
Let me know what your testing throws up, thanks.
/Bruce
Re: [dpdk-dev] [PATCH v2 2/2] build: support building ABI versioned files twice
Quoted 19 lines · Bruce Richardson
Quoted 5 lines · Andrzej Ostruszka
Thanks Bruce for the patch. I like the idea of splitting versioning out of
rte_compat.h, but I have some comments.On 9/27/19 10:59 PM, Bruce Richardson wrote:
[...]Quoted 12 lines
config/common_base CONFIG_RTE_MAX_VFIO_CONTAINERS=64 · line 114
Quoted 7 lines
config/common_base +1
-0@@ -111,6 +111,7 @@ CONFIG_RTE_MAX_VFIO_CONTAINERS=64 CONFIG_RTE_MALLOC_DEBUG=n CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES=n CONFIG_RTE_USE_LIBBSD=n +CONFIG_RTE_USE_FUNCTION_VERSIONING=yI'm not fond of this config option - it is not really an option to be changed by the user. I would prefer to just add flag to CFLAGS in
mk/target/generic/rte.vars.mk.
Quoted 1 line
Ok, that sounds reasonable enough.
Done in V3.
Quoted 22 lines
Quoted 11 lines
Quoted 11 lines
config/common_base +0
-0# # Recognize/ignore the AVX/AVX512 CPU flags for performance/power testing.config/rte_config.h +0
-0@@ -31,9 +31,6 @@/****** library defines ********/ Quoted 10 lines
config/rte_config.h line 36
Quoted 3 lines
config/rte_config.h +0
-3-/* compat defines */ -#define RTE_BUILD_SHARED_LIB -So now everything builds "as static lib" (but with "-fPIC") apart from those libraries that use symbol versioning. I'm OK with that however I'd like to note that code might be using
RTE_BUILD_SHARED_LIBand do different things e.g.app/test-bbdev/test_bbdev_perf.c. I know that was already the case - just wanted to say that aloud to make sure we are all aware of this :).
Quoted 6 lines
Thanks for pointing this out, I wasn't aware of it! Doing a git grep this seems to be the only place in a C file (other than
rte_config.handrte_compat.h) where theSHARED_LIBflag is being checked. I'll need to follow up on that to see what the logic is there, because it seems strange to require such a check.
This #ifdef can be removed, see patchset:
http://patches.dpdk.org/project/dpdk/list/?series=6699
This then leaves function versioning as the only use-case where we need different code paths for static vs shared builds.
Quoted 22 lines
Quoted 9 lines
Quoted 9 lines
doc/guides/contributing/coding_style.rst +0
-0@@ -948,6 +948,13 @@ reason built. For missing dependencies this should be of the form ``'missing dependency, "libname"'``.Quoted 12 lines
doc/guides/contributing/coding_style.rst reason · line 957
Quoted 7 lines
doc/guides/contributing/coding_style.rst +7
-0+use_function_versioning + **Default Value = false**. + Specifies if the library in question has ABI versioned functions. If it + has, this value should be set to ensure that the C files are compiled + twice with suitable parameters for each of shared or static library + builds. +Maybe a different name for this option? In general an "ideal theoretical" solution would be for build system to figure out on its own that separate build is necessary automatically - but that might incur some performance penalty (additional grep'ing of sources or so).
Quoted 6 lines
I was thinking about that, and how we can do it automatically. The trouble is that for correctness we would need to recheck every file after it had changed, and since the result of the check means that we have different build steps it would basically mean doing a full reconfiguration for every file change. That's not really practical, hence this proposed solution.
I've not made any changes here for the V3. However, if we want to reduce the number of changes required, we could always switch to having the rte_function_versioning.h header file included on the basis of the flag in the meson.build file. Having the C flag compile vary based on the meson one is normal, having the inverse is problematic because of what I explained above - you'd basically need to reconfigure to check after each file change.
Personally, I don't think changing things to auto-include the header is worth it, hence the fact of no-change in v3.
Regards,
/Bruce
v3 cover Improve function versioning meson support
In reply to Bruce RichardsonAdding support for LTO has exposed some issues with how the functions versioning was supported by meson, which was always set to build both shared and static libraries.
For plain C code, so long as the -fPIC compiler flag was passed, the output is identical whether or not the code is to be included in a static library or a dynamic one. Unfortunately, when using function versioning that no longer held as different macros were used for the versioned functions depending on which type of build it was. This means that any files that use versioning need to be built twice, with different defines in each case.
While the trivial solution here is just to rebuild everything twice, that involves a lot of unnecessary work when building DPDK. A better option is to identify those files or components which need multiple builds and rebuild only those. To do this, we add a new meson.build setting for libraries "use_function_versioning" and when that is set, we rebuild all source files twice, initially for static library and then with -DRTE_BUILD_SHARED_LIB for the shared library.
If the flag is not set, then the static versioning setting only is used, which could lead to the build succeeding but later causing problems. To avoid that, we add a new define which must be set when the versioning header is included. This addition while solving 1 problem raises 2 other, more minor problems:
* what to do with make builds? since make only builds one library type, we can just always define the new value.
* what about files that include rte_compat.h for the macro for "experimental"? To solve this, we can split compat.h in two, since the versioning macro should be internal only to DPDK (as no public header should expose anything but the latest APIs), while the experimental macros are primarily for public use.
V3: following feedback, moved the define for make from the common_base
build config file to one of the global makefiles, as it's not user tunable.
V2: added in file that missed a "git add" when doing V1
Bruce Richardson (2):
eal: split compat header file
build: support building ABI versioned files twice
config/rte_config.h | 3 - doc/api/doxy-api-index.md | 3 +- doc/guides/contributing/coding_style.rst | 7 ++ doc/guides/contributing/versioning.rst | 4 +- lib/librte_distributor/meson.build | 1 + lib/librte_distributor/rte_distributor.c | 2 +- lib/librte_distributor/rte_distributor_v20.c | 2 +- lib/librte_eal/common/Makefile | 1 + lib/librte_eal/common/include/rte_compat.h | 70 ---------------- .../common/include/rte_function_versioning.h | 83 +++++++++++++++++++ lib/librte_lpm/meson.build | 1 + lib/librte_lpm/rte_lpm.c | 1 + lib/librte_lpm/rte_lpm.h | 1 - lib/librte_lpm/rte_lpm6.c | 1 + lib/librte_timer/meson.build | 1 + lib/librte_timer/rte_timer.c | 2 +- lib/meson.build | 16 +++- mk/target/generic/rte.vars.mk | 8 ++ 18 files changed, 124 insertions(+), 83 deletions(-) create mode 100644 lib/librte_eal/common/include/rte_function_versioning.h
Signature · 1 line
2.21.0
PATCH v3 1/2 eal: split compat header file
The compat.h header file provided macros for two purposes:
1. it provided the macros for marking functions as rte_experimental
2. it provided the macros for doing function versioning
Although these were in the same file, #1 is something that is for use by public header files, which #2 is for internal use only. Therefore, we can split these into two headers, keeping #1 in rte_compat.h and #2 in a new file rte_function_versioning.h. For "make" builds, since internal objects pick up the headers from the "include/" folder, we need to add the new header to the installation list, but for "meson" builds it does not need to be installed as it's not for public use.
The rework also serves to allow the use of the function versioning macros to files that actually need them, so the use of experimental functions does not need including of the versioning code.
--- doc/api/doxy-api-index.md | 3 +- doc/guides/contributing/versioning.rst | 4 +- lib/librte_distributor/rte_distributor.c | 2 +- lib/librte_distributor/rte_distributor_v20.c | 2 +- lib/librte_eal/common/Makefile | 1 + lib/librte_eal/common/include/rte_compat.h | 70 ---------------- .../common/include/rte_function_versioning.h | 79 +++++++++++++++++++ lib/librte_lpm/rte_lpm.c | 1 + lib/librte_lpm/rte_lpm.h | 1 - lib/librte_lpm/rte_lpm6.c | 1 + lib/librte_timer/rte_timer.c | 2 +- 11 files changed, 89 insertions(+), 77 deletions(-) create mode 100644 lib/librte_eal/common/include/rte_function_versioning.h
doc/api/doxy-api-index.md +2 -1
@@ -171,5 +171,6 @@ The public API headers are grouped by topics: - **misc**: [EAL config] (@ref rte_eal.h), [common] (@ref rte_common.h), - [ABI compat] (@ref rte_compat.h), + [experimental APIs] (@ref rte_compat.h), + [ABI versioning] (@ref rte_function_versioning.h), [version] (@ref rte_version.h)
doc/guides/contributing/versioning.rst +2 -2
@@ -206,7 +206,7 @@ functionality or behavior. When that occurs, it is desirable to allow for backward compatibility for a time with older binaries that are dynamically linked to the DPDK. -To support backward compatibility the ``rte_compat.h`` +To support backward compatibility the ``rte_function_versioning.h`` header file provides macros to use when updating exported functions. These macros are used in conjunction with the ``rte_<library>_version.map`` file for a given library to allow multiple versions of a symbol to exist in a shared
@@ -362,7 +362,7 @@ the function, we add this line of code VERSION_SYMBOL(rte_acl_create, _v20, 2.0); -Remembering to also add the rte_compat.h header to the requisite c file where +Remembering to also add the rte_function_versioning.h header to the requisite c file where these changes are being made. The above macro instructs the linker to create a new symbol ``rte_acl_create@DPDK_2.0``, which matches the symbol created in older builds, but now points to the above newly named function. We have now mapped
lib/librte_distributor/rte_distributor.c +1 -1
@@ -8,7 +8,7 @@#include <rte_mbuf.h> #include <rte_memory.h> #include <rte_cycles.h> -#include <rte_compat.h> +#include <rte_function_versioning.h> #include <rte_memzone.h> #include <rte_errno.h> #include <rte_string_fns.h>
lib/librte_distributor/rte_distributor_v20.c +1 -1
@@ -9,7 +9,7 @@#include <rte_memory.h> #include <rte_memzone.h> #include <rte_errno.h> -#include <rte_compat.h> +#include <rte_function_versioning.h> #include <rte_string_fns.h> #include <rte_eal_memconfig.h> #include <rte_pause.h>
lib/librte_eal/common/Makefile +1 -0
@@ -4,6 +4,7 @@ include $(RTE_SDK)/mk/rte.vars.mk INC := rte_branch_prediction.h rte_common.h rte_compat.h +INC += rte_function_versioning.h INC += rte_debug.h rte_eal.h rte_eal_interrupts.h INC += rte_errno.h rte_launch.h rte_lcore.h INC += rte_log.h rte_memory.h rte_memzone.h
lib/librte_eal/common/include/rte_compat.h +0 -70
76 lines not on this page · raw message
lib/librte_eal/common/include/rte_function_versioning.h new file +79 -0
79 lines not on this page · raw message
lib/librte_lpm/rte_lpm.c +1 -0
@@ -22,6 +22,7 @@#include <rte_rwlock.h> #include <rte_spinlock.h> #include <rte_tailq.h> +#include <rte_function_versioning.h> #include "rte_lpm.h"
lib/librte_lpm/rte_lpm.h +0 -1
@@ -20,7 +20,6 @@#include <rte_memory.h> #include <rte_common.h> #include <rte_vect.h> -#include <rte_compat.h> #ifdef __cplusplusextern "C" {
lib/librte_lpm/rte_lpm6.c +1 -0
@@ -25,6 +25,7 @@#include <assert.h> #include <rte_jhash.h> #include <rte_tailq.h> +#include <rte_function_versioning.h> #include "rte_lpm6.h"
lib/librte_timer/rte_timer.c +1 -1
@@ -25,8 +25,8 @@#include <rte_pause.h> #include <rte_memzone.h> #include <rte_malloc.h> -#include <rte_compat.h> #include <rte_errno.h> +#include <rte_function_versioning.h> #include "rte_timer.h"
Signature · 1 line
2.21.0
Re: [dpdk-dev] [PATCH v3 1/2] eal: split compat header file
07/10/2019 17:45, Bruce Richardson:
Quoted 3 lines
The
compat.hheader file provided macros for two purposes:
1. it provided the macros for marking functions asrte_experimental
2. it provided the macros for doing function versioning
[...]
Quoted 2 lines
lib/librte_eal/common/include/rte_compat.h | 70 ---------------- .../common/include/rte_function_versioning.h | 79 +++++++++++++++++++
I take this opportunity to add (while merging) the files in MAINTAINERS, in the section "ABI versioning" of Neil Horman,
so it is clear what are the pieces of ABI code.
PATCH v3 2/2 build: support building ABI versioned files twice
In reply to Bruce RichardsonAny file with ABI versioned functions needs different macros for shared and static builds, so we need to accomodate that. Rather than building everything twice, we just flag to the build system which libraries need that handling, by setting use_function_versioning in the meson.build files.
To ensure we don't get silent errors at build time due to this meson flag being missed, we add an explicit error to the function versioning header file if a known C macro is not defined. Since "make" builds always only build one of shared or static libraries, this define can be always set, and so is added to the global CFLAGS. For meson, the build flag - and therefore the C define - is set for the three libraries that need the function
versioning: "distributor", "lpm" and "timer".
---
v3: move define from common_base to generic/rte.vars.mk
--- config/rte_config.h | 3 --- doc/guides/contributing/coding_style.rst | 7 +++++++ lib/librte_distributor/meson.build | 1 + .../common/include/rte_function_versioning.h | 4 ++++ lib/librte_lpm/meson.build | 1 + lib/librte_timer/meson.build | 1 + lib/meson.build | 16 +++++++++++++--- mk/target/generic/rte.vars.mk | 8 ++++++++ 8 files changed, 35 insertions(+), 6 deletions(-)
config/rte_config.h +0 -3
@@ -31,9 +31,6 @@/****** library defines ********/ -/* compat defines */ -#define RTE_BUILD_SHARED_LIB -/* EAL defines */ #define RTE_MAX_HEAPS 32#define RTE_MAX_MEMSEG_LISTS 128
doc/guides/contributing/coding_style.rst +7 -0
@@ -948,6 +948,13 @@ reason built. For missing dependencies this should be of the form ``'missing dependency, "libname"'``. +use_function_versioning + **Default Value = false**. + Specifies if the library in question has ABI versioned functions. If it + has, this value should be set to ensure that the C files are compiled + twice with suitable parameters for each of shared or static library + builds. + version **Default Value = 1**. Specifies the ABI version of the library, and is used as the major
lib/librte_distributor/meson.build +1 -0
@@ -9,3 +9,4 @@ else endif headers = files('rte_distributor.h') deps += ['mbuf'] +use_function_versioning = true
lib/librte_eal/common/include/rte_function_versioning.h +4 -0
@@ -7,6 +7,10 @@#define _RTE_FUNCTION_VERSIONING_H_#include <rte_common.h> +#ifndef RTE_USE_FUNCTION_VERSIONING +#error Use of function versioning disabled, is "use_function_versioning=true" in meson.build? +#endif +#ifdef RTE_BUILD_SHARED_LIB /*
lib/librte_lpm/meson.build +1 -0
@@ -8,3 +8,4 @@ headers = files('rte_lpm.h', 'rte_lpm6.h') # without worrying about which architecture we actually need headers += files('rte_lpm_altivec.h', 'rte_lpm_neon.h', 'rte_lpm_sse.h') deps += ['hash'] +use_function_versioning = true
lib/librte_timer/meson.build +1 -0
@@ -4,3 +4,4 @@ sources = files('rte_timer.c') headers = files('rte_timer.h') allow_experimental_apis = true +use_function_versioning = true
lib/meson.build +13 -3
@@ -47,6 +47,7 @@ foreach l:libraries name = l version = 1 allow_experimental_apis = false + use_function_versioning = false sources = [] headers = [] includes = []
@@ -96,6 +97,9 @@ foreach l:libraries if allow_experimental_apis cflags += '-DALLOW_EXPERIMENTAL_API' endif + if use_function_versioning + cflags += '-DRTE_USE_FUNCTION_VERSIONING' + endif if get_option('per_library_versions') lib_version = '@[email protected]'.format(version)
@@ -117,9 +121,15 @@ foreach l:libraries include_directories: includes, dependencies: static_deps) - # then use pre-build objects to build shared lib - sources = [] - objs += static_lib.extract_all_objects(recursive: false) + if not use_function_versioning + # use pre-build objects to build shared lib + sources = [] + objs += static_lib.extract_all_objects(recursive: false) + else + # for compat we need to rebuild with + # RTE_BUILD_SHARED_LIB defined + cflags += '-DRTE_BUILD_SHARED_LIB' + endif version_map = '@0@/@1@/rte_@2@_version.map'.format( meson.current_source_dir(), dir_name, name) implib = dir_name + '.dll.a'
mk/target/generic/rte.vars.mk +8 -0
@@ -90,6 +90,14 @@ ASFLAGS += $(TARGET_ASFLAGS) CFLAGS += -I$(RTE_OUTPUT)/include LDFLAGS += -L$(RTE_OUTPUT)/lib +# add in flag for supporting function versioning. The define is used in meson +# builds to ensure that the user has properly flagged the unit in question as +# using function versioning so it can be built twice - once for static lib and +# then a second time for the shared lib. Since make only builds one library +# type at a time, such precautions aren't necessary, so we can globally define +# the flag +CFLAGS += -DRTE_USE_FUNCTION_VERSIONING + # always include rte_config.h: the one in $(RTE_OUTPUT)/include is # the configuration of SDK when $(BUILDING_RTE_SDK) is true, or the # configuration of the application if $(BUILDING_RTE_SDK) is not
Signature · 1 line
2.21.0
Quoted 3 lines · Bruce Richardson
Adding support for LTO has exposed some issues with how the functions versioning was supported by meson, which was always set to build both shared and static libraries.
[...]
Quoted 4 lines
V3: following feedback, moved the define for make from the
common_base
build config file to one of the global makefiles, as it's not user tunable.
V2: added in file that missed a "git add" when doing V1
Thomas Monjalon
Applied23/10/2019 12:19, Andrzej Ostruszka:
Quoted 11 lines · Bruce Richardson
Quoted 3 lines
Adding support for LTO has exposed some issues with how the functions versioning was supported by meson, which was always set to build both shared and static libraries.
[...]
Quoted 4 lines
V3: following feedback, moved the define for make from the
common_base
build config file to one of the global makefiles, as it's not user tunable.
V2: added in file that missed a "git add" when doing V1Reviewed-by Andrzej Ostruszka <[email protected]>
Applied, thanks
Quoted 48 lines · On Fri, 27 Sep 2019 20:49:30 +0100 Bruce Richardson…
Adding support for LTO has exposed some issues with how the functions versioning was supported by meson, which was always set to build both shared and static libraries.
For plain C code, so long as the -fPIC compiler flag was passed, the output is identical whether or not the code is to be included in a static library or a dynamic one. Unfortunately, when using function versioning that no longer held as different macros were used for the versioned functions depending on which type of build it was. This means that any files that use versioning need to be built twice, with different defines in each case.
While the trivial solution here is just to rebuild everything twice, that involves a lot of unnecessary work when building DPDK. A better option is to identify those files or components which need multiple builds and rebuild only those. To do this, we add a new meson.build setting for libraries "
use_function_versioning" and when that is set, we rebuild all source files twice, initially for static library and then with -DRTE_BUILD_SHARED_LIB for the shared library.If the flag is not set, then the static versioning setting only is used, which could lead to the build succeeding but later causing problems. To avoid that, we add a new define which must be set when the versioning header is included. This addition while solving 1 problem raises 2 other, more minor problems:
* what to do with make builds? since make only builds one library type, we can just always define the new value.
* what about files that includerte_compat.hfor the macro for "experimental"? To solve this, we can splitcompat.hin two, since the versioning macro should be internal only to DPDK (as no public header should expose anything but the latest APIs), while the experimental macros are primarily for public use.Bruce Richardson (2):
eal: split compat header file
build: support building ABI versioned files twiceconfig/common_base | 1 + config/rte_config.h | 3 --- doc/api/doxy-api-index.md | 3 ++- doc/guides/contributing/coding_style.rst | 7 ++++++ doc/guides/contributing/versioning.rst | 4 ++-- lib/librte_distributor/meson.build | 1 + lib/librte_distributor/rte_distributor.c | 2 +- lib/librte_distributor/rte_distributor_v20.c | 2 +- lib/librte_eal/common/Makefile | 1 + ...rte_compat.h => rte_function_versioning.h} | 23 ++++++------------- lib/librte_lpm/meson.build | 1 +
Quoted 9 lines
lib/librte_lpm/rte_lpm.c | 1 +
lib/librte_lpm/rte_lpm.h | 1 -
lib/librte_lpm/rte_lpm6.c | 1 +
lib/librte_timer/meson.build | 1 +
lib/librte_timer/rte_timer.c | 2 +-
lib/meson.build | 16 ++++++++++---
17 files changed, 41 insertions(+), 29 deletions(-)
rename lib/librte_eal/common/include/{rte_compat.h => rte_function_versioning.h} (89%)Looks fine.