introduce __rte_internal tag
Subsystems (4)
cover introduce __rte_internal tag
Hey-
Based on our recent conversations regarding the use of symbols only
meant for internal dpdk consumption (between dpdk libraries), this is an idea that I've come up with that I'd like to get some feedback on
Summary:
1) We have symbols in the DPDK that are meant to be used between DPDK libraries, but not by applications linking to them
2) We would like to document those symbols in the code, so as to note them clearly as for being meant for internal use only
3) Linker symbol visibility is a very coarse grained tool, and so there is no good way in a single library to mark items as being meant for use only by other DPDK libraries, at least not without some extensive runtime checking
Proposal:
I'm proposing that we introduce the __rte_internal tag. From a coding standpoint it works a great deal like the __rte_experimental tag in that it expempts the tagged symbol from ABI constraints (as the only users should be represented in the DPDK build environment). Additionally, the __rte_internal macro resolves differently based on the definition of the BUILDING_RTE_SDK flag (working under the assumption that said flag should only ever be set if we are actually building DPDK libraries which will make use of internal calls). If the BUILDING_RTE_SDK flag is set __rte_internal resolves to __attribute__((section "text.internal)), placing it in a special text section which is then used to validate that the the symbol appears in the INTERNAL section of the corresponding library version map). If BUILDING_RTE_SDK is not set, then __rte_internal resolves to __attribute__((error("..."))), which causes any caller of the tagged function to throw an error at compile time, indicating that the symbol is not available for external use.
This isn't a perfect solution, as applications can still hack around it of course, but I think it hits some of the high points, restricting symbol access for any library that prototypes its public and private symbols in the same header file, excluding the internal symbols from ABI constraints, and clearly documenting those symbols which we wish to limit to internal usage.
I've included a patch to the dpaa library to demonstrate its usage. If there is consensus on this approach, I'll expand and repost the patch, pulling in the other libraries which have internal-only symbol usage.
Regards
Neil
RFC PATCH 1/2 Add __rte_internal tag for functions and version target
This tag is meant to be used on function prototypes to identify functions that are only meant to be used by internal DPDK libraries (i.e. libraries that are built while building the SDK itself, as identified by the defining of the BUILDING_RTE_SDK macro). When that flag is not set, it will resolve to an error function attribute, causing build breakage for any compilation unit attempting to build it
Validate the use of this tag in much the same way we validate __rte_experimental. By adding an INTERNAL version to library map files, we can exempt internal-only functions from ABI checking, and handle them to ensure that symbols we wish to only be for internal use between dpdk libraries are properly tagged with __rte_experimental
Note this patch updates the check-experimental-syms.sh script, which normally only check the EXPERIMENTAL section to also check the INTERNAL section now. As such its been renamed to the now more appropriate check-special-syms.sh
--- buildtools/check-experimental-syms.sh | 24 +++++++++++++++++++++- lib/librte_eal/common/include/rte_compat.h | 12 +++++++++++ mk/internal/rte.compile-pre.mk | 6 +++--- mk/target/generic/rte.vars.mk | 2 +- 4 files changed, 39 insertions(+), 5 deletions(-)
buildtools/check-experimental-syms.sh +23 -1
@@ -31,10 +31,32 @@ do cat >&2 <<- END_OF_MESSAGE $SYM is not flagged as experimental but is listed in version map - Please add __rte_experimental to the definition of $SYM + Please add __rte_experimental to the definition/prototype of $SYM END_OF_MESSAGE exit 1 fi done + +for i in `awk 'BEGIN {found=0} + /.*INTERNAL.*/ {found=1} + /.*}.*;/ {found=0} + /.*;/ {if (found == 1) print $1}' $MAPFILE` +do + SYM=`echo $i | sed -e"s/;//"` + objdump -t $OBJFILE | grep -q "\.text.*$SYM$" + IN_TEXT=$? + objdump -t $OBJFILE | grep -q "\.text\.internal.*$SYM$" + IN_EXP=$? + if [ $IN_TEXT -eq 0 -a $IN_EXP -ne 0 ] + then + cat >&2 <<- END_OF_MESSAGE + $SYM is not flagged as internal + but is listed in version map + Please add __rte_internal to the definition/prototype of $SYM + END_OF_MESSAGE + exit 1 + fi +done + exit 0
lib/librte_eal/common/include/rte_compat.h +12 -0
@@ -89,4 +89,16 @@ __attribute__((section(".text.experimental")))#endif +/* +* __rte_internal tags mark functions as internal only, If specified in public +* header files, this tag will resolve to an error directive, preventing +* external applications from attempting to make calls to functions not meant +* for consumption outside the dpdk library +*/ +#ifdef BUILDING_RTE_SDK +#define __rte_internal __attribute__((section(".text.internal"))) +#else +#define __rte_internal __attribute__((error("This function cannot be used outside of the core DPDK library"), \ + section(".text.internal"))) +#endif #endif /* _RTE_COMPAT_H_ */
mk/internal/rte.compile-pre.mk +3 -3
@@ -56,8 +56,8 @@ C_TO_O = $(CC) -Wp,-MD,$(call obj2dep,$(@)).tmp $(CPPFLAGS) $(CFLAGS) \ C_TO_O_STR = $(subst ','\'',$(C_TO_O)) #'# fix syntax highlight C_TO_O_DISP = $(if $(V),"$(C_TO_O_STR)"," CC $(@)") endif -EXPERIMENTAL_CHECK = $(RTE_SDK)/buildtools/check-experimental-syms.sh -CHECK_EXPERIMENTAL = $(EXPERIMENTAL_CHECK) $(SRCDIR)/$(EXPORT_MAP) $@ +SPECIAL_SYM_CHECK = $(RTE_SDK)/buildtools/check-special-syms.sh +CHECK_SPECIAL_SYMS = $(SPECIAL_SYM_CHECK) $(SRCDIR)/$(EXPORT_MAP) $@ PMDINFO_GEN = $(RTE_SDK_BIN)/app/dpdk-pmdinfogen $@ [email protected] PMDINFO_CC = $(CC) $(CPPFLAGS) $(CFLAGS) $(EXTRA_CFLAGS) -c -o [email protected] [email protected]
@@ -75,7 +75,7 @@ C_TO_O_DO = @set -e; \ echo $(C_TO_O_DISP); \ $(C_TO_O) && \ $(PMDINFO_TO_O) && \ - $(CHECK_EXPERIMENTAL) && \ + $(CHECK_SPECIAL_SYMS) && \ echo $(C_TO_O_CMD) > $(call obj2cmd,$(@)) && \ sed 's,'$@':,dep_'$@' =,' $(call obj2dep,$(@)).tmp > $(call obj2dep,$(@)) && \ rm -f $(call obj2dep,$(@)).tmp
mk/target/generic/rte.vars.mk +1 -1
@@ -96,7 +96,7 @@ LDFLAGS += -L$(RTE_OUTPUT)/lib # defined. ifeq ($(BUILDING_RTE_SDK),1) # building sdk -CFLAGS += -include $(RTE_OUTPUT)/include/rte_config.h +CFLAGS += -include $(RTE_OUTPUT)/include/rte_config.h -DBUILDING_RTE_SDK else # if we are building an external application, include SDK's lib and # includes too
Signature · 1 line
2.20.1
Re: [dpdk-dev] [EXT] [RFC PATCH 1/2] Add __rte_internal tag for functions and version target
Quoted 53 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Sunday, May 26, 2019 12:14 AM
To: [email protected]
Cc: Neil Horman <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>; Bruce Richardson <[email protected]>; Thomas Monjalon <[email protected]>
Subject: [EXT] [RFC PATCH 1/2] Add__rte_internaltag for functions and version target
This tag is meant to be used on function prototypes to identify functions that are only meant to be used by internal DPDK libraries (i.e. libraries that are built while building the SDK itself, as identified by the defining of theBUILDING_RTE_SDKmacro). When that flag is not set, it will resolve to an error function attribute, causing build breakage for any compilation unit attempting to build itValidate the use of this tag in much the same way we validate
__rte_experimental. By adding an INTERNAL version to library map files, we can exempt internal-only functions from ABI checking, and handle them to ensure that symbols we wish to only be for internal use between dpdk libraries are properly tagged with__rte_experimentalNote this patch updates the
check-experimental-syms.shscript, which normally only check the EXPERIMENTAL section to also check the INTERNAL section now. As such its been renamed to the now more appropriate check-special-syms.shSigned-off-by Neil Horman <[email protected]>CC Jerin Jacob Kollanukkaran <[email protected]>CC Bruce Richardson <[email protected]>CC Thomas Monjalon <[email protected]>--- buildtools/check-experimental-syms.sh | 24 +++++++++++++++++++++- lib/librte_eal/common/include/rte_compat.h | 12 +++++++++++ mk/internal/rte.compile-pre.mk | 6 +++--- mk/target/generic/rte.vars.mk | 2 +- 4 files changed, 39 insertions(+), 5 deletions(-)buildtools/check- renamed from buildtools/check-experimental-syms.sh +0
-0
experimental-syms.sh
index7d1f3a568..63682c677100755buildtools/check-experimental-syms.sh +1
-1@@ -31,10 +31,32 @@ do cat >&2 <<- END_OF_MESSAGE $SYM is not flagged as experimental but is listed in version map - Please add __rte_experimental to the definition of $SYM + Please add __rte_experimental to the definition/prototypeof $SYM
buildtools/check-experimental-syms.sh +0
-0END_OF_MESSAGE exit 1 fi
buildtools/check-experimental-syms.sh do · line 48
Quoted 10 lines
buildtools/check-experimental-syms.sh +9
-0done + +for i in `awk 'BEGIN {found=0} + /.*INTERNAL.*/ {found=1} + /.*}.*;/ {found=0} + /.*;/ {if (found == 1) print $1}' $MAPFILE` do + SYM=`echo $i | sed -e"s/;//"` + objdump -t $OBJFILE | grep -q "\.text.*$SYM$" + IN_TEXT=$? + objdump -t $OBJFILE | grep -q "\.text\.internal.*$SYM$"
I think, it should be OK for cross compilation case. But please cross check.
lib/librte_eal/common/include/rte_compat.h
Quoted 20 lines
buildtools/check-experimental-syms.sh +7
-0+ IN_EXP=$? + if [ $IN_TEXT -eq 0 -a $IN_EXP -ne 0 ] + then + cat >&2 <<- END_OF_MESSAGE + $SYM is not flagged as internal + but is listed in version map + Please add __rte_internal to the definition/prototype of$SYM
buildtools/check-experimental-syms.sh +5
-0+ END_OF_MESSAGE + exit 1 + fi +done + exit 0lib/librte_eal/common/include/rte_compat.h +0
-0
It is a public header file, How about, having __rte_internal In internal header file so that only SDK components can use it.
lib/librte_eal/common/include/rte_compat.h __attribute__((section(".text.experimental"))) · line 100
Quoted 13 lines
lib/librte_eal/common/include/rte_compat.h +9
-0@@ -89,4 +89,16 @@ __attribute__((section(".text.experimental")))#endif +/* +* __rte_internal tags mark functions as internal only, If specified in +public +* header files, this tag will resolve to an error directive, +preventing +* external applications from attempting to make calls to functions not +meant +* for consumption outside the dpdk library */ #ifdef BUILDING_RTE_SDK +#define __rte_internal __attribute__((section(".text.internal")))
Sometimes, we may have global variables like variable for dynamic logging etc. If so, I think, it is better to change something not starting with .text
lib/librte_eal/common/include/rte_compat.h __attribute__((section(".text.experimental"))) · line 102
Quoted 4 lines
lib/librte_eal/common/include/rte_compat.h +3
-0+#else +#define __rte_internal __attribute__((error("This function cannot be used outside of the core DPDK library"), \ + section(".text.internal")))
Since the above statement is an error, Do we need section attribute here?
RFC PATCH 2/2 Convert dpaa driver to tag internal-only symbols appropriately
In reply to Neil Hormanmake use of the new __rte_internal tag to specify symbols that should only be used by dpdk provided libraries (as specified by the BUILDING_RTE_SDK cflag
---
...rimental-syms.sh => check-special-syms.sh} | 0
drivers/bus/dpaa/include/fsl_bman.h | 12 ++--
drivers/bus/dpaa/include/fsl_fman.h | 50 ++++++++--------
drivers/bus/dpaa/include/fsl_qman.h | 60 +++++++++----------
drivers/bus/dpaa/include/fsl_usd.h | 12 ++--
drivers/bus/dpaa/include/netcfg.h | 4 +-
drivers/bus/dpaa/include/of.h | 6 +-
drivers/bus/dpaa/rte_bus_dpaa_version.map | 47 ++++++---------
8 files changed, 90 insertions(+), 101 deletions(-)
rename buildtools/{check-experimental-syms.sh => check-special-syms.sh} (100%)buildtools/check-special-syms.sh renamed from buildtools/check-experimental-syms.sh +0 -0
drivers/bus/dpaa/include/fsl_bman.h +6 -6
@@ -264,13 +264,13 @@ int bman_shutdown_pool(u32 bpid);* the structure provided by the caller can be released or reused after the * function returns. */ -struct bman_pool *bman_new_pool (const struct bman_pool_params *params); +struct bman_pool __rte_internal *bman_new_pool(const struct bman_pool_params *params);/** * bman_free_pool - Deallocates a Buffer Pool object * @pool: the pool object to release */ -void bman_free_pool (struct bman_pool *pool); +void __rte_internal bman_free_pool(struct bman_pool *pool);/** * bman_get_params - Returns a pool object's parameters.
@@ -279,7 +279,7 @@ void bman_free_pool(struct bman_pool *pool);* The returned pointer refers to state within the pool object so must not be * modified and can no longer be read once the pool object is destroyed. */ -const struct bman_pool_params *bman_get_params (const struct bman_pool *pool); +const struct bman_pool_params __rte_internal *bman_get_params(const struct bman_pool *pool);/** * bman_release - Release buffer(s) to the buffer pool
@@ -289,7 +289,7 @@ const struct bman_pool_params *bman_get_params(const struct bman_pool *pool);* @flags: bit-mask of BMAN_RELEASE_FLAG_*** options * */ -int bman_release (struct bman_pool *pool,const struct bm_buffer *bufs,u8 num, +int __rte_internal bman_release(struct bman_pool *pool, const struct bm_buffer *bufs, u8 num,u32 flags);/**
@@ -302,7 +302,7 @@ int bman_release(struct bman_pool *pool, const struct bm_buffer *bufs, u8 num,* The return value will be the number of buffers obtained from the pool, or a * negative error code if a h/w error or pool starvation was encountered. */ -int bman_acquire (struct bman_pool *pool,struct bm_buffer *bufs,u8 num, +int __rte_internal bman_acquire(struct bman_pool *pool, struct bm_buffer *bufs, u8 num,u32 flags);/**
@@ -317,7 +317,7 @@ int bman_query_pools(struct bm_pool_state *state);* * Return the number of the free buffers */ -u32 bman_query_free_buffers (struct bman_pool *pool); +u32 __rte_internal bman_query_free_buffers(struct bman_pool *pool);/** * bman_update_pool_thresholds - Change the buffer pool's depletion thresholds
drivers/bus/dpaa/include/fsl_fman.h +25 -25
113 lines not on this page · raw message
drivers/bus/dpaa/include/fsl_qman.h +30 -30
235 lines not on this page · raw message
drivers/bus/dpaa/include/fsl_usd.h +6 -6
@@ -56,7 +56,7 @@ int bman_allocate_raw_portal(struct dpaa_raw_portal *portal);int bman_free_raw_portal (struct dpaa_raw_portal *portal);/* Obtain thread-local UIO file-descriptors */ -int qman_thread_fd (void ); +int __rte_internal qman_thread_fd(void);int bman_thread_fd (void );/* Post-process interrupts. NB, the kernel IRQ handler disables the interrupt
@@ -64,14 +64,14 @@ int bman_thread_fd(void);* processing is complete. As such, it is essential to call this before going * into another blocking read/select/poll. */ -void qman_thread_irq (void ); -void bman_thread_irq (void ); +void __rte_internal qman_thread_irq(void); +void __rte_internal bman_thread_irq(void); -void qman_clear_irq (void ); +void __rte_internal qman_clear_irq(void);/* Global setup */ -int qman_global_init (void ); -int bman_global_init (void ); +int __rte_internal qman_global_init(void); +int __rte_internal bman_global_init(void);/* Direct portal create and destroy */ struct qman_portal *fsl_qman_portal_create (void );
drivers/bus/dpaa/include/netcfg.h +2 -2
@@ -46,12 +46,12 @@ struct netcfg_interface {* cfg_file: FMC config XML file * Returns the configuration information in newly allocated memory. */ -struct netcfg_info *netcfg_acquire (void ); +struct netcfg_info __rte_internal *netcfg_acquire(void);/* cfg_ptr: configuration information pointer. * Frees the resources allocated by the configuration layer. */ -void netcfg_release (struct netcfg_info *cfg_ptr); +void __rte_internal netcfg_release(struct netcfg_info *cfg_ptr);#ifdef RTE_LIBRTE_DPAA_DEBUG_DRIVER /* cfg_ptr: configuration information pointer.
drivers/bus/dpaa/include/of.h +3 -3
@@ -87,7 +87,7 @@ struct dt_file {uint64_t buf[OF_FILE_BUF_MAX >>3 ]; }; -const struct device_node *of_find_compatible_node ( +const __rte_internal struct device_node *of_find_compatible_node(const struct device_node *from,const char *type__always_unused ,const char *compatible)
@@ -98,7 +98,7 @@ const struct device_node *of_find_compatible_node( dev_node !=NULL ; \ dev_node = of_find_compatible_node(dev_node, type, compatible)) -const void *of_get_property (const struct device_node *from,const char *name, +const __rte_internal void *of_get_property(const struct device_node *from, const char *name,size_t *lenp) __attribute__((nonnull (2 )));bool of_device_is_available (const struct device_node *dev_node);
@@ -109,7 +109,7 @@ const struct device_node *of_get_parent(const struct device_node *dev_node);const struct device_node *of_get_next_child (const struct device_node *dev_node,const struct device_node *prev); -const void *of_get_mac_address (const struct device_node *np); +const void __rte_internal *of_get_mac_address(const struct device_node *np);#define for_each_child_node (parent, child) \ for (child = of_get_next_child(parent, NULL); child != NULL; \
drivers/bus/dpaa/rte_bus_dpaa_version.map +18 -29
75 lines not on this page · raw message
Signature · 1 line
2.20.1
Quoted 40 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Sunday, May 26, 2019 12:14 AM
To: [email protected]
Cc: Neil Horman <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>; Bruce Richardson <[email protected]>; Thomas Monjalon <[email protected]>
Subject: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagHey-
Based on our recent conversations regarding the use of symbols only
meant for internal dpdk consumption (between dpdk libraries), this is an idea that I've come up with that I'd like to get some feedback onSummary:
1) We have symbols in the DPDK that are meant to be used between DPDK libraries, but not by applications linking to them
2) We would like to document those symbols in the code, so as to note them clearly as for being meant for internal use only
3) Linker symbol visibility is a very coarse grained tool, and so there is no good way in a single library to mark items as being meant for use only by other DPDK libraries, at least not without some extensive runtime checkingProposal:
I'm proposing that we introduce the__rte_internaltag. From a coding standpoint it works a great deal like the__rte_experimentaltag in that it expempts the tagged symbol from ABI constraints (as the only users should be represented in the DPDK build environment). Additionally, the__rte_internalmacro resolves differently based on the definition of theBUILDING_RTE_SDKflag (working under the assumption that said flag should only ever be set if we are actually building DPDK libraries which will make use of internal calls). If theBUILDING_RTE_SDKflag is set__rte_internalresolves to__attribute__((section "text.internal)), placing it in a special text section which is then used to validate that the the symbol appears in the INTERNAL section of the corresponding library version map). IfBUILDING_RTE_SDKis not set, then__rte_internalresolves to __attribute__((error("..."))), which causes any caller of the tagged function to throw an error at compile time, indicating that the symbol is not available for external use.
Quoted 1 line
This isn't a perfect solution, as applications can still hack around it of course,
I think, one way to, avoid, hack around could be to,
1) at config stage, create a random number for the build
2) introduce RTE_CALL_INTERNAL macro for calling internal function, compare
the generated random number for allowing the calls to make within the library. i.e leverage the fact that external library would never know the random number generated
for the DPDK build and internal driver code does.
Quoted 16 lines
but I think it hits some of the high points, restricting symbol access for any library that prototypes its public and private symbols in the same header file, excluding the internal symbols from ABI constraints, and clearly documenting those symbols which we wish to limit to internal usage.
I've included a patch to the dpaa library to demonstrate its usage. If there is consensus on this approach, I'll expand and repost the patch, pulling in the other libraries which have internal-only symbol usage.
Regards
NeilSigned-off-by Neil Horman <[email protected]>CC Jerin Jacob Kollanukkaran <[email protected]>CC Bruce Richardson <[email protected]>CC Thomas Monjalon <[email protected]>
Quoted 42 lines · Jerin Jacob Kollanukkaran
Quoted 40 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Sunday, May 26, 2019 12:14 AM
To: [email protected]
Cc: Neil Horman <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>; Bruce Richardson <[email protected]>; Thomas Monjalon <[email protected]>
Subject: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagHey-
Based on our recent conversations regarding the use of symbols only
meant for internal dpdk consumption (between dpdk libraries), this is an idea that I've come up with that I'd like to get some feedback onSummary:
1) We have symbols in the DPDK that are meant to be used between DPDK libraries, but not by applications linking to them
2) We would like to document those symbols in the code, so as to note them clearly as for being meant for internal use only
3) Linker symbol visibility is a very coarse grained tool, and so there is no good way in a single library to mark items as being meant for use only by other DPDK libraries, at least not without some extensive runtime checkingProposal:
I'm proposing that we introduce the__rte_internaltag. From a coding standpoint it works a great deal like the__rte_experimentaltag in that it expempts the tagged symbol from ABI constraints (as the only users should be represented in the DPDK build environment). Additionally, the__rte_internalmacro resolves differently based on the definition of theBUILDING_RTE_SDKflag (working under the assumption that said flag should only ever be set if we are actually building DPDK libraries which will make use of internal calls). If theBUILDING_RTE_SDKflag is set__rte_internalresolves to__attribute__((section "text.internal)), placing it in a special text section which is then used to validate that the the symbol appears in the INTERNAL section of the corresponding library version map). IfBUILDING_RTE_SDKis not set, then__rte_internalresolves to __attribute__((error("..."))), which causes any caller of the tagged function to throw an error at compile time, indicating that the symbol is not available for external use.Quoted 1 line
This isn't a perfect solution, as applications can still hack around it of course,
Quoted 8 lines
I think, one way to, avoid, hack around could be to,
1) at config stage, create a random number for the build
2) introduceRTE_CALL_INTERNALmacro for calling internal function, compare
the generated random number for allowing the calls to make within the library. i.e leverage the fact that external library would never know the random number generated
for the DPDK build and internal driver code does.
Do we really need to care about this. If have some determined enough to hack around our limitations, then they surely know that they have an unsupported configuration. We just need to protect against inadvertent use of internals, IMHO.
/Bruce
Quoted 51 lines · Bruce Richardson
Quoted 42 lines · Jerin Jacob Kollanukkaran
Quoted 40 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Sunday, May 26, 2019 12:14 AM
To: [email protected]
Cc: Neil Horman <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>; Bruce Richardson <[email protected]>; Thomas Monjalon <[email protected]>
Subject: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagHey-
Based on our recent conversations regarding the use of symbols only
meant for internal dpdk consumption (between dpdk libraries), this is an idea that I've come up with that I'd like to get some feedback onSummary:
1) We have symbols in the DPDK that are meant to be used between DPDK libraries, but not by applications linking to them
2) We would like to document those symbols in the code, so as to note them clearly as for being meant for internal use only
3) Linker symbol visibility is a very coarse grained tool, and so there is no good way in a single library to mark items as being meant for use only by other DPDK libraries, at least not without some extensive runtime checkingProposal:
I'm proposing that we introduce the__rte_internaltag. From a coding standpoint it works a great deal like the__rte_experimentaltag in that it expempts the tagged symbol from ABI constraints (as the only users should be represented in the DPDK build environment). Additionally, the__rte_internalmacro resolves differently based on the definition of theBUILDING_RTE_SDKflag (working under the assumption that said flag should only ever be set if we are actually building DPDK libraries which will make use of internal calls). If theBUILDING_RTE_SDKflag is set__rte_internalresolves to__attribute__((section "text.internal)), placing it in a special text section which is then used to validate that the the symbol appears in the INTERNAL section of the corresponding library version map). IfBUILDING_RTE_SDKis not set, then__rte_internalresolves to __attribute__((error("..."))), which causes any caller of the tagged function to throw an error at compile time, indicating that the symbol is not available for external use.Quoted 1 line
This isn't a perfect solution, as applications can still hack around it of course,
Quoted 8 lines
I think, one way to, avoid, hack around could be to,
1) at config stage, create a random number for the build
2) introduceRTE_CALL_INTERNALmacro for calling internal function, compare
the generated random number for allowing the calls to make within the library. i.e leverage the fact that external library would never know the random number generated
for the DPDK build and internal driver code does.
Quoted 5 lines
Do we really need to care about this. If have some determined enough to hack around our limitations, then they surely know that they have an unsupported configuration. We just need to protect against inadvertent use of internals, IMHO.
I agree, I too had thought about doing some sort of internal runtime checking to match internal only symbols, such that they were only accessable by internally approved users, but it started to feel like a great deal of overhead. Its a good idea for a general mechanism I think, but I believe the value here is more to internally document which apis we want to mark as being for internal use only, and create a lightweight roadblock at build time to catch users inadvertently using them. Determined users will get around anything, and theres not much we can do to stop them.
If we really wanted to go down that road, we could use a mechainsm simmilar to the EXPORT_SYMBOL / EXPORT_SYMBOL_GPL infrastructure that the kernel uses, but that would required building our own custom linker script, which seems like overkill here.
Best
Neil
Quoted 2 lines
/Bruce
Quoted 74 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Wednesday, June 5, 2019 11:41 PM
To: Bruce Richardson <[email protected]>
Cc: Jerin Jacob Kollanukkaran <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Subject: Re: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagQuoted 1 line · Bruce Richardson
On Wed, Jun 05, 2019 at 04:24:09PM +0000, Jerin Jacob Kollanukkaran
Quoted 27 lines · wrote:
Quoted 27 lines
Quoted 27 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Sunday, May 26, 2019 12:14 AM
To: [email protected]
Cc: Neil Horman <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>; Bruce Richardson <[email protected]>; Thomas Monjalon <[email protected]>
Subject: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagHey-
Based on our recent conversations regarding the use of symbols
only meant for internal dpdk consumption (between dpdk libraries), this is an idea that I've come up with that I'd like to get some feedback onSummary:
1) We have symbols in the DPDK that are meant to be used between DPDK libraries, but not by applications linking to them
2) We would like to document those symbols in the code, so as to note them clearly as for being meant for internal use only
3) Linker symbol visibility is a very coarse grained tool, and so there is no good way in a single library to mark items as being meant for use only by other DPDK libraries, at least not without some extensive runtime checkingQuoted 15 lines
Quoted 7 lines
Quoted 7 lines
Proposal:
I'm proposing that we introduce the__rte_internaltag. From a coding standpoint it works a great deal like the__rte_experimentaltag in that it expempts the tagged symbol from ABI constraints (as the only users should be represented in the DPDK build environment). Additionally, the__rte_internalmacro resolves differently based on the definition of theQuoted 8 lines
Quoted 8 lines
BUILDING_RTE_SDKflag (working under the assumption that said flag should only ever be set if we are actually building DPDK libraries which will make use of internal calls). If theBUILDING_RTE_SDKflag is set__rte_internalresolves to__attribute__((section "text.internal)), placing it in a special text section which is then used to validate that the the symbol appears in the INTERNAL section of the corresponding library version map). IfBUILDING_RTE_SDKis not set, then__rte_internalresolves to__attribute__((error("..."))), which causes any caller of the tagged function to throw an error at compile time, indicating that the symbol is not available for external use.
Quoted 11 lines
Quoted 11 lines
Quoted 3 lines
This isn't a perfect solution, as applications can still hack around it of course,
I think, one way to, avoid, hack around could be to,
1) at config stage, create a random number for the build
2) introduceRTE_CALL_INTERNALmacro for calling internal function, compare the generated random number for allowing the calls to make within the library. i.e leverage the fact that external library would never know the random number generated for the DPDK buildand internal driver code does.
Quoted 6 lines
Quoted 1 line
Do we really need to care about this. If have some determined enough to hack around our limitations, then they surely know that they have an unsupported configuration. We just need to protect against inadvertent use of internals, IMHO.
Quoted 8 lines
I agree, I too had thought about doing some sort of internal runtime checking to match internal only symbols, such that they were only accessable by internally approved users, but it started to feel like a great deal of overhead. Its a good idea for a general mechanism I think, but I believe the value here is more to internally document which apis we want to mark as being for internal use only, and create a lightweight roadblock at build time to catch users inadvertently using them. Determined users will get around anything, and theres not much we can do to stop them.
I agree too. IMHO, Simply having following items would be enough
1) Avoid exposing the internal function prototype through public header files
2) Add @internal to API documentation
3) Just decide the name space for internal API for tooling(i.e not start with rte_ or so) Using objdump scheme to detect internal functions requires the the library to build prior to run the checkpatch.
Quoted 11 lines
If we really wanted to go down that road, we could use a mechainsm simmilar to the
EXPORT_SYMBOL/EXPORT_SYMBOL_GPLinfrastructure that the kernel uses, but that would required building our own custom linker script, which seems like overkill here.Best
NeilQuoted 2 lines
/Bruce
Quoted 83 lines · Jerin Jacob Kollanukkaran
Quoted 74 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Wednesday, June 5, 2019 11:41 PM
To: Bruce Richardson <[email protected]>
Cc: Jerin Jacob Kollanukkaran <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Subject: Re: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagQuoted 1 line · Bruce Richardson
On Wed, Jun 05, 2019 at 04:24:09PM +0000, Jerin Jacob Kollanukkaran
Quoted 27 lines · wrote:
Quoted 27 lines
Quoted 27 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Sunday, May 26, 2019 12:14 AM
To: [email protected]
Cc: Neil Horman <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>; Bruce Richardson <[email protected]>; Thomas Monjalon <[email protected]>
Subject: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagHey-
Based on our recent conversations regarding the use of symbols
only meant for internal dpdk consumption (between dpdk libraries), this is an idea that I've come up with that I'd like to get some feedback onSummary:
1) We have symbols in the DPDK that are meant to be used between DPDK libraries, but not by applications linking to them
2) We would like to document those symbols in the code, so as to note them clearly as for being meant for internal use only
3) Linker symbol visibility is a very coarse grained tool, and so there is no good way in a single library to mark items as being meant for use only by other DPDK libraries, at least not without some extensive runtime checkingQuoted 15 lines
Quoted 7 lines
Quoted 7 lines
Proposal:
I'm proposing that we introduce the__rte_internaltag. From a coding standpoint it works a great deal like the__rte_experimentaltag in that it expempts the tagged symbol from ABI constraints (as the only users should be represented in the DPDK build environment). Additionally, the__rte_internalmacro resolves differently based on the definition of theQuoted 8 lines
Quoted 8 lines
BUILDING_RTE_SDKflag (working under the assumption that said flag should only ever be set if we are actually building DPDK libraries which will make use of internal calls). If theBUILDING_RTE_SDKflag is set__rte_internalresolves to__attribute__((section "text.internal)), placing it in a special text section which is then used to validate that the the symbol appears in the INTERNAL section of the corresponding library version map). IfBUILDING_RTE_SDKis not set, then__rte_internalresolves to__attribute__((error("..."))), which causes any caller of the tagged function to throw an error at compile time, indicating that the symbol is not available for external use.
Quoted 11 lines
Quoted 11 lines
Quoted 3 lines
This isn't a perfect solution, as applications can still hack around it of course,
I think, one way to, avoid, hack around could be to,
1) at config stage, create a random number for the build
2) introduceRTE_CALL_INTERNALmacro for calling internal function, compare the generated random number for allowing the calls to make within the library. i.e leverage the fact that external library would never know the random number generated for the DPDK buildand internal driver code does.
Quoted 6 lines
Quoted 1 line
Do we really need to care about this. If have some determined enough to hack around our limitations, then they surely know that they have an unsupported configuration. We just need to protect against inadvertent use of internals, IMHO.
Quoted 8 lines
I agree, I too had thought about doing some sort of internal runtime checking to match internal only symbols, such that they were only accessable by internally approved users, but it started to feel like a great deal of overhead. Its a good idea for a general mechanism I think, but I believe the value here is more to internally document which apis we want to mark as being for internal use only, and create a lightweight roadblock at build time to catch users inadvertently using them. Determined users will get around anything, and theres not much we can do to stop them.
Quoted 7 lines
I agree too. IMHO, Simply having following items would be enough
1) Avoid exposing the internal function prototype through public header files
2) Add @internal to API documentation
3) Just decide the name space for internal API fortooling(i.e not start with rte_ or so)Using objdump scheme to detect internal functions requires the the library to build prior to run the checkpatch.
No, I'm not comfortable with that approach, and I've stated why:
1) Not exposing the functions via header files is a fine start
2) Adding internal documentation is also fine, but does nothing to correlate the code implementing those functions to the documentation. Its valuable to have a tag on a function identifying it as internal only.
3) Using naming conventions to separate internal only from non-internal functions is a vague approach, requiring future developers to be cogniscent of the convention and make the appropriate naming choices. It also implicitly restricts the abliity for future developers to make naming changes in conflict with that convention
4) Adding a tag like __rte_internal creates an interlock whereby, not only are internal functions excused from ABI constraints, but forces developers to intentionally mark their internal functions as being internal in the code, which is beneficial to clarlity of understanding during the development process.
5) Adding a tag like __rte_internal is explicit, and allows developers to use a single header file instead of multiple header files if they so choose
We went through this with experimental symbols as well[1], and it just makes more sense to me to clearly document in the code what constitutes an internal symbol rather than relying on naming conventions and hoping that developers read the documentation before exporting a symbol publically.
[1] https://mails.dpdk.org/archives/dev/2017-December/083828.html
Quoted 12 lines
Quoted 11 lines
If we really wanted to go down that road, we could use a mechainsm simmilar to the
EXPORT_SYMBOL/EXPORT_SYMBOL_GPLinfrastructure that the kernel uses, but that would required building our own custom linker script, which seems like overkill here.Best
NeilQuoted 2 lines
/Bruce
Quoted 116 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Thursday, June 6, 2019 5:04 PM
To: Jerin Jacob Kollanukkaran <[email protected]>
Cc: Bruce Richardson <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Subject: Re: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagQuoted 74 lines · Jerin Jacob Kollanukkaran
Quoted 74 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Wednesday, June 5, 2019 11:41 PM
To: Bruce Richardson <[email protected]>
Cc: Jerin Jacob Kollanukkaran <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Subject: Re: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagQuoted 2 lines · Bruce Richardson
On Wed, Jun 05, 2019 at 04:24:09PM +0000, Jerin Jacob Kollanukkaran
Quoted 37 lines · wrote:
Quoted 37 lines
Quoted 37 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Sunday, May 26, 2019 12:14 AM
To: [email protected]
Cc: Neil Horman <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>; Bruce Richardson <[email protected]>; Thomas Monjalon <[email protected]>
Subject: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagHey-
Based on our recent conversations regarding the use of
symbols only meant for internal dpdk consumption (between dpdk libraries), this is an idea that I've come up with that I'd like to get some feedback onSummary:
1) We have symbols in the DPDK that are meant to be used between DPDK libraries, but not by applications linking to them
2) We would like to document those symbols in the code, so as to note them clearly as for being meant for internal use only
3) Linker symbol visibility is a very coarse grained tool, and so there is no good way in a single library to mark items as being meant for use only by other DPDK libraries, at least not without some extensive runtime checkingProposal:
I'm proposing that we introduce the__rte_internaltag. From a coding standpoint it works a great deal like the__rte_experimentaltag in that it expempts the tagged symbol from ABI constraints (as the only users should be represented in the DPDK build environment). Additionally, the__rte_internalmacro resolves differently based on the definition of theBUILDING_RTE_SDKflag (working under the assumption that said flag should only ever be set if we areQuoted 8 lines
Quoted 8 lines
Quoted 8 lines
actually building DPDK libraries which will make use of internal calls). If the
BUILDING_RTE_SDKflag is set__rte_internalresolves to__attribute__((section "text.internal)), placing it in a special text section which is then used to validate that the the symbol appears in the INTERNAL section of the corresponding library version map). IfBUILDING_RTE_SDKis not set, then__rte_internalresolves to__attribute__((error("..."))), which causes any caller of the tagged function to throw an error at compile time, indicating that the symbol is not available for external use.
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
This isn't a perfect solution, as applications can still hack around it of course,
Quoted 8 lines
Quoted 8 lines
I think, one way to, avoid, hack around could be to,
1) at config stage, create a random number for the build
2) introduceRTE_CALL_INTERNALmacro for calling internal function, compare the generated random number for allowing the calls to make within the library. i.e leverage the fact that external library would never know the random number generated for the DPDK buildand internal driver code does.
Quoted 1 line
Quoted 1 line
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Do we really need to care about this. If have some determined enough to hack around our limitations, then they surely know that they have an unsupported configuration. We just need to protect against inadvertent use of internals, IMHO.
I agree, I too had thought about doing some sort of internal runtime checking to match internal only symbols, such that they were only accessable by internally approved users, but it started to feel like a great
deal of overhead.
Quoted 5 lines
Quoted 5 lines
Its a good idea for a general mechanism I think, but I believe the value here is more to internally document which apis we want to mark as being for internal use only, and create a lightweight roadblock at build time to catch users inadvertently using them. Determined users will get around anything, and theres not much we can do to stop
them.
Quoted 8 lines
I agree too. IMHO, Simply having following items would be enough
1) Avoid exposing the internal function prototype through public header files
2) Add @internal to API documentation
3) Just decide the name space for internal API fortooling(i.e not start with rte_ or so)Using objdump scheme to detect internal functionsrequires the the library to build prior to run the checkpatch.
Quoted 1 line
No, I'm not comfortable with that approach, and I've stated why:
1) Not exposing the functions via header files is a fine start2) Adding internal documentation is also fine, but does nothing to correlate the code implementing those functions to the documentation. Its valuable to have a tag on a function identifying it as internal only.
Quoted 5 lines
3) Using naming conventions to separate internal only from non-internal functions is a vague approach, requiring future developers to be cogniscent of the convention and make the appropriate naming choices. It also implicitly restricts the abliity for future developers to make naming changes in conflict with that convention
Enforcing the naming convention can be achieved through tooling as well.
Quoted 5 lines
4) Adding a tag like
__rte_internalcreates an interlock whereby, not only are internal functions excused from ABI constraints, but forces developers to intentionally mark their internal functions as being internal in the code, which is beneficial to clarlity of understanding during the development process.
No issues in adding __rte_internal. But, I am against current implementaion, Ie. adding objdump dependency
to checkpatch i.e developer has to build the library first so that checkpatch can can know, Is it belongs to internal section or not?
Quoted 24 lines
5) Adding a tag like
__rte_internalis explicit, and allows developers to use a single header file instead of multiple header files if they so chooseWe went through this with experimental symbols as well[1], and it just makes more sense to me to clearly document in the code what constitutes an internal symbol rather than relying on naming conventions and hoping that developers read the documentation before exporting a symbol publically.
[1] https://mails.dpdk.org/archives/dev/2017-December/083828.html
Quoted 12 lines
Quoted 11 lines
If we really wanted to go down that road, we could use a mechainsm simmilar to the
EXPORT_SYMBOL/EXPORT_SYMBOL_GPLinfrastructure that the kernel uses, but that would required building our own custom linker script, which seems like overkill here.Best
NeilQuoted 2 lines
/Bruce
Quoted 134 lines · Jerin Jacob Kollanukkaran
Quoted 117 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Thursday, June 6, 2019 5:04 PM
To: Jerin Jacob Kollanukkaran <[email protected]>
Cc: Bruce Richardson <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Subject: Re: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagQuoted 74 lines · Jerin Jacob Kollanukkaran
Quoted 74 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Wednesday, June 5, 2019 11:41 PM
To: Bruce Richardson <[email protected]>
Cc: Jerin Jacob Kollanukkaran <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Subject: Re: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagQuoted 2 lines · Bruce Richardson
On Wed, Jun 05, 2019 at 04:24:09PM +0000, Jerin Jacob Kollanukkaran
Quoted 37 lines · wrote:
Quoted 37 lines
Quoted 37 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Sunday, May 26, 2019 12:14 AM
To: [email protected]
Cc: Neil Horman <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>; Bruce Richardson <[email protected]>; Thomas Monjalon <[email protected]>
Subject: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagHey-
Based on our recent conversations regarding the use of
symbols only meant for internal dpdk consumption (between dpdk libraries), this is an idea that I've come up with that I'd like to get some feedback onSummary:
1) We have symbols in the DPDK that are meant to be used between DPDK libraries, but not by applications linking to them
2) We would like to document those symbols in the code, so as to note them clearly as for being meant for internal use only
3) Linker symbol visibility is a very coarse grained tool, and so there is no good way in a single library to mark items as being meant for use only by other DPDK libraries, at least not without some extensive runtime checkingProposal:
I'm proposing that we introduce the__rte_internaltag. From a coding standpoint it works a great deal like the__rte_experimentaltag in that it expempts the tagged symbol from ABI constraints (as the only users should be represented in the DPDK build environment). Additionally, the__rte_internalmacro resolves differently based on the definition of theBUILDING_RTE_SDKflag (working under the assumption that said flag should only ever be set if we areQuoted 8 lines
Quoted 8 lines
Quoted 8 lines
actually building DPDK libraries which will make use of internal calls). If the
BUILDING_RTE_SDKflag is set__rte_internalresolves to__attribute__((section "text.internal)), placing it in a special text section which is then used to validate that the the symbol appears in the INTERNAL section of the corresponding library version map). IfBUILDING_RTE_SDKis not set, then__rte_internalresolves to__attribute__((error("..."))), which causes any caller of the tagged function to throw an error at compile time, indicating that the symbol is not available for external use.
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
This isn't a perfect solution, as applications can still hack around it of course,
Quoted 8 lines
Quoted 8 lines
I think, one way to, avoid, hack around could be to,
1) at config stage, create a random number for the build
2) introduceRTE_CALL_INTERNALmacro for calling internal function, compare the generated random number for allowing the calls to make within the library. i.e leverage the fact that external library would never know the random number generated for the DPDK buildand internal driver code does.
Quoted 1 line
Quoted 1 line
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Do we really need to care about this. If have some determined enough to hack around our limitations, then they surely know that they have an unsupported configuration. We just need to protect against inadvertent use of internals, IMHO.
I agree, I too had thought about doing some sort of internal runtime checking to match internal only symbols, such that they were only accessable by internally approved users, but it started to feel like a great
deal of overhead.
Quoted 5 lines
Quoted 5 lines
Its a good idea for a general mechanism I think, but I believe the value here is more to internally document which apis we want to mark as being for internal use only, and create a lightweight roadblock at build time to catch users inadvertently using them. Determined users will get around anything, and theres not much we can do to stop
them.
Quoted 8 lines
I agree too. IMHO, Simply having following items would be enough
1) Avoid exposing the internal function prototype through public header files
2) Add @internal to API documentation
3) Just decide the name space for internal API fortooling(i.e not start with rte_ or so)Using objdump scheme to detect internal functionsrequires the the library to build prior to run the checkpatch.
Quoted 1 line
No, I'm not comfortable with that approach, and I've stated why:
1) Not exposing the functions via header files is a fine start2) Adding internal documentation is also fine, but does nothing to correlate the code implementing those functions to the documentation. Its valuable to have a tag on a function identifying it as internal only.
Quoted 5 lines
3) Using naming conventions to separate internal only from non-internal functions is a vague approach, requiring future developers to be cogniscent of the convention and make the appropriate naming choices. It also implicitly restricts the abliity for future developers to make naming changes in conflict with that convention
Enforcing the naming convention can be achieved through tooling as well.
Quoted 5 lines
4) Adding a tag like
__rte_internalcreates an interlock whereby, not only are internal functions excused from ABI constraints, but forces developers to intentionally mark their internal functions as being internal in the code, which is beneficial to clarlity of understanding during the development process.
Quoted 14 lines
No issues in adding
__rte_internal. But, I am against current implementaion, Ie. adding objdump dependency
to checkpatch i.e developer has to build the library first so that checkpatch can can know, Is it belongs to internal section or not?Quoted 9 lines
5) Adding a tag like
__rte_internalis explicit, and allows developers to use a single header file instead of multiple header files if they so chooseWe went through this with experimental symbols as well[1], and it just makes more sense to me to clearly document in the code what constitutes an internal symbol rather than relying on naming conventions and hoping that developers read the documentation before exporting a symbol publically.
I feel like we are creating a lot of extra work for the developer and adding a number of constraints to getting code patches submitted as the tools all have to be working together. The versioning file and __rte_experimental stuff today has always being handle wrong or not done by the developer. Altering the tools to detect these changes works and it seemed to take a while to iron out. To me we should be doing the minimum steps to reasonably isolate internal API and data from the user. If someone wants to access the those APIs that is their choice and enforcing with new macros and tools is over kill IMHO.
1) Adding @internal to documentation is a great start along with more docs to explain what internal functions/data should be handled.
2) Hiding/moving internal function prototypes in private headers.
3) Adding setters/getters for internal data.
4) Make sure we review and reject direct use of internal functions and data.
The goal here is to handle the 80% rule and make it very obvious to the developer these are internal functions and data. Using these or similar minimum steps should be reasonable IMHO.
Quoted 9 lines
Quoted 9 lines
[1] https://mails.dpdk.org/archives/dev/2017-December/083828.html
Quoted 6 lines
Quoted 6 lines
If we really wanted to go down that road, we could use a mechainsm simmilar to the
EXPORT_SYMBOL/EXPORT_SYMBOL_GPLinfrastructure that the kernel uses, but that would required building our own custom linker script, which seems like overkill here.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Best
NeilQuoted 1 line
/Bruce
Regards,
Keith
Quoted 149 lines · Keith
Quoted 134 lines · Jerin Jacob Kollanukkaran
Quoted 117 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Thursday, June 6, 2019 5:04 PM
To: Jerin Jacob Kollanukkaran <[email protected]>
Cc: Bruce Richardson <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Subject: Re: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagQuoted 74 lines · Jerin Jacob Kollanukkaran
Quoted 74 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Wednesday, June 5, 2019 11:41 PM
To: Bruce Richardson <[email protected]>
Cc: Jerin Jacob Kollanukkaran <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Subject: Re: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagQuoted 2 lines · Bruce Richardson
On Wed, Jun 05, 2019 at 04:24:09PM +0000, Jerin Jacob Kollanukkaran
Quoted 37 lines · wrote:
Quoted 37 lines
Quoted 37 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Sunday, May 26, 2019 12:14 AM
To: [email protected]
Cc: Neil Horman <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>; Bruce Richardson <[email protected]>; Thomas Monjalon <[email protected]>
Subject: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagHey-
Based on our recent conversations regarding the use of
symbols only meant for internal dpdk consumption (between dpdk libraries), this is an idea that I've come up with that I'd like to get some feedback onSummary:
1) We have symbols in the DPDK that are meant to be used between DPDK libraries, but not by applications linking to them
2) We would like to document those symbols in the code, so as to note them clearly as for being meant for internal use only
3) Linker symbol visibility is a very coarse grained tool, and so there is no good way in a single library to mark items as being meant for use only by other DPDK libraries, at least not without some extensive runtime checkingProposal:
I'm proposing that we introduce the__rte_internaltag. From a coding standpoint it works a great deal like the__rte_experimentaltag in that it expempts the tagged symbol from ABI constraints (as the only users should be represented in the DPDK build environment). Additionally, the__rte_internalmacro resolves differently based on the definition of theBUILDING_RTE_SDKflag (working under the assumption that said flag should only ever be set if we areQuoted 8 lines
Quoted 8 lines
Quoted 8 lines
actually building DPDK libraries which will make use of internal calls). If the
BUILDING_RTE_SDKflag is set__rte_internalresolves to__attribute__((section "text.internal)), placing it in a special text section which is then used to validate that the the symbol appears in the INTERNAL section of the corresponding library version map). IfBUILDING_RTE_SDKis not set, then__rte_internalresolves to__attribute__((error("..."))), which causes any caller of the tagged function to throw an error at compile time, indicating that the symbol is not available for external use.
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
This isn't a perfect solution, as applications can still hack around it of course,
Quoted 8 lines
Quoted 8 lines
I think, one way to, avoid, hack around could be to,
1) at config stage, create a random number for the build
2) introduceRTE_CALL_INTERNALmacro for calling internal function, compare the generated random number for allowing the calls to make within the library. i.e leverage the fact that external library would never know the random number generated for the DPDK buildand internal driver code does.
Quoted 1 line
Quoted 1 line
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Do we really need to care about this. If have some determined enough to hack around our limitations, then they surely know that they have an unsupported configuration. We just need to protect against inadvertent use of internals, IMHO.
I agree, I too had thought about doing some sort of internal runtime checking to match internal only symbols, such that they were only accessable by internally approved users, but it started to feel like a great
deal of overhead.
Quoted 5 lines
Quoted 5 lines
Its a good idea for a general mechanism I think, but I believe the value here is more to internally document which apis we want to mark as being for internal use only, and create a lightweight roadblock at build time to catch users inadvertently using them. Determined users will get around anything, and theres not much we can do to stop
them.
Quoted 8 lines
I agree too. IMHO, Simply having following items would be enough
1) Avoid exposing the internal function prototype through public header files
2) Add @internal to API documentation
3) Just decide the name space for internal API fortooling(i.e not start with rte_ or so)Using objdump scheme to detect internal functionsrequires the the library to build prior to run the checkpatch.
Quoted 1 line
No, I'm not comfortable with that approach, and I've stated why:
1) Not exposing the functions via header files is a fine start2) Adding internal documentation is also fine, but does nothing to correlate the code implementing those functions to the documentation. Its valuable to have a tag on a function identifying it as internal only.
Quoted 5 lines
3) Using naming conventions to separate internal only from non-internal functions is a vague approach, requiring future developers to be cogniscent of the convention and make the appropriate naming choices. It also implicitly restricts the abliity for future developers to make naming changes in conflict with that convention
Enforcing the naming convention can be achieved through tooling as well.
Quoted 5 lines
4) Adding a tag like
__rte_internalcreates an interlock whereby, not only are internal functions excused from ABI constraints, but forces developers to intentionally mark their internal functions as being internal in the code, which is beneficial to clarlity of understanding during the development process.Quoted 14 lines
No issues in adding
__rte_internal. But, I am against current implementaion, Ie. adding objdump dependency
to checkpatch i.e developer has to build the library first so that checkpatch can can know, Is it belongs to internal section or not?Quoted 9 lines
5) Adding a tag like
__rte_internalis explicit, and allows developers to use a single header file instead of multiple header files if they so chooseWe went through this with experimental symbols as well[1], and it just makes more sense to me to clearly document in the code what constitutes an internal symbol rather than relying on naming conventions and hoping that developers read the documentation before exporting a symbol publically.
Quoted 3 lines
I feel like we are creating a lot of extra work for the developer and adding a number of constraints to getting code patches submitted as the tools all have to be working together. The versioning file and
__rte_experimentalstuff today has always being handle wrong or not done by the developer. Altering the tools to detect these changes works and it seemed to take a while to iron out. To me we should be doing the minimum steps to reasonably isolate internal API and data from the user. If someone wants to access the those APIs that is their choice and enforcing with new macros and tools is over kill IMHO.1) Adding @internal to documentation is a great start along with more docs to explain what internal functions/data should be handled.
I've got no issue with this
Quoted 1 line
2) Hiding/moving internal function prototypes in private headers.
Nor this, though if you want to do it, ensuring that every library has a private header that doesn't get exported has to be part of the review process, and I'm not confident that that will be a focus of anyones review
Quoted 1 line
3) Adding setters/getters for internal data.
Sure, no issue here
Quoted 1 line
4) Make sure we review and reject direct use of internal functions and data.
How exactly do you intend to enforce this? By definition, the use of internal functions is restricted only to dpdk core libraries, and those are the only patches we ever see on the list. This change is meant to enforce usage prevention for users writing applications whos patches will never be seen for review (save for our example programs)
We already have this mechanism available for experimental abi, and from my perspective it works quite well, and is fairly well understood. I don't see whats wrong with expanding that mechanism to internal functions.
Neil
Quoted 20 lines
The goal here is to handle the 80% rule and make it very obvious to the developer these are internal functions and data. Using these or similar minimum steps should be reasonable IMHO.
Quoted 9 lines
Quoted 9 lines
[1] https://mails.dpdk.org/archives/dev/2017-December/083828.html
Quoted 6 lines
Quoted 6 lines
If we really wanted to go down that road, we could use a mechainsm simmilar to the
EXPORT_SYMBOL/EXPORT_SYMBOL_GPLinfrastructure that the kernel uses, but that would required building our own custom linker script, which seems like overkill here.Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Best
NeilQuoted 1 line
/Bruce
Regards,
Keith
Quoted 172 lines · Neil Horman
Quoted 149 lines · Keith
Quoted 134 lines · Jerin Jacob Kollanukkaran
Quoted 117 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Thursday, June 6, 2019 5:04 PM
To: Jerin Jacob Kollanukkaran <[email protected]>
Cc: Bruce Richardson <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Subject: Re: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagQuoted 74 lines · Jerin Jacob Kollanukkaran
Quoted 74 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Wednesday, June 5, 2019 11:41 PM
To: Bruce Richardson <[email protected]>
Cc: Jerin Jacob Kollanukkaran <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Subject: Re: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagQuoted 2 lines · Bruce Richardson
On Wed, Jun 05, 2019 at 04:24:09PM +0000, Jerin Jacob Kollanukkaran
Quoted 37 lines · wrote:
Quoted 37 lines
Quoted 37 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Sunday, May 26, 2019 12:14 AM
To: [email protected]
Cc: Neil Horman <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>; Bruce Richardson <[email protected]>; Thomas Monjalon <[email protected]>
Subject: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagHey-
Based on our recent conversations regarding the use of
symbols only meant for internal dpdk consumption (between dpdk libraries), this is an idea that I've come up with that I'd like to get some feedback onSummary:
1) We have symbols in the DPDK that are meant to be used between DPDK libraries, but not by applications linking to them
2) We would like to document those symbols in the code, so as to note them clearly as for being meant for internal use only
3) Linker symbol visibility is a very coarse grained tool, and so there is no good way in a single library to mark items as being meant for use only by other DPDK libraries, at least not without some extensive runtime checkingProposal:
I'm proposing that we introduce the__rte_internaltag. From a coding standpoint it works a great deal like the__rte_experimentaltag in that it expempts the tagged symbol from ABI constraints (as the only users should be represented in the DPDK build environment). Additionally, the__rte_internalmacro resolves differently based on the definition of theBUILDING_RTE_SDKflag (working under the assumption that said flag should only ever be set if we areQuoted 8 lines
Quoted 8 lines
Quoted 8 lines
actually building DPDK libraries which will make use of internal calls). If the
BUILDING_RTE_SDKflag is set__rte_internalresolves to__attribute__((section "text.internal)), placing it in a special text section which is then used to validate that the the symbol appears in the INTERNAL section of the corresponding library version map). IfBUILDING_RTE_SDKis not set, then__rte_internalresolves to__attribute__((error("..."))), which causes any caller of the tagged function to throw an error at compile time, indicating that the symbol is not available for external use.
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
This isn't a perfect solution, as applications can still hack around it of course,
Quoted 8 lines
Quoted 8 lines
I think, one way to, avoid, hack around could be to,
1) at config stage, create a random number for the build
2) introduceRTE_CALL_INTERNALmacro for calling internal function, compare the generated random number for allowing the calls to make within the library. i.e leverage the fact that external library would never know the random number generated for the DPDK buildand internal driver code does.
Quoted 1 line
Quoted 1 line
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Do we really need to care about this. If have some determined enough to hack around our limitations, then they surely know that they have an unsupported configuration. We just need to protect against inadvertent use of internals, IMHO.
I agree, I too had thought about doing some sort of internal runtime checking to match internal only symbols, such that they were only accessable by internally approved users, but it started to feel like a great
deal of overhead.
Quoted 5 lines
Quoted 5 lines
Its a good idea for a general mechanism I think, but I believe the value here is more to internally document which apis we want to mark as being for internal use only, and create a lightweight roadblock at build time to catch users inadvertently using them. Determined users will get around anything, and theres not much we can do to stop
them.
Quoted 8 lines
I agree too. IMHO, Simply having following items would be enough
1) Avoid exposing the internal function prototype through public header files
2) Add @internal to API documentation
3) Just decide the name space for internal API fortooling(i.e not start with rte_ or so)Using objdump scheme to detect internal functionsrequires the the library to build prior to run the checkpatch.
Quoted 1 line
No, I'm not comfortable with that approach, and I've stated why:
1) Not exposing the functions via header files is a fine start2) Adding internal documentation is also fine, but does nothing to correlate the code implementing those functions to the documentation. Its valuable to have a tag on a function identifying it as internal only.
Quoted 5 lines
3) Using naming conventions to separate internal only from non-internal functions is a vague approach, requiring future developers to be cogniscent of the convention and make the appropriate naming choices. It also implicitly restricts the abliity for future developers to make naming changes in conflict with that convention
Enforcing the naming convention can be achieved through tooling as well.
Quoted 5 lines
4) Adding a tag like
__rte_internalcreates an interlock whereby, not only are internal functions excused from ABI constraints, but forces developers to intentionally mark their internal functions as being internal in the code, which is beneficial to clarlity of understanding during the development process.Quoted 14 lines
No issues in adding
__rte_internal. But, I am against current implementaion, Ie. adding objdump dependency
to checkpatch i.e developer has to build the library first so that checkpatch can can know, Is it belongs to internal section or not?Quoted 9 lines
5) Adding a tag like
__rte_internalis explicit, and allows developers to use a single header file instead of multiple header files if they so chooseWe went through this with experimental symbols as well[1], and it just makes more sense to me to clearly document in the code what constitutes an internal symbol rather than relying on naming conventions and hoping that developers read the documentation before exporting a symbol publically.
Quoted 3 lines
I feel like we are creating a lot of extra work for the developer and adding a number of constraints to getting code patches submitted as the tools all have to be working together. The versioning file and
__rte_experimentalstuff today has always being handle wrong or not done by the developer. Altering the tools to detect these changes works and it seemed to take a while to iron out. To me we should be doing the minimum steps to reasonably isolate internal API and data from the user. If someone wants to access the those APIs that is their choice and enforcing with new macros and tools is over kill IMHO.1) Adding @internal to documentation is a great start along with more docs to explain what internal functions/data should be handled.
I've got no issue with this
Quoted 1 line
2) Hiding/moving internal function prototypes in private headers.
Nor this, though if you want to do it, ensuring that every library has a private header that doesn't get exported has to be part of the review process, and I'm not confident that that will be a focus of anyones review
Quoted 1 line
3) Adding setters/getters for internal data.
Sure, no issue here
Quoted 1 line
4) Make sure we review and reject direct use of internal functions and data.
How exactly do you intend to enforce this? By definition, the use of internal functions is restricted only to dpdk core libraries, and those are the only patches we ever see on the list. This change is meant to enforce usage prevention for users writing applications whos patches will never be seen for review (save for our example programs)
Quoted 3 lines
We already have this mechanism available for experimental abi, and from my perspective it works quite well, and is fairly well understood. I don't see whats wrong with expanding that mechanism to internal functions.
On this case I was talking more about accessing internal data directly instead of using getters/setters.
I understand your other points, but if we use the items above it should solve most of the issues. Adding another tag and then adding tools to make it work is just going a bit too far. The reason that experimental works is normally the developer only has a few APIs to mark and it works along with versioning you added. The versioning and experimental parts are kind of a requirement with shared libs and identifying experimental APIs. In the case of marking all of the APIs as internal could be a huge list for all of the libs, we should move them to private headers instead.
Quoted 21 lines
Neil
Quoted 18 lines
The goal here is to handle the 80% rule and make it very obvious to the developer these are internal functions and data. Using these or similar minimum steps should be reasonable IMHO.
Quoted 9 lines
Quoted 9 lines
[1] https://mails.dpdk.org/archives/dev/2017-December/083828.html
Quoted 6 lines
Quoted 6 lines
If we really wanted to go down that road, we could use a mechainsm simmilar to the
EXPORT_SYMBOL/EXPORT_SYMBOL_GPLinfrastructure that the kernel uses, but that would required building our own custom linker script, which seems like overkill here.Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Best
NeilQuoted 1 line
/Bruce
Regards,
Keith
Regards,
Keith
Quoted 172 lines · Keith
Quoted 172 lines · Neil Horman
Quoted 149 lines · Keith
Quoted 134 lines · Jerin Jacob Kollanukkaran
Quoted 117 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Thursday, June 6, 2019 5:04 PM
To: Jerin Jacob Kollanukkaran <[email protected]>
Cc: Bruce Richardson <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Subject: Re: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagQuoted 74 lines · Jerin Jacob Kollanukkaran
Quoted 74 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Wednesday, June 5, 2019 11:41 PM
To: Bruce Richardson <[email protected]>
Cc: Jerin Jacob Kollanukkaran <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Subject: Re: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagQuoted 2 lines · Bruce Richardson
On Wed, Jun 05, 2019 at 04:24:09PM +0000, Jerin Jacob Kollanukkaran
Quoted 37 lines · wrote:
Quoted 37 lines
Quoted 37 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Sunday, May 26, 2019 12:14 AM
To: [email protected]
Cc: Neil Horman <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>; Bruce Richardson <[email protected]>; Thomas Monjalon <[email protected]>
Subject: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagHey-
Based on our recent conversations regarding the use of
symbols only meant for internal dpdk consumption (between dpdk libraries), this is an idea that I've come up with that I'd like to get some feedback onSummary:
1) We have symbols in the DPDK that are meant to be used between DPDK libraries, but not by applications linking to them
2) We would like to document those symbols in the code, so as to note them clearly as for being meant for internal use only
3) Linker symbol visibility is a very coarse grained tool, and so there is no good way in a single library to mark items as being meant for use only by other DPDK libraries, at least not without some extensive runtime checkingProposal:
I'm proposing that we introduce the__rte_internaltag. From a coding standpoint it works a great deal like the__rte_experimentaltag in that it expempts the tagged symbol from ABI constraints (as the only users should be represented in the DPDK build environment). Additionally, the__rte_internalmacro resolves differently based on the definition of theBUILDING_RTE_SDKflag (working under the assumption that said flag should only ever be set if we areQuoted 8 lines
Quoted 8 lines
Quoted 8 lines
actually building DPDK libraries which will make use of internal calls). If the
BUILDING_RTE_SDKflag is set__rte_internalresolves to__attribute__((section "text.internal)), placing it in a special text section which is then used to validate that the the symbol appears in the INTERNAL section of the corresponding library version map). IfBUILDING_RTE_SDKis not set, then__rte_internalresolves to__attribute__((error("..."))), which causes any caller of the tagged function to throw an error at compile time, indicating that the symbol is not available for external use.
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
This isn't a perfect solution, as applications can still hack around it of course,
Quoted 8 lines
Quoted 8 lines
I think, one way to, avoid, hack around could be to,
1) at config stage, create a random number for the build
2) introduceRTE_CALL_INTERNALmacro for calling internal function, compare the generated random number for allowing the calls to make within the library. i.e leverage the fact that external library would never know the random number generated for the DPDK buildand internal driver code does.
Quoted 1 line
Quoted 1 line
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Do we really need to care about this. If have some determined enough to hack around our limitations, then they surely know that they have an unsupported configuration. We just need to protect against inadvertent use of internals, IMHO.
I agree, I too had thought about doing some sort of internal runtime checking to match internal only symbols, such that they were only accessable by internally approved users, but it started to feel like a great
deal of overhead.
Quoted 5 lines
Quoted 5 lines
Its a good idea for a general mechanism I think, but I believe the value here is more to internally document which apis we want to mark as being for internal use only, and create a lightweight roadblock at build time to catch users inadvertently using them. Determined users will get around anything, and theres not much we can do to stop
them.
Quoted 8 lines
I agree too. IMHO, Simply having following items would be enough
1) Avoid exposing the internal function prototype through public header files
2) Add @internal to API documentation
3) Just decide the name space for internal API fortooling(i.e not start with rte_ or so)Using objdump scheme to detect internal functionsrequires the the library to build prior to run the checkpatch.
Quoted 1 line
No, I'm not comfortable with that approach, and I've stated why:
1) Not exposing the functions via header files is a fine start2) Adding internal documentation is also fine, but does nothing to correlate the code implementing those functions to the documentation. Its valuable to have a tag on a function identifying it as internal only.
Quoted 5 lines
3) Using naming conventions to separate internal only from non-internal functions is a vague approach, requiring future developers to be cogniscent of the convention and make the appropriate naming choices. It also implicitly restricts the abliity for future developers to make naming changes in conflict with that convention
Enforcing the naming convention can be achieved through tooling as well.
Quoted 5 lines
4) Adding a tag like
__rte_internalcreates an interlock whereby, not only are internal functions excused from ABI constraints, but forces developers to intentionally mark their internal functions as being internal in the code, which is beneficial to clarlity of understanding during the development process.Quoted 14 lines
No issues in adding
__rte_internal. But, I am against current implementaion, Ie. adding objdump dependency
to checkpatch i.e developer has to build the library first so that checkpatch can can know, Is it belongs to internal section or not?Quoted 9 lines
5) Adding a tag like
__rte_internalis explicit, and allows developers to use a single header file instead of multiple header files if they so chooseWe went through this with experimental symbols as well[1], and it just makes more sense to me to clearly document in the code what constitutes an internal symbol rather than relying on naming conventions and hoping that developers read the documentation before exporting a symbol publically.
Quoted 3 lines
I feel like we are creating a lot of extra work for the developer and adding a number of constraints to getting code patches submitted as the tools all have to be working together. The versioning file and
__rte_experimentalstuff today has always being handle wrong or not done by the developer. Altering the tools to detect these changes works and it seemed to take a while to iron out. To me we should be doing the minimum steps to reasonably isolate internal API and data from the user. If someone wants to access the those APIs that is their choice and enforcing with new macros and tools is over kill IMHO.1) Adding @internal to documentation is a great start along with more docs to explain what internal functions/data should be handled.
I've got no issue with this
Quoted 1 line
2) Hiding/moving internal function prototypes in private headers.
Nor this, though if you want to do it, ensuring that every library has a private header that doesn't get exported has to be part of the review process, and I'm not confident that that will be a focus of anyones review
Quoted 1 line
3) Adding setters/getters for internal data.
Sure, no issue here
Quoted 1 line
4) Make sure we review and reject direct use of internal functions and data.
How exactly do you intend to enforce this? By definition, the use of internal functions is restricted only to dpdk core libraries, and those are the only patches we ever see on the list. This change is meant to enforce usage prevention for users writing applications whos patches will never be seen for review (save for our example programs)
Quoted 6 lines
Quoted 3 lines
We already have this mechanism available for experimental abi, and from my perspective it works quite well, and is fairly well understood. I don't see whats wrong with expanding that mechanism to internal functions.
On this case I was talking more about accessing internal data directly instead of using getters/setters.
Sure, we want to prevent that, but the linker version map should prevent it by not exporting those globals already, meaning we're required to implement getters and setters. You can get around that of course by adding a global variable to the linker map file, but thats strongly discouraged.
Quoted 1 line
I understand your other points, but if we use the items above it should solve most of the issues. Adding another tag and then adding tools to make it work is just going a bit too far. The reason that experimental works is normally the developer only has a few APIs to mark and it works along with versioning you added. The versioning and experimental parts are kind of a requirement with shared libs and identifying experimental APIs. In the case of marking all of the APIs as internal could be a huge list for all of the libs, we should move them to private headers instead.
We're not adding tools, we're modifying existing tools to make this work.
If you're concerned about the added onus of tagging internal only functions, I wouldn't be opposed to adding tooling to automatically document functions marked with __rte_internal with @internal in the documentation tree, so that the overhead is no larger than what we expect now (documenting @internal in the docs directly)
As for the effort to tag functions as internal only, I think we might disagree on what the size of the internal-only api is. If we start with Jerins assumption that all internal functions do not start with rte_, then a quick scan of the version maps in the dpdk suggests there are currently 266 functions that we would consider to be internal only. Of those 266 a little less than half are already addressed by the RFC patch I sent. Getting the rest under tagged as __rte_internal is about a days worth of work. From there its just standard maintenence.
Neil
Quoted 26 lines
Quoted 21 lines
Neil
Quoted 18 lines
The goal here is to handle the 80% rule and make it very obvious to the developer these are internal functions and data. Using these or similar minimum steps should be reasonable IMHO.
Quoted 9 lines
Quoted 9 lines
[1] https://mails.dpdk.org/archives/dev/2017-December/083828.html
Quoted 6 lines
Quoted 6 lines
If we really wanted to go down that road, we could use a mechainsm simmilar to the
EXPORT_SYMBOL/EXPORT_SYMBOL_GPLinfrastructure that the kernel uses, but that would required building our own custom linker script, which seems like overkill here.Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Best
NeilQuoted 1 line
/Bruce
Regards,
KeithRegards,
Keith
Quoted 116 lines · Jerin Jacob Kollanukkaran
Quoted 116 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Thursday, June 6, 2019 5:04 PM
To: Jerin Jacob Kollanukkaran <[email protected]>
Cc: Bruce Richardson <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Subject: Re: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagQuoted 74 lines · Jerin Jacob Kollanukkaran
Quoted 74 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Wednesday, June 5, 2019 11:41 PM
To: Bruce Richardson <[email protected]>
Cc: Jerin Jacob Kollanukkaran <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Subject: Re: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagQuoted 2 lines · Bruce Richardson
On Wed, Jun 05, 2019 at 04:24:09PM +0000, Jerin Jacob Kollanukkaran
Quoted 37 lines · wrote:
Quoted 37 lines
Quoted 37 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Sunday, May 26, 2019 12:14 AM
To: [email protected]
Cc: Neil Horman <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>; Bruce Richardson <[email protected]>; Thomas Monjalon <[email protected]>
Subject: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagHey-
Based on our recent conversations regarding the use of
symbols only meant for internal dpdk consumption (between dpdk libraries), this is an idea that I've come up with that I'd like to get some feedback onSummary:
1) We have symbols in the DPDK that are meant to be used between DPDK libraries, but not by applications linking to them
2) We would like to document those symbols in the code, so as to note them clearly as for being meant for internal use only
3) Linker symbol visibility is a very coarse grained tool, and so there is no good way in a single library to mark items as being meant for use only by other DPDK libraries, at least not without some extensive runtime checkingProposal:
I'm proposing that we introduce the__rte_internaltag. From a coding standpoint it works a great deal like the__rte_experimentaltag in that it expempts the tagged symbol from ABI constraints (as the only users should be represented in the DPDK build environment). Additionally, the__rte_internalmacro resolves differently based on the definition of theBUILDING_RTE_SDKflag (working under the assumption that said flag should only ever be set if we areQuoted 8 lines
Quoted 8 lines
Quoted 8 lines
actually building DPDK libraries which will make use of internal calls). If the
BUILDING_RTE_SDKflag is set__rte_internalresolves to__attribute__((section "text.internal)), placing it in a special text section which is then used to validate that the the symbol appears in the INTERNAL section of the corresponding library version map). IfBUILDING_RTE_SDKis not set, then__rte_internalresolves to__attribute__((error("..."))), which causes any caller of the tagged function to throw an error at compile time, indicating that the symbol is not available for external use.
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
This isn't a perfect solution, as applications can still hack around it of course,
Quoted 8 lines
Quoted 8 lines
I think, one way to, avoid, hack around could be to,
1) at config stage, create a random number for the build
2) introduceRTE_CALL_INTERNALmacro for calling internal function, compare the generated random number for allowing the calls to make within the library. i.e leverage the fact that external library would never know the random number generated for the DPDK buildand internal driver code does.
Quoted 1 line
Quoted 1 line
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Do we really need to care about this. If have some determined enough to hack around our limitations, then they surely know that they have an unsupported configuration. We just need to protect against inadvertent use of internals, IMHO.
I agree, I too had thought about doing some sort of internal runtime checking to match internal only symbols, such that they were only accessable by internally approved users, but it started to feel like a great
deal of overhead.
Quoted 5 lines
Quoted 5 lines
Its a good idea for a general mechanism I think, but I believe the value here is more to internally document which apis we want to mark as being for internal use only, and create a lightweight roadblock at build time to catch users inadvertently using them. Determined users will get around anything, and theres not much we can do to stop
them.
Quoted 8 lines
I agree too. IMHO, Simply having following items would be enough
1) Avoid exposing the internal function prototype through public header files
2) Add @internal to API documentation
3) Just decide the name space for internal API fortooling(i.e not start with rte_ or so)Using objdump scheme to detect internal functionsrequires the the library to build prior to run the checkpatch.
Quoted 1 line
No, I'm not comfortable with that approach, and I've stated why:
1) Not exposing the functions via header files is a fine start2) Adding internal documentation is also fine, but does nothing to correlate the code implementing those functions to the documentation. Its valuable to have a tag on a function identifying it as internal only.
Quoted 8 lines
Quoted 5 lines
3) Using naming conventions to separate internal only from non-internal functions is a vague approach, requiring future developers to be cogniscent of the convention and make the appropriate naming choices. It also implicitly restricts the abliity for future developers to make naming changes in conflict with that convention
Enforcing the naming convention can be achieved through tooling as well.
Sure, but why enforce any function naming at all, when you don't have to.
Quoted 8 lines
Quoted 5 lines
4) Adding a tag like
__rte_internalcreates an interlock whereby, not only are internal functions excused from ABI constraints, but forces developers to intentionally mark their internal functions as being internal in the code, which is beneficial to clarlity of understanding during the development process.No issues in adding
__rte_internal. But, I am against current implementaion, Ie. adding objdump dependency
That dependency already exists for the __rte_external flag
Quoted 3 lines
to checkpatch i.e developer has to build the library first so that checkpatch can can know, Is it belongs to internal section or not?
What developer is running checkpatch/posting patches without first building their changes?
Quoted 25 lines
Quoted 24 lines
5) Adding a tag like
__rte_internalis explicit, and allows developers to use a single header file instead of multiple header files if they so chooseWe went through this with experimental symbols as well[1], and it just makes more sense to me to clearly document in the code what constitutes an internal symbol rather than relying on naming conventions and hoping that developers read the documentation before exporting a symbol publically.
[1] https://mails.dpdk.org/archives/dev/2017-December/083828.html
Quoted 12 lines
Quoted 11 lines
If we really wanted to go down that road, we could use a mechainsm simmilar to the
EXPORT_SYMBOL/EXPORT_SYMBOL_GPLinfrastructure that the kernel uses, but that would required building our own custom linker script, which seems like overkill here.Best
NeilQuoted 2 lines
/Bruce
Quoted 140 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Thursday, June 6, 2019 7:05 PM
To: Jerin Jacob Kollanukkaran <[email protected]>
Cc: Bruce Richardson <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Subject: Re: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagQuoted 9 lines · Jerin Jacob Kollanukkaran
Quoted 9 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Thursday, June 6, 2019 5:04 PM
To: Jerin Jacob Kollanukkaran <[email protected]>
Cc: Bruce Richardson <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Subject: Re: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagOn Thu, Jun 06, 2019 at 09:44:52AM +0000, Jerin Jacob Kollanukkaran
Quoted 42 lines · wrote:
Quoted 42 lines
Quoted 42 lines
Quoted 42 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Wednesday, June 5, 2019 11:41 PM
To: Bruce Richardson <[email protected]>
Cc: Jerin Jacob Kollanukkaran <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Subject: Re: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagQuoted 2 lines · Bruce Richardson
On Wed, Jun 05, 2019 at 04:24:09PM +0000, Jerin Jacob Kollanukkaran
Quoted 30 lines · wrote:
Quoted 30 lines
Quoted 30 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Sunday, May 26, 2019 12:14 AM
To: [email protected]
Cc: Neil Horman <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>; Bruce Richardson <[email protected]>; Thomas Monjalon <[email protected]>
Subject: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagHey-
Based on our recent conversations regarding the use of
symbols only meant for internal dpdk consumption (between dpdk libraries), this is an idea that I've come up with that I'd like to get some feedback onSummary:
1) We have symbols in the DPDK that are meant to be used between DPDK libraries, but not by applications linking to them
2) We would like to document those symbols in the code, so as to note them clearly as for being meant for internal use only
3) Linker symbol visibility is a very coarse grained tool, and so there is no good way in a single library to mark items as being meant for use only by other DPDK libraries, at least not without some extensive runtime checkingQuoted 15 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Proposal:
I'm proposing that we introduce the__rte_internaltag. From a coding standpoint it works a great deal like the__rte_experimentaltag in that it expempts the tagged symbol from ABI constraints (as the only users should be represented in the DPDK build environment). Additionally, the__rte_internalmacro resolves differently based on theQuoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
definition of the
BUILDING_RTE_SDKflag (working under the assumption that said flag should only ever be set if we are actually building DPDK libraries which will make use of internal calls). If theBUILDING_RTE_SDKflag is set__rte_internalresolves to__attribute__((section "text.internal)), placing it in a special text section which is then used to validate that the the symbol appears in the INTERNAL section of the corresponding library versionmap).
Quoted 48 lines
Quoted 48 lines
Quoted 24 lines
Quoted 24 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
If
BUILDING_RTE_SDKis not set, then__rte_internalresolves to__attribute__((error("..."))), which causes any caller of the tagged function to throw an error at compile time, indicating that the symbol is not available for external use.
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
This isn't a perfect solution, as applications can still hack around it of course,
Quoted 8 lines
Quoted 8 lines
I think, one way to, avoid, hack around could be to,
1) at config stage, create a random number for the build
2) introduceRTE_CALL_INTERNALmacro for calling internal function, compare the generated random number for allowing the calls to make within the library. i.e leverage the fact that external library would never know the random number generated for the DPDK buildand internal driver code does.
Quoted 6 lines
Quoted 1 line
Do we really need to care about this. If have some determined enough to hack around our limitations, then they surely know that they have an unsupported configuration. We just need to protect against inadvertent use of internals, IMHO.
Quoted 4 lines
Quoted 4 lines
I agree, I too had thought about doing some sort of internal runtime checking to match internal only symbols, such that they were only accessable by internally approved users, but it started to feel like a great
deal of overhead.
Quoted 6 lines
Quoted 6 lines
Its a good idea for a general mechanism I think, but I believe the value here is more to internally document which apis we want to mark as being for internal use only, and create a lightweight roadblock at build time to catch users inadvertently using them. Determined users will get around anything, and theres not much we can do to stop
them.
Quoted 9 lines
I agree too. IMHO, Simply having following items would be enough
1) Avoid exposing the internal function prototype through public header files
2) Add @internal to API documentation
3) Just decide the name space for internal API fortooling(i.e not start with rte_ or so)Using objdump scheme to detect internal functionsrequires the the library to build prior to run the checkpatch.
Quoted 1 line
Quoted 6 lines
Quoted 6 lines
No, I'm not comfortable with that approach, and I've stated why:
1) Not exposing the functions via header files is a fine start2) Adding internal documentation is also fine, but does nothing to correlate the code implementing those functions to the documentation. Its valuable to have a tag on a function identifying it as
internal only.
Quoted 8 lines
Quoted 7 lines
3) Using naming conventions to separate internal only from non-internal functions is a vague approach, requiring future developers to be cogniscent of the convention and make the appropriate naming choices. It also implicitly restricts the abliity for future developers to make naming changes in conflict with that convention
Quoted 3 lines
Quoted 2 lines
Enforcing the naming convention can be achieved through tooling as well.
Sure, but why enforce any function naming at all, when you don't have to.
May I ask, why to enforce __rte_internal, when you don't have to
Quoted 11 lines
Quoted 6 lines
Quoted 6 lines
4) Adding a tag like
__rte_internalcreates an interlock whereby, not only are internal functions excused from ABI constraints, but forces developers to intentionally mark their internal functions as being internal in the code, which is beneficial to clarlity of understandingduring the development process.
Quoted 3 lines
No issues in adding
__rte_internal. But, I am against current implementaion, Ie. adding objdump dependencyThat dependency already exists for the
__rte_externalflag
Sorry, I could not see the dependency.
[master][dpdk.org] $ grep -ri "objdump" devtools/ [master][dpdk.org] $ grep -ri "objdump" usertools/ [master][dpdk.org] $ grep -ri "__rte_external" *
Quoted 6 lines
Quoted 4 lines
to checkpatch i.e developer has to build the library first so that checkpatch can can know, Is it belongs to internal section or not?
What developer is running checkpatch/posting patches without first building their changes?
# it is not developer, The CI/CD tools can quicky check the sanity of patches before the build itself. Why to add unnecessary dependency? # If some PMD is not building if the requirements are not meet(say AES NI PMD for crypto) then how do take care of the dependency.
Quoted 28 lines
Quoted 28 lines
Quoted 27 lines
5) Adding a tag like
__rte_internalis explicit, and allows developers to use a single header file instead of multiple header files if they so chooseWe went through this with experimental symbols as well[1], and it just makes more sense to me to clearly document in the code what constitutes an internal symbol rather than relying on naming conventions and hoping that developers read the documentation before exporting a symbol publically.
[1] https://mails.dpdk.org/archives/dev/2017-December/083828.html
Quoted 12 lines
Quoted 11 lines
If we really wanted to go down that road, we could use a mechainsm simmilar to the
EXPORT_SYMBOL/EXPORT_SYMBOL_GPLinfrastructure that the kernel uses, but that would required building our own custom linker script, which seems like overkill here.Best
NeilQuoted 2 lines
/Bruce
Quoted 140 lines · Jerin Jacob Kollanukkaran
Quoted 140 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Thursday, June 6, 2019 7:05 PM
To: Jerin Jacob Kollanukkaran <[email protected]>
Cc: Bruce Richardson <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Subject: Re: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagQuoted 9 lines · Jerin Jacob Kollanukkaran
Quoted 9 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Thursday, June 6, 2019 5:04 PM
To: Jerin Jacob Kollanukkaran <[email protected]>
Cc: Bruce Richardson <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Subject: Re: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagOn Thu, Jun 06, 2019 at 09:44:52AM +0000, Jerin Jacob Kollanukkaran
Quoted 42 lines · wrote:
Quoted 42 lines
Quoted 42 lines
Quoted 42 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Wednesday, June 5, 2019 11:41 PM
To: Bruce Richardson <[email protected]>
Cc: Jerin Jacob Kollanukkaran <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Subject: Re: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagQuoted 2 lines · Bruce Richardson
On Wed, Jun 05, 2019 at 04:24:09PM +0000, Jerin Jacob Kollanukkaran
Quoted 30 lines · wrote:
Quoted 30 lines
Quoted 30 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Sunday, May 26, 2019 12:14 AM
To: [email protected]
Cc: Neil Horman <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>; Bruce Richardson <[email protected]>; Thomas Monjalon <[email protected]>
Subject: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagHey-
Based on our recent conversations regarding the use of
symbols only meant for internal dpdk consumption (between dpdk libraries), this is an idea that I've come up with that I'd like to get some feedback onSummary:
1) We have symbols in the DPDK that are meant to be used between DPDK libraries, but not by applications linking to them
2) We would like to document those symbols in the code, so as to note them clearly as for being meant for internal use only
3) Linker symbol visibility is a very coarse grained tool, and so there is no good way in a single library to mark items as being meant for use only by other DPDK libraries, at least not without some extensive runtime checkingQuoted 15 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Proposal:
I'm proposing that we introduce the__rte_internaltag. From a coding standpoint it works a great deal like the__rte_experimentaltag in that it expempts the tagged symbol from ABI constraints (as the only users should be represented in the DPDK build environment). Additionally, the__rte_internalmacro resolves differently based on theQuoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
definition of the
BUILDING_RTE_SDKflag (working under the assumption that said flag should only ever be set if we are actually building DPDK libraries which will make use of internal calls). If theBUILDING_RTE_SDKflag is set__rte_internalresolves to__attribute__((section "text.internal)), placing it in a special text section which is then used to validate that the the symbol appears in the INTERNAL section of the corresponding library versionmap).
Quoted 48 lines
Quoted 48 lines
Quoted 24 lines
Quoted 24 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
If
BUILDING_RTE_SDKis not set, then__rte_internalresolves to__attribute__((error("..."))), which causes any caller of the tagged function to throw an error at compile time, indicating that the symbol is not available for external use.
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
This isn't a perfect solution, as applications can still hack around it of course,
Quoted 8 lines
Quoted 8 lines
I think, one way to, avoid, hack around could be to,
1) at config stage, create a random number for the build
2) introduceRTE_CALL_INTERNALmacro for calling internal function, compare the generated random number for allowing the calls to make within the library. i.e leverage the fact that external library would never know the random number generated for the DPDK buildand internal driver code does.
Quoted 6 lines
Quoted 1 line
Do we really need to care about this. If have some determined enough to hack around our limitations, then they surely know that they have an unsupported configuration. We just need to protect against inadvertent use of internals, IMHO.
Quoted 4 lines
Quoted 4 lines
I agree, I too had thought about doing some sort of internal runtime checking to match internal only symbols, such that they were only accessable by internally approved users, but it started to feel like a great
deal of overhead.
Quoted 6 lines
Quoted 6 lines
Its a good idea for a general mechanism I think, but I believe the value here is more to internally document which apis we want to mark as being for internal use only, and create a lightweight roadblock at build time to catch users inadvertently using them. Determined users will get around anything, and theres not much we can do to stop
them.
Quoted 9 lines
I agree too. IMHO, Simply having following items would be enough
1) Avoid exposing the internal function prototype through public header files
2) Add @internal to API documentation
3) Just decide the name space for internal API fortooling(i.e not start with rte_ or so)Using objdump scheme to detect internal functionsrequires the the library to build prior to run the checkpatch.
Quoted 1 line
Quoted 6 lines
Quoted 6 lines
No, I'm not comfortable with that approach, and I've stated why:
1) Not exposing the functions via header files is a fine start2) Adding internal documentation is also fine, but does nothing to correlate the code implementing those functions to the documentation. Its valuable to have a tag on a function identifying it as
internal only.
Quoted 8 lines
Quoted 7 lines
3) Using naming conventions to separate internal only from non-internal functions is a vague approach, requiring future developers to be cogniscent of the convention and make the appropriate naming choices. It also implicitly restricts the abliity for future developers to make naming changes in conflict with that convention
Quoted 6 lines
Quoted 3 lines
Quoted 2 lines
Enforcing the naming convention can be achieved through tooling as well.
Sure, but why enforce any function naming at all, when you don't have to.
May I ask, why to enforce
__rte_internal, when you don't have to
Because its more clear. Implicitly deciding that any function not prefixed with rte_ is internal only does nothing to prevent a developer from accidentally naming a function incorrectly, exporting it, and allowing a user to call it. We can move headers all you want, but we provide an ABI guarantee to end users, and developers should have a way to clearly record that without having to check the documentation for each function that an application developer wants to use.
The long and the short of it for me is that I want a way for developers to opt their code into an internal only condition, not to just document it as such and hope its up to date. If they tag a function as __rte_internal then its clearly marked as internal only, they have checks to ensure that its in the INTERNAL section of the version map, and should that header somehow get externally exported (see rte_mempool_check_cookies for an example of how thats happened), users are prevented from using them at build time, rather than having to ask questions on the list, or read documentation after an error to find out "oops, shouldn't have done that".
I think you'll find that going through all the header files, and bifurcating them to public and private headers is a much larger undertaking than just tagging those functions accordingly. a quick scan of all our header file for the @internal tag shows about 260 instances of such functions, almost all of which are published to applications. All of those functions would have to be moved to private headers, and their requisite C files would need to be updated to include the new header. with the use of __rte_internal, we just have tag the functions as such, which can be handled with a cocinelle or awk script.
Neil
Quoted 60 lines
Quoted 11 lines
Quoted 6 lines
Quoted 6 lines
4) Adding a tag like
__rte_internalcreates an interlock whereby, not only are internal functions excused from ABI constraints, but forces developers to intentionally mark their internal functions as being internal in the code, which is beneficial to clarlity of understandingduring the development process.
Quoted 3 lines
No issues in adding
__rte_internal. But, I am against current implementaion, Ie. adding objdump dependencyThat dependency already exists for the
__rte_externalflagSorry, I could not see the dependency.
[master][dpdk.org] $ grep -ri "objdump" devtools/ [master][dpdk.org] $ grep -ri "objdump" usertools/ [master][dpdk.org] $ grep -ri "
__rte_external" *Quoted 6 lines
Quoted 4 lines
to checkpatch i.e developer has to build the library first so that checkpatch can can know, Is it belongs to internal section or not?
What developer is running checkpatch/posting patches without first building their changes?
# it is not developer, The CI/CD tools can quicky check the sanity of patches before the build itself. Why to add unnecessary dependency? # If some PMD is not building if the requirements are not meet(say AES NI PMD for crypto) then how do take care of the dependency.Quoted 28 lines
Quoted 28 lines
Quoted 27 lines
5) Adding a tag like
__rte_internalis explicit, and allows developers to use a single header file instead of multiple header files if they so chooseWe went through this with experimental symbols as well[1], and it just makes more sense to me to clearly document in the code what constitutes an internal symbol rather than relying on naming conventions and hoping that developers read the documentation before exporting a symbol publically.
[1] https://mails.dpdk.org/archives/dev/2017-December/083828.html
Quoted 12 lines
Quoted 11 lines
If we really wanted to go down that road, we could use a mechainsm simmilar to the
EXPORT_SYMBOL/EXPORT_SYMBOL_GPLinfrastructure that the kernel uses, but that would required building our own custom linker script, which seems like overkill here.Best
NeilQuoted 2 lines
/Bruce
Quoted 188 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Thursday, June 6, 2019 8:34 PM
To: Jerin Jacob Kollanukkaran <[email protected]>
Cc: Bruce Richardson <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Subject: Re: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagQuoted 9 lines · Jerin Jacob Kollanukkaran
Quoted 9 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Thursday, June 6, 2019 7:05 PM
To: Jerin Jacob Kollanukkaran <[email protected]>
Cc: Bruce Richardson <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Subject: Re: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagOn Thu, Jun 06, 2019 at 12:04:57PM +0000, Jerin Jacob Kollanukkaran
Quoted 21 lines · wrote:
Quoted 21 lines
Quoted 10 lines
Quoted 10 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Thursday, June 6, 2019 5:04 PM
To: Jerin Jacob Kollanukkaran <[email protected]>
Cc: Bruce Richardson <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Subject: Re: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagOn Thu, Jun 06, 2019 at 09:44:52AM +0000, Jerin Jacob Kollanukkaran
Quoted 10 lines · wrote:
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Wednesday, June 5, 2019 11:41 PM
To: Bruce Richardson <[email protected]>
Cc: Jerin Jacob Kollanukkaran <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Subject: Re: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagOn Wed, Jun 05, 2019 at 05:45:41PM +0100, Bruce Richardson
Quoted 70 lines · wrote:
Quoted 70 lines
Quoted 43 lines
Quoted 43 lines
Quoted 43 lines
Quoted 43 lines
Quoted 2 lines
On Wed, Jun 05, 2019 at 04:24:09PM +0000, Jerin Jacob Kollanukkaran
Quoted 40 lines · wrote:
Quoted 40 lines
Quoted 40 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Sunday, May 26, 2019 12:14 AM
To: [email protected]
Cc: Neil Horman <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>; Bruce Richardson <[email protected]>; Thomas Monjalon <[email protected]>
Subject: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagHey-
Based on our recent conversations regarding the use
of symbols only meant for internal dpdk consumption (between dpdk libraries), this is an idea that I've come up with that I'd like to get some feedback onSummary:
1) We have symbols in the DPDK that are meant to be used between DPDK libraries, but not by applications linking to them
2) We would like to document those symbols in the code, so as to note them clearly as for being meant for internal use only
3) Linker symbol visibility is a very coarse grained tool, and so there is no good way in a single library to mark items as being meant for use only by other DPDK libraries, at least not without some extensive runtime checkingProposal:
I'm proposing that we introduce the__rte_internaltag. From a coding standpoint it works a great deal like the__rte_experimentaltag in that it expempts the tagged symbol from ABI constraints (as the only users should be represented in the DPDK build environment). Additionally, the__rte_internalmacro resolves differently based on the definition of theBUILDING_RTE_SDKflag (working under the assumptionQuoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
that said flag should only ever be set if we are actually building DPDK libraries which will make use of internal calls). If the
BUILDING_RTE_SDKflag is set__rte_internalresolves to__attribute__((section "text.internal)), placing it in a special text section which is then used to validate that the the symbol appears in the INTERNAL section of the corresponding library versionmap).
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
If
BUILDING_RTE_SDKis not set, then__rte_internalresolves to__attribute__((error("..."))), which causes any caller of the tagged function to throw an error at compile time, indicating that the symbol is not available for external use.
Quoted 13 lines
Quoted 13 lines
Quoted 3 lines
This isn't a perfect solution, as applications can still hack around it of course,
I think, one way to, avoid, hack around could be to,
1) at config stage, create a random number for the build
2) introduceRTE_CALL_INTERNALmacro for calling internal function, compare the generated random number for allowing the calls to make within the library. i.e leverage the fact that external library would never know the random number generated for the DPDK buildQuoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
and internal driver code does.
Quoted 5 lines
Quoted 1 line
Do we really need to care about this. If have some determined enough to hack around our limitations, then they surely know that they have an unsupported configuration. We just need to protect against inadvertent use
of internals, IMHO.
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
I agree, I too had thought about doing some sort of internal runtime checking to match internal only symbols, such that they were only accessable by internally approved users, but it started to feel like a great
deal of overhead.
Quoted 4 lines
Quoted 4 lines
Its a good idea for a general mechanism I think, but I believe the value here is more to internally document which apis we want to mark as being for internal use only, and create a lightweight roadblock at build time to catch users
inadvertently using them.
Quoted 25 lines
Quoted 25 lines
Quoted 16 lines
Quoted 16 lines
Quoted 2 lines
Quoted 2 lines
Determined users will get around anything, and theres not much we can do to stop
them.
Quoted 10 lines
I agree too. IMHO, Simply having following items would be enough
1) Avoid exposing the internal function prototype through public header files
2) Add @internal to API documentation
3) Just decide the name space for internal API fortooling(i.e not start with rte_ or so)Using objdump scheme to detect internal functionsrequires the the library to build prior to run the checkpatch.
Quoted 1 line
Quoted 7 lines
Quoted 7 lines
No, I'm not comfortable with that approach, and I've stated why:
1) Not exposing the functions via header files is a fine start2) Adding internal documentation is also fine, but does nothing to correlate the code implementing those functions to the documentation. Its valuable to have a tag on a function identifying it as
internal only.
Quoted 1 line
Quoted 1 line
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 6 lines
3) Using naming conventions to separate internal only from non-internal functions is a vague approach, requiring future developers to be cogniscent of the convention and make the appropriate naming choices. It also implicitly restricts the abliity for future developers to make naming changes in conflict with that convention
Enforcing the naming convention can be achieved through tooling as
well.
Quoted 2 lines
Quoted 2 lines
Quoted 1 line
Sure, but why enforce any function naming at all, when you don't have
to.
Quoted 3 lines
May I ask, why to enforce
__rte_internal, when you don't have toBecause its more clear. Implicitly deciding that any function not prefixed with
rte_is internal only does nothing to prevent a developer from accidentally naming a function incorrectly, exporting it, and allowing a user to call it. We can move headers all you want, but we provide an ABI guarantee to end users, and developers should have a way to clearly record that without having to check the documentation for each function that an application developer wants to use.The long and the short of it for me is that I want a way for developers to opt their code into an internal only condition, not to just document it as such and hope its up to date. If they tag a function as
__rte_internalthen its clearly marked as internal only, they have checks to ensure that its in the INTERNAL section of the version map, and should that header somehow get externally exported (seerte_mempool_check_cookiesfor an example of how thats happened), users are prevented from using them at build time, rather than having to ask questions on the list, or read documentation after an error to find out "oops, shouldn't have done that".
Quoted 9 lines
I think you'll find that going through all the header files, and bifurcating them to public and private headers is a much larger undertaking than just tagging those functions accordingly. a quick scan of all our header file for the @internal tag shows about 260 instances of such functions, almost all of which are published to applications. All of those functions would have to be moved to private headers, and their requisite C files would need to be updated to include the new header. with the use of
__rte_internal, we just have tag the functions as such, which can be handled with a cocinelle or awk script.
I don't have any strong opinion on name prefix vs marking as __rte_internal. Or combination of both. I am fine any approach.
I have only strong option on not to induce objdump dependency for checkpatch. For the reason mentioned in http://mails.dpdk.org/archives/dev/2019-June/134160.html.
Quoted 68 lines
Neil
Quoted 23 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
4) Adding a tag like
__rte_internalcreates an interlock whereby, not only are internal functions excused from ABI constraints, but forces developers to intentionally mark their internal functions as being internal in the code, which is beneficial to clarlity of understandingQuoted 5 lines
during the development process.
Quoted 3 lines
No issues in adding
__rte_internal. But, I am against current implementaion, Ie. adding objdump dependencyThat dependency already exists for the
__rte_externalflagSorry, I could not see the dependency.
[master][dpdk.org] $ grep -ri "objdump" devtools/ [master][dpdk.org] $ grep -ri "objdump" usertools/ [master][dpdk.org] $ grep -ri "
__rte_external" *Quoted 4 lines
Quoted 4 lines
to checkpatch i.e developer has to build the library first so that checkpatch can can know, Is it belongs to internal section or not?
Quoted 28 lines
Quoted 2 lines
What developer is running checkpatch/posting patches without first building their changes?
# it is not developer, The CI/CD tools can quicky check the sanity of patches before the build itself. Why to add unnecessary dependency? # If some PMD is not building if the requirements are not meet(say AES NI PMD for crypto) then how do take care of the dependency.Quoted 19 lines
Quoted 19 lines
Quoted 19 lines
5) Adding a tag like
__rte_internalis explicit, and allows developers to use a single header file instead of multiple header files if they so chooseWe went through this with experimental symbols as well[1], and it just makes more sense to me to clearly document in the code what constitutes an internal symbol rather than relying on naming conventions and hoping that developers read the documentation before exporting a symbol publically.
[1]
https://mails.dpdk.org/archives/dev/2017-December/083828.htmlQuoted 3 lines
Quoted 3 lines
If we really wanted to go down that road, we could use a mechainsm simmilar to the
EXPORT_SYMBOL/
EXPORT_SYMBOL_GPLQuoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
infrastructure that the kernel uses, but that would required building our own custom linker script, which seems like overkill
here.
Quoted 9 lines
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
Best
NeilQuoted 2 lines
/Bruce
Quoted 8 lines · Jerin Jacob Kollanukkaran
<snip as this is getting long>
I don't have any strong opinion on name prefix vs marking as
__rte_internal. Or combination of both. I am fine any approach.I have only strong option on not to induce objdump dependency for checkpatch. For the reason mentioned in http://mails.dpdk.org/archives/dev/2019-June/134160.html.
Sorry, in my haste I didn't fully adress this in your previous email
I'm really uncertain what you mean by introducing a checkpatch dependency on objdump here. Theres nothing preventing you from running checkpatch before you build the library. The only thing checkpatch does in dpdk is scan the patches for sytle violations, and for changes in the map file for movement to and from the EXPERIMENTAL section (i.e. no use of objdump).
My patch modifies check-experimental-syms.sh (adding an objdump scan for INTERNAL symbols, and renaming the script to check-special-syms.sh to be more meaningful). That script however, is not run by checkpatch, its run during compilation of the library to ensure that any symbol in a map file is also tagged with __rte_internal in the corresponding object). Theres no path from checkpatches to check-experimental-syms.sh
What I meant in my last comment was that any dependency on objdump in check-[experimental|special]-syms.sh already existed prior to this patch.
So I'm unsure why you think checkpatches has a dependency.
Neil
Quoted 36 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Thursday, June 6, 2019 8:57 PM
To: Jerin Jacob Kollanukkaran <[email protected]>
Cc: Bruce Richardson <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Subject: Re: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagQuoted 3 lines · Jerin Jacob Kollanukkaran
<snip as this is getting long>
I don't have any strong opinion on name prefix vs marking as
__rte_internal.Quoted 3 lines
Or combination of both. I am fine any approach.
I have only strong option on not to induce objdump dependency for
checkpatch.
Quoted 1 line
For the reason mentioned in http://mails.dpdk.org/archives/dev/2019-
June/134160.html.
Quoted 1 line
Sorry, in my haste I didn't fully adress this in your previous email
I'm really uncertain what you mean by introducing a checkpatch dependency on objdump here. Theres nothing preventing you from running checkpatch before you build the library. The only thing checkpatch does in dpdk is scan the patches for sytle violations, and for changes in the map file for movement to and from the EXPERIMENTAL section (i.e. no use of objdump).
My patch modifies
check-experimental-syms.sh(adding an objdump scan for INTERNAL symbols, and renaming the script tocheck-special-syms.shto be more meaningful). That script however, is not run by checkpatch, its run during compilation of the library to ensure that any symbol in a map file is also tagged with__rte_internalin the corresponding object). Theres no path from checkpatches tocheck-experimental-syms.sh
Quoted 2 lines
What I meant in my last comment was that any dependency on objdump in check-[experimental|special]-syms.sh already existed prior to this patch.
I see. I thought your patches addressing issue related to
http://patches.dpdk.org/patch/53590/
Where checkpatch.sh complaints when we add new internal driver API with out rte_experimental? That's where all the discussion started. The reason why I was saying the API name prefix to mark as internal API is that checkpatch can detect that case.
Example:
ERROR: symbol otx2_mbox_alloc_msg_rsp is added in the DPDK_19.05 section, but is expected to be added in the EXPERIMENTAL section of the version map
Quoted 4 lines
So I'm unsure why you think checkpatches has a dependency.
Neil
Quoted 36 lines · Jerin Jacob Kollanukkaran
Quoted 36 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Thursday, June 6, 2019 8:57 PM
To: Jerin Jacob Kollanukkaran <[email protected]>
Cc: Bruce Richardson <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Subject: Re: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagQuoted 3 lines · Jerin Jacob Kollanukkaran
<snip as this is getting long>
I don't have any strong opinion on name prefix vs marking as
__rte_internal.Quoted 3 lines
Or combination of both. I am fine any approach.
I have only strong option on not to induce objdump dependency for
checkpatch.
Quoted 1 line
For the reason mentioned in http://mails.dpdk.org/archives/dev/2019-
June/134160.html.
Quoted 1 line
Sorry, in my haste I didn't fully adress this in your previous email
I'm really uncertain what you mean by introducing a checkpatch dependency on objdump here. Theres nothing preventing you from running checkpatch before you build the library. The only thing checkpatch does in dpdk is scan the patches for sytle violations, and for changes in the map file for movement to and from the EXPERIMENTAL section (i.e. no use of objdump).
My patch modifies
check-experimental-syms.sh(adding an objdump scan for INTERNAL symbols, and renaming the script tocheck-special-syms.shto be more meaningful). That script however, is not run by checkpatch, its run during compilation of the library to ensure that any symbol in a map file is also tagged with__rte_internalin the corresponding object). Theres no path from checkpatches tocheck-experimental-syms.sh
Quoted 6 lines
Quoted 2 lines
What I meant in my last comment was that any dependency on objdump in check-[experimental|special]-syms.sh already existed prior to this patch.
I see. I thought your patches addressing issue related to
http://patches.dpdk.org/patch/53590/
It does, it just does it in a different way than you do it.
Quoted 5 lines
Where
checkpatch.shcomplaints when we add new internal driver API with outrte_experimental? That's where all the discussion started. The reason why I was saying the API name prefix to mark as internal API is that checkpatch can detect that case.
Ah, thats a fair point, with my approach we need to add some code to check-symbol-change.sh such that any symbols listed in an INTERNAL section were ignored. That would provide behavioral compatibility to what you were doing in your patch.
But regardless, there is no dependency on objdump that wasn't there before, are we in agreement on that?
Neil
Quoted 10 lines
Example:
ERROR: symbolotx2_mbox_alloc_msg_rspis added in the DPDK_19.05 section, but is expected to be added in the EXPERIMENTAL section of the version mapQuoted 4 lines
So I'm unsure why you think checkpatches has a dependency.
Neil
Quoted 63 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Thursday, June 6, 2019 10:26 PM
To: Jerin Jacob Kollanukkaran <[email protected]>
Cc: Bruce Richardson <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Subject: Re: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagQuoted 9 lines · Jerin Jacob Kollanukkaran
Quoted 9 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Thursday, June 6, 2019 8:57 PM
To: Jerin Jacob Kollanukkaran <[email protected]>
Cc: Bruce Richardson <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Subject: Re: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagOn Thu, Jun 06, 2019 at 03:14:42PM +0000, Jerin Jacob Kollanukkaran
Quoted 13 lines · wrote:
Quoted 13 lines
Quoted 3 lines
<snip as this is getting long>
I don't have any strong opinion on name prefix vs marking as
__rte_internal.Quoted 3 lines
Or combination of both. I am fine any approach.
I have only strong option on not to induce objdump dependency for
checkpatch.
Quoted 2 lines
For the reason mentioned in
http://mails.dpdk.org/archives/dev/2019-June/134160.html.
Quoted 1 line
Quoted 7 lines
Quoted 7 lines
Sorry, in my haste I didn't fully adress this in your previous email
I'm really uncertain what you mean by introducing a checkpatch dependency on objdump here. Theres nothing preventing you from running checkpatch before you build the library. The only thing checkpatch does in dpdk is scan the patches for sytle violations, and for changes in the map file for movement to and from the
EXPERIMENTAL section (i.e. no use of objdump).
Quoted 11 lines
Quoted 11 lines
My patch modifies
check-experimental-syms.sh(adding an objdump scan for INTERNAL symbols, and renaming the script tocheck-special-syms.shto be more meaningful). That script however, is not run by checkpatch, its run during compilation of the library to ensure that any symbol in a map file is also tagged with__rte_internalin the corresponding object). Theres no path from checkpatches tocheck-experimental-syms.shWhat I meant in my last comment was that any dependency on objdump in check-[experimental|special]-syms.sh already existed prior to this
patch.
Quoted 4 lines
I see. I thought your patches addressing issue related to
http://patches.dpdk.org/patch/53590/It does, it just does it in a different way than you do it.
Quoted 5 lines
Where
checkpatch.shcomplaints when we add new internal driver API with outrte_experimental? That's where all the discussion started. The reason why I was saying the API name prefix to mark as internal API is that checkpatch can detect that case.
Quoted 4 lines
Ah, thats a fair point, with my approach we need to add some code to check-
symbol-change.shsuch that any symbols listed in an INTERNAL section were ignored. That would provide behavioral compatibility to what you were doing in your patch.
OK.
Quoted 2 lines
But regardless, there is no dependency on objdump that wasn't there before, are we in agreement on that?
Yes.
Quoted 15 lines
Neil
Quoted 12 lines
Example:
ERROR: symbolotx2_mbox_alloc_msg_rspis added in the DPDK_19.05 section, but is expected to be added in the EXPERIMENTAL section of the version mapQuoted 4 lines
So I'm unsure why you think checkpatches has a dependency.
Neil
Quoted 68 lines · Jerin Jacob Kollanukkaran
Quoted 63 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Thursday, June 6, 2019 10:26 PM
To: Jerin Jacob Kollanukkaran <[email protected]>
Cc: Bruce Richardson <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Subject: Re: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagQuoted 9 lines · Jerin Jacob Kollanukkaran
Quoted 9 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Thursday, June 6, 2019 8:57 PM
To: Jerin Jacob Kollanukkaran <[email protected]>
Cc: Bruce Richardson <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Subject: Re: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagOn Thu, Jun 06, 2019 at 03:14:42PM +0000, Jerin Jacob Kollanukkaran
Quoted 13 lines · wrote:
Quoted 13 lines
Quoted 3 lines
<snip as this is getting long>
I don't have any strong opinion on name prefix vs marking as
__rte_internal.Quoted 3 lines
Or combination of both. I am fine any approach.
I have only strong option on not to induce objdump dependency for
checkpatch.
Quoted 2 lines
For the reason mentioned in
http://mails.dpdk.org/archives/dev/2019-June/134160.html.
Quoted 1 line
Quoted 7 lines
Quoted 7 lines
Sorry, in my haste I didn't fully adress this in your previous email
I'm really uncertain what you mean by introducing a checkpatch dependency on objdump here. Theres nothing preventing you from running checkpatch before you build the library. The only thing checkpatch does in dpdk is scan the patches for sytle violations, and for changes in the map file for movement to and from the
EXPERIMENTAL section (i.e. no use of objdump).
Quoted 11 lines
Quoted 11 lines
My patch modifies
check-experimental-syms.sh(adding an objdump scan for INTERNAL symbols, and renaming the script tocheck-special-syms.shto be more meaningful). That script however, is not run by checkpatch, its run during compilation of the library to ensure that any symbol in a map file is also tagged with__rte_internalin the corresponding object). Theres no path from checkpatches tocheck-experimental-syms.shWhat I meant in my last comment was that any dependency on objdump in check-[experimental|special]-syms.sh already existed prior to this
patch.
Quoted 4 lines
I see. I thought your patches addressing issue related to
http://patches.dpdk.org/patch/53590/It does, it just does it in a different way than you do it.
Quoted 5 lines
Where
checkpatch.shcomplaints when we add new internal driver API with outrte_experimental? That's where all the discussion started. The reason why I was saying the API name prefix to mark as internal API is that checkpatch can detect that case.Quoted 4 lines
Ah, thats a fair point, with my approach we need to add some code to check-
symbol-change.shsuch that any symbols listed in an INTERNAL section were ignored. That would provide behavioral compatibility to what you were doing in your patch.
Quoted 7 lines
OK.
Quoted 2 lines
But regardless, there is no dependency on objdump that wasn't there before, are we in agreement on that?
Yes.
ok, let me revise this patch with that change in place. I'll also complete tagging of the affected internal functions so we have a better idea of what this change entails
Neil
Quoted 174 lines · Neil Horman
Quoted 140 lines · Jerin Jacob Kollanukkaran
Quoted 140 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Thursday, June 6, 2019 7:05 PM
To: Jerin Jacob Kollanukkaran <[email protected]>
Cc: Bruce Richardson <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Subject: Re: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagQuoted 9 lines · Jerin Jacob Kollanukkaran
Quoted 9 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Thursday, June 6, 2019 5:04 PM
To: Jerin Jacob Kollanukkaran <[email protected]>
Cc: Bruce Richardson <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Subject: Re: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagOn Thu, Jun 06, 2019 at 09:44:52AM +0000, Jerin Jacob Kollanukkaran
Quoted 42 lines · wrote:
Quoted 42 lines
Quoted 42 lines
Quoted 42 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Wednesday, June 5, 2019 11:41 PM
To: Bruce Richardson <[email protected]>
Cc: Jerin Jacob Kollanukkaran <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Subject: Re: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagQuoted 2 lines · Bruce Richardson
On Wed, Jun 05, 2019 at 04:24:09PM +0000, Jerin Jacob Kollanukkaran
Quoted 30 lines · wrote:
Quoted 30 lines
Quoted 30 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Sunday, May 26, 2019 12:14 AM
To: [email protected]
Cc: Neil Horman <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>; Bruce Richardson <[email protected]>; Thomas Monjalon <[email protected]>
Subject: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagHey-
Based on our recent conversations regarding the use of
symbols only meant for internal dpdk consumption (between dpdk libraries), this is an idea that I've come up with that I'd like to get some feedback onSummary:
1) We have symbols in the DPDK that are meant to be used between DPDK libraries, but not by applications linking to them
2) We would like to document those symbols in the code, so as to note them clearly as for being meant for internal use only
3) Linker symbol visibility is a very coarse grained tool, and so there is no good way in a single library to mark items as being meant for use only by other DPDK libraries, at least not without some extensive runtime checkingQuoted 15 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Proposal:
I'm proposing that we introduce the__rte_internaltag. From a coding standpoint it works a great deal like the__rte_experimentaltag in that it expempts the tagged symbol from ABI constraints (as the only users should be represented in the DPDK build environment). Additionally, the__rte_internalmacro resolves differently based on theQuoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
definition of the
BUILDING_RTE_SDKflag (working under the assumption that said flag should only ever be set if we are actually building DPDK libraries which will make use of internal calls). If theBUILDING_RTE_SDKflag is set__rte_internalresolves to__attribute__((section "text.internal)), placing it in a special text section which is then used to validate that the the symbol appears in the INTERNAL section of the corresponding library versionmap).
Quoted 48 lines
Quoted 48 lines
Quoted 24 lines
Quoted 24 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
If
BUILDING_RTE_SDKis not set, then__rte_internalresolves to__attribute__((error("..."))), which causes any caller of the tagged function to throw an error at compile time, indicating that the symbol is not available for external use.
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
This isn't a perfect solution, as applications can still hack around it of course,
Quoted 8 lines
Quoted 8 lines
I think, one way to, avoid, hack around could be to,
1) at config stage, create a random number for the build
2) introduceRTE_CALL_INTERNALmacro for calling internal function, compare the generated random number for allowing the calls to make within the library. i.e leverage the fact that external library would never know the random number generated for the DPDK buildand internal driver code does.
Quoted 6 lines
Quoted 1 line
Do we really need to care about this. If have some determined enough to hack around our limitations, then they surely know that they have an unsupported configuration. We just need to protect against inadvertent use of internals, IMHO.
Quoted 4 lines
Quoted 4 lines
I agree, I too had thought about doing some sort of internal runtime checking to match internal only symbols, such that they were only accessable by internally approved users, but it started to feel like a great
deal of overhead.
Quoted 6 lines
Quoted 6 lines
Its a good idea for a general mechanism I think, but I believe the value here is more to internally document which apis we want to mark as being for internal use only, and create a lightweight roadblock at build time to catch users inadvertently using them. Determined users will get around anything, and theres not much we can do to stop
them.
Quoted 9 lines
I agree too. IMHO, Simply having following items would be enough
1) Avoid exposing the internal function prototype through public header files
2) Add @internal to API documentation
3) Just decide the name space for internal API fortooling(i.e not start with rte_ or so)Using objdump scheme to detect internal functionsrequires the the library to build prior to run the checkpatch.
Quoted 1 line
Quoted 6 lines
Quoted 6 lines
No, I'm not comfortable with that approach, and I've stated why:
1) Not exposing the functions via header files is a fine start2) Adding internal documentation is also fine, but does nothing to correlate the code implementing those functions to the documentation. Its valuable to have a tag on a function identifying it as
internal only.
Quoted 8 lines
Quoted 7 lines
3) Using naming conventions to separate internal only from non-internal functions is a vague approach, requiring future developers to be cogniscent of the convention and make the appropriate naming choices. It also implicitly restricts the abliity for future developers to make naming changes in conflict with that convention
Quoted 6 lines
Quoted 3 lines
Quoted 2 lines
Enforcing the naming convention can be achieved through tooling as well.
Sure, but why enforce any function naming at all, when you don't have to.
May I ask, why to enforce
__rte_internal, when you don't have toBecause its more clear. Implicitly deciding that any function not prefixed with
rte_is internal only does nothing to prevent a developer from accidentally naming a function incorrectly, exporting it, and allowing a user to call it. We can move headers all you want, but we provide an ABI guarantee to end users, and developers should have a way to clearly record that without having to check the documentation for each function that an application developer wants to use.The long and the short of it for me is that I want a way for developers to opt their code into an internal only condition, not to just document it as such and hope its up to date. If they tag a function as
__rte_internalthen its clearly marked as internal only, they have checks to ensure that its in the INTERNAL section of the version map, and should that header somehow get externally exported (seerte_mempool_check_cookiesfor an example of how thats happened), users are prevented from using them at build time, rather than having to ask questions on the list, or read documentation after an error to find out "oops, shouldn't have done that".I think you'll find that going through all the header files, and bifurcating them to public and private headers is a much larger undertaking than just tagging those functions accordingly. a quick scan of all our header file for the @internal tag shows about 260 instances of such functions, almost all of which are published to applications. All of those functions would have to be moved to private headers, and their requisite C files would need to be updated to include the new header. with the use of
__rte_internal, we just have tag the functions as such, which can be handled with a cocinelle or awk script.
Quoted 1 line
Neil
This is good, I like alot about this, especially the build system complaining loudly when the developer does something they shouldn't - I think anything that we can add that promotes good behaviors is to be 100% welcomed.
I also agree with the points made elsewhere that this is essentially trying to solve a header problem, the mixing of public and private symbols in what are public headers, with __rte_internal. Adding __rte_internal would essentially ratify that behavior, whereas I would argue that something inherently private, should never see the light of day in a public header.
I completely get that it may be more work, however for me it is better way to fix this problem. It would also add completely clarity, all the extra hassle around does it have the rte_ prefix goes away - if it is in a "public header" it is part of the ABI/API, end of discussion.
Finally, not opposed to also asking folks putting symbols in the private header to mark those symbols with __rte_internal.
Quoted 63 lines
Quoted 62 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
4) Adding a tag like
__rte_internalcreates an interlock whereby, not only are internal functions excused from ABI constraints, but forces developers to intentionally mark their internal functions as being internal in the code, which is beneficial to clarlity of understandingQuoted 5 lines
during the development process.
Quoted 3 lines
No issues in adding
__rte_internal. But, I am against current implementaion, Ie. adding objdump dependencyThat dependency already exists for the
__rte_externalflagSorry, I could not see the dependency.
[master][dpdk.org] $ grep -ri "objdump" devtools/ [master][dpdk.org] $ grep -ri "objdump" usertools/ [master][dpdk.org] $ grep -ri "
__rte_external" *Quoted 6 lines
Quoted 4 lines
to checkpatch i.e developer has to build the library first so that checkpatch can can know, Is it belongs to internal section or not?
What developer is running checkpatch/posting patches without first building their changes?
# it is not developer, The CI/CD tools can quicky check the sanity of patches before the build itself. Why to add unnecessary dependency? # If some PMD is not building if the requirements are not meet(say AES NI PMD for crypto) then how do take care of the dependency.Quoted 28 lines
Quoted 28 lines
Quoted 27 lines
5) Adding a tag like
__rte_internalis explicit, and allows developers to use a single header file instead of multiple header files if they so chooseWe went through this with experimental symbols as well[1], and it just makes more sense to me to clearly document in the code what constitutes an internal symbol rather than relying on naming conventions and hoping that developers read the documentation before exporting a symbol publically.
[1] https://mails.dpdk.org/archives/dev/2017-December/083828.html
Quoted 12 lines
Quoted 11 lines
If we really wanted to go down that road, we could use a mechainsm simmilar to the
EXPORT_SYMBOL/EXPORT_SYMBOL_GPLinfrastructure that the kernel uses, but that would required building our own custom linker script, which seems like overkill here.Best
NeilQuoted 2 lines
/Bruce
Quoted 195 lines · Ray Kinsella
Quoted 174 lines · Neil Horman
Quoted 140 lines · Jerin Jacob Kollanukkaran
Quoted 140 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Thursday, June 6, 2019 7:05 PM
To: Jerin Jacob Kollanukkaran <[email protected]>
Cc: Bruce Richardson <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Subject: Re: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagQuoted 9 lines · Jerin Jacob Kollanukkaran
Quoted 9 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Thursday, June 6, 2019 5:04 PM
To: Jerin Jacob Kollanukkaran <[email protected]>
Cc: Bruce Richardson <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Subject: Re: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagOn Thu, Jun 06, 2019 at 09:44:52AM +0000, Jerin Jacob Kollanukkaran
Quoted 42 lines · wrote:
Quoted 42 lines
Quoted 42 lines
Quoted 42 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Wednesday, June 5, 2019 11:41 PM
To: Bruce Richardson <[email protected]>
Cc: Jerin Jacob Kollanukkaran <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Subject: Re: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagQuoted 2 lines · Bruce Richardson
On Wed, Jun 05, 2019 at 04:24:09PM +0000, Jerin Jacob Kollanukkaran
Quoted 30 lines · wrote:
Quoted 30 lines
Quoted 30 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Sunday, May 26, 2019 12:14 AM
To: [email protected]
Cc: Neil Horman <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>; Bruce Richardson <[email protected]>; Thomas Monjalon <[email protected]>
Subject: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagHey-
Based on our recent conversations regarding the use of
symbols only meant for internal dpdk consumption (between dpdk libraries), this is an idea that I've come up with that I'd like to get some feedback onSummary:
1) We have symbols in the DPDK that are meant to be used between DPDK libraries, but not by applications linking to them
2) We would like to document those symbols in the code, so as to note them clearly as for being meant for internal use only
3) Linker symbol visibility is a very coarse grained tool, and so there is no good way in a single library to mark items as being meant for use only by other DPDK libraries, at least not without some extensive runtime checkingQuoted 15 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Proposal:
I'm proposing that we introduce the__rte_internaltag. From a coding standpoint it works a great deal like the__rte_experimentaltag in that it expempts the tagged symbol from ABI constraints (as the only users should be represented in the DPDK build environment). Additionally, the__rte_internalmacro resolves differently based on theQuoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
definition of the
BUILDING_RTE_SDKflag (working under the assumption that said flag should only ever be set if we are actually building DPDK libraries which will make use of internal calls). If theBUILDING_RTE_SDKflag is set__rte_internalresolves to__attribute__((section "text.internal)), placing it in a special text section which is then used to validate that the the symbol appears in the INTERNAL section of the corresponding library versionmap).
Quoted 48 lines
Quoted 48 lines
Quoted 24 lines
Quoted 24 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
If
BUILDING_RTE_SDKis not set, then__rte_internalresolves to__attribute__((error("..."))), which causes any caller of the tagged function to throw an error at compile time, indicating that the symbol is not available for external use.
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
This isn't a perfect solution, as applications can still hack around it of course,
Quoted 8 lines
Quoted 8 lines
I think, one way to, avoid, hack around could be to,
1) at config stage, create a random number for the build
2) introduceRTE_CALL_INTERNALmacro for calling internal function, compare the generated random number for allowing the calls to make within the library. i.e leverage the fact that external library would never know the random number generated for the DPDK buildand internal driver code does.
Quoted 6 lines
Quoted 1 line
Do we really need to care about this. If have some determined enough to hack around our limitations, then they surely know that they have an unsupported configuration. We just need to protect against inadvertent use of internals, IMHO.
Quoted 4 lines
Quoted 4 lines
I agree, I too had thought about doing some sort of internal runtime checking to match internal only symbols, such that they were only accessable by internally approved users, but it started to feel like a great
deal of overhead.
Quoted 6 lines
Quoted 6 lines
Its a good idea for a general mechanism I think, but I believe the value here is more to internally document which apis we want to mark as being for internal use only, and create a lightweight roadblock at build time to catch users inadvertently using them. Determined users will get around anything, and theres not much we can do to stop
them.
Quoted 9 lines
I agree too. IMHO, Simply having following items would be enough
1) Avoid exposing the internal function prototype through public header files
2) Add @internal to API documentation
3) Just decide the name space for internal API fortooling(i.e not start with rte_ or so)Using objdump scheme to detect internal functionsrequires the the library to build prior to run the checkpatch.
Quoted 1 line
Quoted 6 lines
Quoted 6 lines
No, I'm not comfortable with that approach, and I've stated why:
1) Not exposing the functions via header files is a fine start2) Adding internal documentation is also fine, but does nothing to correlate the code implementing those functions to the documentation. Its valuable to have a tag on a function identifying it as
internal only.
Quoted 8 lines
Quoted 7 lines
3) Using naming conventions to separate internal only from non-internal functions is a vague approach, requiring future developers to be cogniscent of the convention and make the appropriate naming choices. It also implicitly restricts the abliity for future developers to make naming changes in conflict with that convention
Quoted 6 lines
Quoted 3 lines
Quoted 2 lines
Enforcing the naming convention can be achieved through tooling as well.
Sure, but why enforce any function naming at all, when you don't have to.
May I ask, why to enforce
__rte_internal, when you don't have toBecause its more clear. Implicitly deciding that any function not prefixed with
rte_is internal only does nothing to prevent a developer from accidentally naming a function incorrectly, exporting it, and allowing a user to call it. We can move headers all you want, but we provide an ABI guarantee to end users, and developers should have a way to clearly record that without having to check the documentation for each function that an application developer wants to use.The long and the short of it for me is that I want a way for developers to opt their code into an internal only condition, not to just document it as such and hope its up to date. If they tag a function as
__rte_internalthen its clearly marked as internal only, they have checks to ensure that its in the INTERNAL section of the version map, and should that header somehow get externally exported (seerte_mempool_check_cookiesfor an example of how thats happened), users are prevented from using them at build time, rather than having to ask questions on the list, or read documentation after an error to find out "oops, shouldn't have done that".I think you'll find that going through all the header files, and bifurcating them to public and private headers is a much larger undertaking than just tagging those functions accordingly. a quick scan of all our header file for the @internal tag shows about 260 instances of such functions, almost all of which are published to applications. All of those functions would have to be moved to private headers, and their requisite C files would need to be updated to include the new header. with the use of
__rte_internal, we just have tag the functions as such, which can be handled with a cocinelle or awk script.Quoted 1 line
Neil
This is good, I like alot about this, especially the build system complaining loudly when the developer does something they shouldn't - I think anything that we can add that promotes good behaviors is to be 100% welcomed.
I also agree with the points made elsewhere that this is essentially trying to solve a header problem, the mixing of public and private symbols in what are public headers, with
__rte_internal. Adding__rte_internalwould essentially ratify that behavior, whereas I would argue that something inherently private, should never see the light of day in a public header.
Quoted 7 lines
I completely get that it may be more work, however for me it is better way to fix this problem. It would also add completely clarity, all the extra hassle around does it have the
rte_prefix goes away - if it is in a "public header" it is part of the ABI/API, end of discussion.Finally, not opposed to also asking folks putting symbols in the private header to mark those symbols with
__rte_internal.
+1 I think we need to do both split headers and __rte_internal for extra measure. I am still concerned we are adding more work for the developer, if not then at least we split the headers.
Quoted 55 lines
Quoted 55 lines
Quoted 55 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
4) Adding a tag like
__rte_internalcreates an interlock whereby, not only are internal functions excused from ABI constraints, but forces developers to intentionally mark their internal functions as being internal in the code, which is beneficial to clarlity of understandingQuoted 5 lines
during the development process.
Quoted 3 lines
No issues in adding
__rte_internal. But, I am against current implementaion, Ie. adding objdump dependencyThat dependency already exists for the
__rte_externalflagSorry, I could not see the dependency.
[master][dpdk.org] $ grep -ri "objdump" devtools/ [master][dpdk.org] $ grep -ri "objdump" usertools/ [master][dpdk.org] $ grep -ri "
__rte_external" *Quoted 6 lines
Quoted 4 lines
to checkpatch i.e developer has to build the library first so that checkpatch can can know, Is it belongs to internal section or not?
What developer is running checkpatch/posting patches without first building their changes?
# it is not developer, The CI/CD tools can quicky check the sanity of patches before the build itself. Why to add unnecessary dependency? # If some PMD is not building if the requirements are not meet(say AES NI PMD for crypto) then how do take care of the dependency.Quoted 21 lines
Quoted 21 lines
Quoted 21 lines
5) Adding a tag like
__rte_internalis explicit, and allows developers to use a single header file instead of multiple header files if they so chooseWe went through this with experimental symbols as well[1], and it just makes more sense to me to clearly document in the code what constitutes an internal symbol rather than relying on naming conventions and hoping that developers read the documentation before exporting a symbol publically.
[1] https://mails.dpdk.org/archives/dev/2017-December/083828.html
Quoted 6 lines
Quoted 6 lines
If we really wanted to go down that road, we could use a mechainsm simmilar to the
EXPORT_SYMBOL/EXPORT_SYMBOL_GPLinfrastructure that the kernel uses, but that would required building our own custom linker script, which seems like overkill here.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Best
NeilQuoted 1 line
/Bruce
Regards,
Keith
Quoted 200 lines · Keith
Quoted 200 lines · Ray Kinsella
Quoted 174 lines · Neil Horman
Quoted 140 lines · Jerin Jacob Kollanukkaran
Quoted 140 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Thursday, June 6, 2019 7:05 PM
To: Jerin Jacob Kollanukkaran <[email protected]>
Cc: Bruce Richardson <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Subject: Re: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagQuoted 9 lines · Jerin Jacob Kollanukkaran
Quoted 9 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Thursday, June 6, 2019 5:04 PM
To: Jerin Jacob Kollanukkaran <[email protected]>
Cc: Bruce Richardson <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Subject: Re: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagOn Thu, Jun 06, 2019 at 09:44:52AM +0000, Jerin Jacob Kollanukkaran
Quoted 42 lines · wrote:
Quoted 42 lines
Quoted 42 lines
Quoted 42 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Wednesday, June 5, 2019 11:41 PM
To: Bruce Richardson <[email protected]>
Cc: Jerin Jacob Kollanukkaran <[email protected]>; [email protected]; Thomas Monjalon <[email protected]>
Subject: Re: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagQuoted 2 lines · Bruce Richardson
On Wed, Jun 05, 2019 at 04:24:09PM +0000, Jerin Jacob Kollanukkaran
Quoted 30 lines · wrote:
Quoted 30 lines
Quoted 30 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Sunday, May 26, 2019 12:14 AM
To: [email protected]
Cc: Neil Horman <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>; Bruce Richardson <[email protected]>; Thomas Monjalon <[email protected]>
Subject: [EXT] [RFC PATCH 0/2] introduce__rte_internaltagHey-
Based on our recent conversations regarding the use of
symbols only meant for internal dpdk consumption (between dpdk libraries), this is an idea that I've come up with that I'd like to get some feedback onSummary:
1) We have symbols in the DPDK that are meant to be used between DPDK libraries, but not by applications linking to them
2) We would like to document those symbols in the code, so as to note them clearly as for being meant for internal use only
3) Linker symbol visibility is a very coarse grained tool, and so there is no good way in a single library to mark items as being meant for use only by other DPDK libraries, at least not without some extensive runtime checkingQuoted 15 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Proposal:
I'm proposing that we introduce the__rte_internaltag. From a coding standpoint it works a great deal like the__rte_experimentaltag in that it expempts the tagged symbol from ABI constraints (as the only users should be represented in the DPDK build environment). Additionally, the__rte_internalmacro resolves differently based on theQuoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
definition of the
BUILDING_RTE_SDKflag (working under the assumption that said flag should only ever be set if we are actually building DPDK libraries which will make use of internal calls). If theBUILDING_RTE_SDKflag is set__rte_internalresolves to__attribute__((section "text.internal)), placing it in a special text section which is then used to validate that the the symbol appears in the INTERNAL section of the corresponding library versionmap).
Quoted 48 lines
Quoted 48 lines
Quoted 24 lines
Quoted 24 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
If
BUILDING_RTE_SDKis not set, then__rte_internalresolves to__attribute__((error("..."))), which causes any caller of the tagged function to throw an error at compile time, indicating that the symbol is not available for external use.
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
This isn't a perfect solution, as applications can still hack around it of course,
Quoted 8 lines
Quoted 8 lines
I think, one way to, avoid, hack around could be to,
1) at config stage, create a random number for the build
2) introduceRTE_CALL_INTERNALmacro for calling internal function, compare the generated random number for allowing the calls to make within the library. i.e leverage the fact that external library would never know the random number generated for the DPDK buildand internal driver code does.
Quoted 6 lines
Quoted 1 line
Do we really need to care about this. If have some determined enough to hack around our limitations, then they surely know that they have an unsupported configuration. We just need to protect against inadvertent use of internals, IMHO.
Quoted 4 lines
Quoted 4 lines
I agree, I too had thought about doing some sort of internal runtime checking to match internal only symbols, such that they were only accessable by internally approved users, but it started to feel like a great
deal of overhead.
Quoted 6 lines
Quoted 6 lines
Its a good idea for a general mechanism I think, but I believe the value here is more to internally document which apis we want to mark as being for internal use only, and create a lightweight roadblock at build time to catch users inadvertently using them. Determined users will get around anything, and theres not much we can do to stop
them.
Quoted 9 lines
I agree too. IMHO, Simply having following items would be enough
1) Avoid exposing the internal function prototype through public header files
2) Add @internal to API documentation
3) Just decide the name space for internal API fortooling(i.e not start with rte_ or so)Using objdump scheme to detect internal functionsrequires the the library to build prior to run the checkpatch.
Quoted 1 line
Quoted 6 lines
Quoted 6 lines
No, I'm not comfortable with that approach, and I've stated why:
1) Not exposing the functions via header files is a fine start2) Adding internal documentation is also fine, but does nothing to correlate the code implementing those functions to the documentation. Its valuable to have a tag on a function identifying it as
internal only.
Quoted 8 lines
Quoted 7 lines
3) Using naming conventions to separate internal only from non-internal functions is a vague approach, requiring future developers to be cogniscent of the convention and make the appropriate naming choices. It also implicitly restricts the abliity for future developers to make naming changes in conflict with that convention
Quoted 6 lines
Quoted 3 lines
Quoted 2 lines
Enforcing the naming convention can be achieved through tooling as well.
Sure, but why enforce any function naming at all, when you don't have to.
May I ask, why to enforce
__rte_internal, when you don't have toBecause its more clear. Implicitly deciding that any function not prefixed with
rte_is internal only does nothing to prevent a developer from accidentally naming a function incorrectly, exporting it, and allowing a user to call it. We can move headers all you want, but we provide an ABI guarantee to end users, and developers should have a way to clearly record that without having to check the documentation for each function that an application developer wants to use.The long and the short of it for me is that I want a way for developers to opt their code into an internal only condition, not to just document it as such and hope its up to date. If they tag a function as
__rte_internalthen its clearly marked as internal only, they have checks to ensure that its in the INTERNAL section of the version map, and should that header somehow get externally exported (seerte_mempool_check_cookiesfor an example of how thats happened), users are prevented from using them at build time, rather than having to ask questions on the list, or read documentation after an error to find out "oops, shouldn't have done that".I think you'll find that going through all the header files, and bifurcating them to public and private headers is a much larger undertaking than just tagging those functions accordingly. a quick scan of all our header file for the @internal tag shows about 260 instances of such functions, almost all of which are published to applications. All of those functions would have to be moved to private headers, and their requisite C files would need to be updated to include the new header. with the use of
__rte_internal, we just have tag the functions as such, which can be handled with a cocinelle or awk script.Quoted 1 line
Neil
This is good, I like alot about this, especially the build system complaining loudly when the developer does something they shouldn't - I think anything that we can add that promotes good behaviors is to be 100% welcomed.
I also agree with the points made elsewhere that this is essentially trying to solve a header problem, the mixing of public and private symbols in what are public headers, with
__rte_internal. Adding__rte_internalwould essentially ratify that behavior, whereas I would argue that something inherently private, should never see the light of day in a public header.I completely get that it may be more work, however for me it is better way to fix this problem. It would also add completely clarity, all the extra hassle around does it have the
rte_prefix goes away - if it is in a "public header" it is part of the ABI/API, end of discussion.
Quoted 4 lines
Quoted 2 lines
Finally, not opposed to also asking folks putting symbols in the private header to mark those symbols with
__rte_internal.+1 I think we need to do both split headers and
__rte_internalfor extra measure. I am still concerned we are adding more work for the developer, if not then at least we split the headers.
I think this makes sense. Perhaps we could add a check in checkpatch to warn a user if the __rte_internal tag is present in a header that has been copied to the builds include directory (i.e. was specified as SYMLINK-$(VAR) in the makefile). Would that help?
Neil
cover dpdk: introduce __rte_internal tag
In reply to Neil HormanHey-
Based on our recent conversations regarding the use of symbols only
meant for internal dpdk consumption (between dpdk libraries), this is an idea that I've come up with that I'd like to get some feedback on
Summary:
1) We have symbols in the DPDK that are meant to be used between DPDK libraries, but not by applications linking to them
2) We would like to document those symbols in the code, so as to note them clearly as for being meant for internal use only
3) Linker symbol visibility is a very coarse grained tool, and so there is no good way in a single library to mark items as being meant for use only by other DPDK libraries, at least not without some extensive runtime checking
Proposal:
I'm proposing that we introduce the __rte_internal tag. From a coding standpoint it works a great deal like the __rte_experimental tag in that it expempts the tagged symbol from ABI constraints (as the only users should be represented in the DPDK build environment). Additionally, the __rte_internal macro resolves differently based on the definition of the BUILDING_RTE_SDK flag (working under the assumption that said flag should only ever be set if we are actually building DPDK libraries which will make use of internal calls). If the BUILDING_RTE_SDK flag is set __rte_internal resolves to __attribute__((section "text.internal)), placing it in a special text section which is then used to validate that the the symbol appears in the INTERNAL section of the corresponding library version map). If BUILDING_RTE_SDK is not set, then __rte_internal resolves to __attribute__((error("..."))), which causes any caller of the tagged function to throw an error at compile time, indicating that the symbol is not available for external use.
This isn't a perfect solution, as applications can still hack around it of course, but I think it hits some of the high points, restricting symbol access for any library that prototypes its public and private symbols in the same header file, excluding the internal symbols from ABI constraints, and clearly documenting those symbols which we wish to limit to internal usage.
PATCH 1/9 Add __rte_internal tag for functions and version target
This tag is meant to be used on function prototypes to identify functions that are only meant to be used by internal DPDK libraries (i.e. libraries that are built while building the SDK itself, as identified by the defining of the BUILDING_RTE_SDK macro). When that flag is not set, it will resolve to an error function attribute, causing build breakage for any compilation unit attempting to build it
Validate the use of this tag in much the same way we validate __rte_experimental. By adding an INTERNAL version to library map files, we can exempt internal-only functions from ABI checking, and handle them to ensure that symbols we wish to only be for internal use between dpdk libraries are properly tagged with __rte_experimental
Note this patch updates the check-experimental-syms.sh script, which normally only check the EXPERIMENTAL section to also check the INTERNAL section now. As such its been renamed to the now more appropriate check-special-syms.sh
--- buildtools/check-experimental-syms.sh | 24 +++++++++++++++++++++- lib/librte_eal/common/include/rte_compat.h | 12 +++++++++++ mk/internal/rte.compile-pre.mk | 6 +++--- mk/target/generic/rte.vars.mk | 2 +- 4 files changed, 39 insertions(+), 5 deletions(-)
buildtools/check-experimental-syms.sh +23 -1
@@ -31,10 +31,32 @@ do cat >&2 <<- END_OF_MESSAGE $SYM is not flagged as experimental but is listed in version map - Please add __rte_experimental to the definition of $SYM + Please add __rte_experimental to the definition/prototype of $SYM END_OF_MESSAGE exit 1 fi done + +for i in `awk 'BEGIN {found=0} + /.*INTERNAL.*/ {found=1} + /.*}.*;/ {found=0} + /.*;/ {if (found == 1) print $1}' $MAPFILE` +do + SYM=`echo $i | sed -e"s/;//"` + objdump -t $OBJFILE | grep -q "\.text.*$SYM$" + IN_TEXT=$? + objdump -t $OBJFILE | grep -q "\.text\.internal.*$SYM$" + IN_EXP=$? + if [ $IN_TEXT -eq 0 -a $IN_EXP -ne 0 ] + then + cat >&2 <<- END_OF_MESSAGE + $SYM is not flagged as internal + but is listed in version map + Please add __rte_internal to the definition/prototype of $SYM + END_OF_MESSAGE + exit 1 + fi +done + exit 0
lib/librte_eal/common/include/rte_compat.h +12 -0
@@ -89,4 +89,16 @@ __attribute__((section(".text.experimental")))#endif +/* +* __rte_internal tags mark functions as internal only, If specified in public +* header files, this tag will resolve to an error directive, preventing +* external applications from attempting to make calls to functions not meant +* for consumption outside the dpdk library +*/ +#ifdef BUILDING_RTE_SDK +#define __rte_internal __attribute__((section(".text.internal"))) +#else +#define __rte_internal __attribute__((error("This function cannot be used outside of the core DPDK library"), \ + section(".text.internal"))) +#endif #endif /* _RTE_COMPAT_H_ */
mk/internal/rte.compile-pre.mk +3 -3
@@ -56,8 +56,8 @@ C_TO_O = $(CC) -Wp,-MD,$(call obj2dep,$(@)).tmp $(CPPFLAGS) $(CFLAGS) \ C_TO_O_STR = $(subst ','\'',$(C_TO_O)) #'# fix syntax highlight C_TO_O_DISP = $(if $(V),"$(C_TO_O_STR)"," CC $(@)") endif -EXPERIMENTAL_CHECK = $(RTE_SDK)/buildtools/check-experimental-syms.sh -CHECK_EXPERIMENTAL = $(EXPERIMENTAL_CHECK) $(SRCDIR)/$(EXPORT_MAP) $@ +SPECIAL_SYM_CHECK = $(RTE_SDK)/buildtools/check-special-syms.sh +CHECK_SPECIAL_SYMS = $(SPECIAL_SYM_CHECK) $(SRCDIR)/$(EXPORT_MAP) $@ PMDINFO_GEN = $(RTE_SDK_BIN)/app/dpdk-pmdinfogen $@ [email protected] PMDINFO_CC = $(CC) $(CPPFLAGS) $(CFLAGS) $(EXTRA_CFLAGS) -c -o [email protected] [email protected]
@@ -75,7 +75,7 @@ C_TO_O_DO = @set -e; \ echo $(C_TO_O_DISP); \ $(C_TO_O) && \ $(PMDINFO_TO_O) && \ - $(CHECK_EXPERIMENTAL) && \ + $(CHECK_SPECIAL_SYMS) && \ echo $(C_TO_O_CMD) > $(call obj2cmd,$(@)) && \ sed 's,'$@':,dep_'$@' =,' $(call obj2dep,$(@)).tmp > $(call obj2dep,$(@)) && \ rm -f $(call obj2dep,$(@)).tmp
mk/target/generic/rte.vars.mk +1 -1
@@ -96,7 +96,7 @@ LDFLAGS += -L$(RTE_OUTPUT)/lib # defined. ifeq ($(BUILDING_RTE_SDK),1) # building sdk -CFLAGS += -include $(RTE_OUTPUT)/include/rte_config.h +CFLAGS += -include $(RTE_OUTPUT)/include/rte_config.h -DBUILDING_RTE_SDK else # if we are building an external application, include SDK's lib and # includes too
Signature · 1 line
2.20.1
PATCH 2/9 Exempt INTERNAL symbols from checking
In reply to Neil HormanNo need to restrict the ABI on symbols that are only used by core libraries
--- devtools/check-symbol-change.sh | 7 +++++++ 1 file changed, 7 insertions(+)
devtools/check-symbol-change.sh +7 -0
@@ -93,6 +93,13 @@ check_for_rule_violations() if [ "$ar" = "add" ] then + if [ "$secname" = "INTERNAL" ] + then + # these are absolved from any further checking + echo "Skipping symbol $symname in INTERNAL" + continue + fi + if [ "$secname" = "unknown" ] then # Just inform the user of this occurrence, but
Signature · 1 line
2.20.1
PATCH 3/9 mark dpaa driver internal-only symbols with __rte_internal
In reply to Neil Hormanmake use of the new __rte_internal tag to specify symbols that should only be used by dpdk provided libraries (as specified by the BUILDING_RTE_SDK cflag
---
...rimental-syms.sh => check-special-syms.sh} | 0
drivers/bus/dpaa/include/fsl_bman.h | 12 ++--
drivers/bus/dpaa/include/fsl_fman.h | 50 ++++++++--------
drivers/bus/dpaa/include/fsl_qman.h | 60 +++++++++----------
drivers/bus/dpaa/include/fsl_usd.h | 12 ++--
drivers/bus/dpaa/include/netcfg.h | 4 +-
drivers/bus/dpaa/include/of.h | 6 +-
drivers/bus/dpaa/rte_bus_dpaa_version.map | 47 ++++++---------
drivers/net/dpaa/dpaa_ethdev.c | 4 +-
drivers/net/dpaa/dpaa_ethdev.h | 4 +-
drivers/net/dpaa/rte_pmd_dpaa_version.map | 8 ++-
11 files changed, 99 insertions(+), 108 deletions(-)
rename buildtools/{check-experimental-syms.sh => check-special-syms.sh} (100%)buildtools/check-special-syms.sh renamed from buildtools/check-experimental-syms.sh +0 -0
drivers/bus/dpaa/include/fsl_bman.h +6 -6
@@ -264,13 +264,13 @@ int bman_shutdown_pool(u32 bpid);* the structure provided by the caller can be released or reused after the * function returns. */ -struct bman_pool *bman_new_pool (const struct bman_pool_params *params); +struct bman_pool __rte_internal *bman_new_pool(const struct bman_pool_params *params);/** * bman_free_pool - Deallocates a Buffer Pool object * @pool: the pool object to release */ -void bman_free_pool (struct bman_pool *pool); +void __rte_internal bman_free_pool(struct bman_pool *pool);/** * bman_get_params - Returns a pool object's parameters.
@@ -279,7 +279,7 @@ void bman_free_pool(struct bman_pool *pool);* The returned pointer refers to state within the pool object so must not be * modified and can no longer be read once the pool object is destroyed. */ -const struct bman_pool_params *bman_get_params (const struct bman_pool *pool); +const struct bman_pool_params __rte_internal *bman_get_params(const struct bman_pool *pool);/** * bman_release - Release buffer(s) to the buffer pool
@@ -289,7 +289,7 @@ const struct bman_pool_params *bman_get_params(const struct bman_pool *pool);* @flags: bit-mask of BMAN_RELEASE_FLAG_*** options * */ -int bman_release (struct bman_pool *pool,const struct bm_buffer *bufs,u8 num, +int __rte_internal bman_release(struct bman_pool *pool, const struct bm_buffer *bufs, u8 num,u32 flags);/**
@@ -302,7 +302,7 @@ int bman_release(struct bman_pool *pool, const struct bm_buffer *bufs, u8 num,* The return value will be the number of buffers obtained from the pool, or a * negative error code if a h/w error or pool starvation was encountered. */ -int bman_acquire (struct bman_pool *pool,struct bm_buffer *bufs,u8 num, +int __rte_internal bman_acquire(struct bman_pool *pool, struct bm_buffer *bufs, u8 num,u32 flags);/**
@@ -317,7 +317,7 @@ int bman_query_pools(struct bm_pool_state *state);* * Return the number of the free buffers */ -u32 bman_query_free_buffers (struct bman_pool *pool); +u32 __rte_internal bman_query_free_buffers(struct bman_pool *pool);/** * bman_update_pool_thresholds - Change the buffer pool's depletion thresholds
drivers/bus/dpaa/include/fsl_fman.h +25 -25
113 lines not on this page · raw message
drivers/bus/dpaa/include/fsl_qman.h +30 -30
235 lines not on this page · raw message
drivers/bus/dpaa/include/fsl_usd.h +6 -6
@@ -56,7 +56,7 @@ int bman_allocate_raw_portal(struct dpaa_raw_portal *portal);int bman_free_raw_portal (struct dpaa_raw_portal *portal);/* Obtain thread-local UIO file-descriptors */ -int qman_thread_fd (void ); +int __rte_internal qman_thread_fd(void);int bman_thread_fd (void );/* Post-process interrupts. NB, the kernel IRQ handler disables the interrupt
@@ -64,14 +64,14 @@ int bman_thread_fd(void);* processing is complete. As such, it is essential to call this before going * into another blocking read/select/poll. */ -void qman_thread_irq (void ); -void bman_thread_irq (void ); +void __rte_internal qman_thread_irq(void); +void __rte_internal bman_thread_irq(void); -void qman_clear_irq (void ); +void __rte_internal qman_clear_irq(void);/* Global setup */ -int qman_global_init (void ); -int bman_global_init (void ); +int __rte_internal qman_global_init(void); +int __rte_internal bman_global_init(void);/* Direct portal create and destroy */ struct qman_portal *fsl_qman_portal_create (void );
drivers/bus/dpaa/include/netcfg.h +2 -2
@@ -46,12 +46,12 @@ struct netcfg_interface {* cfg_file: FMC config XML file * Returns the configuration information in newly allocated memory. */ -struct netcfg_info *netcfg_acquire (void ); +struct netcfg_info __rte_internal *netcfg_acquire(void);/* cfg_ptr: configuration information pointer. * Frees the resources allocated by the configuration layer. */ -void netcfg_release (struct netcfg_info *cfg_ptr); +void __rte_internal netcfg_release(struct netcfg_info *cfg_ptr);#ifdef RTE_LIBRTE_DPAA_DEBUG_DRIVER /* cfg_ptr: configuration information pointer.
drivers/bus/dpaa/include/of.h +3 -3
@@ -87,7 +87,7 @@ struct dt_file {uint64_t buf[OF_FILE_BUF_MAX >>3 ]; }; -const struct device_node *of_find_compatible_node ( +const __rte_internal struct device_node *of_find_compatible_node(const struct device_node *from,const char *type__always_unused ,const char *compatible)
@@ -98,7 +98,7 @@ const struct device_node *of_find_compatible_node( dev_node !=NULL ; \ dev_node = of_find_compatible_node(dev_node, type, compatible)) -const void *of_get_property (const struct device_node *from,const char *name, +const __rte_internal void *of_get_property(const struct device_node *from, const char *name,size_t *lenp) __attribute__((nonnull (2 )));bool of_device_is_available (const struct device_node *dev_node);
@@ -109,7 +109,7 @@ const struct device_node *of_get_parent(const struct device_node *dev_node);const struct device_node *of_get_next_child (const struct device_node *dev_node,const struct device_node *prev); -const void *of_get_mac_address (const struct device_node *np); +const void __rte_internal *of_get_mac_address(const struct device_node *np);#define for_each_child_node (parent, child) \ for (child = of_get_next_child(parent, NULL); child != NULL; \
drivers/bus/dpaa/rte_bus_dpaa_version.map +18 -29
75 lines not on this page · raw message
drivers/net/dpaa/dpaa_ethdev.c +2 -2
@@ -694,7 +694,7 @@ int dpaa_eth_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, }int -dpaa_eth_eventq_attach (const struct rte_eth_dev *dev, +__rte_internal dpaa_eth_eventq_attach(const struct rte_eth_dev *dev,int eth_rx_queue_id,u16 ch_id,const struct rte_event_eth_rx_adapter_queue_conf *queue_conf)
@@ -758,7 +758,7 @@ dpaa_eth_eventq_attach(const struct rte_eth_dev *dev, }int -dpaa_eth_eventq_detach (const struct rte_eth_dev *dev, +__rte_internal dpaa_eth_eventq_detach(const struct rte_eth_dev *dev,int eth_rx_queue_id) {struct qm_mcc_initfq opts;
drivers/net/dpaa/dpaa_ethdev.h +2 -2
@@ -166,13 +166,13 @@ struct dpaa_if_stats { };int -dpaa_eth_eventq_attach (const struct rte_eth_dev *dev, +__rte_internal dpaa_eth_eventq_attach(const struct rte_eth_dev *dev,int eth_rx_queue_id,u16 ch_id,const struct rte_event_eth_rx_adapter_queue_conf *queue_conf);int -dpaa_eth_eventq_detach (const struct rte_eth_dev *dev, +__rte_internal dpaa_eth_eventq_detach(const struct rte_eth_dev *dev,int eth_rx_queue_id);enum qman_cb_dqrr_result
drivers/net/dpaa/rte_pmd_dpaa_version.map +5 -3
@@ -5,8 +5,10 @@ DPDK_17.11 { DPDK_18.08 { global: - - dpaa_eth_eventq_attach; - dpaa_eth_eventq_detach; rte_pmd_dpaa_set_tx_loopback; } DPDK_17.11; + +INTERNAL { + dpaa_eth_eventq_attach; + dpaa_eth_eventq_detach; +};
Signature · 1 line
2.20.1
PATCH 4/9 fslmc: identify internal only functions and tag them as __rte_internal
In reply to Neil HormanIdentify functions in fslmc bus driver which are internal (based on their not having an rte_ prefix) and tag them with __rte_internal
--- drivers/bus/fslmc/fslmc_bus.c | 2 +- drivers/bus/fslmc/mc/dpbp.c | 12 +- drivers/bus/fslmc/mc/dpci.c | 6 +- drivers/bus/fslmc/mc/dpcon.c | 4 +- drivers/bus/fslmc/mc/dpdmai.c | 16 +- drivers/bus/fslmc/mc/dpio.c | 18 +- drivers/bus/fslmc/mc/dpmng.c | 4 +- drivers/bus/fslmc/mc/fsl_dpbp.h | 12 +- drivers/bus/fslmc/mc/fsl_dpci.h | 6 +- drivers/bus/fslmc/mc/fsl_dpcon.h | 4 +- drivers/bus/fslmc/mc/fsl_dpdmai.h | 16 +- drivers/bus/fslmc/mc/fsl_dpio.h | 18 +- drivers/bus/fslmc/mc/fsl_dpmng.h | 4 +- drivers/bus/fslmc/mc/fsl_mc_cmd.h | 4 +- drivers/bus/fslmc/mc/mc_sys.c | 2 +- drivers/bus/fslmc/portal/dpaa2_hw_dpbp.c | 6 +- drivers/bus/fslmc/portal/dpaa2_hw_dpio.c | 12 +- drivers/bus/fslmc/portal/dpaa2_hw_dpio.h | 10 +- drivers/bus/fslmc/portal/dpaa2_hw_pvt.h | 6 +- .../bus/fslmc/qbman/include/fsl_qbman_debug.h | 4 +- .../fslmc/qbman/include/fsl_qbman_portal.h | 80 +++---- drivers/bus/fslmc/qbman/qbman_debug.c | 4 +- drivers/bus/fslmc/qbman/qbman_portal.c | 82 +++---- drivers/bus/fslmc/rte_bus_fslmc_version.map | 201 +++++++++--------- 24 files changed, 269 insertions(+), 264 deletions(-)
drivers/bus/fslmc/fslmc_bus.c +1 -1
@@ -28,7 +28,7 @@ int dpaa2_logtype_bus;#define FSLMC_BUS_NAME fslmcstruct rte_fslmc_bus rte_fslmc_bus; -uint8_t dpaa2_virt_mode; +uint8_t __rte_internal dpaa2_virt_mode;uint32_t rte_fslmc_get_device_count (enum rte_dpaa2_dev_type device_type)
drivers/bus/fslmc/mc/dpbp.c +6 -6
@@ -26,7 +26,7 @@* * Return: '0' on Success; Error code otherwise. */ -int dpbp_open (struct fsl_mc_io *mc_io, +int __rte_internal dpbp_open(struct fsl_mc_io *mc_io,uint32_t cmd_flags,int dpbp_id,uint16_t *token)
@@ -157,7 +157,7 @@ int dpbp_destroy(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int dpbp_enable (struct fsl_mc_io *mc_io, +int __rte_internal dpbp_enable(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token) {
@@ -179,7 +179,7 @@ int dpbp_enable(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int dpbp_disable (struct fsl_mc_io *mc_io, +int __rte_internal dpbp_disable(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token) {
@@ -235,7 +235,7 @@ int dpbp_is_enabled(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int dpbp_reset (struct fsl_mc_io *mc_io, +int __rte_internal dpbp_reset(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token) {
@@ -258,7 +258,7 @@ int dpbp_reset(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int dpbp_get_attributes (struct fsl_mc_io *mc_io, +int __rte_internal dpbp_get_attributes(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token,struct dpbp_attr *attr)
@@ -329,7 +329,7 @@ int dpbp_get_api_version(struct fsl_mc_io *mc_io,* Return: '0' on Success; Error code otherwise. */ -int dpbp_get_num_free_bufs (struct fsl_mc_io *mc_io, +int __rte_internal dpbp_get_num_free_bufs(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token,uint32_t *num_free_bufs)
drivers/bus/fslmc/mc/dpci.c +3 -3
@@ -314,7 +314,7 @@ int dpci_get_attributes(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int dpci_set_rx_queue(struct fsl_mc_io *mc_io, +int __rte_internal dpci_set_rx_queue(struct fsl_mc_io *mc_io, uint32_t cmd_flags, uint16_t token, uint8_t priority,
@@ -476,7 +476,7 @@ int dpci_get_api_version(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int dpci_set_opr(struct fsl_mc_io *mc_io, +int __rte_internal dpci_set_opr(struct fsl_mc_io *mc_io, uint32_t cmd_flags, uint16_t token, uint8_t index,
@@ -514,7 +514,7 @@ int dpci_set_opr(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int dpci_get_opr(struct fsl_mc_io *mc_io, +int __rte_internal dpci_get_opr(struct fsl_mc_io *mc_io, uint32_t cmd_flags, uint16_t token, uint8_t index,
drivers/bus/fslmc/mc/dpcon.c +2 -2
@@ -25,7 +25,7 @@* * Return: '0' on Success; Error code otherwise. */ -int dpcon_open (struct fsl_mc_io *mc_io, +int __rte_internal dpcon_open(struct fsl_mc_io *mc_io,uint32_t cmd_flags,int dpcon_id,uint16_t *token)
@@ -267,7 +267,7 @@ int dpcon_reset(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int dpcon_get_attributes (struct fsl_mc_io *mc_io, +int __rte_internal dpcon_get_attributes(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token,struct dpcon_attr *attr)
drivers/bus/fslmc/mc/dpdmai.c +8 -8
@@ -24,7 +24,7 @@* * Return: '0' on Success; Error code otherwise. */ -int dpdmai_open (struct fsl_mc_io *mc_io, +int __rte_internal dpdmai_open(struct fsl_mc_io *mc_io,uint32_t cmd_flags,int dpdmai_id,uint16_t *token)
@@ -62,7 +62,7 @@ int dpdmai_open(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int dpdmai_close (struct fsl_mc_io *mc_io, +int __rte_internal dpdmai_close(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token) {
@@ -170,7 +170,7 @@ int dpdmai_destroy(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int dpdmai_enable (struct fsl_mc_io *mc_io, +int __rte_internal dpdmai_enable(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token) {
@@ -193,7 +193,7 @@ int dpdmai_enable(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int dpdmai_disable (struct fsl_mc_io *mc_io, +int __rte_internal dpdmai_disable(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token) {
@@ -275,7 +275,7 @@ int dpdmai_reset(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int dpdmai_get_attributes (struct fsl_mc_io *mc_io, +int __rte_internal dpdmai_get_attributes(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token,struct dpdmai_attr *attr)
@@ -318,7 +318,7 @@ int dpdmai_get_attributes(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int dpdmai_set_rx_queue(struct fsl_mc_io *mc_io, +int __rte_internal dpdmai_set_rx_queue(struct fsl_mc_io *mc_io, uint32_t cmd_flags, uint16_t token, uint8_t queue_idx,
@@ -360,7 +360,7 @@ int dpdmai_set_rx_queue(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int dpdmai_get_rx_queue(struct fsl_mc_io *mc_io, +int __rte_internal dpdmai_get_rx_queue(struct fsl_mc_io *mc_io, uint32_t cmd_flags, uint16_t token, uint8_t queue_idx,
@@ -410,7 +410,7 @@ int dpdmai_get_rx_queue(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int dpdmai_get_tx_queue(struct fsl_mc_io *mc_io, +int __rte_internal dpdmai_get_tx_queue(struct fsl_mc_io *mc_io, uint32_t cmd_flags, uint16_t token, uint8_t queue_idx,
drivers/bus/fslmc/mc/dpio.c +9 -9
@@ -26,7 +26,7 @@* * Return: '0' on Success; Error code otherwise. */ -int dpio_open (struct fsl_mc_io *mc_io, +int __rte_internal dpio_open(struct fsl_mc_io *mc_io,uint32_t cmd_flags,int dpio_id,uint16_t *token)
@@ -61,7 +61,7 @@ int dpio_open(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int dpio_close (struct fsl_mc_io *mc_io, +int __rte_internal dpio_close(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token) {
@@ -173,7 +173,7 @@ int dpio_destroy(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise */ -int dpio_enable (struct fsl_mc_io *mc_io, +int __rte_internal dpio_enable(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token) {
@@ -196,7 +196,7 @@ int dpio_enable(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise */ -int dpio_disable (struct fsl_mc_io *mc_io, +int __rte_internal dpio_disable(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token) {
@@ -253,7 +253,7 @@ int dpio_is_enabled(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int dpio_reset (struct fsl_mc_io *mc_io, +int __rte_internal dpio_reset(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token) {
@@ -277,7 +277,7 @@ int dpio_reset(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise */ -int dpio_get_attributes (struct fsl_mc_io *mc_io, +int __rte_internal dpio_get_attributes(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token,struct dpio_attr *attr)
@@ -322,7 +322,7 @@ int dpio_get_attributes(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int dpio_set_stashing_destination (struct fsl_mc_io *mc_io, +int __rte_internal dpio_set_stashing_destination(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token,uint8_t sdest)
@@ -386,7 +386,7 @@ int dpio_get_stashing_destination(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int dpio_add_static_dequeue_channel(struct fsl_mc_io *mc_io, +int __rte_internal dpio_add_static_dequeue_channel(struct fsl_mc_io *mc_io, uint32_t cmd_flags, uint16_t token, int dpcon_id,
@@ -425,7 +425,7 @@ int dpio_add_static_dequeue_channel(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int dpio_remove_static_dequeue_channel (struct fsl_mc_io *mc_io, +int __rte_internal dpio_remove_static_dequeue_channel(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token,int dpcon_id)
drivers/bus/fslmc/mc/dpmng.c +2 -2
@@ -18,7 +18,7 @@* * Return: '0' on Success; Error code otherwise. */ -int mc_get_version (struct fsl_mc_io *mc_io, +int __rte_internal mc_get_version(struct fsl_mc_io *mc_io,uint32_t cmd_flags,struct mc_version *mc_ver_info) {
@@ -57,7 +57,7 @@ int mc_get_version(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int mc_get_soc_version (struct fsl_mc_io *mc_io, +int __rte_internal mc_get_soc_version(struct fsl_mc_io *mc_io,uint32_t cmd_flags,struct mc_soc_version *mc_platform_info) {
drivers/bus/fslmc/mc/fsl_dpbp.h +6 -6
@@ -14,7 +14,7 @@struct fsl_mc_io ; -int dpbp_open (struct fsl_mc_io *mc_io, +int __rte_internal dpbp_open(struct fsl_mc_io *mc_io,uint32_t cmd_flags,int dpbp_id,uint16_t *token);
@@ -42,11 +42,11 @@ int dpbp_destroy(struct fsl_mc_io *mc_io,uint32_t cmd_flags, uint32_t obj_id); -int dpbp_enable (struct fsl_mc_io *mc_io, +int __rte_internal dpbp_enable(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token); -int dpbp_disable (struct fsl_mc_io *mc_io, +int __rte_internal dpbp_disable(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token);
@@ -55,7 +55,7 @@ int dpbp_is_enabled(struct fsl_mc_io *mc_io,uint16_t token, int *en); -int dpbp_reset (struct fsl_mc_io *mc_io, +int __rte_internal dpbp_reset(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token);
@@ -70,7 +70,7 @@ struct dpbp_attr {uint16_t bpid; }; -int dpbp_get_attributes (struct fsl_mc_io *mc_io, +int __rte_internal dpbp_get_attributes(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token,struct dpbp_attr *attr);
@@ -88,7 +88,7 @@ int dpbp_get_api_version(struct fsl_mc_io *mc_io,uint16_t *major_ver, uint16_t *minor_ver); -int dpbp_get_num_free_bufs (struct fsl_mc_io *mc_io, +int __rte_internal dpbp_get_num_free_bufs(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token,uint32_t *num_free_bufs);
drivers/bus/fslmc/mc/fsl_dpci.h +3 -3
@@ -180,7 +180,7 @@ struct dpci_rx_queue_cfg {int order_preservation_en; }; -int dpci_set_rx_queue(struct fsl_mc_io *mc_io, +int __rte_internal dpci_set_rx_queue(struct fsl_mc_io *mc_io, uint32_t cmd_flags, uint16_t token, uint8_t priority,
@@ -227,14 +227,14 @@ int dpci_get_api_version(struct fsl_mc_io *mc_io,uint16_t *major_ver, uint16_t *minor_ver); -int dpci_set_opr (struct fsl_mc_io *mc_io, +int __rte_internal dpci_set_opr(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token,uint8_t index,uint8_t options,struct opr_cfg *cfg); -int dpci_get_opr(struct fsl_mc_io *mc_io, +int __rte_internal dpci_get_opr(struct fsl_mc_io *mc_io, uint32_t cmd_flags, uint16_t token, uint8_t index,
drivers/bus/fslmc/mc/fsl_dpcon.h +2 -2
@@ -19,7 +19,7 @@ struct fsl_mc_io;*/ #define DPCON_INVALID_DPIO_ID (int)(-1) -int dpcon_open (struct fsl_mc_io *mc_io, +int __rte_internal dpcon_open(struct fsl_mc_io *mc_io,uint32_t cmd_flags,int dpcon_id,uint16_t *token);
@@ -76,7 +76,7 @@ struct dpcon_attr {uint8_t num_priorities; }; -int dpcon_get_attributes (struct fsl_mc_io *mc_io, +int __rte_internal dpcon_get_attributes(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token,struct dpcon_attr *attr);
drivers/bus/fslmc/mc/fsl_dpdmai.h +8 -8
@@ -23,12 +23,12 @@ struct fsl_mc_io;*/ #define DPDMAI_ALL_QUEUES (uint8_t)(-1) -int dpdmai_open (struct fsl_mc_io *mc_io, +int __rte_internal dpdmai_open(struct fsl_mc_io *mc_io,uint32_t cmd_flags,int dpdmai_id,uint16_t *token); -int dpdmai_close (struct fsl_mc_io *mc_io, +int __rte_internal dpdmai_close(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token);
@@ -54,11 +54,11 @@ int dpdmai_destroy(struct fsl_mc_io *mc_io,uint32_t cmd_flags, uint32_t object_id); -int dpdmai_enable (struct fsl_mc_io *mc_io, +int __rte_internal dpdmai_enable(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token); -int dpdmai_disable (struct fsl_mc_io *mc_io, +int __rte_internal dpdmai_disable(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token);
@@ -82,7 +82,7 @@ struct dpdmai_attr {uint8_t num_of_queues; }; -int dpdmai_get_attributes (struct fsl_mc_io *mc_io, +int __rte_internal dpdmai_get_attributes(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token,struct dpdmai_attr *attr);
@@ -148,7 +148,7 @@ struct dpdmai_rx_queue_cfg { }; -int dpdmai_set_rx_queue(struct fsl_mc_io *mc_io, +int __rte_internal dpdmai_set_rx_queue(struct fsl_mc_io *mc_io, uint32_t cmd_flags, uint16_t token, uint8_t queue_idx,
@@ -168,7 +168,7 @@ struct dpdmai_rx_queue_attr {uint32_t fqid; }; -int dpdmai_get_rx_queue(struct fsl_mc_io *mc_io, +int __rte_internal dpdmai_get_rx_queue(struct fsl_mc_io *mc_io, uint32_t cmd_flags, uint16_t token, uint8_t queue_idx,
@@ -184,7 +184,7 @@ struct dpdmai_tx_queue_attr {uint32_t fqid; }; -int dpdmai_get_tx_queue(struct fsl_mc_io *mc_io, +int __rte_internal dpdmai_get_tx_queue(struct fsl_mc_io *mc_io, uint32_t cmd_flags, uint16_t token, uint8_t queue_idx,
drivers/bus/fslmc/mc/fsl_dpio.h +9 -9
@@ -13,12 +13,12 @@struct fsl_mc_io ; -int dpio_open (struct fsl_mc_io *mc_io, +int __rte_internal dpio_open(struct fsl_mc_io *mc_io,uint32_t cmd_flags,int dpio_id,uint16_t *token); -int dpio_close (struct fsl_mc_io *mc_io, +int __rte_internal dpio_close(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token);
@@ -57,11 +57,11 @@ int dpio_destroy(struct fsl_mc_io *mc_io,uint32_t cmd_flags, uint32_t object_id); -int dpio_enable (struct fsl_mc_io *mc_io, +int __rte_internal dpio_enable(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token); -int dpio_disable (struct fsl_mc_io *mc_io, +int __rte_internal dpio_disable(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token);
@@ -70,11 +70,11 @@ int dpio_is_enabled(struct fsl_mc_io *mc_io,uint16_t token, int *en); -int dpio_reset (struct fsl_mc_io *mc_io, +int __rte_internal dpio_reset(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token); -int dpio_set_stashing_destination (struct fsl_mc_io *mc_io, +int __rte_internal dpio_set_stashing_destination(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token,uint8_t sdest);
@@ -84,13 +84,13 @@ int dpio_get_stashing_destination(struct fsl_mc_io *mc_io,uint16_t token, uint8_t *sdest); -int dpio_add_static_dequeue_channel (struct fsl_mc_io *mc_io, +int __rte_internal dpio_add_static_dequeue_channel(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token,int dpcon_id,uint8_t *channel_index); -int dpio_remove_static_dequeue_channel (struct fsl_mc_io *mc_io, +int __rte_internal dpio_remove_static_dequeue_channel(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token,int dpcon_id);
@@ -119,7 +119,7 @@ struct dpio_attr {uint32_t clk; }; -int dpio_get_attributes (struct fsl_mc_io *mc_io, +int __rte_internal dpio_get_attributes(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token,struct dpio_attr *attr);
drivers/bus/fslmc/mc/fsl_dpmng.h +2 -2
@@ -34,7 +34,7 @@ struct mc_version {uint32_t revision; }; -int mc_get_version (struct fsl_mc_io *mc_io, +int __rte_internal mc_get_version(struct fsl_mc_io *mc_io,uint32_t cmd_flags,struct mc_version *mc_ver_info);
@@ -48,7 +48,7 @@ struct mc_soc_version {uint32_t pvr; }; -int mc_get_soc_version (struct fsl_mc_io *mc_io, +int __rte_internal mc_get_soc_version(struct fsl_mc_io *mc_io,uint32_t cmd_flags,struct mc_soc_version *mc_platform_info);#endif /* __FSL_DPMNG_H */
drivers/bus/fslmc/mc/fsl_mc_cmd.h +3 -1
@@ -10,6 +10,8 @@#include <rte_byteorder.h> #include <stdint.h> +#include "rte_compat.h" +#define MC_CMD_NUM_OF_PARAMS 7#define phys_addr_t uint64_t
@@ -80,7 +82,7 @@ enum mc_cmd_status {#define MC_CMD_HDR_FLAGS_MASK 0xFF00FF00 -int mc_send_command (struct fsl_mc_io *mc_io,struct mc_command *cmd); +int __rte_internal mc_send_command(struct fsl_mc_io *mc_io, struct mc_command *cmd); static inline uint64_t mc_encode_cmd_header(uint16_t cmd_id, uint32_t cmd_flags,
drivers/bus/fslmc/mc/mc_sys.c +1 -1
@@ -51,7 +51,7 @@ static int mc_status_to_error(enum mc_cmd_status status)return -EINVAL ; } -int mc_send_command (struct fsl_mc_io *mc_io,struct mc_command *cmd) +int __rte_internal mc_send_command(struct fsl_mc_io *mc_io, struct mc_command *cmd) {enum mc_cmd_status status;uint64_t response;
drivers/bus/fslmc/portal/dpaa2_hw_dpbp.c +3 -3
@@ -89,7 +89,7 @@ dpaa2_create_dpbp_device(int vdev_fd __rte_unused,return 0 ; } -struct dpaa2_dpbp_dev *dpaa2_alloc_dpbp_dev (void ) +struct dpaa2_dpbp_dev *__rte_internal dpaa2_alloc_dpbp_dev(void) {struct dpaa2_dpbp_dev *dpbp_dev =NULL ;
@@ -102,7 +102,7 @@ struct dpaa2_dpbp_dev *dpaa2_alloc_dpbp_dev(void)return dpbp_dev; } -void dpaa2_free_dpbp_dev (struct dpaa2_dpbp_dev *dpbp) +void __rte_internal dpaa2_free_dpbp_dev(struct dpaa2_dpbp_dev *dpbp) {struct dpaa2_dpbp_dev *dpbp_dev =NULL ;
@@ -115,7 +115,7 @@ void dpaa2_free_dpbp_dev(struct dpaa2_dpbp_dev *dpbp) } } -int dpaa2_dpbp_supported (void ) +int __rte_internal dpaa2_dpbp_supported(void) {if (TAILQ_EMPTY (&dpbp_dev_list))return -1 ;
drivers/bus/fslmc/portal/dpaa2_hw_dpio.c +6 -6
@@ -229,7 +229,7 @@ dpaa2_configure_stashing(struct dpaa2_dpio_dev *dpio_dev, int lcoreid)return 0 ; } -static struct dpaa2_dpio_dev *dpaa2_get_qbman_swp (int lcoreid) +static struct dpaa2_dpio_dev *__rte_internal dpaa2_get_qbman_swp(int lcoreid) {struct dpaa2_dpio_dev *dpio_dev =NULL ;int ret;
@@ -253,7 +253,7 @@ static struct dpaa2_dpio_dev *dpaa2_get_qbman_swp(int lcoreid) }int -dpaa2_affine_qbman_swp (void ) +__rte_internal dpaa2_affine_qbman_swp(void) {unsigned int lcore_id =rte_lcore_id ();uint64_t tid =syscall (SYS_gettid);
@@ -301,7 +301,7 @@ dpaa2_affine_qbman_swp(void) }int -dpaa2_affine_qbman_ethrx_swp (void ) +__rte_internal dpaa2_affine_qbman_ethrx_swp(void) {unsigned int lcore_id =rte_lcore_id ();uint64_t tid =syscall (SYS_gettid);
@@ -570,7 +570,7 @@ dpaa2_create_dpio_device(int vdev_fd, }void -dpaa2_free_dq_storage (struct queue_storage_info_t *q_storage) +__rte_internal dpaa2_free_dq_storage(struct queue_storage_info_t *q_storage) {int i =0 ;
@@ -581,7 +581,7 @@ dpaa2_free_dq_storage(struct queue_storage_info_t *q_storage) }int -dpaa2_alloc_dq_storage (struct queue_storage_info_t *q_storage) +__rte_internal dpaa2_alloc_dq_storage(struct queue_storage_info_t *q_storage) {int i =0 ;
@@ -601,7 +601,7 @@ dpaa2_alloc_dq_storage(struct queue_storage_info_t *q_storage) }uint32_t -dpaa2_free_eq_descriptors (void ) +__rte_internal dpaa2_free_eq_descriptors(void) {struct dpaa2_dpio_dev *dpio_dev =DPAA2_PER_LCORE_DPIO ;struct qbman_result *eqresp;
drivers/bus/fslmc/portal/dpaa2_hw_dpio.h +5 -5
@@ -38,21 +38,21 @@ extern uint8_t dpaa2_eqcr_size;extern struct dpaa2_io_portal_t dpaa2_io_portal[RTE_MAX_LCORE ];/* Affine a DPIO portal to current processing thread */ -int dpaa2_affine_qbman_swp (void ); +int __rte_internal dpaa2_affine_qbman_swp(void);/* Affine additional DPIO portal to current crypto processing thread */ -int dpaa2_affine_qbman_ethrx_swp (void ); +int __rte_internal dpaa2_affine_qbman_ethrx_swp(void);/* allocate memory for FQ - dq storage */ int -dpaa2_alloc_dq_storage (struct queue_storage_info_t *q_storage); +__rte_internal dpaa2_alloc_dq_storage(struct queue_storage_info_t *q_storage);/* free memory for FQ- dq storage */ void -dpaa2_free_dq_storage (struct queue_storage_info_t *q_storage); +__rte_internal dpaa2_free_dq_storage(struct queue_storage_info_t *q_storage);/* free the enqueue response descriptors */ uint32_t -dpaa2_free_eq_descriptors (void ); +__rte_internal dpaa2_free_eq_descriptors(void);#endif /* _DPAA2_HW_DPIO_H_ */
drivers/bus/fslmc/portal/dpaa2_hw_pvt.h +3 -3
@@ -418,9 +418,9 @@ void set_swp_active_dqs(uint16_t dpio_index, struct qbman_result *dqs) { rte_global_active_dqs_list[dpio_index].global_active_dqs = dqs; } -struct dpaa2_dpbp_dev *dpaa2_alloc_dpbp_dev (void ); -void dpaa2_free_dpbp_dev (struct dpaa2_dpbp_dev *dpbp); -int dpaa2_dpbp_supported (void ); +struct dpaa2_dpbp_dev *__rte_internal dpaa2_alloc_dpbp_dev(void); +void __rte_internal dpaa2_free_dpbp_dev(struct dpaa2_dpbp_dev *dpbp); +int __rte_internal dpaa2_dpbp_supported(void);struct dpaa2_dpci_dev *rte_dpaa2_alloc_dpci_dev (void );void rte_dpaa2_free_dpci_dev (struct dpaa2_dpci_dev *dpci);
drivers/bus/fslmc/qbman/include/fsl_qbman_debug.h +2 -2
@@ -24,7 +24,7 @@ uint8_t verb;uint8_t reserved2[29 ]; }; -int qbman_fq_query_state (struct qbman_swp *s,uint32_t fqid, +int __rte_internal qbman_fq_query_state(struct qbman_swp *s, uint32_t fqid,struct qbman_fq_query_np_rslt *r); -uint32_t qbman_fq_state_frame_count (const struct qbman_fq_query_np_rslt *r); +uint32_t __rte_internal qbman_fq_state_frame_count(const struct qbman_fq_query_np_rslt *r);uint32_t qbman_fq_state_byte_count (const struct qbman_fq_query_np_rslt *r);
drivers/bus/fslmc/qbman/include/fsl_qbman_portal.h +40 -40
313 lines not on this page · raw message
drivers/bus/fslmc/qbman/qbman_debug.c +2 -2
@@ -23,7 +23,7 @@ struct qbman_fq_query_desc {uint8_t reserved2[57 ]; }; -int qbman_fq_query_state (struct qbman_swp *s,uint32_t fqid, +int __rte_internal qbman_fq_query_state(struct qbman_swp *s, uint32_t fqid,struct qbman_fq_query_np_rslt *r) {struct qbman_fq_query_desc *p;
@@ -54,7 +54,7 @@ int qbman_fq_query_state(struct qbman_swp *s, uint32_t fqid,return 0 ; } -uint32_t qbman_fq_state_frame_count (const struct qbman_fq_query_np_rslt *r) +uint32_t __rte_internal qbman_fq_state_frame_count(const struct qbman_fq_query_np_rslt *r) {return (r->frm_cnt &0x00FFFFFF ); }
drivers/bus/fslmc/qbman/qbman_portal.c +41 -41
306 lines not on this page · raw message
drivers/bus/fslmc/rte_bus_fslmc_version.map +102 -99
233 lines not on this page · raw message
Signature · 1 line
2.20.1
PATCH 5/9 dpaa2: Adjust dpaa2 driver to mark internal symbols with __rte_internal
In reply to Neil HormanIdentify functions in the dpaa2 driver which are internal (based on their not having an rte_ prefix) and tag them with __rte_internal
--- drivers/net/dpaa2/dpaa2_ethdev.c | 4 ++-- drivers/net/dpaa2/dpaa2_ethdev.h | 4 ++-- drivers/net/dpaa2/rte_pmd_dpaa2_version.map | 14 ++++++-------- 3 files changed, 10 insertions(+), 12 deletions(-)
drivers/net/dpaa2/dpaa2_ethdev.c +2 -2
@@ -1798,7 +1798,7 @@ dpaa2_dev_rss_hash_conf_get(struct rte_eth_dev *dev,return 0 ; } -int dpaa2_eth_eventq_attach (const struct rte_eth_dev *dev, +int __rte_internal dpaa2_eth_eventq_attach(const struct rte_eth_dev *dev,int eth_rx_queue_id,uint16_t dpcon_id,const struct rte_event_eth_rx_adapter_queue_conf *queue_conf)
@@ -1881,7 +1881,7 @@ int dpaa2_eth_eventq_attach(const struct rte_eth_dev *dev,return 0 ; } -int dpaa2_eth_eventq_detach (const struct rte_eth_dev *dev, +int __rte_internal dpaa2_eth_eventq_detach(const struct rte_eth_dev *dev,int eth_rx_queue_id) {struct dpaa2_dev_priv *eth_priv = dev->data->dev_private;
drivers/net/dpaa2/dpaa2_ethdev.h +2 -2
@@ -140,12 +140,12 @@ int dpaa2_remove_flow_dist(struct rte_eth_dev *eth_dev,int dpaa2_attach_bp_list (struct dpaa2_dev_priv *priv,void *blist); -int dpaa2_eth_eventq_attach (const struct rte_eth_dev *dev, +int __rte_internal dpaa2_eth_eventq_attach(const struct rte_eth_dev *dev,int eth_rx_queue_id,uint16_t dpcon_id,const struct rte_event_eth_rx_adapter_queue_conf *queue_conf); -int dpaa2_eth_eventq_detach (const struct rte_eth_dev *dev, +int __rte_internal dpaa2_eth_eventq_detach(const struct rte_eth_dev *dev,int eth_rx_queue_id); uint16_t dpaa2_dev_loopback_rx(void *queue, struct rte_mbuf **bufs,
drivers/net/dpaa2/rte_pmd_dpaa2_version.map +6 -8
@@ -1,15 +1,13 @@ -DPDK_17.05 { - - local: *; -}; - -DPDK_17.11 { +INTERNAL { global: dpaa2_eth_eventq_attach; dpaa2_eth_eventq_detach; +}; -} DPDK_17.05; +DPDK_17.05 { + local: *; +}; EXPERIMENTAL { global:
@@ -17,4 +15,4 @@ EXPERIMENTAL { rte_pmd_dpaa2_mux_flow_create; rte_pmd_dpaa2_set_custom_hash; rte_pmd_dpaa2_set_timestamp; -} DPDK_17.11; +} DPDK_17.05;
Signature · 1 line
2.20.1
PATCH 6/9 dpaax: mark internal functions with __rte_internal
In reply to Neil HormanIdentify functions in the dpaa2 driver which are internal (based on
their not having an rte_ prefix) and tag them with __rte_internal
--- drivers/common/dpaax/dpaax_iova_table.c | 8 ++++---- drivers/common/dpaax/dpaax_iova_table.h | 8 ++++---- drivers/common/dpaax/rte_common_dpaax_version.map | 4 +++- 3 files changed, 11 insertions(+), 9 deletions(-)
drivers/common/dpaax/dpaax_iova_table.c +4 -4
@@ -151,7 +151,7 @@ read_memory_node(unsigned int *count) }int -dpaax_iova_table_populate (void ) +__rte_internal dpaax_iova_table_populate(void) {int ret;unsigned int i, node_count;
@@ -252,7 +252,7 @@ dpaax_iova_table_populate(void) }void -dpaax_iova_table_depopulate (void ) +__rte_internal dpaax_iova_table_depopulate(void) {if (dpaax_iova_table_p ==NULL )return ;
@@ -264,7 +264,7 @@ dpaax_iova_table_depopulate(void) }int -dpaax_iova_table_update (phys_addr_t paddr,void *vaddr,size_t length) +__rte_internal dpaax_iova_table_update(phys_addr_t paddr, void *vaddr, size_t length) {int found =0 ;unsigned int i;
@@ -348,7 +348,7 @@ dpaax_iova_table_update(phys_addr_t paddr, void *vaddr, size_t length)* Not for weak hearted - the tables can get quite large */ void -dpaax_iova_table_dump (void ) +__rte_internal dpaax_iova_table_dump(void) {unsigned int i, j;struct dpaax_iovat_element *entry;
drivers/common/dpaax/dpaax_iova_table.h +4 -4
@@ -59,10 +59,10 @@ extern struct dpaax_iova_table *dpaax_iova_table_p;#define DPAAX_MEM_SPLIT_MASK_OFF (DPAAX_MEM_SPLIT - 1)/**< Offset */ /* APIs exposed */ -int dpaax_iova_table_populate (void ); -void dpaax_iova_table_depopulate (void ); -int dpaax_iova_table_update (phys_addr_t paddr,void *vaddr,size_t length); -void dpaax_iova_table_dump (void ); +int __rte_internal dpaax_iova_table_populate(void); +void __rte_internal dpaax_iova_table_depopulate(void); +int __rte_internal dpaax_iova_table_update(phys_addr_t paddr, void *vaddr, size_t length); +void __rte_internal dpaax_iova_table_dump(void);static inline void *dpaax_iova_table_get_va (phys_addr_t paddr) __attribute__((hot));
drivers/common/dpaax/rte_common_dpaax_version.map +3 -1
@@ -1,4 +1,4 @@ -DPDK_18.11 { +INTERNAL { global: dpaax_iova_table_update;
@@ -6,6 +6,8 @@ DPDK_18.11 { dpaax_iova_table_dump; dpaax_iova_table_p; dpaax_iova_table_populate; +}; +DPDK_18.11 { local: *; };
Signature · 1 line
2.20.1
PATCH 7/9 cpt: mark internal functions with __rte_internal
In reply to Neil HormanIdentify functions in the cpt driver which are internal (based on their not having an rte_ prefix) and tag them with __rte_internal
--- drivers/common/cpt/cpt_pmd_ops_helper.c | 4 ++-- drivers/common/cpt/cpt_pmd_ops_helper.h | 4 ++-- drivers/common/cpt/rte_common_cpt_version.map | 6 +++++- 3 files changed, 9 insertions(+), 5 deletions(-)
drivers/common/cpt/cpt_pmd_ops_helper.c +2 -2
@@ -13,7 +13,7 @@#define CPT_OFFSET_CONTROL_BYTES 8int32_t -cpt_pmd_ops_helper_get_mlen_direct_mode (void ) +__rte_internal cpt_pmd_ops_helper_get_mlen_direct_mode(void) {uint32_t len =0 ;
@@ -27,7 +27,7 @@ cpt_pmd_ops_helper_get_mlen_direct_mode(void) }int -cpt_pmd_ops_helper_get_mlen_sg_mode (void ) +__rte_internal cpt_pmd_ops_helper_get_mlen_sg_mode(void) {uint32_t len =0 ;
drivers/common/cpt/cpt_pmd_ops_helper.h +2 -2
@@ -20,7 +20,7 @@*/ int32_t -cpt_pmd_ops_helper_get_mlen_direct_mode (void ); +__rte_internal cpt_pmd_ops_helper_get_mlen_direct_mode(void);/* * Get size of contiguous meta buffer to be allocated when working in scatter
@@ -30,5 +30,5 @@ cpt_pmd_ops_helper_get_mlen_direct_mode(void);* - length */ int -cpt_pmd_ops_helper_get_mlen_sg_mode (void ); +__rte_internal cpt_pmd_ops_helper_get_mlen_sg_mode(void);#endif /* _CPT_PMD_OPS_HELPER_H_ */
drivers/common/cpt/rte_common_cpt_version.map +5 -1
@@ -1,6 +1,10 @@ -DPDK_18.11 { +INTERNAL { global: cpt_pmd_ops_helper_get_mlen_direct_mode; cpt_pmd_ops_helper_get_mlen_sg_mode; }; + +DPDK_18.11 { + local: *; +};
Signature · 1 line
2.20.1
Re: [dpdk-dev] [EXT] [PATCH v1 7/9] cpt: mark internal functions with __rte_internal
Quoted 91 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Thursday, June 13, 2019 2:09 AM
To: [email protected]
Cc: Neil Horman <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>; Bruce Richardson <[email protected]>; Thomas Monjalon <[email protected]>; Anoob Joseph <[email protected]>
Subject: [EXT] [PATCH v1 7/9] cpt: mark internal functions with__rte_internalExternal Email
----------------------------------------------------------------------
Identify functions in the cpt driver which are internal (based on their not having anrte_prefix) and tag them with__rte_internalSigned-off-by Neil Horman <[email protected]>CC Jerin Jacob Kollanukkaran <[email protected]>CC Bruce Richardson <[email protected]>CC Thomas Monjalon <[email protected]>CC Anoob Joseph <[email protected]>--- drivers/common/cpt/cpt_pmd_ops_helper.c | 4 ++-- drivers/common/cpt/cpt_pmd_ops_helper.h | 4 ++-- drivers/common/cpt/rte_common_cpt_version.map | 6 +++++- 3 files changed, 9 insertions(+), 5 deletions(-)drivers/common/cpt/cpt_pmd_ops_helper.c +2
-2@@ -13,7 +13,7 @@#define CPT_OFFSET_CONTROL_BYTES 8int32_t -cpt_pmd_ops_helper_get_mlen_direct_mode (void ) +__rte_internal cpt_pmd_ops_helper_get_mlen_direct_mode(void) {uint32_t len =0 ;@@ -27,7 +27,7 @@ cpt_pmd_ops_helper_get_mlen_direct_mode(void) }int -cpt_pmd_ops_helper_get_mlen_sg_mode (void ) +__rte_internal cpt_pmd_ops_helper_get_mlen_sg_mode(void) {uint32_t len =0 ;drivers/common/cpt/cpt_pmd_ops_helper.h +1
-1@@ -20,7 +20,7 @@*/ int32_t -cpt_pmd_ops_helper_get_mlen_direct_mode (void ); +__rte_internal cpt_pmd_ops_helper_get_mlen_direct_mode(void);/* * Get size of contiguous meta buffer to be allocated when working in scatter @@ -30,5 +30,5 @@
cpt_pmd_ops_helper_get_mlen_direct_mode(void);
* - length
*/
intdrivers/common/cpt/cpt_pmd_ops_helper.h +1
-1-cpt_pmd_ops_helper_get_mlen_sg_mode (void); +__rte_internal cpt_pmd_ops_helper_get_mlen_sg_mode (void );#endif /* _CPT_PMD_OPS_HELPER_H_ */ drivers/common/cpt/rte_common_cpt_version.map +5
-2@@ -1,6 +1,10 @@ -DPDK_18.11 { +INTERNAL { global: cpt_pmd_ops_helper_get_mlen_direct_mode; cpt_pmd_ops_helper_get_mlen_sg_mode; }; + +DPDK_18.11 { + local: *; +}; --
Quoted 1 line
2.20.1
PATCH 8/9 octeonx: mark internal functions with __rte_internal
In reply to Neil HormanIdentify functions in the octeon driver which are internal (based on their not having an rte_ prefix) and tag them with __rte_internal
--- drivers/common/octeontx/octeontx_mbox.c | 6 +++--- drivers/common/octeontx/octeontx_mbox.h | 6 +++--- drivers/common/octeontx/rte_common_octeontx_version.map | 9 ++++++++- 3 files changed, 14 insertions(+), 7 deletions(-)
drivers/common/octeontx/octeontx_mbox.c +3 -3
@@ -190,7 +190,7 @@ mbox_send(struct mbox *m, struct octeontx_mbox_hdr *hdr, const void *txmsg, }int -octeontx_mbox_set_ram_mbox_base (uint8_t *ram_mbox_base) +__rte_internal octeontx_mbox_set_ram_mbox_base(uint8_t *ram_mbox_base) {struct mbox *m = &octeontx_mbox;
@@ -213,7 +213,7 @@ octeontx_mbox_set_ram_mbox_base(uint8_t *ram_mbox_base) }int -octeontx_mbox_set_reg (uint8_t *reg) +__rte_internal octeontx_mbox_set_reg(uint8_t *reg) {struct mbox *m = &octeontx_mbox;
@@ -236,7 +236,7 @@ octeontx_mbox_set_reg(uint8_t *reg) }int -octeontx_mbox_send (struct octeontx_mbox_hdr *hdr,void *txdata, +__rte_internal octeontx_mbox_send(struct octeontx_mbox_hdr *hdr, void *txdata,uint16_t txlen,void *rxdata,uint16_t rxlen) {struct mbox *m = &octeontx_mbox;
drivers/common/octeontx/octeontx_mbox.h +3 -3
@@ -29,9 +29,9 @@ struct octeontx_mbox_hdr {uint8_t res_code;/* Functional layer response code */ }; -int octeontx_mbox_set_ram_mbox_base (uint8_t *ram_mbox_base); -int octeontx_mbox_set_reg (uint8_t *reg); -int octeontx_mbox_send (struct octeontx_mbox_hdr *hdr, +int __rte_internal octeontx_mbox_set_ram_mbox_base(uint8_t *ram_mbox_base); +int __rte_internal octeontx_mbox_set_reg(uint8_t *reg); +int __rte_internal octeontx_mbox_send(struct octeontx_mbox_hdr *hdr,void *txdata,uint16_t txlen,void *rxdata,uint16_t rxlen);#endif /* __OCTEONTX_MBOX_H__ */
drivers/common/octeontx/rte_common_octeontx_version.map +8 -1
@@ -1,7 +1,14 @@ -DPDK_18.05 { +INTERNAL { global: octeontx_mbox_set_ram_mbox_base; octeontx_mbox_set_reg; octeontx_mbox_send; }; + +DPDK_18.05 { + global: + octeontx_logtype_mbox; + + local: *; +};
Signature · 1 line
2.20.1
PATCH 9/9 dpaa2: mark internal functions with __rte_internal
In reply to Neil HormanIdentify functions in the dpaa2 driver which are internal (based on their not having an rte_ prefix) and tag them with __rte_internal
--- drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c | 4 ++-- drivers/crypto/dpaa2_sec/dpaa2_sec_event.h | 4 ++-- .../crypto/dpaa2_sec/rte_pmd_dpaa2_sec_version.map | 13 ++++++------- 3 files changed, 10 insertions(+), 11 deletions(-)
drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c +2 -2
@@ -3152,7 +3152,7 @@ dpaa2_sec_process_atomic_event(struct qbman_swp *swp __attribute__((unused)), }int -dpaa2_sec_eventq_attach (const struct rte_cryptodev *dev, +__rte_internal dpaa2_sec_eventq_attach(const struct rte_cryptodev *dev,int qp_id,uint16_t dpcon_id,const struct rte_event *event)
@@ -3195,7 +3195,7 @@ dpaa2_sec_eventq_attach(const struct rte_cryptodev *dev, }int -dpaa2_sec_eventq_detach (const struct rte_cryptodev *dev, +__rte_internal dpaa2_sec_eventq_detach(const struct rte_cryptodev *dev,int qp_id) {struct dpaa2_sec_dev_private *priv = dev->data->dev_private;
drivers/crypto/dpaa2_sec/dpaa2_sec_event.h +2 -2
@@ -7,12 +7,12 @@#define _DPAA2_SEC_EVENT_H_int -dpaa2_sec_eventq_attach (const struct rte_cryptodev *dev, +__rte_internal dpaa2_sec_eventq_attach(const struct rte_cryptodev *dev,int qp_id,uint16_t dpcon_id,const struct rte_event *event); -int dpaa2_sec_eventq_detach (const struct rte_cryptodev *dev, +int __rte_internal dpaa2_sec_eventq_detach(const struct rte_cryptodev *dev,int qp_id);#endif /* _DPAA2_SEC_EVENT_H_ */
drivers/crypto/dpaa2_sec/rte_pmd_dpaa2_sec_version.map +6 -7
@@ -1,12 +1,11 @@ -DPDK_17.05 { - - local: *; -}; - -DPDK_18.11 { +INTERNAL { global: dpaa2_sec_eventq_attach; dpaa2_sec_eventq_detach; +}; + +DPDK_17.05 { -} DPDK_17.05; + local: *; +};
Signature · 1 line
2.20.1
Re: [dpdk-dev] [PATCH v1 9/9] dpaa2: mark internal functions with __rte_internal
Quoted 10 lines · Neil Horman
Identify functions in the dpaa2 driver which are internal (based on their not having an
rte_prefix) and tag them with__rte_internalSigned-off-by Neil Horman <[email protected]>CC Jerin Jacob Kollanukkaran <[email protected]>CC Bruce Richardson <[email protected]>CC Thomas Monjalon <[email protected]>CC Akhil Goyal <[email protected]>CC Hemant Agrawal <[email protected]>---
I think it's incomplete.
Re: [dpdk-dev] [PATCH v1 9/9] dpaa2: mark internal functions with __rte_internal
Quoted 13 lines · Aaron Conole
Quoted 10 lines · Neil Horman
Identify functions in the dpaa2 driver which are internal (based on their not having an
rte_prefix) and tag them with__rte_internalSigned-off-by Neil Horman <[email protected]>CC Jerin Jacob Kollanukkaran <[email protected]>CC Bruce Richardson <[email protected]>CC Thomas Monjalon <[email protected]>CC Akhil Goyal <[email protected]>CC Hemant Agrawal <[email protected]>---
Quoted 4 lines
I think it's incomplete.
It appears the travis environment is using the meson build system, while I was building with the traditional Make system, and meson doesn't yet set BUILDING_RTE_SDK in its environment. I'll fix that
Neil
Re: [dpdk-dev] [PATCH v1 0/9] dpdk: introduce __rte_internal tag
In reply to Neil HormanQuoted 51 lines · Neil Horman
Hey-
Based on our recent conversations regarding the use of symbols only
meant for internal dpdk consumption (between dpdk libraries), this is an idea
that I've come up with that I'd like to get some feedback onSummary:
1) We have symbols in the DPDK that are meant to be used between DPDK libraries,
but not by applications linking to them
2) We would like to document those symbols in the code, so as to note them clearly as for being meant for internal use only
3) Linker symbol visibility is a very coarse grained tool, and so there is no
good way in a single library to mark items as being meant for use only by other
DPDK libraries, at least not without some extensive runtime checkingProposal:
I'm proposing that we introduce the__rte_internaltag. From a coding standpoint it works a great deal like the__rte_experimentaltag in that it expempts the tagged symbol from ABI constraints (as the only users should be
represented in the DPDK build environment). Additionally, the__rte_internal
macro resolves differently based on the definition of theBUILDING_RTE_SDKflag
(working under the assumption that said flag should only ever be set if we are
actually building DPDK libraries which will make use of internal calls). If theBUILDING_RTE_SDKflag is set__rte_internalresolves to __attribute__((section
"text.internal)), placing it in a special text section which is then used to
validate that the the symbol appears in the INTERNAL section of the corresponding library version map). IfBUILDING_RTE_SDKis not set, then__rte_internalresolves to __attribute__((error("..."))), which causes any caller of the tagged function to throw an error at compile time, indicating that
the symbol is not available for external use.This isn't a perfect solution, as applications can still hack around it of course, but I think it hits some of the high points, restricting symbol access
for any library that prototypes its public and private symbols in the same header file, excluding the internal symbols from ABI constraints, and clearly
documenting those symbols which we wish to limit to internal usage.
Quoted 7 lines
Signed-off-by Neil Horman <[email protected]>CC Jerin Jacob Kollanukkaran <[email protected]>CC Bruce Richardson <[email protected]>CC Thomas Monjalon <[email protected]>
On the principle, are we not breaking the ABI for the libraries that we touch with this?
Compilation is broken in patch 1 and 2 because the script rename is part of patch 3.
Signature · 1 line
David Marchand
Re: [dpdk-dev] [PATCH v1 0/9] dpdk: introduce __rte_internal tag
Quoted 60 lines · David Marchand
Quoted 51 lines · Neil Horman
Hey-
Based on our recent conversations regarding the use of symbols only
meant for internal dpdk consumption (between dpdk libraries), this is an idea
that I've come up with that I'd like to get some feedback onSummary:
1) We have symbols in the DPDK that are meant to be used between DPDK libraries,
but not by applications linking to them
2) We would like to document those symbols in the code, so as to note them clearly as for being meant for internal use only
3) Linker symbol visibility is a very coarse grained tool, and so there is no
good way in a single library to mark items as being meant for use only by other
DPDK libraries, at least not without some extensive runtime checkingProposal:
I'm proposing that we introduce the__rte_internaltag. From a coding standpoint it works a great deal like the__rte_experimentaltag in that it expempts the tagged symbol from ABI constraints (as the only users should be
represented in the DPDK build environment). Additionally, the__rte_internal
macro resolves differently based on the definition of theBUILDING_RTE_SDKflag
(working under the assumption that said flag should only ever be set if we are
actually building DPDK libraries which will make use of internal calls). If theBUILDING_RTE_SDKflag is set__rte_internalresolves to __attribute__((section
"text.internal)), placing it in a special text section which is then used to
validate that the the symbol appears in the INTERNAL section of the corresponding library version map). IfBUILDING_RTE_SDKis not set, then__rte_internalresolves to __attribute__((error("..."))), which causes any caller of the tagged function to throw an error at compile time, indicating that
the symbol is not available for external use.This isn't a perfect solution, as applications can still hack around it of course, but I think it hits some of the high points, restricting symbol access
for any library that prototypes its public and private symbols in the same header file, excluding the internal symbols from ABI constraints, and clearly
documenting those symbols which we wish to limit to internal usage.Quoted 7 lines
Signed-off-by Neil Horman <[email protected]>CC Jerin Jacob Kollanukkaran <[email protected]>CC Bruce Richardson <[email protected]>CC Thomas Monjalon <[email protected]>
Quoted 3 lines
On the principle, are we not breaking the ABI for the libraries that we touch with this?
Yes, strictly speaking on principle, we are, because we are removing symbols from the exported symbol list, but the purpose of this patch set was to identify those symbols which, by (convention/documentation/etc), we had intended to mandate should never be used by applications outside of the DPDK. Since this patch is meant to identify those symbols, and somewhat more formally isolate and prevent users from accessing them, I was making the argument that it was ok to do so. If you like we can go through the process of deprecating them, prior to moving them to an internal space, but given that I was under the impression anyone reporting a bug about the symbol disappearing would be responded to with a note indicating that they never should have been accessed to begin with, I had assumed the movement would be ok.
Quoted 3 lines
Compilation is broken in patch 1 and 2 because the script rename is part of patch 3.
Good point, I'll reshuffle those to be more correct. Neil
Quoted 3 lines
Signature · 2 lines
-- David Marchand
v2 cover dpdk: introduce __rte_internal tag
In reply to Neil HormanHey-
Based on our recent conversations regarding the use of symbols only
meant for internal dpdk consumption (between dpdk libraries), this is an idea that I've come up with that I'd like to get some feedback on
Summary:
1) We have symbols in the DPDK that are meant to be used between DPDK libraries, but not by applications linking to them
2) We would like to document those symbols in the code, so as to note them clearly as for being meant for internal use only
3) Linker symbol visibility is a very coarse grained tool, and so there is no good way in a single library to mark items as being meant for use only by other DPDK libraries, at least not without some extensive runtime checking
Proposal:
I'm proposing that we introduce the __rte_internal tag. From a coding standpoint it works a great deal like the __rte_experimental tag in that it expempts the tagged symbol from ABI constraints (as the only users should be represented in the DPDK build environment). Additionally, the __rte_internal macro resolves differently based on the definition of the BUILDING_RTE_SDK flag (working under the assumption that said flag should only ever be set if we are actually building DPDK libraries which will make use of internal calls). If the BUILDING_RTE_SDK flag is set __rte_internal resolves to __attribute__((section "text.internal)), placing it in a special text section which is then used to validate that the the symbol appears in the INTERNAL section of the corresponding library version map). If BUILDING_RTE_SDK is not set, then __rte_internal resolves to __attribute__((error("..."))), which causes any caller of the tagged function to throw an error at compile time, indicating that the symbol is not available for external use.
This isn't a perfect solution, as applications can still hack around it of course, but I think it hits some of the high points, restricting symbol access for any library that prototypes its public and private symbols in the same header file, excluding the internal symbols from ABI constraints, and clearly documenting those symbols which we wish to limit to internal usage.
---
Change notes
v1->v2
Moved check-experimental-syms.sh rename to earlier in the series Updated meson build environment to support BUILDING_RTE_SDK
PATCH v2 1/10 Add __rte_internal tag for functions and version target
This tag is meant to be used on function prototypes to identify functions that are only meant to be used by internal DPDK libraries (i.e. libraries that are built while building the SDK itself, as identified by the defining of the BUILDING_RTE_SDK macro). When that flag is not set, it will resolve to an error function attribute, causing build breakage for any compilation unit attempting to build it
Validate the use of this tag in much the same way we validate __rte_experimental. By adding an INTERNAL version to library map files, we can exempt internal-only functions from ABI checking, and handle them to ensure that symbols we wish to only be for internal use between dpdk libraries are properly tagged with __rte_experimental
Note this patch updates the check-experimental-syms.sh script, which normally only check the EXPERIMENTAL section to also check the INTERNAL section now. As such its been renamed to the now more appropriate check-special-syms.sh
---
...rimental-syms.sh => check-special-syms.sh} | 24 ++++++++++++++++++-
lib/librte_eal/common/include/rte_compat.h | 12 ++++++++++
mk/internal/rte.compile-pre.mk | 6 ++---
mk/target/generic/rte.vars.mk | 2 +-
4 files changed, 39 insertions(+), 5 deletions(-)
rename buildtools/{check-experimental-syms.sh => check-special-syms.sh} (53%)buildtools/check-special-syms.sh renamed from buildtools/check-experimental-syms.sh +23 -1
@@ -31,10 +31,32 @@ do cat >&2 <<- END_OF_MESSAGE $SYM is not flagged as experimental but is listed in version map - Please add __rte_experimental to the definition of $SYM + Please add __rte_experimental to the definition/prototype of $SYM END_OF_MESSAGE exit 1 fi done + +for i in `awk 'BEGIN {found=0} + /.*INTERNAL.*/ {found=1} + /.*}.*;/ {found=0} + /.*;/ {if (found == 1) print $1}' $MAPFILE` +do + SYM=`echo $i | sed -e"s/;//"` + objdump -t $OBJFILE | grep -q "\.text.*$SYM$" + IN_TEXT=$? + objdump -t $OBJFILE | grep -q "\.text\.internal.*$SYM$" + IN_EXP=$? + if [ $IN_TEXT -eq 0 -a $IN_EXP -ne 0 ] + then + cat >&2 <<- END_OF_MESSAGE + $SYM is not flagged as internal + but is listed in version map + Please add __rte_internal to the definition/prototype of $SYM + END_OF_MESSAGE + exit 1 + fi +done + exit 0
lib/librte_eal/common/include/rte_compat.h +12 -0
@@ -89,4 +89,16 @@ __attribute__((section(".text.experimental")))#endif +/* +* __rte_internal tags mark functions as internal only, If specified in public +* header files, this tag will resolve to an error directive, preventing +* external applications from attempting to make calls to functions not meant +* for consumption outside the dpdk library +*/ +#ifdef BUILDING_RTE_SDK +#define __rte_internal __attribute__((section(".text.internal"))) +#else +#define __rte_internal __attribute__((error("This function cannot be used outside of the core DPDK library"), \ + section(".text.internal"))) +#endif #endif /* _RTE_COMPAT_H_ */
mk/internal/rte.compile-pre.mk +3 -3
@@ -56,8 +56,8 @@ C_TO_O = $(CC) -Wp,-MD,$(call obj2dep,$(@)).tmp $(CPPFLAGS) $(CFLAGS) \ C_TO_O_STR = $(subst ','\'',$(C_TO_O)) #'# fix syntax highlight C_TO_O_DISP = $(if $(V),"$(C_TO_O_STR)"," CC $(@)") endif -EXPERIMENTAL_CHECK = $(RTE_SDK)/buildtools/check-experimental-syms.sh -CHECK_EXPERIMENTAL = $(EXPERIMENTAL_CHECK) $(SRCDIR)/$(EXPORT_MAP) $@ +SPECIAL_SYM_CHECK = $(RTE_SDK)/buildtools/check-special-syms.sh +CHECK_SPECIAL_SYMS = $(SPECIAL_SYM_CHECK) $(SRCDIR)/$(EXPORT_MAP) $@ PMDINFO_GEN = $(RTE_SDK_BIN)/app/dpdk-pmdinfogen $@ [email protected] PMDINFO_CC = $(CC) $(CPPFLAGS) $(CFLAGS) $(EXTRA_CFLAGS) -c -o [email protected] [email protected]
@@ -75,7 +75,7 @@ C_TO_O_DO = @set -e; \ echo $(C_TO_O_DISP); \ $(C_TO_O) && \ $(PMDINFO_TO_O) && \ - $(CHECK_EXPERIMENTAL) && \ + $(CHECK_SPECIAL_SYMS) && \ echo $(C_TO_O_CMD) > $(call obj2cmd,$(@)) && \ sed 's,'$@':,dep_'$@' =,' $(call obj2dep,$(@)).tmp > $(call obj2dep,$(@)) && \ rm -f $(call obj2dep,$(@)).tmp
mk/target/generic/rte.vars.mk +1 -1
@@ -96,7 +96,7 @@ LDFLAGS += -L$(RTE_OUTPUT)/lib # defined. ifeq ($(BUILDING_RTE_SDK),1) # building sdk -CFLAGS += -include $(RTE_OUTPUT)/include/rte_config.h +CFLAGS += -include $(RTE_OUTPUT)/include/rte_config.h -DBUILDING_RTE_SDK else # if we are building an external application, include SDK's lib and # includes too
Signature · 1 line
2.20.1
Re: [dpdk-dev] [PATCH v2 01/10] Add __rte_internal tag for functions and version target
Hi Neil,
Quoted 38 lines
-----Original Message-----
From: dev <[email protected]> On Behalf Of Neil Horman
Sent: Thursday, June 13, 2019 22:24
To: [email protected]
Cc: Neil Horman <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>; Richardson, Bruce <[email protected]>; Thomas Monjalon <[email protected]>
Subject: [dpdk-dev] [PATCH v2 01/10] Add__rte_internaltag for functions and version targetThis tag is meant to be used on function prototypes to identify functions that are only meant to be used by internal DPDK libraries (i.e. libraries that are built while building the SDK itself, as identified by the defining of the
BUILDING_RTE_SDKmacro). When that flag is not set, it will resolve to an error function attribute, causing build breakage for any compilation unit attempting to build itValidate the use of this tag in much the same way we validate
__rte_experimental. By adding an INTERNAL version to library map files, we can exempt internal-only functions from ABI checking, and handle them to ensure that symbols we wish to only be for internal use between dpdk libraries are properly tagged with__rte_experimentalNote this patch updates the
check-experimental-syms.shscript, which normally only check the EXPERIMENTAL section to also check the INTERNAL section now. As such its been renamed to the now more appropriatecheck-special-syms.shSigned-off-by Neil Horman <[email protected]>CC Jerin Jacob Kollanukkaran <[email protected]>CC Bruce Richardson <[email protected]>CC Thomas Monjalon <[email protected]>--- ...rimental-syms.sh => check-special-syms.sh} | 24 ++++++++++++++++++- lib/librte_eal/common/include/rte_compat.h | 12 ++++++++++ mk/internal/rte.compile-pre.mk | 6 ++--- mk/target/generic/rte.vars.mk | 2 +- 4 files changed, 39 insertions(+), 5 deletions(-) rename buildtools/{check-experimental-syms.sh => check-special-syms.sh} (53%)
....
Quoted 21 lines
lib/librte_eal/common/include/rte_compat.h +12
-0@@ -89,4 +89,16 @@ __attribute__((section(".text.experimental")))#endif +/* +* __rte_internal tags mark functions as internal only, If specified in public +* header files, this tag will resolve to an error directive, preventing +* external applications from attempting to make calls to functions not meant +* for consumption outside the dpdk library +*/ +#ifdef BUILDING_RTE_SDK +#define __rte_internal __attribute__((section(".text.internal"))) +#else +#define __rte_internal __attribute__((error("This function cannot be used outside of the core DPDK library"), \ + section(".text.internal"))) +#endif
Quoted 1 line
#endif /* _RTE_COMPAT_H_ */
Since struct definition is also a kind of ABI (am I right ? ;-) ), like:
drivers/bus/pci/rte_bus_pci.h
struct rte_pci_device {
...
struct rte_intr_handle vfio_req_intr_handle;
/**< Handler of VFIO request interrupt */
} __rte_internal;Then will capture the errors anyway by using one of __rte_internal definition.
error: 'section' attribute does not apply to types [-Werror=attributes]
error: 'error' attribute does not apply to types
Quoted 1 line
2.20.1
Re: [dpdk-dev] [PATCH v2 01/10] Add __rte_internal tag for functions and version target
Quoted 50 lines · Haiyue
Hi Neil,
Quoted 38 lines
-----Original Message-----
From: dev <[email protected]> On Behalf Of Neil Horman
Sent: Thursday, June 13, 2019 22:24
To: [email protected]
Cc: Neil Horman <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>; Richardson, Bruce <[email protected]>; Thomas Monjalon <[email protected]>
Subject: [dpdk-dev] [PATCH v2 01/10] Add__rte_internaltag for functions and version targetThis tag is meant to be used on function prototypes to identify functions that are only meant to be used by internal DPDK libraries (i.e. libraries that are built while building the SDK itself, as identified by the defining of the
BUILDING_RTE_SDKmacro). When that flag is not set, it will resolve to an error function attribute, causing build breakage for any compilation unit attempting to build itValidate the use of this tag in much the same way we validate
__rte_experimental. By adding an INTERNAL version to library map files, we can exempt internal-only functions from ABI checking, and handle them to ensure that symbols we wish to only be for internal use between dpdk libraries are properly tagged with__rte_experimentalNote this patch updates the
check-experimental-syms.shscript, which normally only check the EXPERIMENTAL section to also check the INTERNAL section now. As such its been renamed to the now more appropriatecheck-special-syms.shSigned-off-by Neil Horman <[email protected]>CC Jerin Jacob Kollanukkaran <[email protected]>CC Bruce Richardson <[email protected]>CC Thomas Monjalon <[email protected]>--- ...rimental-syms.sh => check-special-syms.sh} | 24 ++++++++++++++++++- lib/librte_eal/common/include/rte_compat.h | 12 ++++++++++ mk/internal/rte.compile-pre.mk | 6 ++--- mk/target/generic/rte.vars.mk | 2 +- 4 files changed, 39 insertions(+), 5 deletions(-) rename buildtools/{check-experimental-syms.sh => check-special-syms.sh} (53%)....
Quoted 8 lines
lib/librte_eal/common/include/rte_compat.h +0
-0@@ -89,4 +89,16 @@ __attribute__((section(".text.experimental")))#endif
Quoted 17 lines
Quoted 13 lines
lib/librte_eal/common/include/rte_compat.h +12
-0+/* +* __rte_internal tags mark functions as internal only, If specified in public +* header files, this tag will resolve to an error directive, preventing +* external applications from attempting to make calls to functions not meant +* for consumption outside the dpdk library +*/ +#ifdef BUILDING_RTE_SDK +#define __rte_internal __attribute__((section(".text.internal"))) +#else +#define __rte_internal __attribute__((error("This function cannot be used outside of the core DPDK library"), \ + section(".text.internal"))) +#endif Quoted 1 line
#endif /* _RTE_COMPAT_H_ */
Since struct definition is also a kind of ABI (am I right ? ;-) ), like:
Yes, thats correct, which is why I've advocated for making structs opaque as part of the abi, but I suppose thats not where we are. :)
Quoted 11 lines
drivers/bus/pci/rte_bus_pci.hstruct rte_pci_device { ... struct rte_intr_handle vfio_req_intr_handle; /**< Handler of VFIO request interrupt */ } __rte_internal;Then will capture the errors anyway by using one of
__rte_internaldefinition.
error: 'section' attribute does not apply to types [-Werror=attributes]
error: 'error' attribute does not apply to types
As it is currently written, the __rte_internal macro is only written to work on functions. If you don't want a struct to be part of the ABI, we would need to either:
a) make a simmilar macro (say __rte_internal_data) which uses a simmilar gcc attibute to catch external usage.
or
b) just move the strucute definition to a location that isn't exposed as part of the external ABI
Neil
Quoted 3 lines
Quoted 1 line
2.20.1
Re: [dpdk-dev] [PATCH v2 01/10] Add __rte_internal tag for functions and version target
Quoted 80 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Friday, April 17, 2020 10:38
To: Wang, Haiyue <[email protected]>
Cc: [email protected]; Jerin Jacob Kollanukkaran <[email protected]>; Richardson, Bruce <[email protected]>; Thomas Monjalon <[email protected]>; David Marchand <[email protected]>; Yigit, Ferruh <[email protected]>
Subject: Re: [dpdk-dev] [PATCH v2 01/10] Add__rte_internaltag for functions and version targetQuoted 7 lines · Haiyue
Hi Neil,
Quoted 5 lines
-----Original Message-----
From: dev <[email protected]> On Behalf Of Neil Horman
Sent: Thursday, June 13, 2019 22:24
To: [email protected]
Cc: Neil Horman <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>;Richardson,
Quoted 33 lines
Quoted 25 lines
Bruce <[email protected]>; Thomas Monjalon <[email protected]>
Subject: [dpdk-dev] [PATCH v2 01/10] Add__rte_internaltag for functions and version targetThis tag is meant to be used on function prototypes to identify functions that are only meant to be used by internal DPDK libraries (i.e. libraries that are built while building the SDK itself, as identified by the defining of the
BUILDING_RTE_SDKmacro). When that flag is not set, it will resolve to an error function attribute, causing build breakage for any compilation unit attempting to build itValidate the use of this tag in much the same way we validate
__rte_experimental. By adding an INTERNAL version to library map files, we can exempt internal-only functions from ABI checking, and handle them to ensure that symbols we wish to only be for internal use between dpdk libraries are properly tagged with__rte_experimentalNote this patch updates the
check-experimental-syms.shscript, which normally only check the EXPERIMENTAL section to also check the INTERNAL section now. As such its been renamed to the now more appropriatecheck-special-syms.shSigned-off-by Neil Horman <[email protected]>CC Jerin Jacob Kollanukkaran <[email protected]>CC Bruce Richardson <[email protected]>CC Thomas Monjalon <[email protected]>Quoted 8 lines
--- ...rimental-syms.sh => check-special-syms.sh} | 24 ++++++++++++++++++- lib/librte_eal/common/include/rte_compat.h | 12 ++++++++++ mk/internal/rte.compile-pre.mk | 6 ++--- mk/target/generic/rte.vars.mk | 2 +- 4 files changed, 39 insertions(+), 5 deletions(-) rename buildtools/{check-experimental-syms.sh => check-special-syms.sh} (53%)Quoted 3 lines
....
Review
Quoted 1 line
(quoted diff) +0
-0
b/lib/librte_eal/common/include/rte_compat.hQuoted 7 lines
Quoted 7 lines
index
92ff28faf..739e8485c100644lib/librte_eal/common/include/rte_compat.h +0
-0@@ -89,4 +89,16 @@ __attribute__((section(".text.experimental")))#endif Quoted 10 lines
lib/librte_eal/common/include/rte_compat.h __attribute__((section(".text.experimental")))
Quoted 10 lines
lib/librte_eal/common/include/rte_compat.h +10
-0+/* +* __rte_internal tags mark functions as internal only, If specified in public +* header files, this tag will resolve to an error directive, preventing +* external applications from attempting to make calls to functions not meant +* for consumption outside the dpdk library +*/ +#ifdef BUILDING_RTE_SDK +#define __rte_internal __attribute__((section(".text.internal"))) +#else +#define __rte_internal __attribute__((error("This function cannot be used outside of the coreDPDK
Quoted 7 lines
Quoted 4 lines
lib/librte_eal/common/include/rte_compat.h +2
-0library"), \ + section(".text.internal"))) +#endif #endif /* _RTE_COMPAT_H_ */
Since struct definition is also a kind of ABI (am I right ? ;-) ), like:
Quoted 3 lines
Yes, thats correct, which is why I've advocated for making structs opaque as part of the abi, but I suppose thats not where we are. :)
Make sense, normally structs can't live alone without function API. A little paranoid for type only ABI checking. ;-) And this definition is good for ABI checking as we did it for EXPERIMENTAL.
Thanks!
Haiyue
Quoted 28 lines
Quoted 11 lines
drivers/bus/pci/rte_bus_pci.hstruct rte_pci_device { ... struct rte_intr_handle vfio_req_intr_handle; /**< Handler of VFIO request interrupt */ } __rte_internal;Then will capture the errors anyway by using one of
__rte_internaldefinition.
error: 'section' attribute does not apply to types [-Werror=attributes]
error: 'error' attribute does not apply to typesAs it is currently written, the
__rte_internalmacro is only written to work on functions. If you don't want a struct to be part of the ABI, we would need to either:a) make a simmilar macro (say
__rte_internal_data) which uses a simmilar gcc attibute to catch external usage.or
b) just move the strucute definition to a location that isn't exposed as part of the external ABI
Neil
Quoted 3 lines
Quoted 1 line
2.20.1
PATCH v2 2/10 meson: add BUILDING_RTE_SDK
In reply to Neil HormanThe __rte_internal macro is defined dependent on the value of the build environment variable BUILDING_RTE_SDK. This variable was set in the Makefile environment but not the meson environment, so lets reconcile the two by defining it for meson in the lib and drivers directories, but not the examples/apps directories, which should be treated as they are not part of the core DPDK library
--- drivers/meson.build | 1 + lib/meson.build | 1 + 2 files changed, 2 insertions(+)
drivers/meson.build +1 -0
@@ -23,6 +23,7 @@ endif # specify -D_GNU_SOURCE unconditionally default_cflags += '-D_GNU_SOURCE' +default_cflags += '-DBUILDING_RTE_SDK' foreach class:dpdk_driver_classes drivers = []
lib/meson.build +1 -0
@@ -35,6 +35,7 @@ if is_windows endif default_cflags = machine_args +default_cflags += '-DBUILDING_RTE_SDK' if cc.has_argument('-Wno-format-truncation') default_cflags += '-Wno-format-truncation' endif
Signature · 1 line
2.20.1
Re: [dpdk-dev] [PATCH v2 02/10] meson: add BUILDING_RTE_SDK
Quoted 33 lines · Neil Horman
The
__rte_internalmacro is defined dependent on the value of the build environment variableBUILDING_RTE_SDK. This variable was set in theMakefileenvironment but not the meson environment, so lets reconcile the two by defining it for meson in the lib and drivers directories, but not the examples/apps directories, which should be treated as they are not part of the core DPDK librarySigned-off-by Neil Horman <[email protected]>CC Jerin Jacob Kollanukkaran <[email protected]>CC Bruce Richardson <[email protected]>CC Thomas Monjalon <[email protected]>--- drivers/meson.build | 1 + lib/meson.build | 1 + 2 files changed, 2 insertions(+)drivers/meson.build +1
-0@@ -23,6 +23,7 @@ endif # specify -D_GNU_SOURCE unconditionally default_cflags += '-D_GNU_SOURCE' +default_cflags += '-DBUILDING_RTE_SDK' foreach class:dpdk_driver_classes drivers = []lib/meson.build +0
-0
lib/meson.build if is_windows · line 41
Quoted 7 lines
lib/meson.build +1
-0@@ -35,6 +35,7 @@ if is_windows endif default_cflags = machine_args +default_cflags += '-DBUILDING_RTE_SDK' if cc.has_argument('-Wno-format-truncation') default_cflags += '-Wno-format-truncation' endif
While this will work, it's not something that individual components should ever need to override so I think using "add_project_arguments()" function is a better way to add this to the C builds.
add_project_arguments('-DBUILDING_RTE_SDK', language: 'c')
Ref: http://mesonbuild.com/Adding-arguments.html
Ref: http://mesonbuild.com/Reference-manual.html#add_project_arguments
/Bruce
Re: [dpdk-dev] [PATCH v2 02/10] meson: add BUILDING_RTE_SDK
Quoted 36 lines · Bruce Richardson
Quoted 35 lines · Neil Horman
The
__rte_internalmacro is defined dependent on the value of the build environment variableBUILDING_RTE_SDK. This variable was set in theMakefileenvironment but not the meson environment, so lets reconcile the two by defining it for meson in the lib and drivers directories, but not the examples/apps directories, which should be treated as they are not part of the core DPDK librarySigned-off-by Neil Horman <[email protected]>CC Jerin Jacob Kollanukkaran <[email protected]>CC Bruce Richardson <[email protected]>CC Thomas Monjalon <[email protected]>--- drivers/meson.build | 1 + lib/meson.build | 1 + 2 files changed, 2 insertions(+)drivers/meson.build +1
-0@@ -23,6 +23,7 @@ endif # specify -D_GNU_SOURCE unconditionally default_cflags += '-D_GNU_SOURCE' +default_cflags += '-DBUILDING_RTE_SDK' foreach class:dpdk_driver_classes drivers = []lib/meson.build +0
-0@@ -35,6 +35,7 @@ if is_windows endif
Quoted 10 lines
lib/meson.build if is_windows · line 41
Quoted 5 lines
lib/meson.build +1
-0default_cflags = machine_args +default_cflags += '-DBUILDING_RTE_SDK' if cc.has_argument('-Wno-format-truncation') default_cflags += '-Wno-format-truncation' endifWhile this will work, it's not something that individual components should ever need to override so I think using "
add_project_arguments()" function is a better way to add this to the C builds.
That sounds like it makes sense to me, but reading the documentation for meson, I'm not sure I see the behavioral difference. Can you elaborate on how add_project_arguments behaves differently here?
Neil
Quoted 7 lines
add_project_arguments('-DBUILDING_RTE_SDK', language: 'c')Ref: http://mesonbuild.com/Adding-arguments.html
Ref: http://mesonbuild.com/Reference-manual.html#add_project_arguments/Bruce
Re: [dpdk-dev] [PATCH v2 02/10] meson: add BUILDING_RTE_SDK
Quoted 47 lines · Neil Horman
Quoted 36 lines · Bruce Richardson
Quoted 35 lines · Neil Horman
The
__rte_internalmacro is defined dependent on the value of the build environment variableBUILDING_RTE_SDK. This variable was set in theMakefileenvironment but not the meson environment, so lets reconcile the two by defining it for meson in the lib and drivers directories, but not the examples/apps directories, which should be treated as they are not part of the core DPDK librarySigned-off-by Neil Horman <[email protected]>CC Jerin Jacob Kollanukkaran <[email protected]>CC Bruce Richardson <[email protected]>CC Thomas Monjalon <[email protected]>--- drivers/meson.build | 1 + lib/meson.build | 1 + 2 files changed, 2 insertions(+)drivers/meson.build +1
-0@@ -23,6 +23,7 @@ endif # specify -D_GNU_SOURCE unconditionally default_cflags += '-D_GNU_SOURCE' +default_cflags += '-DBUILDING_RTE_SDK' foreach class:dpdk_driver_classes drivers = []lib/meson.build +0
-0@@ -35,6 +35,7 @@ if is_windows endifQuoted 10 lines
lib/meson.build if is_windows · line 41
Quoted 5 lines
lib/meson.build +1
-0default_cflags = machine_args +default_cflags += '-DBUILDING_RTE_SDK' if cc.has_argument('-Wno-format-truncation') default_cflags += '-Wno-format-truncation' endifWhile this will work, it's not something that individual components should ever need to override so I think using "
add_project_arguments()" function is a better way to add this to the C builds.
Quoted 4 lines
That sounds like it makes sense to me, but reading the documentation for meson, I'm not sure I see the behavioral difference. Can you elaborate on how
add_project_argumentsbehaves differently here?
The end result should be the same. The key differences are:
1. Only ever needs to be set in one place
2. Cannot be overridden by the individual objects in the build.
So it's just slightly cleaner. If you prefer your existing approach, I'm ok with that.
/Bruce
Re: [dpdk-dev] [PATCH v2 02/10] meson: add BUILDING_RTE_SDK
Quoted 52 lines · Bruce Richardson
Quoted 47 lines · Neil Horman
Quoted 36 lines · Bruce Richardson
Quoted 35 lines · Neil Horman
The
__rte_internalmacro is defined dependent on the value of the build environment variableBUILDING_RTE_SDK. This variable was set in theMakefileenvironment but not the meson environment, so lets reconcile the two by defining it for meson in the lib and drivers directories, but not the examples/apps directories, which should be treated as they are not part of the core DPDK librarySigned-off-by Neil Horman <[email protected]>CC Jerin Jacob Kollanukkaran <[email protected]>CC Bruce Richardson <[email protected]>CC Thomas Monjalon <[email protected]>--- drivers/meson.build | 1 + lib/meson.build | 1 + 2 files changed, 2 insertions(+)drivers/meson.build +1
-0@@ -23,6 +23,7 @@ endif # specify -D_GNU_SOURCE unconditionally default_cflags += '-D_GNU_SOURCE' +default_cflags += '-DBUILDING_RTE_SDK' foreach class:dpdk_driver_classes drivers = []lib/meson.build +0
-0@@ -35,6 +35,7 @@ if is_windows endifQuoted 10 lines
lib/meson.build if is_windows · line 41
Quoted 5 lines
lib/meson.build +1
-0default_cflags = machine_args +default_cflags += '-DBUILDING_RTE_SDK' if cc.has_argument('-Wno-format-truncation') default_cflags += '-Wno-format-truncation' endifWhile this will work, it's not something that individual components should ever need to override so I think using "
add_project_arguments()" function is a better way to add this to the C builds.Quoted 4 lines
That sounds like it makes sense to me, but reading the documentation for meson, I'm not sure I see the behavioral difference. Can you elaborate on how
add_project_argumentsbehaves differently here?
Quoted 7 lines
The end result should be the same. The key differences are:
1. Only ever needs to be set in one place
2. Cannot be overridden by the individual objects in the build.So it's just slightly cleaner. If you prefer your existing approach, I'm ok with that.
No, I like the aspect of item 2, though, just for clarification, do we really only need to set it at the top level? I ask because it seems that BUILDING_RTE_SDK should be set for the lib and drivers subtrees, but not the app subtree, as that tree, being the location of our example code, should be treated as non-core dpdk libraries, and so should throw an error if any code there attempts to use a dpdk internal only function. Or is there some other magic afoot in that subtree?
Neil
Quoted 2 lines
/Bruce
Re: [dpdk-dev] [PATCH v2 02/10] meson: add BUILDING_RTE_SDK
Quoted 61 lines · Neil Horman
Quoted 52 lines · Bruce Richardson
Quoted 47 lines · Neil Horman
Quoted 36 lines · Bruce Richardson
Quoted 35 lines · Neil Horman
The
__rte_internalmacro is defined dependent on the value of the build environment variableBUILDING_RTE_SDK. This variable was set in theMakefileenvironment but not the meson environment, so lets reconcile the two by defining it for meson in the lib and drivers directories, but not the examples/apps directories, which should be treated as they are not part of the core DPDK librarySigned-off-by Neil Horman <[email protected]>CC Jerin Jacob Kollanukkaran <[email protected]>CC Bruce Richardson <[email protected]>CC Thomas Monjalon <[email protected]>--- drivers/meson.build | 1 + lib/meson.build | 1 + 2 files changed, 2 insertions(+)drivers/meson.build +1
-0@@ -23,6 +23,7 @@ endif # specify -D_GNU_SOURCE unconditionally default_cflags += '-D_GNU_SOURCE' +default_cflags += '-DBUILDING_RTE_SDK' foreach class:dpdk_driver_classes drivers = []lib/meson.build +0
-0@@ -35,6 +35,7 @@ if is_windows endifQuoted 10 lines
lib/meson.build if is_windows · line 41
Quoted 5 lines
lib/meson.build +1
-0default_cflags = machine_args +default_cflags += '-DBUILDING_RTE_SDK' if cc.has_argument('-Wno-format-truncation') default_cflags += '-Wno-format-truncation' endifWhile this will work, it's not something that individual components should ever need to override so I think using "
add_project_arguments()" function is a better way to add this to the C builds.Quoted 4 lines
That sounds like it makes sense to me, but reading the documentation for meson, I'm not sure I see the behavioral difference. Can you elaborate on how
add_project_argumentsbehaves differently here?Quoted 7 lines
The end result should be the same. The key differences are:
1. Only ever needs to be set in one place
2. Cannot be overridden by the individual objects in the build.So it's just slightly cleaner. If you prefer your existing approach, I'm ok with that.
Quoted 8 lines
No, I like the aspect of item 2, though, just for clarification, do we really only need to set it at the top level? I ask because it seems that
BUILDING_RTE_SDKshould be set for the lib and drivers subtrees, but not the app subtree, as that tree, being the location of our example code, should be treated as non-core dpdk libraries, and so should throw an error if any code there attempts to use a dpdk internal only function. Or is there some other magic afoot in that subtree?
Good point. Keep your original patch as-is so.
/Bruce
Re: [dpdk-dev] [PATCH v2 02/10] meson: add BUILDING_RTE_SDK
In reply to Neil HormanQuoted 11 lines · Neil Horman
The
__rte_internalmacro is defined dependent on the value of the build environment variableBUILDING_RTE_SDK. This variable was set in theMakefileenvironment but not the meson environment, so lets reconcile the two by defining it for meson in the lib and drivers directories, but not the examples/apps directories, which should be treated as they are not part of the core DPDK librarySigned-off-by Neil Horman <[email protected]>CC Jerin Jacob Kollanukkaran <[email protected]>CC Bruce Richardson <[email protected]>CC Thomas Monjalon <[email protected]>
PATCH v2 3/10 Exempt INTERNAL symbols from checking
In reply to Neil HormanNo need to restrict the ABI on symbols that are only used by core libraries
--- devtools/check-symbol-change.sh | 7 +++++++ 1 file changed, 7 insertions(+)
devtools/check-symbol-change.sh +7 -0
@@ -93,6 +93,13 @@ check_for_rule_violations() if [ "$ar" = "add" ] then + if [ "$secname" = "INTERNAL" ] + then + # these are absolved from any further checking + echo "Skipping symbol $symname in INTERNAL" + continue + fi + if [ "$secname" = "unknown" ] then # Just inform the user of this occurrence, but
Signature · 1 line
2.20.1
PATCH v2 4/10 mark dpaa driver internal-only symbols with __rte_internal
In reply to Neil Hormanmake use of the new __rte_internal tag to specify symbols that should only be used by dpdk provided libraries (as specified by the BUILDING_RTE_SDK cflag
--- drivers/bus/dpaa/include/fsl_bman.h | 12 ++--- drivers/bus/dpaa/include/fsl_fman.h | 50 +++++++++---------- drivers/bus/dpaa/include/fsl_qman.h | 60 +++++++++++------------ drivers/bus/dpaa/include/fsl_usd.h | 12 ++--- drivers/bus/dpaa/include/netcfg.h | 4 +- drivers/bus/dpaa/include/of.h | 6 +-- drivers/bus/dpaa/rte_bus_dpaa_version.map | 47 +++++++----------- drivers/net/dpaa/dpaa_ethdev.c | 4 +- drivers/net/dpaa/dpaa_ethdev.h | 4 +- drivers/net/dpaa/rte_pmd_dpaa_version.map | 8 +-- 10 files changed, 99 insertions(+), 108 deletions(-)
drivers/bus/dpaa/include/fsl_bman.h +6 -6
@@ -264,13 +264,13 @@ int bman_shutdown_pool(u32 bpid);* the structure provided by the caller can be released or reused after the * function returns. */ -struct bman_pool *bman_new_pool (const struct bman_pool_params *params); +struct bman_pool __rte_internal *bman_new_pool(const struct bman_pool_params *params);/** * bman_free_pool - Deallocates a Buffer Pool object * @pool: the pool object to release */ -void bman_free_pool (struct bman_pool *pool); +void __rte_internal bman_free_pool(struct bman_pool *pool);/** * bman_get_params - Returns a pool object's parameters.
@@ -279,7 +279,7 @@ void bman_free_pool(struct bman_pool *pool);* The returned pointer refers to state within the pool object so must not be * modified and can no longer be read once the pool object is destroyed. */ -const struct bman_pool_params *bman_get_params (const struct bman_pool *pool); +const struct bman_pool_params __rte_internal *bman_get_params(const struct bman_pool *pool);/** * bman_release - Release buffer(s) to the buffer pool
@@ -289,7 +289,7 @@ const struct bman_pool_params *bman_get_params(const struct bman_pool *pool);* @flags: bit-mask of BMAN_RELEASE_FLAG_*** options * */ -int bman_release (struct bman_pool *pool,const struct bm_buffer *bufs,u8 num, +int __rte_internal bman_release(struct bman_pool *pool, const struct bm_buffer *bufs, u8 num,u32 flags);/**
@@ -302,7 +302,7 @@ int bman_release(struct bman_pool *pool, const struct bm_buffer *bufs, u8 num,* The return value will be the number of buffers obtained from the pool, or a * negative error code if a h/w error or pool starvation was encountered. */ -int bman_acquire (struct bman_pool *pool,struct bm_buffer *bufs,u8 num, +int __rte_internal bman_acquire(struct bman_pool *pool, struct bm_buffer *bufs, u8 num,u32 flags);/**
@@ -317,7 +317,7 @@ int bman_query_pools(struct bm_pool_state *state);* * Return the number of the free buffers */ -u32 bman_query_free_buffers (struct bman_pool *pool); +u32 __rte_internal bman_query_free_buffers(struct bman_pool *pool);/** * bman_update_pool_thresholds - Change the buffer pool's depletion thresholds
drivers/bus/dpaa/include/fsl_fman.h +25 -25
113 lines not on this page · raw message
drivers/bus/dpaa/include/fsl_qman.h +30 -30
235 lines not on this page · raw message
drivers/bus/dpaa/include/fsl_usd.h +6 -6
@@ -56,7 +56,7 @@ int bman_allocate_raw_portal(struct dpaa_raw_portal *portal);int bman_free_raw_portal (struct dpaa_raw_portal *portal);/* Obtain thread-local UIO file-descriptors */ -int qman_thread_fd (void ); +int __rte_internal qman_thread_fd(void);int bman_thread_fd (void );/* Post-process interrupts. NB, the kernel IRQ handler disables the interrupt
@@ -64,14 +64,14 @@ int bman_thread_fd(void);* processing is complete. As such, it is essential to call this before going * into another blocking read/select/poll. */ -void qman_thread_irq (void ); -void bman_thread_irq (void ); +void __rte_internal qman_thread_irq(void); +void __rte_internal bman_thread_irq(void); -void qman_clear_irq (void ); +void __rte_internal qman_clear_irq(void);/* Global setup */ -int qman_global_init (void ); -int bman_global_init (void ); +int __rte_internal qman_global_init(void); +int __rte_internal bman_global_init(void);/* Direct portal create and destroy */ struct qman_portal *fsl_qman_portal_create (void );
drivers/bus/dpaa/include/netcfg.h +2 -2
@@ -46,12 +46,12 @@ struct netcfg_interface {* cfg_file: FMC config XML file * Returns the configuration information in newly allocated memory. */ -struct netcfg_info *netcfg_acquire (void ); +struct netcfg_info __rte_internal *netcfg_acquire(void);/* cfg_ptr: configuration information pointer. * Frees the resources allocated by the configuration layer. */ -void netcfg_release (struct netcfg_info *cfg_ptr); +void __rte_internal netcfg_release(struct netcfg_info *cfg_ptr);#ifdef RTE_LIBRTE_DPAA_DEBUG_DRIVER /* cfg_ptr: configuration information pointer.
drivers/bus/dpaa/include/of.h +3 -3
@@ -87,7 +87,7 @@ struct dt_file {uint64_t buf[OF_FILE_BUF_MAX >>3 ]; }; -const struct device_node *of_find_compatible_node ( +const __rte_internal struct device_node *of_find_compatible_node(const struct device_node *from,const char *type__always_unused ,const char *compatible)
@@ -98,7 +98,7 @@ const struct device_node *of_find_compatible_node( dev_node !=NULL ; \ dev_node = of_find_compatible_node(dev_node, type, compatible)) -const void *of_get_property (const struct device_node *from,const char *name, +const __rte_internal void *of_get_property(const struct device_node *from, const char *name,size_t *lenp) __attribute__((nonnull (2 )));bool of_device_is_available (const struct device_node *dev_node);
@@ -109,7 +109,7 @@ const struct device_node *of_get_parent(const struct device_node *dev_node);const struct device_node *of_get_next_child (const struct device_node *dev_node,const struct device_node *prev); -const void *of_get_mac_address (const struct device_node *np); +const void __rte_internal *of_get_mac_address(const struct device_node *np);#define for_each_child_node (parent, child) \ for (child = of_get_next_child(parent, NULL); child != NULL; \
drivers/bus/dpaa/rte_bus_dpaa_version.map +18 -29
75 lines not on this page · raw message
drivers/net/dpaa/dpaa_ethdev.c +2 -2
@@ -694,7 +694,7 @@ int dpaa_eth_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_idx, }int -dpaa_eth_eventq_attach (const struct rte_eth_dev *dev, +__rte_internal dpaa_eth_eventq_attach(const struct rte_eth_dev *dev,int eth_rx_queue_id,u16 ch_id,const struct rte_event_eth_rx_adapter_queue_conf *queue_conf)
@@ -758,7 +758,7 @@ dpaa_eth_eventq_attach(const struct rte_eth_dev *dev, }int -dpaa_eth_eventq_detach (const struct rte_eth_dev *dev, +__rte_internal dpaa_eth_eventq_detach(const struct rte_eth_dev *dev,int eth_rx_queue_id) {struct qm_mcc_initfq opts;
drivers/net/dpaa/dpaa_ethdev.h +2 -2
@@ -166,13 +166,13 @@ struct dpaa_if_stats { };int -dpaa_eth_eventq_attach (const struct rte_eth_dev *dev, +__rte_internal dpaa_eth_eventq_attach(const struct rte_eth_dev *dev,int eth_rx_queue_id,u16 ch_id,const struct rte_event_eth_rx_adapter_queue_conf *queue_conf);int -dpaa_eth_eventq_detach (const struct rte_eth_dev *dev, +__rte_internal dpaa_eth_eventq_detach(const struct rte_eth_dev *dev,int eth_rx_queue_id);enum qman_cb_dqrr_result
drivers/net/dpaa/rte_pmd_dpaa_version.map +5 -3
@@ -5,8 +5,10 @@ DPDK_17.11 { DPDK_18.08 { global: - - dpaa_eth_eventq_attach; - dpaa_eth_eventq_detach; rte_pmd_dpaa_set_tx_loopback; } DPDK_17.11; + +INTERNAL { + dpaa_eth_eventq_attach; + dpaa_eth_eventq_detach; +};
Signature · 1 line
2.20.1
PATCH v2 5/10 fslmc: identify internal only functions and tag them as __rte_internal
In reply to Neil HormanIdentify functions in fslmc bus driver which are internal (based on their not having an rte_ prefix) and tag them with __rte_internal
--- drivers/bus/fslmc/fslmc_bus.c | 2 +- drivers/bus/fslmc/mc/dpbp.c | 12 +- drivers/bus/fslmc/mc/dpci.c | 6 +- drivers/bus/fslmc/mc/dpcon.c | 4 +- drivers/bus/fslmc/mc/dpdmai.c | 16 +- drivers/bus/fslmc/mc/dpio.c | 18 +- drivers/bus/fslmc/mc/dpmng.c | 4 +- drivers/bus/fslmc/mc/fsl_dpbp.h | 12 +- drivers/bus/fslmc/mc/fsl_dpci.h | 6 +- drivers/bus/fslmc/mc/fsl_dpcon.h | 4 +- drivers/bus/fslmc/mc/fsl_dpdmai.h | 16 +- drivers/bus/fslmc/mc/fsl_dpio.h | 18 +- drivers/bus/fslmc/mc/fsl_dpmng.h | 4 +- drivers/bus/fslmc/mc/fsl_mc_cmd.h | 4 +- drivers/bus/fslmc/mc/mc_sys.c | 2 +- drivers/bus/fslmc/portal/dpaa2_hw_dpbp.c | 6 +- drivers/bus/fslmc/portal/dpaa2_hw_dpio.c | 12 +- drivers/bus/fslmc/portal/dpaa2_hw_dpio.h | 10 +- drivers/bus/fslmc/portal/dpaa2_hw_pvt.h | 6 +- .../bus/fslmc/qbman/include/fsl_qbman_debug.h | 4 +- .../fslmc/qbman/include/fsl_qbman_portal.h | 80 +++---- drivers/bus/fslmc/qbman/qbman_debug.c | 4 +- drivers/bus/fslmc/qbman/qbman_portal.c | 82 +++---- drivers/bus/fslmc/rte_bus_fslmc_version.map | 201 +++++++++--------- 24 files changed, 269 insertions(+), 264 deletions(-)
drivers/bus/fslmc/fslmc_bus.c +1 -1
@@ -28,7 +28,7 @@ int dpaa2_logtype_bus;#define FSLMC_BUS_NAME fslmcstruct rte_fslmc_bus rte_fslmc_bus; -uint8_t dpaa2_virt_mode; +uint8_t __rte_internal dpaa2_virt_mode;uint32_t rte_fslmc_get_device_count (enum rte_dpaa2_dev_type device_type)
drivers/bus/fslmc/mc/dpbp.c +6 -6
@@ -26,7 +26,7 @@* * Return: '0' on Success; Error code otherwise. */ -int dpbp_open (struct fsl_mc_io *mc_io, +int __rte_internal dpbp_open(struct fsl_mc_io *mc_io,uint32_t cmd_flags,int dpbp_id,uint16_t *token)
@@ -157,7 +157,7 @@ int dpbp_destroy(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int dpbp_enable (struct fsl_mc_io *mc_io, +int __rte_internal dpbp_enable(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token) {
@@ -179,7 +179,7 @@ int dpbp_enable(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int dpbp_disable (struct fsl_mc_io *mc_io, +int __rte_internal dpbp_disable(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token) {
@@ -235,7 +235,7 @@ int dpbp_is_enabled(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int dpbp_reset (struct fsl_mc_io *mc_io, +int __rte_internal dpbp_reset(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token) {
@@ -258,7 +258,7 @@ int dpbp_reset(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int dpbp_get_attributes (struct fsl_mc_io *mc_io, +int __rte_internal dpbp_get_attributes(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token,struct dpbp_attr *attr)
@@ -329,7 +329,7 @@ int dpbp_get_api_version(struct fsl_mc_io *mc_io,* Return: '0' on Success; Error code otherwise. */ -int dpbp_get_num_free_bufs (struct fsl_mc_io *mc_io, +int __rte_internal dpbp_get_num_free_bufs(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token,uint32_t *num_free_bufs)
drivers/bus/fslmc/mc/dpci.c +3 -3
@@ -314,7 +314,7 @@ int dpci_get_attributes(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int dpci_set_rx_queue(struct fsl_mc_io *mc_io, +int __rte_internal dpci_set_rx_queue(struct fsl_mc_io *mc_io, uint32_t cmd_flags, uint16_t token, uint8_t priority,
@@ -476,7 +476,7 @@ int dpci_get_api_version(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int dpci_set_opr(struct fsl_mc_io *mc_io, +int __rte_internal dpci_set_opr(struct fsl_mc_io *mc_io, uint32_t cmd_flags, uint16_t token, uint8_t index,
@@ -514,7 +514,7 @@ int dpci_set_opr(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int dpci_get_opr(struct fsl_mc_io *mc_io, +int __rte_internal dpci_get_opr(struct fsl_mc_io *mc_io, uint32_t cmd_flags, uint16_t token, uint8_t index,
drivers/bus/fslmc/mc/dpcon.c +2 -2
@@ -25,7 +25,7 @@* * Return: '0' on Success; Error code otherwise. */ -int dpcon_open (struct fsl_mc_io *mc_io, +int __rte_internal dpcon_open(struct fsl_mc_io *mc_io,uint32_t cmd_flags,int dpcon_id,uint16_t *token)
@@ -267,7 +267,7 @@ int dpcon_reset(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int dpcon_get_attributes (struct fsl_mc_io *mc_io, +int __rte_internal dpcon_get_attributes(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token,struct dpcon_attr *attr)
drivers/bus/fslmc/mc/dpdmai.c +8 -8
@@ -24,7 +24,7 @@* * Return: '0' on Success; Error code otherwise. */ -int dpdmai_open (struct fsl_mc_io *mc_io, +int __rte_internal dpdmai_open(struct fsl_mc_io *mc_io,uint32_t cmd_flags,int dpdmai_id,uint16_t *token)
@@ -62,7 +62,7 @@ int dpdmai_open(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int dpdmai_close (struct fsl_mc_io *mc_io, +int __rte_internal dpdmai_close(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token) {
@@ -170,7 +170,7 @@ int dpdmai_destroy(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int dpdmai_enable (struct fsl_mc_io *mc_io, +int __rte_internal dpdmai_enable(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token) {
@@ -193,7 +193,7 @@ int dpdmai_enable(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int dpdmai_disable (struct fsl_mc_io *mc_io, +int __rte_internal dpdmai_disable(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token) {
@@ -275,7 +275,7 @@ int dpdmai_reset(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int dpdmai_get_attributes (struct fsl_mc_io *mc_io, +int __rte_internal dpdmai_get_attributes(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token,struct dpdmai_attr *attr)
@@ -318,7 +318,7 @@ int dpdmai_get_attributes(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int dpdmai_set_rx_queue(struct fsl_mc_io *mc_io, +int __rte_internal dpdmai_set_rx_queue(struct fsl_mc_io *mc_io, uint32_t cmd_flags, uint16_t token, uint8_t queue_idx,
@@ -360,7 +360,7 @@ int dpdmai_set_rx_queue(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int dpdmai_get_rx_queue(struct fsl_mc_io *mc_io, +int __rte_internal dpdmai_get_rx_queue(struct fsl_mc_io *mc_io, uint32_t cmd_flags, uint16_t token, uint8_t queue_idx,
@@ -410,7 +410,7 @@ int dpdmai_get_rx_queue(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int dpdmai_get_tx_queue(struct fsl_mc_io *mc_io, +int __rte_internal dpdmai_get_tx_queue(struct fsl_mc_io *mc_io, uint32_t cmd_flags, uint16_t token, uint8_t queue_idx,
drivers/bus/fslmc/mc/dpio.c +9 -9
@@ -26,7 +26,7 @@* * Return: '0' on Success; Error code otherwise. */ -int dpio_open (struct fsl_mc_io *mc_io, +int __rte_internal dpio_open(struct fsl_mc_io *mc_io,uint32_t cmd_flags,int dpio_id,uint16_t *token)
@@ -61,7 +61,7 @@ int dpio_open(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int dpio_close (struct fsl_mc_io *mc_io, +int __rte_internal dpio_close(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token) {
@@ -173,7 +173,7 @@ int dpio_destroy(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise */ -int dpio_enable (struct fsl_mc_io *mc_io, +int __rte_internal dpio_enable(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token) {
@@ -196,7 +196,7 @@ int dpio_enable(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise */ -int dpio_disable (struct fsl_mc_io *mc_io, +int __rte_internal dpio_disable(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token) {
@@ -253,7 +253,7 @@ int dpio_is_enabled(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int dpio_reset (struct fsl_mc_io *mc_io, +int __rte_internal dpio_reset(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token) {
@@ -277,7 +277,7 @@ int dpio_reset(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise */ -int dpio_get_attributes (struct fsl_mc_io *mc_io, +int __rte_internal dpio_get_attributes(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token,struct dpio_attr *attr)
@@ -322,7 +322,7 @@ int dpio_get_attributes(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int dpio_set_stashing_destination (struct fsl_mc_io *mc_io, +int __rte_internal dpio_set_stashing_destination(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token,uint8_t sdest)
@@ -386,7 +386,7 @@ int dpio_get_stashing_destination(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int dpio_add_static_dequeue_channel(struct fsl_mc_io *mc_io, +int __rte_internal dpio_add_static_dequeue_channel(struct fsl_mc_io *mc_io, uint32_t cmd_flags, uint16_t token, int dpcon_id,
@@ -425,7 +425,7 @@ int dpio_add_static_dequeue_channel(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int dpio_remove_static_dequeue_channel (struct fsl_mc_io *mc_io, +int __rte_internal dpio_remove_static_dequeue_channel(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token,int dpcon_id)
drivers/bus/fslmc/mc/dpmng.c +2 -2
@@ -18,7 +18,7 @@* * Return: '0' on Success; Error code otherwise. */ -int mc_get_version (struct fsl_mc_io *mc_io, +int __rte_internal mc_get_version(struct fsl_mc_io *mc_io,uint32_t cmd_flags,struct mc_version *mc_ver_info) {
@@ -57,7 +57,7 @@ int mc_get_version(struct fsl_mc_io *mc_io,* * Return: '0' on Success; Error code otherwise. */ -int mc_get_soc_version (struct fsl_mc_io *mc_io, +int __rte_internal mc_get_soc_version(struct fsl_mc_io *mc_io,uint32_t cmd_flags,struct mc_soc_version *mc_platform_info) {
drivers/bus/fslmc/mc/fsl_dpbp.h +6 -6
@@ -14,7 +14,7 @@struct fsl_mc_io ; -int dpbp_open (struct fsl_mc_io *mc_io, +int __rte_internal dpbp_open(struct fsl_mc_io *mc_io,uint32_t cmd_flags,int dpbp_id,uint16_t *token);
@@ -42,11 +42,11 @@ int dpbp_destroy(struct fsl_mc_io *mc_io,uint32_t cmd_flags, uint32_t obj_id); -int dpbp_enable (struct fsl_mc_io *mc_io, +int __rte_internal dpbp_enable(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token); -int dpbp_disable (struct fsl_mc_io *mc_io, +int __rte_internal dpbp_disable(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token);
@@ -55,7 +55,7 @@ int dpbp_is_enabled(struct fsl_mc_io *mc_io,uint16_t token, int *en); -int dpbp_reset (struct fsl_mc_io *mc_io, +int __rte_internal dpbp_reset(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token);
@@ -70,7 +70,7 @@ struct dpbp_attr {uint16_t bpid; }; -int dpbp_get_attributes (struct fsl_mc_io *mc_io, +int __rte_internal dpbp_get_attributes(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token,struct dpbp_attr *attr);
@@ -88,7 +88,7 @@ int dpbp_get_api_version(struct fsl_mc_io *mc_io,uint16_t *major_ver, uint16_t *minor_ver); -int dpbp_get_num_free_bufs (struct fsl_mc_io *mc_io, +int __rte_internal dpbp_get_num_free_bufs(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token,uint32_t *num_free_bufs);
drivers/bus/fslmc/mc/fsl_dpci.h +3 -3
@@ -180,7 +180,7 @@ struct dpci_rx_queue_cfg {int order_preservation_en; }; -int dpci_set_rx_queue(struct fsl_mc_io *mc_io, +int __rte_internal dpci_set_rx_queue(struct fsl_mc_io *mc_io, uint32_t cmd_flags, uint16_t token, uint8_t priority,
@@ -227,14 +227,14 @@ int dpci_get_api_version(struct fsl_mc_io *mc_io,uint16_t *major_ver, uint16_t *minor_ver); -int dpci_set_opr (struct fsl_mc_io *mc_io, +int __rte_internal dpci_set_opr(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token,uint8_t index,uint8_t options,struct opr_cfg *cfg); -int dpci_get_opr(struct fsl_mc_io *mc_io, +int __rte_internal dpci_get_opr(struct fsl_mc_io *mc_io, uint32_t cmd_flags, uint16_t token, uint8_t index,
drivers/bus/fslmc/mc/fsl_dpcon.h +2 -2
@@ -19,7 +19,7 @@ struct fsl_mc_io;*/ #define DPCON_INVALID_DPIO_ID (int)(-1) -int dpcon_open (struct fsl_mc_io *mc_io, +int __rte_internal dpcon_open(struct fsl_mc_io *mc_io,uint32_t cmd_flags,int dpcon_id,uint16_t *token);
@@ -76,7 +76,7 @@ struct dpcon_attr {uint8_t num_priorities; }; -int dpcon_get_attributes (struct fsl_mc_io *mc_io, +int __rte_internal dpcon_get_attributes(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token,struct dpcon_attr *attr);
drivers/bus/fslmc/mc/fsl_dpdmai.h +8 -8
@@ -23,12 +23,12 @@ struct fsl_mc_io;*/ #define DPDMAI_ALL_QUEUES (uint8_t)(-1) -int dpdmai_open (struct fsl_mc_io *mc_io, +int __rte_internal dpdmai_open(struct fsl_mc_io *mc_io,uint32_t cmd_flags,int dpdmai_id,uint16_t *token); -int dpdmai_close (struct fsl_mc_io *mc_io, +int __rte_internal dpdmai_close(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token);
@@ -54,11 +54,11 @@ int dpdmai_destroy(struct fsl_mc_io *mc_io,uint32_t cmd_flags, uint32_t object_id); -int dpdmai_enable (struct fsl_mc_io *mc_io, +int __rte_internal dpdmai_enable(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token); -int dpdmai_disable (struct fsl_mc_io *mc_io, +int __rte_internal dpdmai_disable(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token);
@@ -82,7 +82,7 @@ struct dpdmai_attr {uint8_t num_of_queues; }; -int dpdmai_get_attributes (struct fsl_mc_io *mc_io, +int __rte_internal dpdmai_get_attributes(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token,struct dpdmai_attr *attr);
@@ -148,7 +148,7 @@ struct dpdmai_rx_queue_cfg { }; -int dpdmai_set_rx_queue(struct fsl_mc_io *mc_io, +int __rte_internal dpdmai_set_rx_queue(struct fsl_mc_io *mc_io, uint32_t cmd_flags, uint16_t token, uint8_t queue_idx,
@@ -168,7 +168,7 @@ struct dpdmai_rx_queue_attr {uint32_t fqid; }; -int dpdmai_get_rx_queue(struct fsl_mc_io *mc_io, +int __rte_internal dpdmai_get_rx_queue(struct fsl_mc_io *mc_io, uint32_t cmd_flags, uint16_t token, uint8_t queue_idx,
@@ -184,7 +184,7 @@ struct dpdmai_tx_queue_attr {uint32_t fqid; }; -int dpdmai_get_tx_queue(struct fsl_mc_io *mc_io, +int __rte_internal dpdmai_get_tx_queue(struct fsl_mc_io *mc_io, uint32_t cmd_flags, uint16_t token, uint8_t queue_idx,
drivers/bus/fslmc/mc/fsl_dpio.h +9 -9
@@ -13,12 +13,12 @@struct fsl_mc_io ; -int dpio_open (struct fsl_mc_io *mc_io, +int __rte_internal dpio_open(struct fsl_mc_io *mc_io,uint32_t cmd_flags,int dpio_id,uint16_t *token); -int dpio_close (struct fsl_mc_io *mc_io, +int __rte_internal dpio_close(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token);
@@ -57,11 +57,11 @@ int dpio_destroy(struct fsl_mc_io *mc_io,uint32_t cmd_flags, uint32_t object_id); -int dpio_enable (struct fsl_mc_io *mc_io, +int __rte_internal dpio_enable(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token); -int dpio_disable (struct fsl_mc_io *mc_io, +int __rte_internal dpio_disable(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token);
@@ -70,11 +70,11 @@ int dpio_is_enabled(struct fsl_mc_io *mc_io,uint16_t token, int *en); -int dpio_reset (struct fsl_mc_io *mc_io, +int __rte_internal dpio_reset(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token); -int dpio_set_stashing_destination (struct fsl_mc_io *mc_io, +int __rte_internal dpio_set_stashing_destination(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token,uint8_t sdest);
@@ -84,13 +84,13 @@ int dpio_get_stashing_destination(struct fsl_mc_io *mc_io,uint16_t token, uint8_t *sdest); -int dpio_add_static_dequeue_channel (struct fsl_mc_io *mc_io, +int __rte_internal dpio_add_static_dequeue_channel(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token,int dpcon_id,uint8_t *channel_index); -int dpio_remove_static_dequeue_channel (struct fsl_mc_io *mc_io, +int __rte_internal dpio_remove_static_dequeue_channel(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token,int dpcon_id);
@@ -119,7 +119,7 @@ struct dpio_attr {uint32_t clk; }; -int dpio_get_attributes (struct fsl_mc_io *mc_io, +int __rte_internal dpio_get_attributes(struct fsl_mc_io *mc_io,uint32_t cmd_flags,uint16_t token,struct dpio_attr *attr);
drivers/bus/fslmc/mc/fsl_dpmng.h +2 -2
@@ -34,7 +34,7 @@ struct mc_version {uint32_t revision; }; -int mc_get_version (struct fsl_mc_io *mc_io, +int __rte_internal mc_get_version(struct fsl_mc_io *mc_io,uint32_t cmd_flags,struct mc_version *mc_ver_info);
@@ -48,7 +48,7 @@ struct mc_soc_version {uint32_t pvr; }; -int mc_get_soc_version (struct fsl_mc_io *mc_io, +int __rte_internal mc_get_soc_version(struct fsl_mc_io *mc_io,uint32_t cmd_flags,struct mc_soc_version *mc_platform_info);#endif /* __FSL_DPMNG_H */
drivers/bus/fslmc/mc/fsl_mc_cmd.h +3 -1
@@ -10,6 +10,8 @@#include <rte_byteorder.h> #include <stdint.h> +#include "rte_compat.h" +#define MC_CMD_NUM_OF_PARAMS 7#define phys_addr_t uint64_t
@@ -80,7 +82,7 @@ enum mc_cmd_status {#define MC_CMD_HDR_FLAGS_MASK 0xFF00FF00 -int mc_send_command (struct fsl_mc_io *mc_io,struct mc_command *cmd); +int __rte_internal mc_send_command(struct fsl_mc_io *mc_io, struct mc_command *cmd); static inline uint64_t mc_encode_cmd_header(uint16_t cmd_id, uint32_t cmd_flags,
drivers/bus/fslmc/mc/mc_sys.c +1 -1
@@ -51,7 +51,7 @@ static int mc_status_to_error(enum mc_cmd_status status)return -EINVAL ; } -int mc_send_command (struct fsl_mc_io *mc_io,struct mc_command *cmd) +int __rte_internal mc_send_command(struct fsl_mc_io *mc_io, struct mc_command *cmd) {enum mc_cmd_status status;uint64_t response;
drivers/bus/fslmc/portal/dpaa2_hw_dpbp.c +3 -3
@@ -89,7 +89,7 @@ dpaa2_create_dpbp_device(int vdev_fd __rte_unused,return 0 ; } -struct dpaa2_dpbp_dev *dpaa2_alloc_dpbp_dev (void ) +struct dpaa2_dpbp_dev *__rte_internal dpaa2_alloc_dpbp_dev(void) {struct dpaa2_dpbp_dev *dpbp_dev =NULL ;
@@ -102,7 +102,7 @@ struct dpaa2_dpbp_dev *dpaa2_alloc_dpbp_dev(void)return dpbp_dev; } -void dpaa2_free_dpbp_dev (struct dpaa2_dpbp_dev *dpbp) +void __rte_internal dpaa2_free_dpbp_dev(struct dpaa2_dpbp_dev *dpbp) {struct dpaa2_dpbp_dev *dpbp_dev =NULL ;
@@ -115,7 +115,7 @@ void dpaa2_free_dpbp_dev(struct dpaa2_dpbp_dev *dpbp) } } -int dpaa2_dpbp_supported (void ) +int __rte_internal dpaa2_dpbp_supported(void) {if (TAILQ_EMPTY (&dpbp_dev_list))return -1 ;
drivers/bus/fslmc/portal/dpaa2_hw_dpio.c +6 -6
@@ -229,7 +229,7 @@ dpaa2_configure_stashing(struct dpaa2_dpio_dev *dpio_dev, int lcoreid)return 0 ; } -static struct dpaa2_dpio_dev *dpaa2_get_qbman_swp (int lcoreid) +static struct dpaa2_dpio_dev *__rte_internal dpaa2_get_qbman_swp(int lcoreid) {struct dpaa2_dpio_dev *dpio_dev =NULL ;int ret;
@@ -253,7 +253,7 @@ static struct dpaa2_dpio_dev *dpaa2_get_qbman_swp(int lcoreid) }int -dpaa2_affine_qbman_swp (void ) +__rte_internal dpaa2_affine_qbman_swp(void) {unsigned int lcore_id =rte_lcore_id ();uint64_t tid =syscall (SYS_gettid);
@@ -301,7 +301,7 @@ dpaa2_affine_qbman_swp(void) }int -dpaa2_affine_qbman_ethrx_swp (void ) +__rte_internal dpaa2_affine_qbman_ethrx_swp(void) {unsigned int lcore_id =rte_lcore_id ();uint64_t tid =syscall (SYS_gettid);
@@ -570,7 +570,7 @@ dpaa2_create_dpio_device(int vdev_fd, }void -dpaa2_free_dq_storage (struct queue_storage_info_t *q_storage) +__rte_internal dpaa2_free_dq_storage(struct queue_storage_info_t *q_storage) {int i =0 ;
@@ -581,7 +581,7 @@ dpaa2_free_dq_storage(struct queue_storage_info_t *q_storage) }int -dpaa2_alloc_dq_storage (struct queue_storage_info_t *q_storage) +__rte_internal dpaa2_alloc_dq_storage(struct queue_storage_info_t *q_storage) {int i =0 ;
@@ -601,7 +601,7 @@ dpaa2_alloc_dq_storage(struct queue_storage_info_t *q_storage) }uint32_t -dpaa2_free_eq_descriptors (void ) +__rte_internal dpaa2_free_eq_descriptors(void) {struct dpaa2_dpio_dev *dpio_dev =DPAA2_PER_LCORE_DPIO ;struct qbman_result *eqresp;
drivers/bus/fslmc/portal/dpaa2_hw_dpio.h +5 -5
@@ -38,21 +38,21 @@ extern uint8_t dpaa2_eqcr_size;extern struct dpaa2_io_portal_t dpaa2_io_portal[RTE_MAX_LCORE ];/* Affine a DPIO portal to current processing thread */ -int dpaa2_affine_qbman_swp (void ); +int __rte_internal dpaa2_affine_qbman_swp(void);/* Affine additional DPIO portal to current crypto processing thread */ -int dpaa2_affine_qbman_ethrx_swp (void ); +int __rte_internal dpaa2_affine_qbman_ethrx_swp(void);/* allocate memory for FQ - dq storage */ int -dpaa2_alloc_dq_storage (struct queue_storage_info_t *q_storage); +__rte_internal dpaa2_alloc_dq_storage(struct queue_storage_info_t *q_storage);/* free memory for FQ- dq storage */ void -dpaa2_free_dq_storage (struct queue_storage_info_t *q_storage); +__rte_internal dpaa2_free_dq_storage(struct queue_storage_info_t *q_storage);/* free the enqueue response descriptors */ uint32_t -dpaa2_free_eq_descriptors (void ); +__rte_internal dpaa2_free_eq_descriptors(void);#endif /* _DPAA2_HW_DPIO_H_ */
drivers/bus/fslmc/portal/dpaa2_hw_pvt.h +3 -3
@@ -418,9 +418,9 @@ void set_swp_active_dqs(uint16_t dpio_index, struct qbman_result *dqs) { rte_global_active_dqs_list[dpio_index].global_active_dqs = dqs; } -struct dpaa2_dpbp_dev *dpaa2_alloc_dpbp_dev (void ); -void dpaa2_free_dpbp_dev (struct dpaa2_dpbp_dev *dpbp); -int dpaa2_dpbp_supported (void ); +struct dpaa2_dpbp_dev *__rte_internal dpaa2_alloc_dpbp_dev(void); +void __rte_internal dpaa2_free_dpbp_dev(struct dpaa2_dpbp_dev *dpbp); +int __rte_internal dpaa2_dpbp_supported(void);struct dpaa2_dpci_dev *rte_dpaa2_alloc_dpci_dev (void );void rte_dpaa2_free_dpci_dev (struct dpaa2_dpci_dev *dpci);
drivers/bus/fslmc/qbman/include/fsl_qbman_debug.h +2 -2
@@ -24,7 +24,7 @@ uint8_t verb;uint8_t reserved2[29 ]; }; -int qbman_fq_query_state (struct qbman_swp *s,uint32_t fqid, +int __rte_internal qbman_fq_query_state(struct qbman_swp *s, uint32_t fqid,struct qbman_fq_query_np_rslt *r); -uint32_t qbman_fq_state_frame_count (const struct qbman_fq_query_np_rslt *r); +uint32_t __rte_internal qbman_fq_state_frame_count(const struct qbman_fq_query_np_rslt *r);uint32_t qbman_fq_state_byte_count (const struct qbman_fq_query_np_rslt *r);
drivers/bus/fslmc/qbman/include/fsl_qbman_portal.h +40 -40
313 lines not on this page · raw message
drivers/bus/fslmc/qbman/qbman_debug.c +2 -2
@@ -23,7 +23,7 @@ struct qbman_fq_query_desc {uint8_t reserved2[57 ]; }; -int qbman_fq_query_state (struct qbman_swp *s,uint32_t fqid, +int __rte_internal qbman_fq_query_state(struct qbman_swp *s, uint32_t fqid,struct qbman_fq_query_np_rslt *r) {struct qbman_fq_query_desc *p;
@@ -54,7 +54,7 @@ int qbman_fq_query_state(struct qbman_swp *s, uint32_t fqid,return 0 ; } -uint32_t qbman_fq_state_frame_count (const struct qbman_fq_query_np_rslt *r) +uint32_t __rte_internal qbman_fq_state_frame_count(const struct qbman_fq_query_np_rslt *r) {return (r->frm_cnt &0x00FFFFFF ); }
drivers/bus/fslmc/qbman/qbman_portal.c +41 -41
306 lines not on this page · raw message
drivers/bus/fslmc/rte_bus_fslmc_version.map +102 -99
233 lines not on this page · raw message
Signature · 1 line
2.20.1
Re: [dpdk-dev] [PATCH v2 05/10] fslmc: identify internal only functions and tag them as __rte_internal
HI Neil,
The same code of low-level HW driver is shared with different OSs (kernel, uboot etc) and different framework. If we introduce rte_internal in these low-level hw files, it will be a big maintenance issue for NXP. Everytime there is a change or upgrade, it will be a pain.
Regards,
Hemant
Quoted 15 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Thursday, June 13, 2019 7:54 PM
To: [email protected]
Cc: Neil Horman <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>; Bruce Richardson <[email protected]>; Thomas Monjalon <[email protected]>; Hemant Agrawal <[email protected]>; Shreyansh Jain <[email protected]>
Subject: [PATCH v2 05/10] fslmc: identify internal only functions and tag them as__rte_internal
Importance: HighIdentify functions in fslmc bus driver which are internal (based on their not having an
rte_prefix) and tag them with__rte_internal
Re: [dpdk-dev] [PATCH v2 05/10] fslmc: identify internal only functions and tag them as __rte_internal
Quoted 3 lines · Hemant Agrawal
HI Neil,
The same code of low-level HW driver is shared with different OSs (kernel, uboot etc) and different framework. If we introducerte_internalin these low-level hw files, it will be a big maintenance issue for NXP. Everytime there is a change or upgrade, it will be a pain.
Why? If you don't want to support the use of a project specific tag in other projects, you can add a global cflag to those like this:
-D__rte_experimental
and the code will be eliminated
I'm not sure what you see as difficult here
Neil
Quoted 19 lines
Regards,
HemantQuoted 15 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Thursday, June 13, 2019 7:54 PM
To: [email protected]
Cc: Neil Horman <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>; Bruce Richardson <[email protected]>; Thomas Monjalon <[email protected]>; Hemant Agrawal <[email protected]>; Shreyansh Jain <[email protected]>
Subject: [PATCH v2 05/10] fslmc: identify internal only functions and tag them as__rte_internal
Importance: HighIdentify functions in fslmc bus driver which are internal (based on their not having an
rte_prefix) and tag them with__rte_internal
Re: [dpdk-dev] [PATCH v2 05/10] fslmc: identify internal only functions and tag them as __rte_internal
HI Neil,
Quoted 7 lines · Hemant Agrawal
Quoted 2 lines
HI Neil,
The same code of low-level HW driver is shared with different OSs(kernel, uboot etc) and different framework. If we introduce
rte_internalin these low-level hw files, it will be a big maintenance issue for NXP. Everytime there is a change or upgrade, it will be a pain.Quoted 1 line
Quoted 7 lines
Why? If you don't want to support the use of a project specific tag in other projects, you can add a global cflag to those like this:
-D__rte_experimentaland the code will be eliminated
I'm not sure what you see as difficult here
[Hemant]
Do, we have an alternate for this approach?
Also, w.r.t to changes, at present you are adding the "__rte_internal" after the function return type. Is it possible to add it before the function return type and limiting the changes to header file only i.e. something similar to the current approach of "__rte_experimental" => current approach
drivers/bus/fslmc/mc/dpbp.c line 32
Quoted 7 lines
* Return: '0' on Success; Error code otherwise.
*/drivers/bus/fslmc/mc/dpbp.c +1
-1-int dpbp_open (struct fsl_mc_io *mc_io, +int __rte_internal dpbp_open(struct fsl_mc_io *mc_io,uint32_t cmd_flags,int dpbp_id,uint16_t *token)
=> approach used in rte_experimental
Quoted 3 lines
__rte_experimentaluint16_trte_bbdev_count(void);
This will help in minimizing the changes.
Regards,
Hemant
Quoted 22 lines
Neil
Quoted 19 lines
Regards,
HemantQuoted 15 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Thursday, June 13, 2019 7:54 PM
To: [email protected]
Cc: Neil Horman <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>; Bruce Richardson <[email protected]>; Thomas Monjalon <[email protected]>; Hemant Agrawal <[email protected]>; Shreyansh Jain <[email protected]>
Subject: [PATCH v2 05/10] fslmc: identify internal only functions and tag them as__rte_internal
Importance: HighIdentify functions in fslmc bus driver which are internal (based on their not having an
rte_prefix) and tag them with__rte_internal
Re: [dpdk-dev] [PATCH v2 05/10] fslmc: identify internal only functions and tag them as __rte_internal
Quoted 12 lines · Hemant Agrawal
HI Neil,
Quoted 11 lines · Hemant Agrawal
Quoted 2 lines
HI Neil,
The same code of low-level HW driver is shared with different OSs(kernel, uboot etc) and different framework. If we introduce
rte_internalin these low-level hw files, it will be a big maintenance issue for NXP. Everytime there is a change or upgrade, it will be a pain.Quoted 1 line
Why? If you don't want to support the use of a project specific tag in other projects, you can add a global cflag to those like this:
-D__rte_experimental
Quoted 6 lines
Quoted 3 lines
and the code will be eliminated
I'm not sure what you see as difficult here
[Hemant]
Do, we have an alternate for this approach?
No, what issue do you see with the approach suggested above?
Quoted 1 line
Also, w.r.t to changes, at present you are adding the "
__rte_internal" after the function return type. Is it possible to add it before the function return type and limiting the changes to header file only i.e. something similar to the current approach of "__rte_experimental"
The __rte_internal macro expands to a function attribute, and the gcc manual mandates that it be inserted between the return type and function name. As for its application in both the prototypes and declarations, I'd need to go back and look again, I can't recall if that can be done.
Quoted 42 lines
=> current approach
drivers/bus/fslmc/mc/dpbp.c line 32
Quoted 7 lines
* Return: '0' on Success; Error code otherwise.
*/drivers/bus/fslmc/mc/dpbp.c +1
-1-int dpbp_open (struct fsl_mc_io *mc_io, +int __rte_internal dpbp_open(struct fsl_mc_io *mc_io,uint32_t cmd_flags,int dpbp_id,uint16_t *token)=> approach used in
rte_experimentalQuoted 3 lines
__rte_experimentaluint16_trte_bbdev_count(void);This will help in minimizing the changes.
Regards,
HemantQuoted 22 lines
Neil
Quoted 19 lines
Regards,
HemantQuoted 15 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Thursday, June 13, 2019 7:54 PM
To: [email protected]
Cc: Neil Horman <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>; Bruce Richardson <[email protected]>; Thomas Monjalon <[email protected]>; Hemant Agrawal <[email protected]>; Shreyansh Jain <[email protected]>
Subject: [PATCH v2 05/10] fslmc: identify internal only functions and tag them as__rte_internal
Importance: HighIdentify functions in fslmc bus driver which are internal (based on their not having an
rte_prefix) and tag them with__rte_internal
Re: [dpdk-dev] [PATCH v2 05/10] fslmc: identify internal only functions and tag them as __rte_internal
HI Neil,
Quoted 13 lines · Neil Horman
Quoted 12 lines · Hemant Agrawal
HI Neil,
Quoted 11 lines · Hemant Agrawal
Quoted 2 lines
HI Neil,
The same code of low-level HW driver is shared with different OSs(kernel, uboot etc) and different framework. If we introduce
rte_internalin these low-level hw files, it will be a big maintenance issue for NXP. Everytime there is a change or upgrade, it will be a pain.Quoted 1 line
Why? If you don't want to support the use of a project specific tag in other projects, you can add a global cflag to those like this:
-D__rte_experimental
Quoted 7 lines
Quoted 6 lines
Quoted 3 lines
and the code will be eliminated
I'm not sure what you see as difficult here
[Hemant]
Do, we have an alternate for this approach?No, what issue do you see with the approach suggested above?
We are just trying to reduce the maintenance cost it brings. Specially dealing with extra changes to the common libraries provided by other teams.
Quoted 7 lines
Quoted 3 lines
Also, w.r.t to changes, at present you are adding the "
__rte_internal" after the function return type. Is it possible to add it before the function return type and limiting the changes to header file only i.e. something similar to the current approach of "__rte_experimental"The
__rte_internalmacro expands to a function attribute, and the gcc manual mandates that it be inserted between the return type and function name. As for its application in both the prototypes and declarations, I'd need to go back and look again, I can't recall if that can be done.
Thanks in advance. Please check if you can make it similar to __rte_experimental support.
Quoted 43 lines
Quoted 43 lines
=> current approach
drivers/bus/fslmc/mc/dpbp.c line 32
Quoted 7 lines
* Return: '0' on Success; Error code otherwise.
*/drivers/bus/fslmc/mc/dpbp.c +1
-1-int dpbp_open (struct fsl_mc_io *mc_io, +int __rte_internal dpbp_open(struct fsl_mc_io *mc_io,uint32_t cmd_flags,int dpbp_id,uint16_t *token)=> approach used in
rte_experimentalQuoted 3 lines
__rte_experimentaluint16_trte_bbdev_count(void);This will help in minimizing the changes.
Regards,
HemantQuoted 22 lines
Neil
Quoted 19 lines
Regards,
HemantQuoted 15 lines
-----Original Message-----
From: Neil Horman <[email protected]>
Sent: Thursday, June 13, 2019 7:54 PM
To: [email protected]
Cc: Neil Horman <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>; Bruce Richardson <[email protected]>; Thomas Monjalon <[email protected]>; Hemant Agrawal <[email protected]>; Shreyansh Jain <[email protected]>
Subject: [PATCH v2 05/10] fslmc: identify internal only functions and tag them as__rte_internal
Importance: HighIdentify functions in fslmc bus driver which are internal (based on their not having an
rte_prefix) and tag them with__rte_internal
PATCH v2 6/10 dpaa2: Adjust dpaa2 driver to mark internal symbols with __rte_internal
In reply to Neil HormanIdentify functions in the dpaa2 driver which are internal (based on their not having an rte_ prefix) and tag them with __rte_internal
--- drivers/net/dpaa2/dpaa2_ethdev.c | 4 ++-- drivers/net/dpaa2/dpaa2_ethdev.h | 4 ++-- drivers/net/dpaa2/rte_pmd_dpaa2_version.map | 14 ++++++-------- 3 files changed, 10 insertions(+), 12 deletions(-)
drivers/net/dpaa2/dpaa2_ethdev.c +2 -2
@@ -1798,7 +1798,7 @@ dpaa2_dev_rss_hash_conf_get(struct rte_eth_dev *dev,return 0 ; } -int dpaa2_eth_eventq_attach (const struct rte_eth_dev *dev, +int __rte_internal dpaa2_eth_eventq_attach(const struct rte_eth_dev *dev,int eth_rx_queue_id,uint16_t dpcon_id,const struct rte_event_eth_rx_adapter_queue_conf *queue_conf)
@@ -1881,7 +1881,7 @@ int dpaa2_eth_eventq_attach(const struct rte_eth_dev *dev,return 0 ; } -int dpaa2_eth_eventq_detach (const struct rte_eth_dev *dev, +int __rte_internal dpaa2_eth_eventq_detach(const struct rte_eth_dev *dev,int eth_rx_queue_id) {struct dpaa2_dev_priv *eth_priv = dev->data->dev_private;
drivers/net/dpaa2/dpaa2_ethdev.h +2 -2
@@ -140,12 +140,12 @@ int dpaa2_remove_flow_dist(struct rte_eth_dev *eth_dev,int dpaa2_attach_bp_list (struct dpaa2_dev_priv *priv,void *blist); -int dpaa2_eth_eventq_attach (const struct rte_eth_dev *dev, +int __rte_internal dpaa2_eth_eventq_attach(const struct rte_eth_dev *dev,int eth_rx_queue_id,uint16_t dpcon_id,const struct rte_event_eth_rx_adapter_queue_conf *queue_conf); -int dpaa2_eth_eventq_detach (const struct rte_eth_dev *dev, +int __rte_internal dpaa2_eth_eventq_detach(const struct rte_eth_dev *dev,int eth_rx_queue_id); uint16_t dpaa2_dev_loopback_rx(void *queue, struct rte_mbuf **bufs,
drivers/net/dpaa2/rte_pmd_dpaa2_version.map +6 -8
@@ -1,15 +1,13 @@ -DPDK_17.05 { - - local: *; -}; - -DPDK_17.11 { +INTERNAL { global: dpaa2_eth_eventq_attach; dpaa2_eth_eventq_detach; +}; -} DPDK_17.05; +DPDK_17.05 { + local: *; +}; EXPERIMENTAL { global:
@@ -17,4 +15,4 @@ EXPERIMENTAL { rte_pmd_dpaa2_mux_flow_create; rte_pmd_dpaa2_set_custom_hash; rte_pmd_dpaa2_set_timestamp; -} DPDK_17.11; +} DPDK_17.05;
Signature · 1 line
2.20.1
PATCH v2 7/10 dpaax: mark internal functions with __rte_internal
In reply to Neil HormanIdentify functions in the dpaa2 driver which are internal (based on
their not having an rte_ prefix) and tag them with __rte_internal
--- drivers/common/dpaax/dpaax_iova_table.c | 8 ++++---- drivers/common/dpaax/dpaax_iova_table.h | 8 ++++---- drivers/common/dpaax/rte_common_dpaax_version.map | 4 +++- 3 files changed, 11 insertions(+), 9 deletions(-)
drivers/common/dpaax/dpaax_iova_table.c +4 -4
@@ -151,7 +151,7 @@ read_memory_node(unsigned int *count) }int -dpaax_iova_table_populate (void ) +__rte_internal dpaax_iova_table_populate(void) {int ret;unsigned int i, node_count;
@@ -252,7 +252,7 @@ dpaax_iova_table_populate(void) }void -dpaax_iova_table_depopulate (void ) +__rte_internal dpaax_iova_table_depopulate(void) {if (dpaax_iova_table_p ==NULL )return ;
@@ -264,7 +264,7 @@ dpaax_iova_table_depopulate(void) }int -dpaax_iova_table_update (phys_addr_t paddr,void *vaddr,size_t length) +__rte_internal dpaax_iova_table_update(phys_addr_t paddr, void *vaddr, size_t length) {int found =0 ;unsigned int i;
@@ -348,7 +348,7 @@ dpaax_iova_table_update(phys_addr_t paddr, void *vaddr, size_t length)* Not for weak hearted - the tables can get quite large */ void -dpaax_iova_table_dump (void ) +__rte_internal dpaax_iova_table_dump(void) {unsigned int i, j;struct dpaax_iovat_element *entry;
drivers/common/dpaax/dpaax_iova_table.h +4 -4
@@ -59,10 +59,10 @@ extern struct dpaax_iova_table *dpaax_iova_table_p;#define DPAAX_MEM_SPLIT_MASK_OFF (DPAAX_MEM_SPLIT - 1)/**< Offset */ /* APIs exposed */ -int dpaax_iova_table_populate (void ); -void dpaax_iova_table_depopulate (void ); -int dpaax_iova_table_update (phys_addr_t paddr,void *vaddr,size_t length); -void dpaax_iova_table_dump (void ); +int __rte_internal dpaax_iova_table_populate(void); +void __rte_internal dpaax_iova_table_depopulate(void); +int __rte_internal dpaax_iova_table_update(phys_addr_t paddr, void *vaddr, size_t length); +void __rte_internal dpaax_iova_table_dump(void);static inline void *dpaax_iova_table_get_va (phys_addr_t paddr) __attribute__((hot));
drivers/common/dpaax/rte_common_dpaax_version.map +3 -1
@@ -1,4 +1,4 @@ -DPDK_18.11 { +INTERNAL { global: dpaax_iova_table_update;
@@ -6,6 +6,8 @@ DPDK_18.11 { dpaax_iova_table_dump; dpaax_iova_table_p; dpaax_iova_table_populate; +}; +DPDK_18.11 { local: *; };
Signature · 1 line
2.20.1
PATCH v2 8/10 cpt: mark internal functions with __rte_internal
In reply to Neil HormanIdentify functions in the cpt driver which are internal (based on their not having an rte_ prefix) and tag them with __rte_internal
--- drivers/common/cpt/cpt_pmd_ops_helper.c | 4 ++-- drivers/common/cpt/cpt_pmd_ops_helper.h | 4 ++-- drivers/common/cpt/rte_common_cpt_version.map | 6 +++++- 3 files changed, 9 insertions(+), 5 deletions(-)
drivers/common/cpt/cpt_pmd_ops_helper.c +2 -2
@@ -13,7 +13,7 @@#define CPT_OFFSET_CONTROL_BYTES 8int32_t -cpt_pmd_ops_helper_get_mlen_direct_mode (void ) +__rte_internal cpt_pmd_ops_helper_get_mlen_direct_mode(void) {uint32_t len =0 ;
@@ -27,7 +27,7 @@ cpt_pmd_ops_helper_get_mlen_direct_mode(void) }int -cpt_pmd_ops_helper_get_mlen_sg_mode (void ) +__rte_internal cpt_pmd_ops_helper_get_mlen_sg_mode(void) {uint32_t len =0 ;
drivers/common/cpt/cpt_pmd_ops_helper.h +2 -2
@@ -20,7 +20,7 @@*/ int32_t -cpt_pmd_ops_helper_get_mlen_direct_mode (void ); +__rte_internal cpt_pmd_ops_helper_get_mlen_direct_mode(void);/* * Get size of contiguous meta buffer to be allocated when working in scatter
@@ -30,5 +30,5 @@ cpt_pmd_ops_helper_get_mlen_direct_mode(void);* - length */ int -cpt_pmd_ops_helper_get_mlen_sg_mode (void ); +__rte_internal cpt_pmd_ops_helper_get_mlen_sg_mode(void);#endif /* _CPT_PMD_OPS_HELPER_H_ */
drivers/common/cpt/rte_common_cpt_version.map +5 -1
@@ -1,6 +1,10 @@ -DPDK_18.11 { +INTERNAL { global: cpt_pmd_ops_helper_get_mlen_direct_mode; cpt_pmd_ops_helper_get_mlen_sg_mode; }; + +DPDK_18.11 { + local: *; +};
Signature · 1 line
2.20.1
PATCH v2 9/10 octeonx: mark internal functions with __rte_internal
In reply to Neil HormanIdentify functions in the octeon driver which are internal (based on their not having an rte_ prefix) and tag them with __rte_internal
--- drivers/common/octeontx/octeontx_mbox.c | 6 +++--- drivers/common/octeontx/octeontx_mbox.h | 6 +++--- drivers/common/octeontx/rte_common_octeontx_version.map | 9 ++++++++- 3 files changed, 14 insertions(+), 7 deletions(-)
drivers/common/octeontx/octeontx_mbox.c +3 -3
@@ -190,7 +190,7 @@ mbox_send(struct mbox *m, struct octeontx_mbox_hdr *hdr, const void *txmsg, }int -octeontx_mbox_set_ram_mbox_base (uint8_t *ram_mbox_base) +__rte_internal octeontx_mbox_set_ram_mbox_base(uint8_t *ram_mbox_base) {struct mbox *m = &octeontx_mbox;
@@ -213,7 +213,7 @@ octeontx_mbox_set_ram_mbox_base(uint8_t *ram_mbox_base) }int -octeontx_mbox_set_reg (uint8_t *reg) +__rte_internal octeontx_mbox_set_reg(uint8_t *reg) {struct mbox *m = &octeontx_mbox;
@@ -236,7 +236,7 @@ octeontx_mbox_set_reg(uint8_t *reg) }int -octeontx_mbox_send (struct octeontx_mbox_hdr *hdr,void *txdata, +__rte_internal octeontx_mbox_send(struct octeontx_mbox_hdr *hdr, void *txdata,uint16_t txlen,void *rxdata,uint16_t rxlen) {struct mbox *m = &octeontx_mbox;
drivers/common/octeontx/octeontx_mbox.h +3 -3
@@ -29,9 +29,9 @@ struct octeontx_mbox_hdr {uint8_t res_code;/* Functional layer response code */ }; -int octeontx_mbox_set_ram_mbox_base (uint8_t *ram_mbox_base); -int octeontx_mbox_set_reg (uint8_t *reg); -int octeontx_mbox_send (struct octeontx_mbox_hdr *hdr, +int __rte_internal octeontx_mbox_set_ram_mbox_base(uint8_t *ram_mbox_base); +int __rte_internal octeontx_mbox_set_reg(uint8_t *reg); +int __rte_internal octeontx_mbox_send(struct octeontx_mbox_hdr *hdr,void *txdata,uint16_t txlen,void *rxdata,uint16_t rxlen);#endif /* __OCTEONTX_MBOX_H__ */
drivers/common/octeontx/rte_common_octeontx_version.map +8 -1
@@ -1,7 +1,14 @@ -DPDK_18.05 { +INTERNAL { global: octeontx_mbox_set_ram_mbox_base; octeontx_mbox_set_reg; octeontx_mbox_send; }; + +DPDK_18.05 { + global: + octeontx_logtype_mbox; + + local: *; +};
Signature · 1 line
2.20.1
PATCH v2 10/10 dpaa2: mark internal functions with __rte_internal
In reply to Neil HormanIdentify functions in the dpaa2 driver which are internal (based on their not having an rte_ prefix) and tag them with __rte_internal
--- drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c | 4 ++-- drivers/crypto/dpaa2_sec/dpaa2_sec_event.h | 4 ++-- .../crypto/dpaa2_sec/rte_pmd_dpaa2_sec_version.map | 13 ++++++------- 3 files changed, 10 insertions(+), 11 deletions(-)
drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c +2 -2
@@ -3152,7 +3152,7 @@ dpaa2_sec_process_atomic_event(struct qbman_swp *swp __attribute__((unused)), }int -dpaa2_sec_eventq_attach (const struct rte_cryptodev *dev, +__rte_internal dpaa2_sec_eventq_attach(const struct rte_cryptodev *dev,int qp_id,uint16_t dpcon_id,const struct rte_event *event)
@@ -3195,7 +3195,7 @@ dpaa2_sec_eventq_attach(const struct rte_cryptodev *dev, }int -dpaa2_sec_eventq_detach (const struct rte_cryptodev *dev, +__rte_internal dpaa2_sec_eventq_detach(const struct rte_cryptodev *dev,int qp_id) {struct dpaa2_sec_dev_private *priv = dev->data->dev_private;
drivers/crypto/dpaa2_sec/dpaa2_sec_event.h +2 -2
@@ -7,12 +7,12 @@#define _DPAA2_SEC_EVENT_H_int -dpaa2_sec_eventq_attach (const struct rte_cryptodev *dev, +__rte_internal dpaa2_sec_eventq_attach(const struct rte_cryptodev *dev,int qp_id,uint16_t dpcon_id,const struct rte_event *event); -int dpaa2_sec_eventq_detach (const struct rte_cryptodev *dev, +int __rte_internal dpaa2_sec_eventq_detach(const struct rte_cryptodev *dev,int qp_id);#endif /* _DPAA2_SEC_EVENT_H_ */
drivers/crypto/dpaa2_sec/rte_pmd_dpaa2_sec_version.map +6 -7
@@ -1,12 +1,11 @@ -DPDK_17.05 { - - local: *; -}; - -DPDK_18.11 { +INTERNAL { global: dpaa2_sec_eventq_attach; dpaa2_sec_eventq_detach; +}; + +DPDK_17.05 { -} DPDK_17.05; + local: *; +};
Signature · 1 line
2.20.1
Re: [dpdk-dev] [PATCH v2 0/10] dpdk: introduce __rte_internal tag
In reply to Neil HormanI think it would be good to rebase and send at the beginning of the 19.11 cycle. Thank you
13/06/2019 16:23, Neil Horman:
Quoted 36 lines
Hey-
Based on our recent conversations regarding the use of symbols only
meant for internal dpdk consumption (between dpdk libraries), this is an idea that I've come up with that I'd like to get some feedback onSummary:
1) We have symbols in the DPDK that are meant to be used between DPDK libraries, but not by applications linking to them
2) We would like to document those symbols in the code, so as to note them clearly as for being meant for internal use only
3) Linker symbol visibility is a very coarse grained tool, and so there is no good way in a single library to mark items as being meant for use only by other DPDK libraries, at least not without some extensive runtime checkingProposal:
I'm proposing that we introduce the__rte_internaltag. From a coding standpoint it works a great deal like the__rte_experimentaltag in that it expempts the tagged symbol from ABI constraints (as the only users should be represented in the DPDK build environment). Additionally, the__rte_internalmacro resolves differently based on the definition of theBUILDING_RTE_SDKflag (working under the assumption that said flag should only ever be set if we are actually building DPDK libraries which will make use of internal calls). If theBUILDING_RTE_SDKflag is set__rte_internalresolves to__attribute__((section "text.internal)), placing it in a special text section which is then used to validate that the the symbol appears in the INTERNAL section of the corresponding library version map). IfBUILDING_RTE_SDKis not set, then__rte_internalresolves to __attribute__((error("..."))), which causes any caller of the tagged function to throw an error at compile time, indicating that the symbol is not available for external use.This isn't a perfect solution, as applications can still hack around it of course, but I think it hits some of the high points, restricting symbol access for any library that prototypes its public and private symbols in the same header file, excluding the internal symbols from ABI constraints, and clearly documenting those symbols which we wish to limit to internal usage.
Re: [dpdk-dev] [PATCH v2 0/10] dpdk: introduce __rte_internal tag
Quoted 3 lines · Thomas Monjalon
I think it would be good to rebase and send at the beginning of the 19.11 cycle. Thank you
I'm on PTO for the next 10 days or so, but yes, I'll take care of that asap.
Thanks
Neil
Quoted 42 lines
13/06/2019 16:23, Neil Horman:
Quoted 36 lines
Hey-
Based on our recent conversations regarding the use of symbols only
meant for internal dpdk consumption (between dpdk libraries), this is an idea that I've come up with that I'd like to get some feedback onSummary:
1) We have symbols in the DPDK that are meant to be used between DPDK libraries, but not by applications linking to them
2) We would like to document those symbols in the code, so as to note them clearly as for being meant for internal use only
3) Linker symbol visibility is a very coarse grained tool, and so there is no good way in a single library to mark items as being meant for use only by other DPDK libraries, at least not without some extensive runtime checkingProposal:
I'm proposing that we introduce the__rte_internaltag. From a coding standpoint it works a great deal like the__rte_experimentaltag in that it expempts the tagged symbol from ABI constraints (as the only users should be represented in the DPDK build environment). Additionally, the__rte_internalmacro resolves differently based on the definition of theBUILDING_RTE_SDKflag (working under the assumption that said flag should only ever be set if we are actually building DPDK libraries which will make use of internal calls). If theBUILDING_RTE_SDKflag is set__rte_internalresolves to__attribute__((section "text.internal)), placing it in a special text section which is then used to validate that the the symbol appears in the INTERNAL section of the corresponding library version map). IfBUILDING_RTE_SDKis not set, then__rte_internalresolves to __attribute__((error("..."))), which causes any caller of the tagged function to throw an error at compile time, indicating that the symbol is not available for external use.This isn't a perfect solution, as applications can still hack around it of course, but I think it hits some of the high points, restricting symbol access for any library that prototypes its public and private symbols in the same header file, excluding the internal symbols from ABI constraints, and clearly documenting those symbols which we wish to limit to internal usage.
Re: [dpdk-dev] [PATCH v2 0/10] dpdk: introduce __rte_internal tag
Hello Neil,
Quoted 6 lines · Neil Horman
Quoted 3 lines · Thomas Monjalon
I think it would be good to rebase and send at the beginning of the 19.11 cycle. Thank you
I'm on PTO for the next 10 days or so, but yes, I'll take care of that asap.
Looks like your PTO were a bit longer than 10 days ;-). Do you have some cycles to resurrect this series?
--
David Marchand
Re: [dpdk-dev] [PATCH v2 0/10] dpdk: introduce __rte_internal tag
Quoted 4 lines · David Marchand
Hello Neil,
Quoted 1 line · Neil Horman
Quoted 9 lines
Quoted 5 lines · Thomas Monjalon
Quoted 3 lines
I think it would be good to rebase and send at the beginning of the 19.11 cycle. Thank you
I'm on PTO for the next 10 days or so, but yes, I'll take care of that asap.
Looks like your PTO were a bit longer than 10 days ;-). Do you have some cycles to resurrect this series?
Shoot, apologies, it completely slipped my mind. Yes, I'll get back to this today
Neil
Quoted 5 lines
--
David Marchand
Re: [dpdk-dev] [PATCH v2 0/10] dpdk: introduce __rte_internal tag
Quoted 10 lines · Neil Horman
Quoted 8 lines · David Marchand
Hello Neil,
Quoted 5 lines · Neil Horman
Quoted 3 lines · Thomas Monjalon
I think it would be good to rebase and send at the beginning of the 19.11 cycle. Thank you
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
I'm on PTO for the next 10 days or so, but yes, I'll take care of that asap.
Looks like your PTO were a bit longer than 10 days ;-). Do you have some cycles to resurrect this series?
Shoot, apologies, it completely slipped my mind. Yes, I'll get back to this today
Cool!
Thanks Neil.
Signature · 1 line
David Marchand
v3 cover dpdk: introduce __rte_internal tag
In reply to Neil HormanMove the internal function into INTERNAL session to avoid the ABI checking, and it is only used for DPDK drivers or related library.
__rte_internal funA
INTERNAL {
global:
funA
};
v3: based on Neil's v2 patch https://patchwork.dpdk.org/cover/54771/
Use the ALLOW_INTERNAL_API to mark this new feature.
Haiyue Wang (1):
eal: add internal ABI mark support
buildtools/check-internal-syms.sh | 57 +++++++++++++++++++++++++++++ devtools/check-symbol-change.sh | 7 ++++ drivers/meson.build | 2 +- lib/librte_eal/include/rte_compat.h | 13 +++++++ lib/meson.build | 2 +- mk/internal/rte.compile-pre.mk | 3 ++ mk/target/generic/rte.vars.mk | 1 + 7 files changed, 83 insertions(+), 2 deletions(-) create mode 100755 buildtools/check-internal-syms.sh
Signature · 1 line
2.26.2
PATCH v3 1/1 eal: add internal ABI mark support
Introduce __rte_internal tag to mark internal ABI function, this kind of function can't be called by external application.
--- buildtools/check-internal-syms.sh | 57 +++++++++++++++++++++++++++++ devtools/check-symbol-change.sh | 7 ++++ drivers/meson.build | 2 +- lib/librte_eal/include/rte_compat.h | 13 +++++++ lib/meson.build | 2 +- mk/internal/rte.compile-pre.mk | 3 ++ mk/target/generic/rte.vars.mk | 1 + 7 files changed, 83 insertions(+), 2 deletions(-) create mode 100755 buildtools/check-internal-syms.sh
buildtools/check-internal-syms.sh new file +57 -0
@@ -0,0 +1,57 @@ +#!/bin/sh + +# SPDX-License-Identifier: BSD-3-Clause + +MAPFILE=$1 +OBJFILE=$2 + +LIST_SYMBOL=$(dirname $(readlink -f $0))/map-list-symbol.sh + +# added check for "make -C test/" usage +if [ ! -e $MAPFILE ] || [ ! -f $OBJFILE ] +then + exit 0 +fi + +if [ -d $MAPFILE ] +then + exit 0 +fi + +DUMPFILE=$(mktemp -t dpdk.${0##*/}.XXX.objdump) +trap 'rm -f "$DUMPFILE"' EXIT +objdump -t $OBJFILE >$DUMPFILE + +ret=0 +for SYM in `$LIST_SYMBOL -S INTERNAL $MAPFILE |cut -d ' ' -f 3` +do + if grep -q "\.text.*[[:space:]]$SYM$" $DUMPFILE && + ! grep -q "\.text\.internal.*[[:space:]]$SYM$" $DUMPFILE + then + cat >&2 <<- END_OF_MESSAGE + $SYM is not flagged as internal + but is listed in version map + Please add __rte_internal to the definition of $SYM + END_OF_MESSAGE + ret=1 + fi +done + +# Filter out symbols suffixed with a . for icc +for SYM in `awk '{ + if ($2 != "l" && $4 == ".text.internal" && !($NF ~ /\.$/)) { + print $NF + } +}' $DUMPFILE` +do + $LIST_SYMBOL -S INTERNAL -s $SYM -q $MAPFILE || { + cat >&2 <<- END_OF_MESSAGE + $SYM is flagged as internal + but is not listed in version map + Please add $SYM to the version map + END_OF_MESSAGE + ret=1 + } +done + +exit $ret
devtools/check-symbol-change.sh +7 -0
@@ -91,6 +91,13 @@ check_for_rule_violations() if [ "$ar" = "add" ] then + if [ "$secname" = "INTERNAL" ] + then + # these are absolved from any further checking + echo "Skipping symbol $symname in INTERNAL" + continue + fi + if [ "$secname" = "unknown" ] then # Just inform the user of this occurrence, but
drivers/meson.build +1 -1
@@ -20,7 +20,7 @@ dpdk_driver_classes = ['common', disabled_drivers = run_command(list_dir_globs, get_option('disable_drivers'), ).stdout().split() -default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API'] +default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API'] + ['-DALLOW_INTERNAL_API'] if cc.has_argument('-Wno-format-truncation') default_cflags += '-Wno-format-truncation' endif
lib/librte_eal/include/rte_compat.h +13 -0
@@ -19,4 +19,17 @@ __attribute__((section(".text.experimental")))#endif +#ifndef ALLOW_INTERNAL_API + +#define __rte_internal \ +__attribute__((error("Symbol is not public ABI"), \ +section(".text.internal"))) + +#else + +#define __rte_internal \ +__attribute__((section(".text.internal"))) + +#endif +#endif /* _RTE_COMPAT_H_ */
lib/meson.build +1 -1
@@ -36,7 +36,7 @@ if is_windows libraries = ['kvargs','eal'] # only supported libraries for windows endif -default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API'] +default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API'] + ['-DALLOW_INTERNAL_API'] if cc.has_argument('-Wno-format-truncation') default_cflags += '-Wno-format-truncation' endif
mk/internal/rte.compile-pre.mk +3 -0
@@ -58,6 +58,8 @@ C_TO_O_DISP = $(if $(V),"$(C_TO_O_STR)"," CC $(@)") endif EXPERIMENTAL_CHECK = $(RTE_SDK)/buildtools/check-experimental-syms.sh CHECK_EXPERIMENTAL = $(EXPERIMENTAL_CHECK) $(SRCDIR)/$(EXPORT_MAP) $@ +INTERNAL_CHECK = $(RTE_SDK)/buildtools/check-internal-syms.sh +CHECK_INTERNAL = $(INTERNAL_CHECK) $(SRCDIR)/$(EXPORT_MAP) $@ PMDINFO_GEN = $(RTE_SDK_BIN)/app/dpdk-pmdinfogen $@ [email protected] PMDINFO_CC = $(CC) $(CPPFLAGS) $(CFLAGS) $(EXTRA_CFLAGS) -c -o [email protected] [email protected]
@@ -76,6 +78,7 @@ C_TO_O_DO = @set -e; \ $(C_TO_O) && \ $(PMDINFO_TO_O) && \ $(CHECK_EXPERIMENTAL) && \ + $(CHECK_INTERNAL) && \ echo $(C_TO_O_CMD) > $(call obj2cmd,$(@)) && \ sed 's,'$@':,dep_'$@' =,' $(call obj2dep,$(@)).tmp > $(call obj2dep,$(@)) && \ rm -f $(call obj2dep,$(@)).tmp
mk/target/generic/rte.vars.mk +1 -0
@@ -106,6 +106,7 @@ ifeq ($(BUILDING_RTE_SDK),1) # building sdk CFLAGS += -include $(RTE_OUTPUT)/include/rte_config.h CFLAGS += -DALLOW_EXPERIMENTAL_API +CFLAGS += -DALLOW_INTERNAL_API else # if we are building an external application, include SDK's lib and # includes too
Signature · 1 line
2.26.2
Re: [dpdk-dev] [PATCH v3 1/1] eal: add internal ABI mark support
Quoted 5 lines · Haiyue Wang
Introduce
__rte_internaltag to mark internal ABI function, this kind of function can't be called by external application.Signed-off-by Haiyue Wang <[email protected]>
Quoted 10 lines
--- buildtools/check-internal-syms.sh | 57 +++++++++++++++++++++++++++++ devtools/check-symbol-change.sh | 7 ++++ drivers/meson.build | 2 +- lib/librte_eal/include/rte_compat.h | 13 +++++++ lib/meson.build | 2 +- mk/internal/rte.compile-pre.mk | 3 ++ mk/target/generic/rte.vars.mk | 1 + 7 files changed, 83 insertions(+), 2 deletions(-) create mode 100755 buildtools/check-internal-syms.sh
I did not enter the details, but I suppose this patch lacks an update on the suppression rules so that INTERNAL versioned symbols are skipped.
https://git.dpdk.org/dpdk/tree/devtools/libabigail.abignore
Signature · 1 line
David Marchand
Re: [dpdk-dev] [PATCH v3 1/1] eal: add internal ABI mark support
Quoted 25 lines
-----Original Message-----
From: David Marchand <[email protected]>
Sent: Wednesday, April 22, 2020 22:14
To: Wang, Haiyue <[email protected]>
Cc: dev <[email protected]>; Thomas Monjalon <[email protected]>; Richardson, Bruce <[email protected]>; Yigit, Ferruh <[email protected]>; Ray Kinsella <[email protected]>
Subject: Re: [PATCH v3 1/1] eal: add internal ABI mark supportQuoted 5 lines · Haiyue Wang
Introduce
__rte_internaltag to mark internal ABI function, this kind of function can't be called by external application.Signed-off-by Haiyue Wang <[email protected]>Quoted 10 lines
--- buildtools/check-internal-syms.sh | 57 +++++++++++++++++++++++++++++ devtools/check-symbol-change.sh | 7 ++++ drivers/meson.build | 2 +- lib/librte_eal/include/rte_compat.h | 13 +++++++ lib/meson.build | 2 +- mk/internal/rte.compile-pre.mk | 3 ++ mk/target/generic/rte.vars.mk | 1 + 7 files changed, 83 insertions(+), 2 deletions(-) create mode 100755 buildtools/check-internal-syms.sh
Quoted 6 lines
I did not enter the details, but I suppose this patch lacks an update on the suppression rules so that INTERNAL versioned symbols are skipped.
https://git.dpdk.org/dpdk/tree/devtools/libabigail.abignore
Added it in v4.
Quoted 2 lines
--
David Marchand
v4 cover dpdk: introduce __rte_internal tag
In reply to Neil HormanMove the internal function into INTERNAL session to avoid the ABI checking, and it is only used for DPDK drivers or related library.
__rte_internal funA
INTERNAL {
global:
funA
};
v4: add the ABI check suppression rules
v3: based on Neil's v2 patch https://patchwork.dpdk.org/cover/54771/
Use the ALLOW_INTERNAL_API to mark this new feature.
Haiyue Wang (1):
eal: add internal ABI mark support
buildtools/check-internal-syms.sh | 57 +++++++++++++++++++++++++++++ devtools/check-symbol-change.sh | 7 ++++ devtools/libabigail.abignore | 5 +++ drivers/meson.build | 2 +- lib/librte_eal/include/rte_compat.h | 13 +++++++ lib/meson.build | 2 +- mk/internal/rte.compile-pre.mk | 3 ++ mk/target/generic/rte.vars.mk | 1 + 8 files changed, 88 insertions(+), 2 deletions(-) create mode 100755 buildtools/check-internal-syms.sh
Signature · 1 line
2.26.2
PATCH v4 1/1 eal: add internal ABI mark support
Introduce __rte_internal tag to mark internal ABI function, this kind of function can't be called by external application.
--- buildtools/check-internal-syms.sh | 57 +++++++++++++++++++++++++++++ devtools/check-symbol-change.sh | 7 ++++ devtools/libabigail.abignore | 5 +++ drivers/meson.build | 2 +- lib/librte_eal/include/rte_compat.h | 13 +++++++ lib/meson.build | 2 +- mk/internal/rte.compile-pre.mk | 3 ++ mk/target/generic/rte.vars.mk | 1 + 8 files changed, 88 insertions(+), 2 deletions(-) create mode 100755 buildtools/check-internal-syms.sh
buildtools/check-internal-syms.sh new file +57 -0
@@ -0,0 +1,57 @@ +#!/bin/sh + +# SPDX-License-Identifier: BSD-3-Clause + +MAPFILE=$1 +OBJFILE=$2 + +LIST_SYMBOL=$(dirname $(readlink -f $0))/map-list-symbol.sh + +# added check for "make -C test/" usage +if [ ! -e $MAPFILE ] || [ ! -f $OBJFILE ] +then + exit 0 +fi + +if [ -d $MAPFILE ] +then + exit 0 +fi + +DUMPFILE=$(mktemp -t dpdk.${0##*/}.XXX.objdump) +trap 'rm -f "$DUMPFILE"' EXIT +objdump -t $OBJFILE >$DUMPFILE + +ret=0 +for SYM in `$LIST_SYMBOL -S INTERNAL $MAPFILE |cut -d ' ' -f 3` +do + if grep -q "\.text.*[[:space:]]$SYM$" $DUMPFILE && + ! grep -q "\.text\.internal.*[[:space:]]$SYM$" $DUMPFILE + then + cat >&2 <<- END_OF_MESSAGE + $SYM is not flagged as internal + but is listed in version map + Please add __rte_internal to the definition of $SYM + END_OF_MESSAGE + ret=1 + fi +done + +# Filter out symbols suffixed with a . for icc +for SYM in `awk '{ + if ($2 != "l" && $4 == ".text.internal" && !($NF ~ /\.$/)) { + print $NF + } +}' $DUMPFILE` +do + $LIST_SYMBOL -S INTERNAL -s $SYM -q $MAPFILE || { + cat >&2 <<- END_OF_MESSAGE + $SYM is flagged as internal + but is not listed in version map + Please add $SYM to the version map + END_OF_MESSAGE + ret=1 + } +done + +exit $ret
devtools/check-symbol-change.sh +7 -0
@@ -91,6 +91,13 @@ check_for_rule_violations() if [ "$ar" = "add" ] then + if [ "$secname" = "INTERNAL" ] + then + # these are absolved from any further checking + echo "Skipping symbol $symname in INTERNAL" + continue + fi + if [ "$secname" = "unknown" ] then # Just inform the user of this occurrence, but
devtools/libabigail.abignore +5 -0
@@ -3,6 +3,11 @@ [suppress_variable] symbol_version = EXPERIMENTAL +[suppress_function] + symbol_version = INTERNAL +[suppress_variable] + symbol_version = INTERNAL + ; Explicit ignore for driver-only ABI [suppress_type] name = rte_cryptodev_ops
drivers/meson.build +1 -1
@@ -20,7 +20,7 @@ dpdk_driver_classes = ['common', disabled_drivers = run_command(list_dir_globs, get_option('disable_drivers'), ).stdout().split() -default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API'] +default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API'] + ['-DALLOW_INTERNAL_API'] if cc.has_argument('-Wno-format-truncation') default_cflags += '-Wno-format-truncation' endif
lib/librte_eal/include/rte_compat.h +13 -0
@@ -19,4 +19,17 @@ __attribute__((section(".text.experimental")))#endif +#ifndef ALLOW_INTERNAL_API + +#define __rte_internal \ +__attribute__((error("Symbol is not public ABI"), \ +section(".text.internal"))) + +#else + +#define __rte_internal \ +__attribute__((section(".text.internal"))) + +#endif +#endif /* _RTE_COMPAT_H_ */
lib/meson.build +1 -1
@@ -36,7 +36,7 @@ if is_windows libraries = ['kvargs','eal'] # only supported libraries for windows endif -default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API'] +default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API'] + ['-DALLOW_INTERNAL_API'] if cc.has_argument('-Wno-format-truncation') default_cflags += '-Wno-format-truncation' endif
mk/internal/rte.compile-pre.mk +3 -0
@@ -58,6 +58,8 @@ C_TO_O_DISP = $(if $(V),"$(C_TO_O_STR)"," CC $(@)") endif EXPERIMENTAL_CHECK = $(RTE_SDK)/buildtools/check-experimental-syms.sh CHECK_EXPERIMENTAL = $(EXPERIMENTAL_CHECK) $(SRCDIR)/$(EXPORT_MAP) $@ +INTERNAL_CHECK = $(RTE_SDK)/buildtools/check-internal-syms.sh +CHECK_INTERNAL = $(INTERNAL_CHECK) $(SRCDIR)/$(EXPORT_MAP) $@ PMDINFO_GEN = $(RTE_SDK_BIN)/app/dpdk-pmdinfogen $@ [email protected] PMDINFO_CC = $(CC) $(CPPFLAGS) $(CFLAGS) $(EXTRA_CFLAGS) -c -o [email protected] [email protected]
@@ -76,6 +78,7 @@ C_TO_O_DO = @set -e; \ $(C_TO_O) && \ $(PMDINFO_TO_O) && \ $(CHECK_EXPERIMENTAL) && \ + $(CHECK_INTERNAL) && \ echo $(C_TO_O_CMD) > $(call obj2cmd,$(@)) && \ sed 's,'$@':,dep_'$@' =,' $(call obj2dep,$(@)).tmp > $(call obj2dep,$(@)) && \ rm -f $(call obj2dep,$(@)).tmp
mk/target/generic/rte.vars.mk +1 -0
@@ -106,6 +106,7 @@ ifeq ($(BUILDING_RTE_SDK),1) # building sdk CFLAGS += -include $(RTE_OUTPUT)/include/rte_config.h CFLAGS += -DALLOW_EXPERIMENTAL_API +CFLAGS += -DALLOW_INTERNAL_API else # if we are building an external application, include SDK's lib and # includes too
Signature · 1 line
2.26.2
v5 cover dpdk: introduce __rte_internal tag
In reply to Neil HormanMove the internal function into INTERNAL session to avoid the ABI checking, and it is only used for DPDK drivers or related library.
__rte_internal funA
INTERNAL {
global:
funA
};
v5: add the checkpatch for __rte_internal style
v4: add the ABI check suppression rules
v3: based on Neil's v2 patch https://patchwork.dpdk.org/cover/54771/
Use the ALLOW_INTERNAL_API to mark this new feature.
Haiyue Wang (1):
eal: add internal ABI marking support
buildtools/check-internal-syms.sh | 57 +++++++++++++++++++++++++++++ devtools/check-symbol-change.sh | 7 ++++ devtools/checkpatches.sh | 38 +++++++++++++++++++ devtools/libabigail.abignore | 5 +++ drivers/meson.build | 2 +- lib/librte_eal/include/rte_compat.h | 13 +++++++ lib/meson.build | 2 +- mk/internal/rte.compile-pre.mk | 3 ++ mk/target/generic/rte.vars.mk | 1 + 9 files changed, 126 insertions(+), 2 deletions(-) create mode 100755 buildtools/check-internal-syms.sh
Signature · 1 line
2.26.2
PATCH v5 1/1 eal: add internal ABI marking support
Introduce __rte_internal tag to mark internal ABI function which is used by the drivers or other libraries.
--- buildtools/check-internal-syms.sh | 57 +++++++++++++++++++++++++++++ devtools/check-symbol-change.sh | 7 ++++ devtools/checkpatches.sh | 38 +++++++++++++++++++ devtools/libabigail.abignore | 5 +++ drivers/meson.build | 2 +- lib/librte_eal/include/rte_compat.h | 13 +++++++ lib/meson.build | 2 +- mk/internal/rte.compile-pre.mk | 3 ++ mk/target/generic/rte.vars.mk | 1 + 9 files changed, 126 insertions(+), 2 deletions(-) create mode 100755 buildtools/check-internal-syms.sh
buildtools/check-internal-syms.sh new file +57 -0
@@ -0,0 +1,57 @@ +#!/bin/sh + +# SPDX-License-Identifier: BSD-3-Clause + +MAPFILE=$1 +OBJFILE=$2 + +LIST_SYMBOL=$(dirname $(readlink -f $0))/map-list-symbol.sh + +# added check for "make -C test/" usage +if [ ! -e $MAPFILE ] || [ ! -f $OBJFILE ] +then + exit 0 +fi + +if [ -d $MAPFILE ] +then + exit 0 +fi + +DUMPFILE=$(mktemp -t dpdk.${0##*/}.XXX.objdump) +trap 'rm -f "$DUMPFILE"' EXIT +objdump -t $OBJFILE >$DUMPFILE + +ret=0 +for SYM in `$LIST_SYMBOL -S INTERNAL $MAPFILE |cut -d ' ' -f 3` +do + if grep -q "\.text.*[[:space:]]$SYM$" $DUMPFILE && + ! grep -q "\.text\.internal.*[[:space:]]$SYM$" $DUMPFILE + then + cat >&2 <<- END_OF_MESSAGE + $SYM is not flagged as internal + but is listed in version map + Please add __rte_internal to the definition of $SYM + END_OF_MESSAGE + ret=1 + fi +done + +# Filter out symbols suffixed with a . for icc +for SYM in `awk '{ + if ($2 != "l" && $4 == ".text.internal" && !($NF ~ /\.$/)) { + print $NF + } +}' $DUMPFILE` +do + $LIST_SYMBOL -S INTERNAL -s $SYM -q $MAPFILE || { + cat >&2 <<- END_OF_MESSAGE + $SYM is flagged as internal + but is not listed in version map + Please add $SYM to the version map + END_OF_MESSAGE + ret=1 + } +done + +exit $ret
devtools/check-symbol-change.sh +7 -0
@@ -91,6 +91,13 @@ check_for_rule_violations() if [ "$ar" = "add" ] then + if [ "$secname" = "INTERNAL" ] + then + # these are absolved from any further checking + echo "Skipping symbol $symname in INTERNAL" + continue + fi + if [ "$secname" = "unknown" ] then # Just inform the user of this occurrence, but
devtools/checkpatches.sh +38 -0
@@ -111,6 +111,36 @@ check_experimental_tags() { # <patch> return $res } +check_internal_tags() { # <patch> + res=0 + + cat "$1" |awk ' + BEGIN { + current_file = ""; + ret = 0; + } + /^+++ b\// { + current_file = $2; + } + /^+.*__rte_internal/ { + if (current_file ~ ".c$" ) { + print "Please only put __rte_internal tags in " \ + "headers ("current_file")"; + ret = 1; + } + if ($1 != "+__rte_internal" || $2 != "") { + print "__rte_internal must appear alone on the line" \ + " immediately preceding the return type of a function." + ret = 1; + } + } + END { + exit ret; + }' || res=1 + + return $res +} + number=0 range='origin/master..' quiet=false
@@ -194,6 +224,14 @@ check () { # <patch> <commit> <title> ret=1 fi + ! $verbose || printf '\nChecking __rte_internal tags:\n' + report=$(check_internal_tags "$tmpinput") + if [ $? -ne 0 ] ; then + $headline_printed || print_headline "$3" + printf '%s\n' "$report" + ret=1 + fi + if [ "$tmpinput" != "$1" ]; then rm -f "$tmpinput" trap - INT
devtools/libabigail.abignore +5 -0
@@ -3,6 +3,11 @@ [suppress_variable] symbol_version = EXPERIMENTAL +[suppress_function] + symbol_version = INTERNAL +[suppress_variable] + symbol_version = INTERNAL + ; Explicit ignore for driver-only ABI [suppress_type] name = rte_cryptodev_ops
drivers/meson.build +1 -1
@@ -20,7 +20,7 @@ dpdk_driver_classes = ['common', disabled_drivers = run_command(list_dir_globs, get_option('disable_drivers'), ).stdout().split() -default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API'] +default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API'] + ['-DALLOW_INTERNAL_API'] if cc.has_argument('-Wno-format-truncation') default_cflags += '-Wno-format-truncation' endif
lib/librte_eal/include/rte_compat.h +13 -0
@@ -19,4 +19,17 @@ __attribute__((section(".text.experimental")))#endif +#ifndef ALLOW_INTERNAL_API + +#define __rte_internal \ +__attribute__((error("Symbol is not public ABI"), \ +section(".text.internal"))) + +#else + +#define __rte_internal \ +__attribute__((section(".text.internal"))) + +#endif +#endif /* _RTE_COMPAT_H_ */
lib/meson.build +1 -1
@@ -38,7 +38,7 @@ if is_windows libraries = ['kvargs','eal'] # only supported libraries for windows endif -default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API'] +default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API'] + ['-DALLOW_INTERNAL_API'] if cc.has_argument('-Wno-format-truncation') default_cflags += '-Wno-format-truncation' endif
mk/internal/rte.compile-pre.mk +3 -0
@@ -58,6 +58,8 @@ C_TO_O_DISP = $(if $(V),"$(C_TO_O_STR)"," CC $(@)") endif EXPERIMENTAL_CHECK = $(RTE_SDK)/buildtools/check-experimental-syms.sh CHECK_EXPERIMENTAL = $(EXPERIMENTAL_CHECK) $(SRCDIR)/$(EXPORT_MAP) $@ +INTERNAL_CHECK = $(RTE_SDK)/buildtools/check-internal-syms.sh +CHECK_INTERNAL = $(INTERNAL_CHECK) $(SRCDIR)/$(EXPORT_MAP) $@ PMDINFO_GEN = $(RTE_SDK_BIN)/app/dpdk-pmdinfogen $@ [email protected] PMDINFO_CC = $(CC) $(CPPFLAGS) $(CFLAGS) $(EXTRA_CFLAGS) -c -o [email protected] [email protected]
@@ -76,6 +78,7 @@ C_TO_O_DO = @set -e; \ $(C_TO_O) && \ $(PMDINFO_TO_O) && \ $(CHECK_EXPERIMENTAL) && \ + $(CHECK_INTERNAL) && \ echo $(C_TO_O_CMD) > $(call obj2cmd,$(@)) && \ sed 's,'$@':,dep_'$@' =,' $(call obj2dep,$(@)).tmp > $(call obj2dep,$(@)) && \ rm -f $(call obj2dep,$(@)).tmp
mk/target/generic/rte.vars.mk +1 -0
@@ -106,6 +106,7 @@ ifeq ($(BUILDING_RTE_SDK),1) # building sdk CFLAGS += -include $(RTE_OUTPUT)/include/rte_config.h CFLAGS += -DALLOW_EXPERIMENTAL_API +CFLAGS += -DALLOW_INTERNAL_API else # if we are building an external application, include SDK's lib and # includes too
Signature · 1 line
2.26.2
Re: [dpdk-dev] [PATCH v5 1/1] eal: add internal ABI marking support
Quoted 5 lines · Haiyue Wang
Introduce
__rte_internaltag to mark internal ABI function which is used by the drivers or other libraries.Signed-off-by Haiyue Wang <[email protected]>
Quoted 12 lines
--- buildtools/check-internal-syms.sh | 57 +++++++++++++++++++++++++++++ devtools/check-symbol-change.sh | 7 ++++ devtools/checkpatches.sh | 38 +++++++++++++++++++ devtools/libabigail.abignore | 5 +++ drivers/meson.build | 2 +- lib/librte_eal/include/rte_compat.h | 13 +++++++ lib/meson.build | 2 +- mk/internal/rte.compile-pre.mk | 3 ++ mk/target/generic/rte.vars.mk | 1 + 9 files changed, 126 insertions(+), 2 deletions(-) create mode 100755 buildtools/check-internal-syms.sh
We can have a single script (let's say devtools/check-symbols.sh) that contains both experimental and internal symbols.
Quoted 8 lines
devtools/check-symbol-change.sh +0
-0@@ -91,6 +91,13 @@ check_for_rule_violations() if [ "$ar" = "add" ] then
devtools/check-symbol-change.sh check_for_rule_violations() · line 103
Quoted 10 lines
devtools/check-symbol-change.sh +7
-0+ if [ "$secname" = "INTERNAL" ] + then + # these are absolved from any further checking + echo "Skipping symbol $symname in INTERNAL" + continue + fi + if [ "$secname" = "unknown" ] then # Just inform the user of this occurrence, but
This only handles symbol additions to the INTERNAL section. Other cases like moving or removing a INTERNAL symbol are not handled.
Quoted 77 lines
devtools/checkpatches.sh +38
-0@@ -111,6 +111,36 @@ check_experimental_tags() { # <patch> return $res } +check_internal_tags() { # <patch> + res=0 + + cat "$1" |awk ' + BEGIN { + current_file = ""; + ret = 0; + } + /^+++ b\// { + current_file = $2; + } + /^+.*__rte_internal/ { + if (current_file ~ ".c$" ) { + print "Please only put __rte_internal tags in " \ + "headers ("current_file")"; + ret = 1; + } + if ($1 != "+__rte_internal" || $2 != "") { + print "__rte_internal must appear alone on the line" \ + " immediately preceding the return type of a function." + ret = 1; + } + } + END { + exit ret; + }' || res=1 + + return $res +} + number=0 range='origin/master..' quiet=false@@ -194,6 +224,14 @@ check () { # <patch> <commit> <title> ret=1 fi + ! $verbose || printf '\nChecking __rte_internal tags:\n' + report=$(check_internal_tags "$tmpinput") + if [ $? -ne 0 ] ; then + $headline_printed || print_headline "$3" + printf '%s\n' "$report" + ret=1 + fi + if [ "$tmpinput" != "$1" ]; then rm -f "$tmpinput" trap - INTdevtools/libabigail.abignore +5
-0@@ -3,6 +3,11 @@ [suppress_variable] symbol_version = EXPERIMENTAL +[suppress_function] + symbol_version = INTERNAL +[suppress_variable] + symbol_version = INTERNAL + ; Explicit ignore for driver-only ABI [suppress_type] name = rte_cryptodev_opsdrivers/meson.build +0
-0
drivers/meson.build dpdk_driver_classes = ['common', · line 23
Quoted 5 lines
drivers/meson.build +1
-1@@ -20,7 +20,7 @@ dpdk_driver_classes = ['common', disabled_drivers = run_command(list_dir_globs, get_option('disable_drivers'), ).stdout().split() -default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API'] +default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API'] + ['-DALLOW_INTERNAL_API']
Nit:default_cflags = machine_argsdefault_cflags += ['-DALLOW_EXPERIMENTAL_API']default_cflags += ['-DALLOW_INTERNAL_API']
drivers/meson.build dpdk_driver_classes = ['common', · line 26
Quoted 3 lines
drivers/meson.build +0
-0if cc.has_argument('-Wno-format-truncation') default_cflags += '-Wno-format-truncation' endif
More importantly on this file:
- drivers/meson.build is not updated to check for internal symbols, see:
https://git.dpdk.org/dpdk/tree/drivers/meson.build#n166
- For fully experimental libraries, we have a special so version:
https://git.dpdk.org/dpdk/tree/drivers/meson.build#n131
This will apply to common drivers that will be 100% internal. Not sure if this is an issue.
Quoted 27 lines
lib/librte_eal/include/rte_compat.h +13
-0@@ -19,4 +19,17 @@ __attribute__((section(".text.experimental")))#endif +#ifndef ALLOW_INTERNAL_API + +#define __rte_internal \ +__attribute__((error("Symbol is not public ABI"), \ +section(".text.internal"))) + +#else + +#define __rte_internal \ +__attribute__((section(".text.internal"))) + +#endif +#endif /* _RTE_COMPAT_H_ */ lib/meson.build +0
-0
lib/meson.build if is_windows · line 41
Quoted 5 lines
lib/meson.build +1
-1@@ -38,7 +38,7 @@ if is_windows libraries = ['kvargs','eal'] # only supported libraries for windows endif -default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API'] +default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API'] + ['-DALLOW_INTERNAL_API']
Same comments as for drivers/meson.build.
mk/internal/rte.compile-pre.mk C_TO_O_DISP = $(if $(V),"$(C_TO_O_STR)"," CC $(@)") · line 62
Quoted 10 lines
mk/internal/rte.compile-pre.mk +2
-0@@ -58,6 +58,8 @@ C_TO_O_DISP = $(if $(V),"$(C_TO_O_STR)"," CC $(@)") endif EXPERIMENTAL_CHECK = $(RTE_SDK)/buildtools/check-experimental-syms.sh CHECK_EXPERIMENTAL = $(EXPERIMENTAL_CHECK) $(SRCDIR)/$(EXPORT_MAP) $@ +INTERNAL_CHECK = $(RTE_SDK)/buildtools/check-internal-syms.sh +CHECK_INTERNAL = $(INTERNAL_CHECK) $(SRCDIR)/$(EXPORT_MAP) $@
With a single script, we can go with:CHECK_SYMBOLS_SCRIPT = $(RTE_SDK)/buildtools/check-symbols.sh CHECK_SYMBOLS = $(CHECK_SYMBOLS_SCRIPT) $(SRCDIR)/$(EXPORT_MAP) $@
mk/internal/rte.compile-pre.mk C_TO_O_DO = @set -e; \ · line 81
Quoted 8 lines
mk/internal/rte.compile-pre.mk +1
-0PMDINFO_GEN = $(RTE_SDK_BIN)/app/dpdk-pmdinfogen $@ [email protected] PMDINFO_CC = $(CC) $(CPPFLAGS) $(CFLAGS) $(EXTRA_CFLAGS) -c -o [email protected] [email protected]@@ -76,6 +78,7 @@ C_TO_O_DO = @set -e; \ $(C_TO_O) && \ $(PMDINFO_TO_O) && \ $(CHECK_EXPERIMENTAL) && \ + $(CHECK_INTERNAL) && \
+ $(CHECK_SYMBOLS) && \
Quoted 16 lines
mk/internal/rte.compile-pre.mk +0
-0echo $(C_TO_O_CMD) > $(call obj2cmd,$(@)) && \ sed 's,'$@':,dep_'$@' =,' $(call obj2dep,$(@)).tmp > $(call obj2dep,$(@)) && \ rm -f $(call obj2dep,$(@)).tmpmk/target/generic/rte.vars.mk +1
-1@@ -106,6 +106,7 @@ ifeq ($(BUILDING_RTE_SDK),1) # building sdk CFLAGS += -include $(RTE_OUTPUT)/include/rte_config.h CFLAGS += -DALLOW_EXPERIMENTAL_API +CFLAGS += -DALLOW_INTERNAL_API else # if we are building an external application, include SDK's lib and # includes too --
Quoted 2 lines
2.26.2
We are missing updates on devtools/check-abi-version.sh and devtools/update_version_map_abi.py.
Signature · 1 line
David Marchand
Re: [dpdk-dev] [PATCH v5 1/1] eal: add internal ABI marking support
Hi David,
Try to fix the issues you mentioned, except below, plan to another patch set, I need more time to test these adding.
====
We are missing updates on devtools/check-abi-version.sh and devtools/update_version_map_abi.py.
More importantly on this file:
- drivers/meson.build is not updated to check for internal symbols, see:
https://git.dpdk.org/dpdk/tree/drivers/meson.build#n166
- For fully experimental libraries, we have a special so version:
https://git.dpdk.org/dpdk/tree/drivers/meson.build#n131
This will apply to common drivers that will be 100% internal. Not sure if this is an issue.
BR,
Haiyue
Quoted 247 lines
-----Original Message-----
From: David Marchand <[email protected]>
Sent: Friday, April 24, 2020 22:53
To: Wang, Haiyue <[email protected]>
Cc: dev <[email protected]>; Thomas Monjalon <[email protected]>; Richardson, Bruce <[email protected]>; Yigit, Ferruh <[email protected]>; Neil Horman <[email protected]>; Ray Kinsella <[email protected]>
Subject: Re: [PATCH v5 1/1] eal: add internal ABI marking supportQuoted 5 lines · Haiyue Wang
Introduce
__rte_internaltag to mark internal ABI function which is used by the drivers or other libraries.Signed-off-by Haiyue Wang <[email protected]>Quoted 12 lines
--- buildtools/check-internal-syms.sh | 57 +++++++++++++++++++++++++++++ devtools/check-symbol-change.sh | 7 ++++ devtools/checkpatches.sh | 38 +++++++++++++++++++ devtools/libabigail.abignore | 5 +++ drivers/meson.build | 2 +- lib/librte_eal/include/rte_compat.h | 13 +++++++ lib/meson.build | 2 +- mk/internal/rte.compile-pre.mk | 3 ++ mk/target/generic/rte.vars.mk | 1 + 9 files changed, 126 insertions(+), 2 deletions(-) create mode 100755 buildtools/check-internal-syms.shWe can have a single script (let's say
devtools/check-symbols.sh) that contains both experimental and internal symbols.Quoted 8 lines
devtools/check-symbol-change.sh +0
-0@@ -91,6 +91,13 @@ check_for_rule_violations() if [ "$ar" = "add" ] thendevtools/check-symbol-change.sh check_for_rule_violations() · line 103
Quoted 10 lines
devtools/check-symbol-change.sh +7
-0+ if [ "$secname" = "INTERNAL" ] + then + # these are absolved from any further checking + echo "Skipping symbol $symname in INTERNAL" + continue + fi + if [ "$secname" = "unknown" ] then # Just inform the user of this occurrence, butThis only handles symbol additions to the INTERNAL section. Other cases like moving or removing a INTERNAL symbol are not handled.
Quoted 77 lines
devtools/checkpatches.sh +38
-0@@ -111,6 +111,36 @@ check_experimental_tags() { # <patch> return $res } +check_internal_tags() { # <patch> + res=0 + + cat "$1" |awk ' + BEGIN { + current_file = ""; + ret = 0; + } + /^+++ b\// { + current_file = $2; + } + /^+.*__rte_internal/ { + if (current_file ~ ".c$" ) { + print "Please only put __rte_internal tags in " \ + "headers ("current_file")"; + ret = 1; + } + if ($1 != "+__rte_internal" || $2 != "") { + print "__rte_internal must appear alone on the line" \ + " immediately preceding the return type of a function." + ret = 1; + } + } + END { + exit ret; + }' || res=1 + + return $res +} + number=0 range='origin/master..' quiet=false@@ -194,6 +224,14 @@ check () { # <patch> <commit> <title> ret=1 fi + ! $verbose || printf '\nChecking __rte_internal tags:\n' + report=$(check_internal_tags "$tmpinput") + if [ $? -ne 0 ] ; then + $headline_printed || print_headline "$3" + printf '%s\n' "$report" + ret=1 + fi + if [ "$tmpinput" != "$1" ]; then rm -f "$tmpinput" trap - INTdevtools/libabigail.abignore +5
-0@@ -3,6 +3,11 @@ [suppress_variable] symbol_version = EXPERIMENTAL +[suppress_function] + symbol_version = INTERNAL +[suppress_variable] + symbol_version = INTERNAL + ; Explicit ignore for driver-only ABI [suppress_type] name = rte_cryptodev_opsdrivers/meson.build +0
-0drivers/meson.build dpdk_driver_classes = ['common', · line 23
Quoted 5 lines
drivers/meson.build +1
-1@@ -20,7 +20,7 @@ dpdk_driver_classes = ['common', disabled_drivers = run_command(list_dir_globs, get_option('disable_drivers'), ).stdout().split() -default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API'] +default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API'] + ['-DALLOW_INTERNAL_API']Nit:
default_cflags=machine_argsdefault_cflags+= ['-DALLOW_EXPERIMENTAL_API']default_cflags+= ['-DALLOW_INTERNAL_API']drivers/meson.build dpdk_driver_classes = ['common', · line 26
Quoted 3 lines
drivers/meson.build +0
-0if cc.has_argument('-Wno-format-truncation') default_cflags += '-Wno-format-truncation' endifMore importantly on this file:
-
drivers/meson.buildis not updated to check for internal symbols, see:
https://git.dpdk.org/dpdk/tree/drivers/meson.build#n166- For fully experimental libraries, we have a special so version:
https://git.dpdk.org/dpdk/tree/drivers/meson.build#n131This will apply to common drivers that will be 100% internal. Not sure if this is an issue.
Quoted 27 lines
lib/librte_eal/include/rte_compat.h +13
-0@@ -19,4 +19,17 @@ __attribute__((section(".text.experimental")))#endif +#ifndef ALLOW_INTERNAL_API + +#define __rte_internal \ +__attribute__((error("Symbol is not public ABI"), \ +section(".text.internal"))) + +#else + +#define __rte_internal \ +__attribute__((section(".text.internal"))) + +#endif +#endif /* _RTE_COMPAT_H_ */ lib/meson.build +0
-0lib/meson.build if is_windows · line 41
Quoted 5 lines
lib/meson.build +1
-1@@ -38,7 +38,7 @@ if is_windows libraries = ['kvargs','eal'] # only supported libraries for windows endif -default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API'] +default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API'] + ['-DALLOW_INTERNAL_API']Same comments as for
drivers/meson.build.mk/internal/rte.compile-pre.mk C_TO_O_DISP = $(if $(V),"$(C_TO_O_STR)"," CC $(@)") · line 62
Quoted 10 lines
mk/internal/rte.compile-pre.mk +2
-0@@ -58,6 +58,8 @@ C_TO_O_DISP = $(if $(V),"$(C_TO_O_STR)"," CC $(@)") endif EXPERIMENTAL_CHECK = $(RTE_SDK)/buildtools/check-experimental-syms.sh CHECK_EXPERIMENTAL = $(EXPERIMENTAL_CHECK) $(SRCDIR)/$(EXPORT_MAP) $@ +INTERNAL_CHECK = $(RTE_SDK)/buildtools/check-internal-syms.sh +CHECK_INTERNAL = $(INTERNAL_CHECK) $(SRCDIR)/$(EXPORT_MAP) $@With a single script, we can go with:
CHECK_SYMBOLS_SCRIPT= $(RTE_SDK)/buildtools/check-symbols.shCHECK_SYMBOLS= $(CHECK_SYMBOLS_SCRIPT) $(SRCDIR)/$(EXPORT_MAP) $@mk/internal/rte.compile-pre.mk C_TO_O_DO = @set -e; \ · line 81
Quoted 8 lines
mk/internal/rte.compile-pre.mk +1
-0PMDINFO_GEN = $(RTE_SDK_BIN)/app/dpdk-pmdinfogen $@ [email protected] PMDINFO_CC = $(CC) $(CPPFLAGS) $(CFLAGS) $(EXTRA_CFLAGS) -c -o [email protected] [email protected]@@ -76,6 +78,7 @@ C_TO_O_DO = @set -e; \ $(C_TO_O) && \ $(PMDINFO_TO_O) && \ $(CHECK_EXPERIMENTAL) && \ + $(CHECK_INTERNAL) && \+ $(CHECK_SYMBOLS) && \
Quoted 16 lines
mk/internal/rte.compile-pre.mk +0
-0echo $(C_TO_O_CMD) > $(call obj2cmd,$(@)) && \ sed 's,'$@':,dep_'$@' =,' $(call obj2dep,$(@)).tmp > $(call obj2dep,$(@)) && \ rm -f $(call obj2dep,$(@)).tmpmk/target/generic/rte.vars.mk +1
-1@@ -106,6 +106,7 @@ ifeq ($(BUILDING_RTE_SDK),1) # building sdk CFLAGS += -include $(RTE_OUTPUT)/include/rte_config.h CFLAGS += -DALLOW_EXPERIMENTAL_API +CFLAGS += -DALLOW_INTERNAL_API else # if we are building an external application, include SDK's lib and # includes too --Quoted 2 lines
2.26.2
We are missing updates on
devtools/check-abi-version.shanddevtools/update_version_map_abi.py.--
David Marchand
Re: [dpdk-dev] [PATCH v5 1/1] eal: add internal ABI marking support
Quoted 5 lines · Haiyue
Hi David,
Try to fix the issues you mentioned, except below, plan to another patch set, I need more time to test these adding.
Thanks for working on this topic.
Quoted 5 lines
====
We are missing updates on
devtools/check-abi-version.shanddevtools/update_version_map_abi.py.
Those two scripts can be updated later:
- I suspect the first one to be broken already,
- the 2nd one is for 20.11 when we will update all map files.
Quoted 5 lines
More importantly on this file:
-
drivers/meson.buildis not updated to check for internal symbols, see:
https://git.dpdk.org/dpdk/tree/drivers/meson.build#n166
On this point, your series is almost good to go, I would just get rid of the "experimental" mentions.
I will reply on the patch.
Quoted 5 lines
- For fully experimental libraries, we have a special so version:
https://git.dpdk.org/dpdk/tree/drivers/meson.build#n131This will apply to common drivers that will be 100% internal. Not sure if this is an issue.
This part should be fine, I want others to be aware of this.
Signature · 1 line
David Marchand
Re: [dpdk-dev] [PATCH v5 1/1] eal: add internal ABI marking support
25/04/2020 16:21, David Marchand:
Quoted 8 lines · Haiyue
Quoted 5 lines
- For fully experimental libraries, we have a special so version:
https://git.dpdk.org/dpdk/tree/drivers/meson.build#n131This will apply to common drivers that will be 100% internal. Not sure if this is an issue.
This part should be fine, I want others to be aware of this.
I am not one of the ABI maintainers, but in my opinion it is OK to have "pure internal" libs with version 0.x.
v6 cover dpdk: introduce __rte_internal tag
In reply to Neil HormanMove the internal function into INTERNAL session to avoid the ABI checking, and it is only used for DPDK drivers or related library.
__rte_internal funA
INTERNAL {
global:
funA
};
v6: split into small patches, and add the missed handling.
v5: add the checkpatch for __rte_internal style
v4: add the ABI check suppression rules
v3: based on Neil's v2 patch https://patchwork.dpdk.org/cover/54771/
Use the ALLOW_INTERNAL_API to mark this new feature.
Haiyue Wang (6):
eal: add internal ABI tag definition
build: enable internal API tag
mk: add internal tag check
devtools: ignore internal ABI check
devtools: exempt internal ABI checking
devtools: enforce internal tag at the beginning
...-experimental-syms.sh => check-symbols.sh} | 31 +++++++++++++++
devtools/check-symbol-change.sh | 8 ++++
devtools/checkpatches.sh | 39 +++++++++++++++++++
devtools/libabigail.abignore | 5 +++
drivers/meson.build | 5 ++-
lib/librte_eal/include/rte_compat.h | 13 +++++++
lib/meson.build | 5 ++-
mk/internal/rte.compile-pre.mk | 6 +--
mk/target/generic/rte.vars.mk | 1 +
9 files changed, 108 insertions(+), 5 deletions(-)
rename buildtools/{check-experimental-syms.sh => check-symbols.sh} (61%)Signature · 1 line
2.26.2
PATCH v6 1/6 eal: add internal ABI tag definition
Introduce the __rte_internal tag to mark internal ABI function which is used only by the drivers or other libraries.
--- lib/librte_eal/include/rte_compat.h | 13 +++++++++++++ 1 file changed, 13 insertions(+)
lib/librte_eal/include/rte_compat.h +13 -0
@@ -19,4 +19,17 @@ __attribute__((section(".text.experimental")))#endif +#ifndef ALLOW_INTERNAL_API + +#define __rte_internal \ +__attribute__((error("Symbol is not public ABI"), \ +section(".text.internal"))) + +#else + +#define __rte_internal \ +__attribute__((section(".text.internal"))) + +#endif +#endif /* _RTE_COMPAT_H_ */
Signature · 1 line
2.26.2
PATCH v6 2/6 build: enable internal API tag
In reply to Haiyue WangAllow the drivers and libraries to use the internal tag for marking internal ABI symbols.
--- drivers/meson.build | 5 ++++- lib/meson.build | 5 ++++- mk/target/generic/rte.vars.mk | 1 + 3 files changed, 9 insertions(+), 2 deletions(-)
drivers/meson.build +4 -1
@@ -20,7 +20,10 @@ dpdk_driver_classes = ['common', disabled_drivers = run_command(list_dir_globs, get_option('disable_drivers'), ).stdout().split() -default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API'] +default_cflags = machine_args +default_cflags += ['-DALLOW_EXPERIMENTAL_API'] +default_cflags += ['-DALLOW_INTERNAL_API'] + if cc.has_argument('-Wno-format-truncation') default_cflags += '-Wno-format-truncation' endif
lib/meson.build +4 -1
@@ -38,7 +38,10 @@ if is_windows libraries = ['kvargs','eal'] # only supported libraries for windows endif -default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API'] +default_cflags = machine_args +default_cflags += ['-DALLOW_EXPERIMENTAL_API'] +default_cflags += ['-DALLOW_INTERNAL_API'] + if cc.has_argument('-Wno-format-truncation') default_cflags += '-Wno-format-truncation' endif
mk/target/generic/rte.vars.mk +1 -0
@@ -106,6 +106,7 @@ ifeq ($(BUILDING_RTE_SDK),1) # building sdk CFLAGS += -include $(RTE_OUTPUT)/include/rte_config.h CFLAGS += -DALLOW_EXPERIMENTAL_API +CFLAGS += -DALLOW_INTERNAL_API else # if we are building an external application, include SDK's lib and # includes too
Signature · 1 line
2.26.2
PATCH v6 3/6 mk: add internal tag check
In reply to Haiyue WangAdd checks during build to ensure that all symbols in the INTERNAL version map section have __internal tags on their definitions, and enable the warnings needed to announce their use.
---
...-experimental-syms.sh => check-symbols.sh} | 31 +++++++++++++++++++
mk/internal/rte.compile-pre.mk | 6 ++--
2 files changed, 34 insertions(+), 3 deletions(-)
rename buildtools/{check-experimental-syms.sh => check-symbols.sh} (61%)buildtools/check-symbols.sh renamed from buildtools/check-experimental-syms.sh +31 -0
@@ -54,4 +54,35 @@ do } done +for SYM in `$LIST_SYMBOL -S INTERNAL $MAPFILE |cut -d ' ' -f 3` +do + if grep -q "\.text.*[[:space:]]$SYM$" $DUMPFILE && + ! grep -q "\.text\.internal.*[[:space:]]$SYM$" $DUMPFILE + then + cat >&2 <<- END_OF_MESSAGE + $SYM is not flagged as internal + but is listed in version map + Please add __rte_internal to the definition of $SYM + END_OF_MESSAGE + ret=1 + fi +done + +# Filter out symbols suffixed with a . for icc +for SYM in `awk '{ + if ($2 != "l" && $4 == ".text.internal" && !($NF ~ /\.$/)) { + print $NF + } +}' $DUMPFILE` +do + $LIST_SYMBOL -S INTERNAL -s $SYM -q $MAPFILE || { + cat >&2 <<- END_OF_MESSAGE + $SYM is flagged as internal + but is not listed in version map + Please add $SYM to the version map + END_OF_MESSAGE + ret=1 + } +done + exit $ret
mk/internal/rte.compile-pre.mk +3 -3
@@ -56,8 +56,8 @@ C_TO_O = $(CC) -Wp,-MD,$(call obj2dep,$(@)).tmp $(CPPFLAGS) $(CFLAGS) \ C_TO_O_STR = $(subst ','\'',$(C_TO_O)) #'# fix syntax highlight C_TO_O_DISP = $(if $(V),"$(C_TO_O_STR)"," CC $(@)") endif -EXPERIMENTAL_CHECK = $(RTE_SDK)/buildtools/check-experimental-syms.sh -CHECK_EXPERIMENTAL = $(EXPERIMENTAL_CHECK) $(SRCDIR)/$(EXPORT_MAP) $@ +CHECK_SYMBOLS_SCRIPT = $(RTE_SDK)/buildtools/check-symbols.sh +CHECK_SYMBOLS = $(CHECK_SYMBOLS_SCRIPT) $(SRCDIR)/$(EXPORT_MAP) $@ PMDINFO_GEN = $(RTE_SDK_BIN)/app/dpdk-pmdinfogen $@ [email protected] PMDINFO_CC = $(CC) $(CPPFLAGS) $(CFLAGS) $(EXTRA_CFLAGS) -c -o [email protected] [email protected]
@@ -75,7 +75,7 @@ C_TO_O_DO = @set -e; \ echo $(C_TO_O_DISP); \ $(C_TO_O) && \ $(PMDINFO_TO_O) && \ - $(CHECK_EXPERIMENTAL) && \ + $(CHECK_SYMBOLS) && \ echo $(C_TO_O_CMD) > $(call obj2cmd,$(@)) && \ sed 's,'$@':,dep_'$@' =,' $(call obj2dep,$(@)).tmp > $(call obj2dep,$(@)) && \ rm -f $(call obj2dep,$(@)).tmp
Signature · 1 line
2.26.2
PATCH v6 4/6 devtools: ignore internal ABI check
In reply to Haiyue WangIgnore the internal version ABI check, this kind of ABI is used only by drivers and libraries.
--- devtools/libabigail.abignore | 5 +++++ 1 file changed, 5 insertions(+)
devtools/libabigail.abignore +5 -0
@@ -3,6 +3,11 @@ [suppress_variable] symbol_version = EXPERIMENTAL +[suppress_function] + symbol_version = INTERNAL +[suppress_variable] + symbol_version = INTERNAL + ; Explicit ignore for driver-only ABI [suppress_type] name = rte_cryptodev_ops
Signature · 1 line
2.26.2
PATCH v6 5/6 devtools: exempt internal ABI checking
In reply to Haiyue WangNo need to restrict the ABI on symbols that are only used by core libraries.
--- devtools/check-symbol-change.sh | 8 ++++++++ 1 file changed, 8 insertions(+)
devtools/check-symbol-change.sh +8 -0
@@ -91,6 +91,13 @@ check_for_rule_violations() if [ "$ar" = "add" ] then + if [ "$secname" = "INTERNAL" ] + then + # these are absolved from any further checking + echo "Skipping symbol $symname in INTERNAL" + continue + fi + if [ "$secname" = "unknown" ] then # Just inform the user of this occurrence, but
@@ -148,6 +155,7 @@ check_for_rule_violations() else if ! grep -q "$mname $symname .* add" "$mapdb" && \ + [ "$secname" != "INTERNAL" ] && \ [ "$secname" != "EXPERIMENTAL" ] then # Just inform users that non-experimenal
Signature · 1 line
2.26.2
PATCH v6 6/6 devtools: enforce internal tag at the beginning
In reply to Haiyue WangMove the internal tag on a separate line and make it the first thing of function prototypes.
--- devtools/checkpatches.sh | 39 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+)
devtools/checkpatches.sh +39 -0
@@ -111,6 +111,37 @@ check_experimental_tags() { # <patch> return $res } +check_internal_tags() { # <patch> + res=0 + + cat "$1" |awk ' + BEGIN { + current_file = ""; + ret = 0; + } + /^+++ b\// { + current_file = $2; + } + /^+.*__rte_internal/ { + if (current_file ~ ".c$" ) { + print "Please only put __rte_internal tags in " \ + "headers ("current_file")"; + ret = 1; + } + if ($1 != "+__rte_internal" || $2 != "") { + print "__rte_internal must appear alone on the line" \ + " immediately preceding the return type of" \ + " a function." + ret = 1; + } + } + END { + exit ret; + }' || res=1 + + return $res +} + number=0 range='origin/master..' quiet=false
@@ -194,6 +225,14 @@ check () { # <patch> <commit> <title> ret=1 fi + ! $verbose || printf '\nChecking __rte_internal tags:\n' + report=$(check_internal_tags "$tmpinput") + if [ $? -ne 0 ] ; then + $headline_printed || print_headline "$3" + printf '%s\n' "$report" + ret=1 + fi + if [ "$tmpinput" != "$1" ]; then rm -f "$tmpinput" trap - INT
Signature · 1 line
2.26.2
v7 cover dpdk: introduce __rte_internal tag
In reply to Neil HormanMove the internal function into INTERNAL session to avoid the ABI checking, and it is only used for DPDK drivers or related library.
__rte_internal funA
INTERNAL {
global:
funA
};
v7: Fix the meson build error
v6: split into small patches, and add the missed handling.
v5: add the checkpatch for __rte_internal style
v4: add the ABI check suppression rules
v3: based on Neil's v2 patch https://patchwork.dpdk.org/cover/54771/
Use the ALLOW_INTERNAL_API to mark this new feature.
Haiyue Wang (6):
eal: add internal ABI tag definition
build: enable internal API tag
mk: add internal tag check
devtools: ignore internal ABI check
devtools: exempt internal ABI checking
devtools: enforce internal tag at the beginning
MAINTAINERS | 2 +-
...-experimental-syms.sh => check-symbols.sh} | 31 +++++++++++++++
buildtools/meson.build | 2 +-
devtools/check-symbol-change.sh | 8 ++++
devtools/checkpatches.sh | 39 +++++++++++++++++++
devtools/libabigail.abignore | 5 +++
drivers/meson.build | 5 ++-
lib/librte_eal/include/rte_compat.h | 13 +++++++
lib/meson.build | 5 ++-
mk/internal/rte.compile-pre.mk | 6 +--
mk/target/generic/rte.vars.mk | 1 +
11 files changed, 110 insertions(+), 7 deletions(-)
rename buildtools/{check-experimental-syms.sh => check-symbols.sh} (61%)Signature · 1 line
2.26.2
PATCH v7 1/6 eal: add internal ABI tag definition
Introduce the __rte_internal tag to mark internal ABI function which is used only by the drivers or other libraries.
--- lib/librte_eal/include/rte_compat.h | 13 +++++++++++++ 1 file changed, 13 insertions(+)
lib/librte_eal/include/rte_compat.h +13 -0
@@ -19,4 +19,17 @@ __attribute__((section(".text.experimental")))#endif +#ifndef ALLOW_INTERNAL_API + +#define __rte_internal \ +__attribute__((error("Symbol is not public ABI"), \ +section(".text.internal"))) + +#else + +#define __rte_internal \ +__attribute__((section(".text.internal"))) + +#endif +#endif /* _RTE_COMPAT_H_ */
Signature · 1 line
2.26.2
PATCH v7 2/6 build: enable internal API tag
In reply to Haiyue WangAllow the drivers and libraries to use the internal tag for marking internal ABI symbols.
--- drivers/meson.build | 5 ++++- lib/meson.build | 5 ++++- mk/target/generic/rte.vars.mk | 1 + 3 files changed, 9 insertions(+), 2 deletions(-)
drivers/meson.build +4 -1
@@ -20,7 +20,10 @@ dpdk_driver_classes = ['common', disabled_drivers = run_command(list_dir_globs, get_option('disable_drivers'), ).stdout().split() -default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API'] +default_cflags = machine_args +default_cflags += ['-DALLOW_EXPERIMENTAL_API'] +default_cflags += ['-DALLOW_INTERNAL_API'] + if cc.has_argument('-Wno-format-truncation') default_cflags += '-Wno-format-truncation' endif
lib/meson.build +4 -1
@@ -38,7 +38,10 @@ if is_windows libraries = ['kvargs','eal'] # only supported libraries for windows endif -default_cflags = machine_args + ['-DALLOW_EXPERIMENTAL_API'] +default_cflags = machine_args +default_cflags += ['-DALLOW_EXPERIMENTAL_API'] +default_cflags += ['-DALLOW_INTERNAL_API'] + if cc.has_argument('-Wno-format-truncation') default_cflags += '-Wno-format-truncation' endif
mk/target/generic/rte.vars.mk +1 -0
@@ -106,6 +106,7 @@ ifeq ($(BUILDING_RTE_SDK),1) # building sdk CFLAGS += -include $(RTE_OUTPUT)/include/rte_config.h CFLAGS += -DALLOW_EXPERIMENTAL_API +CFLAGS += -DALLOW_INTERNAL_API else # if we are building an external application, include SDK's lib and # includes too
Signature · 1 line
2.26.2
PATCH v7 3/6 mk: add internal tag check
In reply to Haiyue WangAdd checks during build to ensure that all symbols in the INTERNAL version map section have __internal tags on their definitions, and enable the warnings needed to announce their use.
---
MAINTAINERS | 2 +-
...-experimental-syms.sh => check-symbols.sh} | 31 +++++++++++++++++++
buildtools/meson.build | 2 +-
mk/internal/rte.compile-pre.mk | 6 ++--
4 files changed, 36 insertions(+), 5 deletions(-)
rename buildtools/{check-experimental-syms.sh => check-symbols.sh} (61%)MAINTAINERS +1 -1
@@ -153,7 +153,7 @@ F: devtools/libabigail.abignore F: devtools/update-abi.sh F: devtools/update_version_map_abi.py F: devtools/validate-abi.sh -F: buildtools/check-experimental-syms.sh +F: buildtools/check-symbols.sh F: buildtools/map-list-symbol.sh Driver information
buildtools/check-symbols.sh renamed from buildtools/check-experimental-syms.sh +31 -0
@@ -54,4 +54,35 @@ do } done +for SYM in `$LIST_SYMBOL -S INTERNAL $MAPFILE |cut -d ' ' -f 3` +do + if grep -q "\.text.*[[:space:]]$SYM$" $DUMPFILE && + ! grep -q "\.text\.internal.*[[:space:]]$SYM$" $DUMPFILE + then + cat >&2 <<- END_OF_MESSAGE + $SYM is not flagged as internal + but is listed in version map + Please add __rte_internal to the definition of $SYM + END_OF_MESSAGE + ret=1 + fi +done + +# Filter out symbols suffixed with a . for icc +for SYM in `awk '{ + if ($2 != "l" && $4 == ".text.internal" && !($NF ~ /\.$/)) { + print $NF + } +}' $DUMPFILE` +do + $LIST_SYMBOL -S INTERNAL -s $SYM -q $MAPFILE || { + cat >&2 <<- END_OF_MESSAGE + $SYM is flagged as internal + but is not listed in version map + Please add $SYM to the version map + END_OF_MESSAGE + ret=1 + } +done + exit $ret
buildtools/meson.build +1 -1
@@ -6,7 +6,7 @@ subdir('pmdinfogen') pkgconf = find_program('pkg-config', 'pkgconf', required: false) pmdinfo = find_program('gen-pmdinfo-cfile.sh') list_dir_globs = find_program('list-dir-globs.py') -check_experimental_syms = find_program('check-experimental-syms.sh') +check_experimental_syms = find_program('check-symbols.sh') ldflags_ibverbs_static = find_program('options-ibverbs-static.sh') # set up map-to-def script using python, either built-in or external
mk/internal/rte.compile-pre.mk +3 -3
@@ -56,8 +56,8 @@ C_TO_O = $(CC) -Wp,-MD,$(call obj2dep,$(@)).tmp $(CPPFLAGS) $(CFLAGS) \ C_TO_O_STR = $(subst ','\'',$(C_TO_O)) #'# fix syntax highlight C_TO_O_DISP = $(if $(V),"$(C_TO_O_STR)"," CC $(@)") endif -EXPERIMENTAL_CHECK = $(RTE_SDK)/buildtools/check-experimental-syms.sh -CHECK_EXPERIMENTAL = $(EXPERIMENTAL_CHECK) $(SRCDIR)/$(EXPORT_MAP) $@ +CHECK_SYMBOLS_SCRIPT = $(RTE_SDK)/buildtools/check-symbols.sh +CHECK_SYMBOLS = $(CHECK_SYMBOLS_SCRIPT) $(SRCDIR)/$(EXPORT_MAP) $@ PMDINFO_GEN = $(RTE_SDK_BIN)/app/dpdk-pmdinfogen $@ [email protected] PMDINFO_CC = $(CC) $(CPPFLAGS) $(CFLAGS) $(EXTRA_CFLAGS) -c -o [email protected] [email protected]
@@ -75,7 +75,7 @@ C_TO_O_DO = @set -e; \ echo $(C_TO_O_DISP); \ $(C_TO_O) && \ $(PMDINFO_TO_O) && \ - $(CHECK_EXPERIMENTAL) && \ + $(CHECK_SYMBOLS) && \ echo $(C_TO_O_CMD) > $(call obj2cmd,$(@)) && \ sed 's,'$@':,dep_'$@' =,' $(call obj2dep,$(@)).tmp > $(call obj2dep,$(@)) && \ rm -f $(call obj2dep,$(@)).tmp
Signature · 1 line
2.26.2
Re: [dpdk-dev] [PATCH v7 3/6] mk: add internal tag check
Quoted 5 lines · Haiyue Wang
Add checks during build to ensure that all symbols in the INTERNAL version map section have
__internaltags on their definitions, and enable the warnings needed to announce their use.
Quoted 8 lines
Signed-off-by Haiyue Wang <[email protected]>--- MAINTAINERS | 2 +- ...-experimental-syms.sh => check-symbols.sh} | 31 +++++++++++++++++++ buildtools/meson.build | 2 +- mk/internal/rte.compile-pre.mk | 6 ++-- 4 files changed, 36 insertions(+), 5 deletions(-) rename buildtools/{check-experimental-syms.sh => check-symbols.sh} (61%)
Just missing a little update on drivers/meson.build and lib/meson.build. Squashed with:
buildtools/meson.build +2 -2
@@ -6,7 +6,7 @@ subdir('pmdinfogen') pkgconf = find_program('pkg-config', 'pkgconf', required: false) pmdinfo = find_program('gen-pmdinfo-cfile.sh') list_dir_globs = find_program('list-dir-globs.py') -check_experimental_syms = find_program('check-symbols.sh') +check_symbols = find_program('check-symbols.sh') ldflags_ibverbs_static = find_program('options-ibverbs-static.sh') # set up map-to-def script using python, either built-in or external
@@ -20,4 +20,4 @@ map_to_def_cmd = py3 + files('map_to_def.py') sphinx_wrapper = py3 + files('call-sphinx-build.py') # stable ABI always starts with "DPDK_" -is_experimental_cmd = [find_program('grep', 'findstr'), '^DPDK_'] +is_stable_cmd = [find_program('grep', 'findstr'), '^DPDK_']
drivers/meson.build +6 -6
@@ -131,15 +131,15 @@ foreach class:dpdk_driver_classes meson.current_source_dir(), drv_path, lib_name) - is_experimental = run_command(is_experimental_cmd, - files(version_map)).returncode() + is_stable = run_command(is_stable_cmd, + files(version_map)).returncode() == 0 - if is_experimental != 0 - lib_version = experimental_abi_version - so_version = experimental_so_version - else + if is_stable lib_version = abi_version so_version = stable_so_version + else + lib_version = experimental_abi_version + so_version = experimental_so_version endif # now build the static driver
@@ -168,14 +168,14 @@ foreach class:dpdk_driver_classes else lk_args = ['-Wl,--version-script=' +
version_map]
# on unix systems check the output of the
- # experimental syms script, using it as a
+ # check-symbols.sh script, using it as a
# dependency of the .so build
- lk_deps += custom_target(lib_name + '.exp_chk',
- command: [check_experimental_syms,
+ lk_deps += custom_target(lib_name + '.sym_chk',
+ command: [check_symbols,
version_map, '@INPUT@'],
capture: true,
input: static_lib,
- output: lib_name + '.exp_chk')
+ output: lib_name + '.sym_chk')
endif shared_lib = shared_library(lib_name,
lib/meson.build +0 -0
@@ -109,15 +109,15 @@ foreach l:libraries version_map = '@0@/@1@/rte_@2@_version.map'.format( meson.current_source_dir(),
dir_name, name)
- is_experimental = run_command(is_experimental_cmd,
- files(version_map)).returncode()
+ is_stable = run_command(is_stable_cmd,
+ files(version_map)).returncode() == 0
- if is_experimental != 0
- lib_version = experimental_abi_version
- so_version = experimental_so_version
- else
+ if is_stable
lib_version = abi_version so_version = stable_so_version
+ else
+ lib_version = experimental_abi_version
+ so_version = experimental_so_version
endif
# first build static lib
@@ -160,14 +160,14 @@ foreach l:libraries
lk_deps = [version_map, def_file]
if not is_windows
# on unix systems check the output of the
- # experimental syms script, using it as a
+ # check-symbols.sh script, using it as a
# dependency of the .so build
- lk_deps += custom_target(name + '.exp_chk',
- command: [check_experimental_syms,
+ lk_deps += custom_target(name + '.sym_chk',
+ command: [check_symbols,
version_map, '@INPUT@'],
capture: true,
input: static_lib,
- output: name + '.exp_chk')
+ output: name + '.sym_chk')
endif shared_lib = shared_library(libname,
Signature · 1 line
David Marchand
PATCH v7 4/6 devtools: ignore internal ABI check
In reply to Haiyue WangIgnore the internal version ABI check, this kind of ABI is used only by drivers and libraries.
--- devtools/libabigail.abignore | 5 +++++ 1 file changed, 5 insertions(+)
devtools/libabigail.abignore +5 -0
@@ -3,6 +3,11 @@ [suppress_variable] symbol_version = EXPERIMENTAL +[suppress_function] + symbol_version = INTERNAL +[suppress_variable] + symbol_version = INTERNAL + ; Explicit ignore for driver-only ABI [suppress_type] name = rte_cryptodev_ops
Signature · 1 line
2.26.2
PATCH v7 5/6 devtools: exempt internal ABI checking
In reply to Haiyue WangNo need to restrict the ABI on symbols that are only used by core libraries.
--- devtools/check-symbol-change.sh | 8 ++++++++ 1 file changed, 8 insertions(+)
devtools/check-symbol-change.sh +8 -0
@@ -91,6 +91,13 @@ check_for_rule_violations() if [ "$ar" = "add" ] then + if [ "$secname" = "INTERNAL" ] + then + # these are absolved from any further checking + echo "Skipping symbol $symname in INTERNAL" + continue + fi + if [ "$secname" = "unknown" ] then # Just inform the user of this occurrence, but
@@ -148,6 +155,7 @@ check_for_rule_violations() else if ! grep -q "$mname $symname .* add" "$mapdb" && \ + [ "$secname" != "INTERNAL" ] && \ [ "$secname" != "EXPERIMENTAL" ] then # Just inform users that non-experimenal
Signature · 1 line
2.26.2
Re: [dpdk-dev] [PATCH v7 5/6] devtools: exempt internal ABI checking
Quoted 5 lines · Haiyue Wang
No need to restrict the ABI on symbols that are only used by core libraries.
Signed-off-by Haiyue Wang <[email protected]>
Rather than add a special case for INTERNAL, we can invert the logic in this script: identify "stable" sections symbol.
I went with the following patch:
devtools/check-symbol-change.sh +9 -5
@@ -77,6 +77,10 @@ build_map_changes() } +is_stable_section() { + [ "$1" != 'EXPERIMENTAL' ] && [ "$1" != 'INTERNAL' ] +} + check_for_rule_violations() { local mapdb="$1"
@@ -110,11 +114,11 @@ check_for_rule_violations() # section directly if [ -z "$oldsecname" ] then - if [ "$secname" = 'EXPERIMENTAL' ] + if ! is_stable_section $secname then echo -n "INFO: symbol $symname has " echo -n "been added to the " - echo -n "EXPERIMENTAL section of the " + echo -n "$secname section of the " echo "version map" continue else
@@ -137,7 +141,7 @@ check_for_rule_violations() # This symbol is moving between two sections (the # original section is not experimental). # This can be legit, just warn. - if [ "$oldsecname" != 'EXPERIMENTAL' ] + if is_stable_section $oldsecname then echo -n "INFO: symbol $symname is being " echo -n "moved from $oldsecname to $secname. "
@@ -148,9 +152,9 @@ check_for_rule_violations() else if ! grep -q "$mname $symname .* add" "$mapdb" && \ - [ "$secname" != "EXPERIMENTAL" ] + is_stable_section $secname then - # Just inform users that non-experimenal + # Just inform users that stable # symbols need to go through a deprecation # process echo -n "INFO: symbol $symname is being "
Signature · 1 line
David Marchand
PATCH v7 6/6 devtools: enforce internal tag at the beginning
In reply to Haiyue WangMove the internal tag on a separate line and make it the first thing of function prototypes.
--- devtools/checkpatches.sh | 39 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+)
devtools/checkpatches.sh +39 -0
@@ -111,6 +111,37 @@ check_experimental_tags() { # <patch> return $res } +check_internal_tags() { # <patch> + res=0 + + cat "$1" |awk ' + BEGIN { + current_file = ""; + ret = 0; + } + /^+++ b\// { + current_file = $2; + } + /^+.*__rte_internal/ { + if (current_file ~ ".c$" ) { + print "Please only put __rte_internal tags in " \ + "headers ("current_file")"; + ret = 1; + } + if ($1 != "+__rte_internal" || $2 != "") { + print "__rte_internal must appear alone on the line" \ + " immediately preceding the return type of" \ + " a function." + ret = 1; + } + } + END { + exit ret; + }' || res=1 + + return $res +} + number=0 range='origin/master..' quiet=false
@@ -194,6 +225,14 @@ check () { # <patch> <commit> <title> ret=1 fi + ! $verbose || printf '\nChecking __rte_internal tags:\n' + report=$(check_internal_tags "$tmpinput") + if [ $? -ne 0 ] ; then + $headline_printed || print_headline "$3" + printf '%s\n' "$report" + ret=1 + fi + if [ "$tmpinput" != "$1" ]; then rm -f "$tmpinput" trap - INT
Signature · 1 line
2.26.2
Re: [dpdk-dev] [PATCH v7 0/6] dpdk: introduce __rte_internal tag
In reply to Haiyue WangQuoted 11 lines · Haiyue Wang
Move the internal function into INTERNAL session to avoid the ABI checking, and it is only used for DPDK drivers or related library.
__rte_internalfunAINTERNAL {
global:funA
};
Thanks a lot for working on this.
I did some modifications (see my replies on patch 3 and 5) and applied this series.
We are just missing the update on the scripts mentioned in a previous mail. Can you work on this for rc2?
Thanks again!
Signature · 1 line
David Marchand
Re: [dpdk-dev] [PATCH v7 0/6] dpdk: introduce __rte_internal tag
Quoted 22 lines
-----Original Message-----
From: David Marchand <[email protected]>
Sent: Saturday, April 25, 2020 22:39
To: Wang, Haiyue <[email protected]>
Cc: dev <[email protected]>; Thomas Monjalon <[email protected]>; Richardson, Bruce <[email protected]>; Yigit, Ferruh <[email protected]>; Neil Horman <[email protected]>; Ray Kinsella <[email protected]>
Subject: Re: [PATCH v7 0/6] dpdk: introduce__rte_internaltagQuoted 11 lines · Haiyue Wang
Move the internal function into INTERNAL session to avoid the ABI checking, and it is only used for DPDK drivers or related library.
__rte_internalfunAINTERNAL {
global:funA
};
Quoted 7 lines
Thanks a lot for working on this.
I did some modifications (see my replies on patch 3 and 5) and applied this series.We are just missing the update on the scripts mentioned in a previous mail. Can you work on this for rc2?
Sure, it's my pleasure. ;-)
Quoted 5 lines
Thanks again!
--
David Marchand
Re: [dpdk-dev] [PATCH v7 0/6] dpdk: introduce __rte_internal tag
In reply to David MarchandHi David,
Quoted 22 lines
-----Original Message-----
From: David Marchand <[email protected]>
Sent: Saturday, April 25, 2020 22:39
To: Wang, Haiyue <[email protected]>
Cc: dev <[email protected]>; Thomas Monjalon <[email protected]>; Richardson, Bruce <[email protected]>; Yigit, Ferruh <[email protected]>; Neil Horman <[email protected]>; Ray Kinsella <[email protected]>
Subject: Re: [PATCH v7 0/6] dpdk: introduce__rte_internaltagQuoted 11 lines · Haiyue Wang
Move the internal function into INTERNAL session to avoid the ABI checking, and it is only used for DPDK drivers or related library.
__rte_internalfunAINTERNAL {
global:funA
};
Quoted 7 lines
Thanks a lot for working on this.
I did some modifications (see my replies on patch 3 and 5) and applied this series.We are just missing the update on the scripts mentioned in a previous mail. Can you work on this for rc2?
Do you mean ?
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
This will apply to common drivers that will be 100% internal. Not sure if this is an issue.
This part should be fine, I want others to be aware of this.
I am not one of the ABI maintainers, but in my opinion it is OK to have "pure internal" libs with version 0.x.
I've tested it with Intel's drivers/common/iavf, it works as expected. a). librte_common_iavf.so.0.200.2
b). Skipped experimental library librte_common_iavf.dump.
This has been updated by your modification.
+ if is_stable
lib_version = abi_version so_version = stable_so_version
+ else
+ lib_version = experimental_abi_version
+ so_version = experimental_so_version
endif
Quoted 5 lines
Thanks again!
--
David Marchand
Re: [dpdk-dev] [PATCH v7 0/6] dpdk: introduce __rte_internal tag
Quoted 29 lines · Haiyue
Hi David,
Quoted 26 lines
-----Original Message-----
From: David Marchand <[email protected]>
Sent: Saturday, April 25, 2020 22:39
To: Wang, Haiyue <[email protected]>
Cc: dev <[email protected]>; Thomas Monjalon <[email protected]>; Richardson, Bruce <[email protected]>; Yigit, Ferruh <[email protected]>; Neil Horman <[email protected]>; Ray Kinsella <[email protected]>
Subject: Re: [PATCH v7 0/6] dpdk: introduce__rte_internaltagQuoted 11 lines · Haiyue Wang
Move the internal function into INTERNAL session to avoid the ABI checking, and it is only used for DPDK drivers or related library.
__rte_internalfunAINTERNAL {
global:funA
};Thanks a lot for working on this.
I did some modifications (see my replies on patch 3 and 5) and applied this series.
Quoted 5 lines
Quoted 3 lines
We are just missing the update on the scripts mentioned in a previous mail. Can you work on this for rc2?
Do you mean ?
Sorry, I was referring to another mail:
"""
Quoted 2 lines
We are missing updates on
devtools/check-abi-version.shanddevtools/update_version_map_abi.py.
"""
Those scripts devtools/check-abi-version.sh and devtools/update_version_map_abi.py are not "internal" aware. Can you take care of them?
Quoted 13 lines
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
This will apply to common drivers that will be 100% internal. Not sure if this is an issue.
This part should be fine, I want others to be aware of this.
I am not one of the ABI maintainers, but in my opinion it is OK to have "pure internal" libs with version 0.x.
I've tested it with Intel's
drivers/common/iavf, it works as expected. a). librte_common_iavf.so.0.200.2
b). Skipped experimental library librte_common_iavf.dump.This has been updated by your modification.
Review
Quoted 7 lines
(quoted diff) +4
-0+ if is_stable lib_version = abi_version so_version = stable_so_version + else + lib_version = experimental_abi_version + so_version = experimental_so_version endif
Thanks for testing.
Signature · 1 line
David Marchand
Re: [dpdk-dev] [PATCH v7 0/6] dpdk: introduce __rte_internal tag
Quoted 47 lines
-----Original Message-----
From: David Marchand <[email protected]>
Sent: Wednesday, April 29, 2020 16:22
To: Wang, Haiyue <[email protected]>
Cc: dev <[email protected]>; Thomas Monjalon <[email protected]>; Richardson, Bruce <[email protected]>; Yigit, Ferruh <[email protected]>; Neil Horman <[email protected]>; Ray Kinsella <[email protected]>
Subject: Re: [PATCH v7 0/6] dpdk: introduce__rte_internaltagQuoted 29 lines · Haiyue
Hi David,
Quoted 26 lines
-----Original Message-----
From: David Marchand <[email protected]>
Sent: Saturday, April 25, 2020 22:39
To: Wang, Haiyue <[email protected]>
Cc: dev <[email protected]>; Thomas Monjalon <[email protected]>; Richardson, Bruce <[email protected]>; Yigit, Ferruh <[email protected]>; Neil Horman <[email protected]>; Ray Kinsella <[email protected]>
Subject: Re: [PATCH v7 0/6] dpdk: introduce__rte_internaltagQuoted 11 lines · Haiyue Wang
Move the internal function into INTERNAL session to avoid the ABI checking, and it is only used for DPDK drivers or related library.
__rte_internalfunAINTERNAL {
global:funA
};Thanks a lot for working on this.
I did some modifications (see my replies on patch 3 and 5) and applied this series.Quoted 5 lines
Quoted 3 lines
We are just missing the update on the scripts mentioned in a previous mail. Can you work on this for rc2?
Do you mean ?
Sorry, I was referring to another mail:
"""
Quoted 8 lines
Quoted 2 lines
We are missing updates on
devtools/check-abi-version.shanddevtools/update_version_map_abi.py."""
Those scripts
devtools/check-abi-version.shanddevtools/update_version_map_abi.pyare not "internal" aware. Can you take care of them?
Got it, I will try.
Quoted 27 lines
Quoted 14 lines
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
This will apply to common drivers that will be 100% internal. Not sure if this is an issue.
This part should be fine, I want others to be aware of this.
I am not one of the ABI maintainers, but in my opinion it is OK to have "pure internal" libs with version 0.x.
I've tested it with Intel's
drivers/common/iavf, it works as expected. a). librte_common_iavf.so.0.200.2
b). Skipped experimental library librte_common_iavf.dump.This has been updated by your modification.
Review
Quoted 7 lines
(quoted diff) +4
-0+ if is_stable lib_version = abi_version so_version = stable_so_version + else + lib_version = experimental_abi_version + so_version = experimental_so_version endifThanks for testing.
--
David Marchand