mbuf: add bulk free function
v9 cover mbuf: add bulk free function
Add function for freeing a bulk of mbufs.
Add unit test for functions for allocating and freeing a bulk of mbufs.
---
v9:
* Fix a typo in a description.
* In the unit test, make chain length a #define and add a few RTE_BUILD_BUG_ON().
* The unit test prints when allocating the mbuf pools for the bulk tests. Also print when freeing them.
v8:
* Add unit test, covering both bulk alloc and bulk free.
v7:
* Squash multiple modifications into one.
v6:
* Remove __rte_always_inline from static function.
The compiler will inline anyway.
v5:
* Rename variables from "free" to "pending" for improved readability.
* Add prefix __ to rte_pktmbuf_free_seg_via_array().
* Add array size parameter to __rte_pktmbuf_free_seg_via_array(). The compiler will optimize the parameter away anyway.
* Add description to __rte_pktmbuf_free_seg_via_array().
* Minor description updates.
v4:
* Mark as experimental by adding __rte_experimental.
* Add function to experimental section of map file.
* Fix source code formatting regarding pointer to pointer.
* Squash multiple modifications into one.
v3:
* Bugfix: Handle pakets with multiple segments.
* Add inline helper function, mainly for readability.
* Fix source code formatting regarding indentation.
v2:
* Function is not inline.
* Optimize to free multible mbufs belonging to the same mempool in bulk. Inspired by ixgbe_tx_free_bufs(), but allowing NULL pointers in the array, just like rte_pktmbuf_free() can take a NULL pointer.
* Use unsigned int instead of unsigned. Passes checkpatch, but mismatches the original coding style of the modified files.
* Fix a typo in the description headline: mempools is plural.
Morten Brørup (2):
mbuf: add bulk free function
mbuf: add unit test for bulk alloc/free functions
app/test/test_mbuf.c | 179 +++++++++++++++++++++++++++ lib/librte_mbuf/rte_mbuf.c | 66 ++++++++++ lib/librte_mbuf/rte_mbuf.h | 15 +++ lib/librte_mbuf/rte_mbuf_version.map | 1 + 4 files changed, 261 insertions(+)
Signature · 1 line
2.17.1
PATCH v9 1/2 mbuf: add bulk free function
Add function for freeing a bulk of mbufs.
---
v9:
* Fix a typo in a description.
* In the unit test, make chain length a #define and add a few RTE_BUILD_BUG_ON().
* The unit test prints when allocating the mbuf pools for the bulk tests. Also print when freeing them.
v8:
* Add unit test, covering both bulk alloc and bulk free.
v7:
* Squash multiple modifications into one.
v6:
* Remove __rte_always_inline from static function.
The compiler will inline anyway.
v5:
* Rename variables from "free" to "pending" for improved readability.
* Add prefix __ to rte_pktmbuf_free_seg_via_array().
* Add array size parameter to __rte_pktmbuf_free_seg_via_array(). The compiler will optimize the parameter away anyway.
* Add description to __rte_pktmbuf_free_seg_via_array().
* Minor description updates.
v4:
* Mark as experimental by adding __rte_experimental.
* Add function to experimental section of map file.
* Fix source code formatting regarding pointer to pointer.
* Squash multiple modifications into one.
v3:
* Bugfix: Handle pakets with multiple segments.
* Add inline helper function, mainly for readability.
* Fix source code formatting regarding indentation.
v2:
* Function is not inline.
* Optimize to free multible mbufs belonging to the same mempool in bulk. Inspired by ixgbe_tx_free_bufs(), but allowing NULL pointers in the array, just like rte_pktmbuf_free() can take a NULL pointer.
* Use unsigned int instead of unsigned. Passes checkpatch, but mismatches the original coding style of the modified files.
* Fix a typo in the description headline: mempools is plural.
--- lib/librte_mbuf/rte_mbuf.c | 66 ++++++++++++++++++++++++++++ lib/librte_mbuf/rte_mbuf.h | 15 +++++++ lib/librte_mbuf/rte_mbuf_version.map | 1 + 3 files changed, 82 insertions(+)
lib/librte_mbuf/rte_mbuf.c +66 -0
@@ -245,6 +245,72 @@ int rte_mbuf_check(const struct rte_mbuf *m, int is_header,return 0 ; } +/** +* @internal helper function for freeing a bulk of packet mbuf segments +* via an array holding the packet mbuf segments from the same mempool +* pending to be freed. +* +* @param m +* The packet mbuf segment to be freed. +* @param pending +* Pointer to the array of packet mbuf segments pending to be freed. +* @param nb_pending +* Pointer to the number of elements held in the array. +* @param pending_sz +* Number of elements the array can hold. +* Note: The compiler should optimize this parameter away when using a +* constant value, such as RTE_PKTMBUF_FREE_PENDING_SZ. +*/ +static void +__rte_pktmbuf_free_seg_via_array (struct rte_mbuf *m, +struct rte_mbuf **const pending,unsigned int *const nb_pending, +const unsigned int pending_sz) +{ + m =rte_pktmbuf_prefree_seg (m); +if (likely (m !=NULL )) { +if (*nb_pending == pending_sz || + (*nb_pending >0 && m->pool != pending[0 ]->pool)) { +rte_mempool_put_bulk (pending[0 ]->pool, + (void **)pending, *nb_pending); + *nb_pending =0 ; + } + + pending[(*nb_pending)++] = m; + } +} + +/** +* Size of the array holding mbufs from the same mempool pending to be freed +* in bulk. +*/ +#define RTE_PKTMBUF_FREE_PENDING_SZ 64 + +/* Free a bulk of packet mbufs back into their original mempools. */ +void rte_pktmbuf_free_bulk (struct rte_mbuf **mbufs,unsigned int count) +{ +struct rte_mbuf *m, *m_next, *pending[RTE_PKTMBUF_FREE_PENDING_SZ ]; +unsigned int idx, nb_pending =0 ; + +for (idx =0 ; idx < count; idx++) { + m = mbufs[idx]; +if (unlikely (m ==NULL )) +continue ; + +__rte_mbuf_sanity_check (m,1 ); + +do { + m_next = m->next; +__rte_pktmbuf_free_seg_via_array (m, + pending, &nb_pending, +RTE_PKTMBUF_FREE_PENDING_SZ ); + m = m_next; + }while (m !=NULL ); + } + +if (nb_pending >0 ) +rte_mempool_put_bulk (pending[0 ]->pool, (void **)pending, nb_pending); +} +/* Creates a shallow copy of mbuf */ struct rte_mbuf *rte_pktmbuf_clone (struct rte_mbuf *md,struct rte_mempool *mp)
lib/librte_mbuf/rte_mbuf.h +15 -0
@@ -1914,6 +1914,21 @@ static inline void rte_pktmbuf_free(struct rte_mbuf *m) } } +/** +* Free a bulk of packet mbufs back into their original mempools. +* +* Free a bulk of mbufs, and all their segments in case of chained buffers. +* Each segment is added back into its original mempool. +* +* @param mbufs +* Array of pointers to packet mbufs. +* The array may contain NULL pointers. +* @param count +* Array size. +*/ +__rte_experimental +void rte_pktmbuf_free_bulk(struct rte_mbuf **mbufs, unsigned int count); +/** * Create a "clone" of the given packet mbuf. *
lib/librte_mbuf/rte_mbuf_version.map +1 -0
@@ -58,6 +58,7 @@ EXPERIMENTAL { global: rte_mbuf_check; + rte_pktmbuf_free_bulk; rte_pktmbuf_copy; } DPDK_18.08;
Signature · 1 line
2.17.1
PATCH v9 2/2 mbuf: add unit test for bulk alloc/free functions
In reply to Morten BrørupAdd unit test for functions for allocating and freeing a bulk of mbufs.
---
v9:
* Fix a typo in a description.
* In the unit test, make chain length a #define and add a few RTE_BUILD_BUG_ON().
* The unit test prints when allocating the mbuf pools for the bulk tests. Also print when freeing them.
v8:
* Add unit test, covering both bulk alloc and bulk free.
v7:
* Squash multiple modifications into one.
v6:
* Remove __rte_always_inline from static function.
The compiler will inline anyway.
v5:
* Rename variables from "free" to "pending" for improved readability.
* Add prefix __ to rte_pktmbuf_free_seg_via_array().
* Add array size parameter to __rte_pktmbuf_free_seg_via_array(). The compiler will optimize the parameter away anyway.
* Add description to __rte_pktmbuf_free_seg_via_array().
* Minor description updates.
v4:
* Mark as experimental by adding __rte_experimental.
* Add function to experimental section of map file.
* Fix source code formatting regarding pointer to pointer.
* Squash multiple modifications into one.
v3:
* Bugfix: Handle pakets with multiple segments.
* Add inline helper function, mainly for readability.
* Fix source code formatting regarding indentation.
v2:
* Function is not inline.
* Optimize to free multible mbufs belonging to the same mempool in bulk. Inspired by ixgbe_tx_free_bufs(), but allowing NULL pointers in the array, just like rte_pktmbuf_free() can take a NULL pointer.
* Use unsigned int instead of unsigned. Passes checkpatch, but mismatches the original coding style of the modified files.
* Fix a typo in the description headline: mempools is plural.
--- app/test/test_mbuf.c | 179 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 179 insertions(+)
app/test/test_mbuf.c +179 -0
@@ -12,6 +12,7 @@#include <sys/queue.h> #include <rte_common.h> +#include <rte_errno.h> #include <rte_debug.h> #include <rte_log.h> #include <rte_memory.h>
@@ -39,6 +40,9 @@#define MBUF_TEST_HDR2_LEN 30#define MBUF_TEST_ALL_HDRS_LEN (MBUF_TEST_HDR1_LEN+MBUF_TEST_HDR2_LEN) +/* chain length in bulk test */ +#define CHAIN_LEN 16 +/* size of private data for mbuf in pktmbuf_pool2 */ #define MBUF2_PRIV_SIZE 128
@@ -700,6 +704,175 @@ test_pktmbuf_pool(struct rte_mempool *pktmbuf_pool)return ret; } +/* +* test bulk allocation and bulk free of mbufs +*/ +static int +test_pktmbuf_pool_bulk (void ) +{ +struct rte_mempool *pool =NULL ; +struct rte_mempool *pool2 =NULL ; +unsigned int i; +struct rte_mbuf *m; +struct rte_mbuf *mbufs[NB_MBUF ]; +int ret =0 ; + +/* We cannot use the preallocated mbuf pools because their caches +* prevent us from bulk allocating all objects in them. +* So we create our own mbuf pools without caches. +*/ +printf ("Create mbuf pools for bulk allocation.\n" ); + pool =rte_pktmbuf_pool_create ("test_pktmbuf_bulk" , +NB_MBUF ,0 ,0 ,MBUF_DATA_SIZE ,SOCKET_ID_ANY ); +if (pool ==NULL ) { +printf ("rte_pktmbuf_pool_create() failed. rte_errno %d\n" , + rte_errno); + goto err; + } + pool2 =rte_pktmbuf_pool_create ("test_pktmbuf_bulk2" , +NB_MBUF ,0 ,0 ,MBUF_DATA_SIZE ,SOCKET_ID_ANY ); +if (pool2 ==NULL ) { +printf ("rte_pktmbuf_pool_create() failed. rte_errno %d\n" , + rte_errno); + goto err; + } + +/* Preconditions: Mempools must be full. */ +if (!(rte_mempool_full (pool) &&rte_mempool_full (pool2))) { +printf ("Test precondition failed: mempools not full\n" ); + goto err; + } +if (!(rte_mempool_avail_count (pool) ==NB_MBUF && +rte_mempool_avail_count (pool2) ==NB_MBUF )) { +printf ("Test precondition failed: mempools: %u+%u != %u+%u" , +rte_mempool_avail_count (pool), +rte_mempool_avail_count (pool2), +NB_MBUF ,NB_MBUF ); + goto err; + } + +printf ("Test single bulk alloc, followed by multiple bulk free.\n" ); + +/* Bulk allocate all mbufs in the pool, in one go. */ + ret =rte_pktmbuf_alloc_bulk (pool, mbufs,NB_MBUF ); +if (ret !=0 ) { +printf ("rte_pktmbuf_alloc_bulk() failed: %d\n" , ret); + goto err; + } +/* Test that they have been removed from the pool. */ +if (!rte_mempool_empty (pool)) { +printf ("mempool not empty\n" ); + goto err; + } +/* Bulk free all mbufs, in four steps. */ +RTE_BUILD_BUG_ON (NB_MBUF %4 !=0 ); +for (i =0 ; i <NB_MBUF ; i +=NB_MBUF /4 ) { +rte_pktmbuf_free_bulk (&mbufs[i],NB_MBUF /4 ); +/* Test that they have been returned to the pool. */ +if (rte_mempool_avail_count (pool) != i +NB_MBUF /4 ) { +printf ("mempool avail count incorrect\n" ); + goto err; + } + } + +printf ("Test multiple bulk alloc, followed by single bulk free.\n" ); + +/* Bulk allocate all mbufs in the pool, in four steps. */ +for (i =0 ; i <NB_MBUF ; i +=NB_MBUF /4 ) { + ret =rte_pktmbuf_alloc_bulk (pool, &mbufs[i],NB_MBUF /4 ); +if (ret !=0 ) { +printf ("rte_pktmbuf_alloc_bulk() failed: %d\n" , ret); + goto err; + } + } +/* Test that they have been removed from the pool. */ +if (!rte_mempool_empty (pool)) { +printf ("mempool not empty\n" ); + goto err; + } +/* Bulk free all mbufs, in one go. */ +rte_pktmbuf_free_bulk (mbufs,NB_MBUF ); +/* Test that they have been returned to the pool. */ +if (!rte_mempool_full (pool)) { +printf ("mempool not full\n" ); + goto err; + } + +printf ("Test bulk free of single long chain.\n" ); + +/* Bulk allocate all mbufs in the pool, in one go. */ + ret =rte_pktmbuf_alloc_bulk (pool, mbufs,NB_MBUF ); +if (ret !=0 ) { +printf ("rte_pktmbuf_alloc_bulk() failed: %d\n" , ret); + goto err; + } +/* Create a long mbuf chain. */ +for (i =1 ; i <NB_MBUF ; i++) { + ret =rte_pktmbuf_chain (mbufs[0 ], mbufs[i]); +if (ret !=0 ) { +printf ("rte_pktmbuf_chain() failed: %d\n" , ret); + goto err; + } + mbufs[i] =NULL ; + } +/* Free the mbuf chain containing all the mbufs. */ +rte_pktmbuf_free_bulk (mbufs,1 ); +/* Test that they have been returned to the pool. */ +if (!rte_mempool_full (pool)) { +printf ("mempool not full\n" ); + goto err; + } + +printf ("Test bulk free of multiple chains using multiple pools.\n" ); + +/* Create mbuf chains containing mbufs from different pools. */ +RTE_BUILD_BUG_ON (CHAIN_LEN %2 !=0 ); +RTE_BUILD_BUG_ON (NB_MBUF % (CHAIN_LEN /2 ) !=0 ); +for (i =0 ; i <NB_MBUF *2 ; i++) { + m =rte_pktmbuf_alloc ((i &4 ) ? pool2 : pool); +if (m ==NULL ) { +printf ("rte_pktmbuf_alloc() failed (%u)\n" , i); + goto err; + } +if ((i %CHAIN_LEN ) ==0 ) + mbufs[i /CHAIN_LEN ] = m; +else +rte_pktmbuf_chain (mbufs[i /CHAIN_LEN ], m); + } +/* Test that both pools have been emptied. */ +if (!(rte_mempool_empty (pool) &&rte_mempool_empty (pool2))) { +printf ("mempools not empty\n" ); + goto err; + } +/* Free one mbuf chain. */ +rte_pktmbuf_free_bulk (mbufs,1 ); +/* Test that the segments have been returned to the pools. */ +if (!(rte_mempool_avail_count (pool) ==CHAIN_LEN /2 && +rte_mempool_avail_count (pool2) ==CHAIN_LEN /2 )) { +printf ("all segments of first mbuf have not been returned\n" ); + goto err; + } +/* Free the remaining mbuf chains. */ +rte_pktmbuf_free_bulk (&mbufs[1 ],NB_MBUF *2 /CHAIN_LEN -1 ); +/* Test that they have been returned to the pools. */ +if (!(rte_mempool_full (pool) &&rte_mempool_full (pool2))) { +printf ("mempools not full\n" ); + goto err; + } + + ret =0 ; + goto done; + +err: + ret =-1 ; + +done: +printf ("Free mbuf pools for bulk allocation.\n" ); +rte_mempool_free (pool); +rte_mempool_free (pool2); +return ret; +} +/* * test that the pointer to the data on a packet mbuf is set properly */
@@ -1317,6 +1490,12 @@ test_mbuf(void) goto err; } +/* test bulk mbuf alloc and free */ +if (test_pktmbuf_pool_bulk () <0 ) { +printf ("test_pktmbuf_pool_bulk() failed\n" ); + goto err; + } +/* test that the pointer to the data on a packet mbuf is set properly */ if (test_pktmbuf_pool_ptr (pktmbuf_pool) <0 ) {printf ("test_pktmbuf_pool_ptr() failed\n" );
Signature · 1 line
2.17.1
Thomas Monjalon
Applied23/10/2019 12:11, Morten Brørup:
Quoted 7 lines
Add function for freeing a bulk of mbufs.
Add unit test for functions for allocating and freeing a bulk of mbufs.
Morten Brørup (2):
mbuf: add bulk free function
mbuf: add unit test for bulk alloc/free functions
Applied, thanks