eal: refactor rte_eal_init into sub-functions
PATCH v2 1/1 eal: refactor rte_eal_init into sub-functions
From: Rahul Gupta <[email protected]>
In continuation to the following email, I am sending this patch.
(https://inbox.dpdk.org/dev/[email protected]/)
Initialization requires rte_eal_init + rte_pktmbuf_pool_create which can consume a total time of 500-600 ms:
a) For many devices FLR may take a significant chunk of time (200-250 ms in our use-case), this FLR is triggered during device probe in rte_eal_init().
b) rte_pktmbuf_pool_create() can consume up to 300-350 ms for applications that require huge memory.
This cost is incurred on each restart (which happens in our use-case during binary updates for servicing).
This patch provides an optimization using pthreads that applications can use and which can save 200-230ms.
In this patch, rte_eal_init() is refactored into two parts-
a) 1st part is dependent code ie- it’s a perquisite of the FLR and mempool creation. So this code needs to be executed before any pthreads. Its named as rte_eal_init_setup()
b) 2nd part of code is independent code ie- it can execute in parallel to mempool creation in a pthread. Its named as rte_eal_init_async().
Existing applications requires to just call- rte_eal_init_wait_async_complete() after rte_eal_init() unless they wish to leverage the optimization.
If the application wants to leverage this optimization, then it should create a thread using rte_eal_remote_launch() to schedule a task it would like todo in parallel rte_eal_init_async(), this task can be a mbuf pool creation using- rte_pktmbuf_pool_create()
After both the above tasks, if next operations require completion of above thread a), then user can use rte_eal_init_wait_async_complete(), or if user wants to just check status of that thread, then use- rte_eal_init_async_done()
---
v2: Address Stephen Hemminger's comment
app/test-pmd/testpmd.c | 20 +++++++++++++++ lib/eal/include/rte_eal.h | 48 ++++++++++++++++++++++++++++++++++++ lib/eal/linux/eal.c | 52 ++++++++++++++++++++++++++++++++++++--- lib/eal/version.map | 6 +++++ 4 files changed, 123 insertions(+), 3 deletions(-)
app/test-pmd/testpmd.c +20 -0
@@ -4531,6 +4531,7 @@ main(int argc, char** argv)portid_t port_id;uint16_t count;int ret; +int lcore_id;#ifdef RTE_EXEC_ENV_WINDOWS signal (SIGINT , signal_handler);
@@ -4555,6 +4556,25 @@ main(int argc, char** argv)rte_exit (EXIT_FAILURE ,"Cannot init EAL: %s\n" ,rte_strerror (rte_errno)); + lcore_id =rte_lcore_id (); + lcore_id =rte_get_next_lcore (lcore_id,0 ,1 ); +/* Gives status of rte_eal_init_async() */ +while (rte_eal_init_async_done (lcore_id) ==0 ) + ; + +/* +* Use rte_eal_init_wait_async_complete() to get return value of +* rte_eal_init_async(). +* Or +* if testpmd application dont want to know progress/status of +* rte_eal_init_async() and just want to wait till it finishes +* then use following function. +*/ + ret =rte_eal_init_wait_async_complete (); +if (ret <0 ) +rte_exit (EXIT_FAILURE ,"Cannot init EAL: " +"rte_eal_init_async() failed: %s\n" , +strerror (ret));/* allocate port structures, and init them */ init_port ();
lib/eal/include/rte_eal.h +48 -0
54 lines not on this page · raw message
lib/eal/linux/eal.c +49 -3
71 lines not on this page · raw message
lib/eal/version.map +6 -0
@@ -393,6 +393,12 @@ EXPERIMENTAL { # added in 23.07 rte_memzone_max_get; rte_memzone_max_set; + + # added in 24.01 + rte_eal_init_async; + rte_eal_init_async_done; + rte_eal_init_setup; + rte_eal_init_wait_async_complete; }; INTERNAL {
Signature · 1 line
2.25.1
app/test-pmd/testpmd.c main(int argc, char** argv) · line 4563
Quoted 5 lines · On Wed, 17 Jan 2024 11:25:02 -0800 Rahul Gupta <[email protected]>…
app/test-pmd/testpmd.c +5
-0+ lcore_id =rte_lcore_id (); + lcore_id =rte_get_next_lcore (lcore_id,0 ,1 ); +/* Gives status of rte_eal_init_async() */ +while (rte_eal_init_async_done (lcore_id) ==0 ) + ;
Will this all work if application is run with single lcore? It would work before this.