linux/eal: Remove most causes of panic on init
v2 cover linux/eal: Remove most causes of panic on init
In many cases, it's enough to simply let the application know that the call to initialize DPDK has failed. A complete halt can then be decided by the application based on error returned (and the app could even attempt a possible re-attempt after some corrective action by the user or application).
Changes ->v2:
- Audited all "RTE_LOG (" calls that were introduced, and converted to "RTE_LOG("
- Added some fprintf(stderr, "") lines to indicate errors before logging is initialized
- Removed assignments to errno.
- Changed patch 14/25 to reflect EFAULT, and document in 25/25
I kept the rte_errno reflection, since this is control-path code and the init function returns a sentinel value of -1.
Aaron Conole (25):
eal: CPU init will no longer panic
eal: return error instead of panic for cpu init
eal: No panic on hugepages info init
eal: do not panic on failed hugepage query
eal: failure to parse args returns error
eal-common: introduce a way to query cpu support
eal: Signal error when CPU isn't supported
eal: do not panic on memzone initialization fails
eal: set errno when exiting for already called
eal: Do not panic on log failures
eal: Do not panic on pci-probe
eal: do not panic on vfio failure
eal: do not panic on memory init
eal: do not panic on tailq init
eal: do not panic on alarm init
eal: convert timer_init not to call panic
eal: change the private pipe call to reflect errno
eal: Do not panic on interrupt thread init
eal: do not error if plugins fail to init
eal_pci: Continue probing even on failures
eal: do not panic on failed PCI probe
eal_common_dev: continue initializing vdevs
eal: do not panic (or abort) if vdev init fails
eal: do not panic when bus probe fails
rte_eal_init: add info about rte_errno codes
lib/librte_eal/common/eal_common_cpuflags.c | 13 ++- lib/librte_eal/common/eal_common_dev.c | 5 +- lib/librte_eal/common/eal_common_lcore.c | 7 +- lib/librte_eal/common/eal_common_pci.c | 15 ++- lib/librte_eal/common/eal_common_tailqs.c | 3 +- .../common/include/generic/rte_cpuflags.h | 9 ++ lib/librte_eal/common/include/rte_eal.h | 27 ++++- lib/librte_eal/linuxapp/eal/eal.c | 122 +++++++++++++++------ lib/librte_eal/linuxapp/eal/eal_hugepage_info.c | 6 +- lib/librte_eal/linuxapp/eal/eal_interrupts.c | 5 +- 10 files changed, 161 insertions(+), 51 deletions(-)
Signature · 1 line
2.9.3
PATCH v2 1/25 eal: CPU init will no longer panic
After this change, the EAL CPU NUMA node resolution step can no longer emit an rte_panic. This aligns with the code in rte_eal_init, which expects failures to return an error code.
--- lib/librte_eal/common/eal_common_lcore.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
lib/librte_eal/common/eal_common_lcore.c +4 -3
@@ -83,16 +83,17 @@ rte_eal_cpu_init(void) config->lcore_role[lcore_id] =ROLE_RTE ; lcore_config[lcore_id].core_id =eal_cpu_core_id (lcore_id); lcore_config[lcore_id].socket_id =eal_cpu_socket_id (lcore_id); - if (lcore_config[lcore_id].socket_id >= RTE_MAX_NUMA_NODES) +if (lcore_config[lcore_id].socket_id >=RTE_MAX_NUMA_NODES ) {#ifdef RTE_EAL_ALLOW_INV_SOCKET_ID lcore_config[lcore_id].socket_id =0 ;#else -rte_panic ("Socket ID (%u) is greater than " +RTE_LOG (ERR ,EAL ,"Socket ID (%u) is greater than " "RTE_MAX_NUMA_NODES (%d)\n" , lcore_config[lcore_id].socket_id,RTE_MAX_NUMA_NODES ); +return -1 ;#endif - + } RTE_LOG(DEBUG, EAL, "Detected lcore %u as " "core %u on socket %u\n", lcore_id, lcore_config[lcore_id].core_id,
Signature · 1 line
2.9.3
PATCH v2 2/25 eal: return error instead of panic for cpu init
In reply to Aaron ConoleThere may be no way to gracefully recover, but the application should be notified that a failure happened, rather than completely aborting. This allows the user to proceed with a "slow-path" type solution.
--- lib/librte_eal/linuxapp/eal/eal.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
@@ -767,8 +767,11 @@ rte_eal_init(int argc, char **argv)/* set log level as early as possible */ rte_set_log_level (internal_config.log_level); -if (rte_eal_cpu_init () <0 ) -rte_panic ("Cannot detect lcores\n" ); +if (rte_eal_cpu_init () <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot detect lcores\n" ); + rte_errno =ENOTSUP ; +return -1 ; + } fctret =eal_parse_args (argc, argv);if (fctret <0 )
Signature · 1 line
2.9.3
PATCH v2 3/25 eal: No panic on hugepages info init
In reply to Aaron ConoleWhen attempting to scan hugepages, signal to the eal.c that an error has occured, rather than performing a panic.
--- lib/librte_eal/linuxapp/eal/eal_hugepage_info.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal_hugepage_info.c +4 -2
@@ -283,9 +283,11 @@ eal_hugepage_info_init(void)struct dirent *dirent; dir =opendir (sys_dir_path); -if (dir ==NULL ) -rte_panic ("Cannot open directory %s to read system hugepage " +if (dir ==NULL ) { +RTE_LOG (ERR ,EAL ,"Cannot open directory %s to read system hugepage " "info\n" , sys_dir_path); +return -1 ; + }for (dirent =readdir (dir); dirent !=NULL ; dirent =readdir (dir)) {struct hugepage_info *hpi;
Signature · 1 line
2.9.3
PATCH v2 4/25 eal: do not panic on failed hugepage query
In reply to Aaron ConoleIf we fail to acquire hugepage information, simply signal an error to the application. This clears the run_once counter, allowing the user or application to take a corrective action and retry.
--- lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
@@ -780,8 +780,12 @@ rte_eal_init(int argc, char **argv)if (internal_config.no_hugetlbfs ==0 && internal_config.process_type !=RTE_PROC_SECONDARY && internal_config.xen_dom0_support ==0 && -eal_hugepage_info_init () <0 ) -rte_panic ("Cannot get hugepage information\n" ); +eal_hugepage_info_init () <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot get hugepage information\n" ); + rte_errno =EACCES ; +rte_atomic32_clear (&run_once); +return -1 ; + }if (internal_config.memory ==0 && internal_config.force_sockets ==0 ) {if (internal_config.no_hugetlbfs)
Signature · 1 line
2.9.3
PATCH v2 5/25 eal: failure to parse args returns error
In reply to Aaron ConoleIt's possible that the application could take a corrective action here, and either prompt the user for different arguments, or at least perform a better logging. Exiting this early prevents any useful information gathering from the application layer.
--- lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
@@ -774,8 +774,12 @@ rte_eal_init(int argc, char **argv) } fctret =eal_parse_args (argc, argv); -if (fctret <0 ) -exit (1 ); +if (fctret <0 ) { +RTE_LOG (ERR ,EAL ,"Invalid 'command line' arguments\n" ); + rte_errno =EINVAL ; +rte_atomic32_clear (&run_once); +return -1 ; + } if (internal_config.no_hugetlbfs == 0 && internal_config.process_type != RTE_PROC_SECONDARY &&
Signature · 1 line
2.9.3
PATCH v2 6/25 eal-common: introduce a way to query cpu support
In reply to Aaron ConoleThis adds a new API to check for the eal cpu versions.
--- lib/librte_eal/common/eal_common_cpuflags.c | 13 +++++++++++-- lib/librte_eal/common/include/generic/rte_cpuflags.h | 9 +++++++++ 2 files changed, 20 insertions(+), 2 deletions(-)
lib/librte_eal/common/eal_common_cpuflags.c +11 -2
@@ -43,6 +43,13 @@void rte_cpu_check_supported (void ) { +if (!rte_cpu_is_supported ()) +exit (1 ); +} + +bool +rte_cpu_is_supported (void ) +{/* This is generated at compile-time by the build system */ static const enum rte_cpu_flag_t compile_time_flags[] = {RTE_COMPILE_TIME_CPUFLAGS
@@ -57,14 +64,16 @@ rte_cpu_check_supported(void)fprintf (stderr,"ERROR: CPU feature flag lookup failed with error %d\n" , ret); -exit (1 ); +return false; }if (!ret) {fprintf (stderr,"ERROR: This system does not support \"%s\".\n" "Please check that RTE_MACHINE is set correctly.\n" ,rte_cpu_get_flag_name (compile_time_flags[i])); -exit (1 ); +return false; } } + +return true; }
lib/librte_eal/common/include/generic/rte_cpuflags.h +9 -0
@@ -40,6 +40,7 @@*/ #include <errno.h> +#include <stdbool.h> /** * Enumeration of all CPU features supported
@@ -82,4 +83,12 @@ rte_cpu_get_flag_enabled(enum rte_cpu_flag_t feature);void rte_cpu_check_supported (void ); +/** +* This function checks that the currently used CPU supports the CPU features +* that were specified at compile time. It is called automatically within the +* EAL, so does not need to be used by applications. This version returns a +* result so that decisions may be made (for instance, graceful shutdowns). +*/ +bool +rte_cpu_is_supported (void );#endif /* _RTE_CPUFLAGS_H_ */
Signature · 1 line
2.9.3
PATCH v2 7/25 eal: Signal error when CPU isn't supported
In reply to Aaron ConoleIt's now possible to gracefully exit the application, or for applications which support non-dpdk datapaths working in concert with DPDK datapaths, there no longer is the possibility of exiting for unsupported CPUs.
--- lib/librte_eal/linuxapp/eal/eal.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -1
@@ -752,7 +752,11 @@ rte_eal_init(int argc, char **argv)char thread_name[RTE_MAX_THREAD_NAME_LEN ];/* checks if the machine is adequate */ -rte_cpu_check_supported (); +if (!rte_cpu_is_supported ()) { +fprintf (stderr,"EAL: FATAL - unsupported cpu type.\n" ); + rte_errno =ENOTSUP ; +return -1 ; + }if (!rte_atomic32_test_and_set (&run_once))return -1 ;
Signature · 1 line
2.9.3
PATCH v2 8/25 eal: do not panic on memzone initialization fails
In reply to Aaron ConoleWhen memzone initialization fails, report the error to the calling application rather than panic(). Without a good way of detaching / releasing hugepages, at this point the application will have to restart.
--- lib/librte_eal/linuxapp/eal/eal.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
@@ -832,8 +832,11 @@ rte_eal_init(int argc, char **argv)/* the directories are locked during eal_hugepage_info_init */ eal_hugedirs_unlock (); -if (rte_eal_memzone_init () <0 ) -rte_panic ("Cannot init memzone\n" ); +if (rte_eal_memzone_init () <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot init memzone\n" ); + rte_errno =ENODEV ; +return -1 ; + }if (rte_eal_tailqs_init () <0 )rte_panic ("Cannot init tail queues for objects\n" );
Signature · 1 line
2.9.3
PATCH v2 9/25 eal: set errno when exiting for already called
In reply to Aaron Conole--- lib/librte_eal/linuxapp/eal/eal.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal.c +4 -1
@@ -758,8 +758,11 @@ rte_eal_init(int argc, char **argv)return -1 ; } -if (!rte_atomic32_test_and_set (&run_once)) +if (!rte_atomic32_test_and_set (&run_once)) { +fprintf (stderr,"EAL: ERROR - already called initialization.\n" ); + rte_errno =EALREADY ;return -1 ; + } logid =strrchr (argv[0 ],'/' ); logid =strdup (logid ? logid +1 : argv[0 ]);
Signature · 1 line
2.9.3
PATCH v2 10/25 eal: Do not panic on log failures
In reply to Aaron ConoleWhen log initialization fails, it's generally because the fopencookie failed. While this is rare in practice, it could happen, and it is likely because of memory pressure. So, flag the error, and allow the user to retry.
--- lib/librte_eal/linuxapp/eal/eal.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +7 -2
@@ -818,8 +818,13 @@ rte_eal_init(int argc, char **argv)rte_config_init (); -if (rte_eal_log_init (logid, internal_config.syslog_facility) <0 ) -rte_panic ("Cannot init logs\n" ); +if (rte_eal_log_init (logid, internal_config.syslog_facility) <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot init logging\n" ); +fprintf (stderr,"EAL: ERROR - cannot init logging.\n" ); + rte_errno =EIO ; +rte_atomic32_clear (&run_once); +return -1 ; + }if (rte_eal_pci_init () <0 )rte_panic ("Cannot init PCI\n" );
Signature · 1 line
2.9.3
PATCH v2 11/25 eal: Do not panic on pci-probe
In reply to Aaron ConoleThis will usually be an issue because of permissions. However, it could also be caused by OOM. In either case, errno will contain the underlying cause. It is safe to re-init the system here, so allow the application to take corrective action and reinit.
--- lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
@@ -826,8 +826,12 @@ rte_eal_init(int argc, char **argv)return -1 ; } -if (rte_eal_pci_init () <0 ) -rte_panic ("Cannot init PCI\n" ); +if (rte_eal_pci_init () <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot init PCI\n" ); + rte_errno =EUNATCH ; +rte_atomic32_clear (&run_once); +return -1 ; + }#ifdef VFIO_PRESENT if (rte_eal_vfio_setup () <0 )
Signature · 1 line
2.9.3
PATCH v2 12/25 eal: do not panic on vfio failure
In reply to Aaron Conole--- lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
@@ -834,8 +834,12 @@ rte_eal_init(int argc, char **argv) }#ifdef VFIO_PRESENT -if (rte_eal_vfio_setup () <0 ) -rte_panic ("Cannot init VFIO\n" ); +if (rte_eal_vfio_setup () <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot init VFIO\n" ); + rte_errno =EAGAIN ; +rte_atomic32_clear (&run_once); +return -1 ; + }#endif if (rte_eal_memory_init () <0 )
Signature · 1 line
2.9.3
PATCH v2 13/25 eal: do not panic on memory init
In reply to Aaron ConoleThis can only happen when access to hugepages (either as primary or secondary process) fails (and that is usually permissions). Since the manner of failure is not reversible, we cannot allow retry.
--- lib/librte_eal/linuxapp/eal/eal.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
@@ -842,8 +842,11 @@ rte_eal_init(int argc, char **argv) }#endif -if (rte_eal_memory_init () <0 ) -rte_panic ("Cannot init memory\n" ); +if (rte_eal_memory_init () <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot init memory\n" ); + rte_errno =EACCES ; +return -1 ; + }/* the directories are locked during eal_hugepage_info_init */ eal_hugedirs_unlock ();
Signature · 1 line
2.9.3
PATCH v2 14/25 eal: do not panic on tailq init
In reply to Aaron ConoleThere are some theoretical racy conditions in the system that _could_ cause early tailq init to fail; however, no need to panic the application. While it can't continue using DPDK, it could make better alerts to the user.
--- lib/librte_eal/common/eal_common_tailqs.c | 3 +-- lib/librte_eal/linuxapp/eal/eal.c | 7 +++++-- 2 files changed, 6 insertions(+), 4 deletions(-)
lib/librte_eal/common/eal_common_tailqs.c +1 -2
@@ -188,8 +188,7 @@ rte_eal_tailqs_init(void)if (t->head ==NULL ) {RTE_LOG (ERR ,EAL ,"Cannot initialize tailq: %s\n" , t->name); -/* no need to TAILQ_REMOVE, we are going to panic in -* rte_eal_init() */ +/* TAILQ_REMOVE not needed, error is already fatal */ goto fail; } }
lib/librte_eal/linuxapp/eal/eal.c +5 -2
@@ -857,8 +857,11 @@ rte_eal_init(int argc, char **argv)return -1 ; } -if (rte_eal_tailqs_init () <0 ) -rte_panic ("Cannot init tail queues for objects\n" ); +if (rte_eal_tailqs_init () <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot init tail queues for objects\n" ); + rte_errno =EFAULT ; +return -1 ; + }if (rte_eal_alarm_init () <0 )rte_panic ("Cannot init interrupt-handling thread\n" );
Signature · 1 line
2.9.3
PATCH v2 15/25 eal: do not panic on alarm init
In reply to Aaron Conolerte_eal_alarm_init() call uses the linux timerfd framework to create a poll()-able timer using standard posix file operations. This could fail for a few reasons given in the man-pages, but many could be corrected by the user application. No need to panic.
--- lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
@@ -61,6 +61,7 @@#include <rte_launch.h> #include <rte_eal.h> #include <rte_eal_memconfig.h> +#include <rte_errno.h> #include <rte_per_lcore.h> #include <rte_lcore.h> #include <rte_log.h>
@@ -863,8 +864,11 @@ rte_eal_init(int argc, char **argv)return -1 ; } -if (rte_eal_alarm_init () <0 ) -rte_panic ("Cannot init interrupt-handling thread\n" ); +if (rte_eal_alarm_init () <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot init interrupt-handling thread\n" ); +/* rte_eal_alarm_init sets rte_errno on failure. */ +return -1 ; + }if (rte_eal_timer_init () <0 )rte_panic ("Cannot init HPET or TSC timers\n" );
Signature · 1 line
2.9.3
PATCH v2 16/25 eal: convert timer_init not to call panic
In reply to Aaron ConoleAfter code inspection, there is no way for eal_timer_init() to fail. It simply returns 0 in all cases. As such, this test could either go-away or stay here as 'future-proofing'.
--- lib/librte_eal/linuxapp/eal/eal.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
@@ -870,8 +870,11 @@ rte_eal_init(int argc, char **argv)return -1 ; } -if (rte_eal_timer_init () <0 ) -rte_panic ("Cannot init HPET or TSC timers\n" ); +if (rte_eal_timer_init () <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot init HPET or TSC timers\n" ); + rte_errno =ENOTSUP ; +return -1 ; + }eal_check_mem_on_local_socket ();
Signature · 1 line
2.9.3
PATCH v2 17/25 eal: change the private pipe call to reflect errno
In reply to Aaron ConoleThere could be some confusion as to why the call failed - this change will always reflect the value of the error in rte_error.
--- lib/librte_eal/linuxapp/eal/eal_interrupts.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal_interrupts.c +4 -1
@@ -896,13 +896,16 @@ rte_eal_intr_init(void)* create a pipe which will be waited by epoll and notified to * rebuild the wait list of epoll. */ -if (pipe (intr_pipe.pipefd) <0 ) +if (pipe (intr_pipe.pipefd) <0 ) { + rte_errno = errno;return -1 ; + }/* create the host thread to wait/handle the interrupt */ ret =pthread_create (&intr_thread,NULL , eal_intr_thread_main,NULL );if (ret !=0 ) { + rte_errno = ret;RTE_LOG (ERR ,EAL ,"Failed to create thread for interrupt handling\n" ); }else {
Signature · 1 line
2.9.3
PATCH v2 18/25 eal: Do not panic on interrupt thread init
In reply to Aaron ConoleWhen initializing the interrupt thread, there are a number of possible reasons for failure - some of which are correctable by the application. Do not panic() needlessly, and give the application a change to reflect this information to the user.
--- lib/librte_eal/linuxapp/eal/eal.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +4 -2
@@ -889,8 +889,10 @@ rte_eal_init(int argc, char **argv) rte_config.master_lcore, (int )thread_id, cpuset, ret == 0 ? "" : "..."); -if (rte_eal_intr_init () <0 ) -rte_panic ("Cannot init interrupt-handling thread\n" ); +if (rte_eal_intr_init () <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot init interrupt-handling thread\n" ); +return -1 ; + }if (rte_bus_scan ())rte_panic ("Cannot scan the buses for devices\n" );
Signature · 1 line
2.9.3
PATCH v2 19/25 eal: do not error if plugins fail to init
In reply to Aaron ConolePlugins are useful and important. However, it seems crazy to abort everything just because they don't initialize properly.
--- lib/librte_eal/linuxapp/eal/eal.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +3 -2
@@ -878,8 +878,9 @@ rte_eal_init(int argc, char **argv)eal_check_mem_on_local_socket (); -if (eal_plugins_init () <0 ) -rte_panic ("Cannot init plugins\n" ); +if (eal_plugins_init () <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot init plugins\n" ); + }eal_thread_init_master (rte_config.master_lcore);
Signature · 1 line
2.9.3
PATCH v2 20/25 eal_pci: Continue probing even on failures
In reply to Aaron ConoleSome devices may be inaccessible for a variety of reasons, or the PCI-bus may be unavailable causing the whole thing to fail. Still, better to continue attempts at probes.
--- lib/librte_eal/common/eal_common_pci.c | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-)
lib/librte_eal/common/eal_common_pci.c +10 -5
@@ -69,6 +69,7 @@#include <sys/queue.h> #include <sys/mman.h> +#include <rte_errno.h> #include <rte_interrupts.h> #include <rte_log.h> #include <rte_pci.h>
@@ -416,6 +417,7 @@ rte_eal_pci_probe(void)struct rte_pci_device *dev =NULL ;struct rte_devargs *devargs;int probe_all =0 ; +int ret_1 =0 ;int ret =0 ;if (rte_eal_devargs_type_count (RTE_DEVTYPE_WHITELISTED_PCI ) ==0 )
@@ -430,17 +432,20 @@ rte_eal_pci_probe(void)/* probe all or only whitelisted devices */ if (probe_all) - ret =pci_probe_all_drivers (dev); + ret_1 =pci_probe_all_drivers (dev);else if (devargs !=NULL && devargs->type ==RTE_DEVTYPE_WHITELISTED_PCI ) - ret =pci_probe_all_drivers (dev); -if (ret <0 ) -rte_exit (EXIT_FAILURE ,"Requested device " PCI_PRI_FMT + ret_1 =pci_probe_all_drivers (dev); +if (ret_1 <0 ) { +RTE_LOG (ERR ,EAL ,"Requested device " PCI_PRI_FMT " cannot be used\n" , dev->addr.domain, dev->addr.bus, dev->addr.devid, dev->addr.function); + rte_errno = errno; + ret =1 ; + } } -return 0 ; +return -ret; }/* dump one device */
Signature · 1 line
2.9.3
PATCH v2 21/25 eal: do not panic on failed PCI probe
In reply to Aaron ConoleIt may even be possible to simply log the error and continue on letting the user check the logs and restart the application when things are failed.
--- lib/librte_eal/linuxapp/eal/eal.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
@@ -939,8 +939,11 @@ rte_eal_init(int argc, char **argv)rte_panic ("Cannot probe devices\n" );/* Probe & Initialize PCI devices */ -if (rte_eal_pci_probe ()) -rte_panic ("Cannot probe PCI\n" ); +if (rte_eal_pci_probe ()) { +RTE_LOG (ERR ,EAL ,"Cannot probe PCI\n" ); + rte_errno =ENOTSUP ; +return -1 ; + }if (rte_eal_dev_init () <0 )rte_panic ("Cannot init pmd devices\n" );
Signature · 1 line
2.9.3
PATCH v2 22/25 eal_common_dev: continue initializing vdevs
In reply to Aaron ConoleEven if one vdev should fail, there's no need to prevent further processing. Log the error, and reflect it to the higher levels to decide.
--- lib/librte_eal/common/eal_common_dev.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
lib/librte_eal/common/eal_common_dev.c +3 -2
@@ -80,6 +80,7 @@ intrte_eal_dev_init (void) {struct rte_devargs *devargs; +int ret =0 ;/* * Note that the dev_driver_list is populated here
@@ -97,11 +98,11 @@ rte_eal_dev_init(void) devargs->args)) {RTE_LOG (ERR ,EAL ,"failed to initialize %s device\n" , devargs->virt.drv_name); -return -1 ; + ret =-1 ; } } -return 0 ; +return ret; }int rte_eal_dev_attach (const char *name,const char *devargs)
Signature · 1 line
2.9.3
PATCH v2 23/25 eal: do not panic (or abort) if vdev init fails
In reply to Aaron ConoleSeems like it's possible to continue. At least, the error is reflected properly in the logs. A user could then go and correct or investigate the situation.
--- lib/librte_eal/linuxapp/eal/eal.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal.c +1 -1
@@ -946,7 +946,7 @@ rte_eal_init(int argc, char **argv) }if (rte_eal_dev_init () <0 ) -rte_panic ("Cannot init pmd devices\n" ); +RTE_LOG (ERR ,EAL ,"Cannot init pmd devices\n" );rte_eal_mcfg_complete ();
Signature · 1 line
2.9.3
PATCH v2 24/25 eal: do not panic when bus probe fails
In reply to Aaron Conole--- lib/librte_eal/linuxapp/eal/eal.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
@@ -935,8 +935,11 @@ rte_eal_init(int argc, char **argv)rte_eal_mp_wait_lcore ();/* Probe all the buses and devices/drivers on them */ -if (rte_bus_probe ()) -rte_panic ("Cannot probe devices\n" ); +if (rte_bus_probe ()) { +RTE_LOG (ERR ,EAL ,"Cannot probe devices\n" ); + rte_errno =ENOTSUP ; +return -1 ; + }/* Probe & Initialize PCI devices */ if (rte_eal_pci_probe ()) {
Signature · 1 line
2.9.3
PATCH v2 25/25 rte_eal_init: add info about rte_errno codes
In reply to Aaron ConoleThe rte_eal_init function will now pass failure reason hints to the application. To help app developers deciper this, add some brief information about what the codes are indicating.
--- lib/librte_eal/common/include/rte_eal.h | 27 ++++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-)
lib/librte_eal/common/include/rte_eal.h +26 -1
@@ -159,7 +159,32 @@ int rte_eal_iopl_init(void);* function call and should not be further interpreted by the * application. The EAL does not take any ownership of the memory used * for either the argv array, or its members. -* - On failure, a negative error value. +* - On failure, -1 and rte_errno is set to a value indicating the cause +* for failure. +* +* The error codes returned via rte_errno: +* EACCES indicates a permissions issue. +* +* EAGAIN indicates either a bus or system resource was not available, +* try again. +* +* EALREADY indicates that the rte_eal_init function has already been +* called, and cannot be called again. +* +* EFAULT indicates the tailq configuration name was not found in +* memory configuration. +* +* EINVAL indicates invalid parameters were passed as argv/argc. +* +* EIO indicates failure to setup the logging handlers. This is usually +* caused by an out-of-memory condition. +* +* ENODEV indicates memory setup issues. +* +* ENOTSUP indicates that the EAL cannot initialize on this system. +* +* EUNATCH indicates that the PCI bus is either not present, or is not +* readable by the eal. */ int rte_eal_init (int argc,char **argv);
Signature · 1 line
2.9.3
Quoted 14 lines · Aaron Conole
In many cases, it's enough to simply let the application know that the call to initialize DPDK has failed. A complete halt can then be decided by the application based on error returned (and the app could even attempt a possible re-attempt after some corrective action by the user or application).
Changes ->v2:
- Audited all "RTE_LOG(" calls that were introduced, and converted to "RTE_LOG("
- Added somefprintf(stderr, "")lines to indicate errors before logging is initialized
- Removed assignments to errno.
- Changed patch 14/25 to reflect EFAULT, and document in 25/25
Quoted 3 lines
I kept the
rte_errnoreflection, since this is control-path code and the init function returns a sentinel value of -1.
I got 3 new checkpatch warnings that seem to have missed my local commit hook. I'll fix them up and send a new series in-reply to this one.
Sorry for the noise.
v3 cover linux/eal: Remove most causes of panic on init
In reply to Aaron ConoleIn many cases, it's enough to simply let the application know that the call to initialize DPDK has failed. A complete halt can then be decided by the application based on error returned (and the app could even attempt a possible re-attempt after some corrective action by the user or application).
Changes ->v2:
- Audited all "RTE_LOG (" calls that were introduced, and converted to "RTE_LOG("
- Added some fprintf(stderr, "") lines to indicate errors before logging is initialized
- Removed assignments to errno.
- Changed patch 14/25 to reflect EFAULT, and document in 25/25
Changes ->v3:
- Checkpatch issues in patches 3 (spelling mistake), 9 (issue with leading spaces), and 19 (braces around single line statement if-condition)
I kept the rte_errno reflection, since this is control-path code and the init function returns a sentinel value of -1.
Aaron Conole (25):
eal: CPU init will no longer panic
eal: return error instead of panic for cpu init
eal: No panic on hugepages info init
eal: do not panic on failed hugepage query
eal: failure to parse args returns error
eal-common: introduce a way to query cpu support
eal: Signal error when CPU isn't supported
eal: do not panic on memzone initialization fails
eal: set errno when exiting for already called
eal: Do not panic on log failures
eal: Do not panic on pci-probe
eal: do not panic on vfio failure
eal: do not panic on memory init
eal: do not panic on tailq init
eal: do not panic on alarm init
eal: convert timer_init not to call panic
eal: change the private pipe call to reflect errno
eal: Do not panic on interrupt thread init
eal: do not error if plugins fail to init
eal_pci: Continue probing even on failures
eal: do not panic on failed PCI probe
eal_common_dev: continue initializing vdevs
eal: do not panic (or abort) if vdev init fails
eal: do not panic when bus probe fails
rte_eal_init: add info about rte_errno codes
lib/librte_eal/common/eal_common_cpuflags.c | 13 ++- lib/librte_eal/common/eal_common_dev.c | 5 +- lib/librte_eal/common/eal_common_lcore.c | 7 +- lib/librte_eal/common/eal_common_pci.c | 15 ++- lib/librte_eal/common/eal_common_tailqs.c | 3 +- .../common/include/generic/rte_cpuflags.h | 9 ++ lib/librte_eal/common/include/rte_eal.h | 27 ++++- lib/librte_eal/linuxapp/eal/eal.c | 122 +++++++++++++++------ lib/librte_eal/linuxapp/eal/eal_hugepage_info.c | 6 +- lib/librte_eal/linuxapp/eal/eal_interrupts.c | 5 +- 10 files changed, 161 insertions(+), 51 deletions(-)
Signature · 1 line
2.9.3
PATCH v3 1/25 eal: CPU init will no longer panic
After this change, the EAL CPU NUMA node resolution step can no longer emit an rte_panic. This aligns with the code in rte_eal_init, which expects failures to return an error code.
--- lib/librte_eal/common/eal_common_lcore.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
lib/librte_eal/common/eal_common_lcore.c +4 -3
@@ -83,16 +83,17 @@ rte_eal_cpu_init(void) config->lcore_role[lcore_id] =ROLE_RTE ; lcore_config[lcore_id].core_id =eal_cpu_core_id (lcore_id); lcore_config[lcore_id].socket_id =eal_cpu_socket_id (lcore_id); - if (lcore_config[lcore_id].socket_id >= RTE_MAX_NUMA_NODES) +if (lcore_config[lcore_id].socket_id >=RTE_MAX_NUMA_NODES ) {#ifdef RTE_EAL_ALLOW_INV_SOCKET_ID lcore_config[lcore_id].socket_id =0 ;#else -rte_panic ("Socket ID (%u) is greater than " +RTE_LOG (ERR ,EAL ,"Socket ID (%u) is greater than " "RTE_MAX_NUMA_NODES (%d)\n" , lcore_config[lcore_id].socket_id,RTE_MAX_NUMA_NODES ); +return -1 ;#endif - + } RTE_LOG(DEBUG, EAL, "Detected lcore %u as " "core %u on socket %u\n", lcore_id, lcore_config[lcore_id].core_id,
Signature · 1 line
2.9.3
PATCH v3 2/25 eal: return error instead of panic for cpu init
In reply to Aaron ConoleThere may be no way to gracefully recover, but the application should be notified that a failure happened, rather than completely aborting. This allows the user to proceed with a "slow-path" type solution.
--- lib/librte_eal/linuxapp/eal/eal.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
@@ -767,8 +767,11 @@ rte_eal_init(int argc, char **argv)/* set log level as early as possible */ rte_set_log_level (internal_config.log_level); -if (rte_eal_cpu_init () <0 ) -rte_panic ("Cannot detect lcores\n" ); +if (rte_eal_cpu_init () <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot detect lcores\n" ); + rte_errno =ENOTSUP ; +return -1 ; + } fctret =eal_parse_args (argc, argv);if (fctret <0 )
Signature · 1 line
2.9.3
PATCH v3 3/25 eal: No panic on hugepages info init
In reply to Aaron ConoleWhen attempting to scan hugepages, signal to the eal.c that an error has occurred, rather than performing a panic.
--- lib/librte_eal/linuxapp/eal/eal_hugepage_info.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal_hugepage_info.c +4 -2
@@ -283,9 +283,11 @@ eal_hugepage_info_init(void)struct dirent *dirent; dir =opendir (sys_dir_path); -if (dir ==NULL ) -rte_panic ("Cannot open directory %s to read system hugepage " +if (dir ==NULL ) { +RTE_LOG (ERR ,EAL ,"Cannot open directory %s to read system hugepage " "info\n" , sys_dir_path); +return -1 ; + }for (dirent =readdir (dir); dirent !=NULL ; dirent =readdir (dir)) {struct hugepage_info *hpi;
Signature · 1 line
2.9.3
PATCH v3 4/25 eal: do not panic on failed hugepage query
In reply to Aaron ConoleIf we fail to acquire hugepage information, simply signal an error to the application. This clears the run_once counter, allowing the user or application to take a corrective action and retry.
--- lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
@@ -780,8 +780,12 @@ rte_eal_init(int argc, char **argv)if (internal_config.no_hugetlbfs ==0 && internal_config.process_type !=RTE_PROC_SECONDARY && internal_config.xen_dom0_support ==0 && -eal_hugepage_info_init () <0 ) -rte_panic ("Cannot get hugepage information\n" ); +eal_hugepage_info_init () <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot get hugepage information\n" ); + rte_errno =EACCES ; +rte_atomic32_clear (&run_once); +return -1 ; + }if (internal_config.memory ==0 && internal_config.force_sockets ==0 ) {if (internal_config.no_hugetlbfs)
Signature · 1 line
2.9.3
PATCH v3 5/25 eal: failure to parse args returns error
In reply to Aaron ConoleIt's possible that the application could take a corrective action here, and either prompt the user for different arguments, or at least perform a better logging. Exiting this early prevents any useful information gathering from the application layer.
--- lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
@@ -774,8 +774,12 @@ rte_eal_init(int argc, char **argv) } fctret =eal_parse_args (argc, argv); -if (fctret <0 ) -exit (1 ); +if (fctret <0 ) { +RTE_LOG (ERR ,EAL ,"Invalid 'command line' arguments\n" ); + rte_errno =EINVAL ; +rte_atomic32_clear (&run_once); +return -1 ; + } if (internal_config.no_hugetlbfs == 0 && internal_config.process_type != RTE_PROC_SECONDARY &&
Signature · 1 line
2.9.3
PATCH v3 6/25 eal-common: introduce a way to query cpu support
In reply to Aaron ConoleThis adds a new API to check for the eal cpu versions.
--- lib/librte_eal/common/eal_common_cpuflags.c | 13 +++++++++++-- lib/librte_eal/common/include/generic/rte_cpuflags.h | 9 +++++++++ 2 files changed, 20 insertions(+), 2 deletions(-)
lib/librte_eal/common/eal_common_cpuflags.c +11 -2
@@ -43,6 +43,13 @@void rte_cpu_check_supported (void ) { +if (!rte_cpu_is_supported ()) +exit (1 ); +} + +bool +rte_cpu_is_supported (void ) +{/* This is generated at compile-time by the build system */ static const enum rte_cpu_flag_t compile_time_flags[] = {RTE_COMPILE_TIME_CPUFLAGS
@@ -57,14 +64,16 @@ rte_cpu_check_supported(void)fprintf (stderr,"ERROR: CPU feature flag lookup failed with error %d\n" , ret); -exit (1 ); +return false; }if (!ret) {fprintf (stderr,"ERROR: This system does not support \"%s\".\n" "Please check that RTE_MACHINE is set correctly.\n" ,rte_cpu_get_flag_name (compile_time_flags[i])); -exit (1 ); +return false; } } + +return true; }
lib/librte_eal/common/include/generic/rte_cpuflags.h +9 -0
@@ -40,6 +40,7 @@*/ #include <errno.h> +#include <stdbool.h> /** * Enumeration of all CPU features supported
@@ -82,4 +83,12 @@ rte_cpu_get_flag_enabled(enum rte_cpu_flag_t feature);void rte_cpu_check_supported (void ); +/** +* This function checks that the currently used CPU supports the CPU features +* that were specified at compile time. It is called automatically within the +* EAL, so does not need to be used by applications. This version returns a +* result so that decisions may be made (for instance, graceful shutdowns). +*/ +bool +rte_cpu_is_supported (void );#endif /* _RTE_CPUFLAGS_H_ */
Signature · 1 line
2.9.3
PATCH v3 7/25 eal: Signal error when CPU isn't supported
In reply to Aaron ConoleIt's now possible to gracefully exit the application, or for applications which support non-dpdk datapaths working in concert with DPDK datapaths, there no longer is the possibility of exiting for unsupported CPUs.
--- lib/librte_eal/linuxapp/eal/eal.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -1
@@ -752,7 +752,11 @@ rte_eal_init(int argc, char **argv)char thread_name[RTE_MAX_THREAD_NAME_LEN ];/* checks if the machine is adequate */ -rte_cpu_check_supported (); +if (!rte_cpu_is_supported ()) { +fprintf (stderr,"EAL: FATAL - unsupported cpu type.\n" ); + rte_errno =ENOTSUP ; +return -1 ; + }if (!rte_atomic32_test_and_set (&run_once))return -1 ;
Signature · 1 line
2.9.3
PATCH v3 8/25 eal: do not panic on memzone initialization fails
In reply to Aaron ConoleWhen memzone initialization fails, report the error to the calling application rather than panic(). Without a good way of detaching / releasing hugepages, at this point the application will have to restart.
--- lib/librte_eal/linuxapp/eal/eal.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
@@ -832,8 +832,11 @@ rte_eal_init(int argc, char **argv)/* the directories are locked during eal_hugepage_info_init */ eal_hugedirs_unlock (); -if (rte_eal_memzone_init () <0 ) -rte_panic ("Cannot init memzone\n" ); +if (rte_eal_memzone_init () <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot init memzone\n" ); + rte_errno =ENODEV ; +return -1 ; + }if (rte_eal_tailqs_init () <0 )rte_panic ("Cannot init tail queues for objects\n" );
Signature · 1 line
2.9.3
PATCH v3 9/25 eal: set errno when exiting for already called
In reply to Aaron Conole--- lib/librte_eal/linuxapp/eal/eal.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal.c +4 -1
@@ -758,8 +758,11 @@ rte_eal_init(int argc, char **argv)return -1 ; } -if (!rte_atomic32_test_and_set (&run_once)) +if (!rte_atomic32_test_and_set (&run_once)) { +fprintf (stderr,"EAL: ERROR - already called initialization.\n" ); + rte_errno =EALREADY ;return -1 ; + } logid =strrchr (argv[0 ],'/' ); logid =strdup (logid ? logid +1 : argv[0 ]);
Signature · 1 line
2.9.3
PATCH v3 10/25 eal: Do not panic on log failures
In reply to Aaron ConoleWhen log initialization fails, it's generally because the fopencookie failed. While this is rare in practice, it could happen, and it is likely because of memory pressure. So, flag the error, and allow the user to retry.
--- lib/librte_eal/linuxapp/eal/eal.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +7 -2
@@ -818,8 +818,13 @@ rte_eal_init(int argc, char **argv)rte_config_init (); -if (rte_eal_log_init (logid, internal_config.syslog_facility) <0 ) -rte_panic ("Cannot init logs\n" ); +if (rte_eal_log_init (logid, internal_config.syslog_facility) <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot init logging\n" ); +fprintf (stderr,"EAL: ERROR - cannot init logging.\n" ); + rte_errno =EIO ; +rte_atomic32_clear (&run_once); +return -1 ; + }if (rte_eal_pci_init () <0 )rte_panic ("Cannot init PCI\n" );
Signature · 1 line
2.9.3
PATCH v3 11/25 eal: Do not panic on pci-probe
In reply to Aaron ConoleThis will usually be an issue because of permissions. However, it could also be caused by OOM. In either case, errno will contain the underlying cause. It is safe to re-init the system here, so allow the application to take corrective action and reinit.
--- lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
@@ -826,8 +826,12 @@ rte_eal_init(int argc, char **argv)return -1 ; } -if (rte_eal_pci_init () <0 ) -rte_panic ("Cannot init PCI\n" ); +if (rte_eal_pci_init () <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot init PCI\n" ); + rte_errno =EUNATCH ; +rte_atomic32_clear (&run_once); +return -1 ; + }#ifdef VFIO_PRESENT if (rte_eal_vfio_setup () <0 )
Signature · 1 line
2.9.3
PATCH v3 12/25 eal: do not panic on vfio failure
In reply to Aaron Conole--- lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
@@ -834,8 +834,12 @@ rte_eal_init(int argc, char **argv) }#ifdef VFIO_PRESENT -if (rte_eal_vfio_setup () <0 ) -rte_panic ("Cannot init VFIO\n" ); +if (rte_eal_vfio_setup () <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot init VFIO\n" ); + rte_errno =EAGAIN ; +rte_atomic32_clear (&run_once); +return -1 ; + }#endif if (rte_eal_memory_init () <0 )
Signature · 1 line
2.9.3
PATCH v3 13/25 eal: do not panic on memory init
In reply to Aaron ConoleThis can only happen when access to hugepages (either as primary or secondary process) fails (and that is usually permissions). Since the manner of failure is not reversible, we cannot allow retry.
--- lib/librte_eal/linuxapp/eal/eal.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
@@ -842,8 +842,11 @@ rte_eal_init(int argc, char **argv) }#endif -if (rte_eal_memory_init () <0 ) -rte_panic ("Cannot init memory\n" ); +if (rte_eal_memory_init () <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot init memory\n" ); + rte_errno =EACCES ; +return -1 ; + }/* the directories are locked during eal_hugepage_info_init */ eal_hugedirs_unlock ();
Signature · 1 line
2.9.3
PATCH v3 14/25 eal: do not panic on tailq init
In reply to Aaron ConoleThere are some theoretical racy conditions in the system that _could_ cause early tailq init to fail; however, no need to panic the application. While it can't continue using DPDK, it could make better alerts to the user.
--- lib/librte_eal/common/eal_common_tailqs.c | 3 +-- lib/librte_eal/linuxapp/eal/eal.c | 7 +++++-- 2 files changed, 6 insertions(+), 4 deletions(-)
lib/librte_eal/common/eal_common_tailqs.c +1 -2
@@ -188,8 +188,7 @@ rte_eal_tailqs_init(void)if (t->head ==NULL ) {RTE_LOG (ERR ,EAL ,"Cannot initialize tailq: %s\n" , t->name); -/* no need to TAILQ_REMOVE, we are going to panic in -* rte_eal_init() */ +/* TAILQ_REMOVE not needed, error is already fatal */ goto fail; } }
lib/librte_eal/linuxapp/eal/eal.c +5 -2
@@ -857,8 +857,11 @@ rte_eal_init(int argc, char **argv)return -1 ; } -if (rte_eal_tailqs_init () <0 ) -rte_panic ("Cannot init tail queues for objects\n" ); +if (rte_eal_tailqs_init () <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot init tail queues for objects\n" ); + rte_errno =EFAULT ; +return -1 ; + }if (rte_eal_alarm_init () <0 )rte_panic ("Cannot init interrupt-handling thread\n" );
Signature · 1 line
2.9.3
PATCH v3 15/25 eal: do not panic on alarm init
In reply to Aaron Conolerte_eal_alarm_init() call uses the linux timerfd framework to create a poll()-able timer using standard posix file operations. This could fail for a few reasons given in the man-pages, but many could be corrected by the user application. No need to panic.
--- lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
@@ -61,6 +61,7 @@#include <rte_launch.h> #include <rte_eal.h> #include <rte_eal_memconfig.h> +#include <rte_errno.h> #include <rte_per_lcore.h> #include <rte_lcore.h> #include <rte_log.h>
@@ -863,8 +864,11 @@ rte_eal_init(int argc, char **argv)return -1 ; } -if (rte_eal_alarm_init () <0 ) -rte_panic ("Cannot init interrupt-handling thread\n" ); +if (rte_eal_alarm_init () <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot init interrupt-handling thread\n" ); +/* rte_eal_alarm_init sets rte_errno on failure. */ +return -1 ; + }if (rte_eal_timer_init () <0 )rte_panic ("Cannot init HPET or TSC timers\n" );
Signature · 1 line
2.9.3
PATCH v3 16/25 eal: convert timer_init not to call panic
In reply to Aaron ConoleAfter code inspection, there is no way for eal_timer_init() to fail. It simply returns 0 in all cases. As such, this test could either go-away or stay here as 'future-proofing'.
--- lib/librte_eal/linuxapp/eal/eal.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
@@ -870,8 +870,11 @@ rte_eal_init(int argc, char **argv)return -1 ; } -if (rte_eal_timer_init () <0 ) -rte_panic ("Cannot init HPET or TSC timers\n" ); +if (rte_eal_timer_init () <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot init HPET or TSC timers\n" ); + rte_errno =ENOTSUP ; +return -1 ; + }eal_check_mem_on_local_socket ();
Signature · 1 line
2.9.3
PATCH v3 17/25 eal: change the private pipe call to reflect errno
In reply to Aaron ConoleThere could be some confusion as to why the call failed - this change will always reflect the value of the error in rte_error.
--- lib/librte_eal/linuxapp/eal/eal_interrupts.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal_interrupts.c +4 -1
@@ -896,13 +896,16 @@ rte_eal_intr_init(void)* create a pipe which will be waited by epoll and notified to * rebuild the wait list of epoll. */ -if (pipe (intr_pipe.pipefd) <0 ) +if (pipe (intr_pipe.pipefd) <0 ) { + rte_errno = errno;return -1 ; + }/* create the host thread to wait/handle the interrupt */ ret =pthread_create (&intr_thread,NULL , eal_intr_thread_main,NULL );if (ret !=0 ) { + rte_errno = ret;RTE_LOG (ERR ,EAL ,"Failed to create thread for interrupt handling\n" ); }else {
Signature · 1 line
2.9.3
PATCH v3 18/25 eal: Do not panic on interrupt thread init
In reply to Aaron ConoleWhen initializing the interrupt thread, there are a number of possible reasons for failure - some of which are correctable by the application. Do not panic() needlessly, and give the application a change to reflect this information to the user.
--- lib/librte_eal/linuxapp/eal/eal.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +4 -2
@@ -889,8 +889,10 @@ rte_eal_init(int argc, char **argv) rte_config.master_lcore, (int )thread_id, cpuset, ret == 0 ? "" : "..."); -if (rte_eal_intr_init () <0 ) -rte_panic ("Cannot init interrupt-handling thread\n" ); +if (rte_eal_intr_init () <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot init interrupt-handling thread\n" ); +return -1 ; + }if (rte_bus_scan ())rte_panic ("Cannot scan the buses for devices\n" );
Signature · 1 line
2.9.3
PATCH v3 19/25 eal: do not error if plugins fail to init
In reply to Aaron ConolePlugins are useful and important. However, it seems crazy to abort everything just because they don't initialize properly.
--- lib/librte_eal/linuxapp/eal/eal.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal.c +1 -1
@@ -879,7 +879,7 @@ rte_eal_init(int argc, char **argv)eal_check_mem_on_local_socket ();if (eal_plugins_init () <0 ) -rte_panic ("Cannot init plugins\n" ); +RTE_LOG (ERR ,EAL ,"Cannot init plugins\n" );eal_thread_init_master (rte_config.master_lcore);
Signature · 1 line
2.9.3
PATCH v3 20/25 eal_pci: Continue probing even on failures
In reply to Aaron ConoleSome devices may be inaccessible for a variety of reasons, or the PCI-bus may be unavailable causing the whole thing to fail. Still, better to continue attempts at probes.
--- lib/librte_eal/common/eal_common_pci.c | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-)
lib/librte_eal/common/eal_common_pci.c +10 -5
@@ -69,6 +69,7 @@#include <sys/queue.h> #include <sys/mman.h> +#include <rte_errno.h> #include <rte_interrupts.h> #include <rte_log.h> #include <rte_pci.h>
@@ -416,6 +417,7 @@ rte_eal_pci_probe(void)struct rte_pci_device *dev =NULL ;struct rte_devargs *devargs;int probe_all =0 ; +int ret_1 =0 ;int ret =0 ;if (rte_eal_devargs_type_count (RTE_DEVTYPE_WHITELISTED_PCI ) ==0 )
@@ -430,17 +432,20 @@ rte_eal_pci_probe(void)/* probe all or only whitelisted devices */ if (probe_all) - ret =pci_probe_all_drivers (dev); + ret_1 =pci_probe_all_drivers (dev);else if (devargs !=NULL && devargs->type ==RTE_DEVTYPE_WHITELISTED_PCI ) - ret =pci_probe_all_drivers (dev); -if (ret <0 ) -rte_exit (EXIT_FAILURE ,"Requested device " PCI_PRI_FMT + ret_1 =pci_probe_all_drivers (dev); +if (ret_1 <0 ) { +RTE_LOG (ERR ,EAL ,"Requested device " PCI_PRI_FMT " cannot be used\n" , dev->addr.domain, dev->addr.bus, dev->addr.devid, dev->addr.function); + rte_errno = errno; + ret =1 ; + } } -return 0 ; +return -ret; }/* dump one device */
Signature · 1 line
2.9.3
PATCH v3 21/25 eal: do not panic on failed PCI probe
In reply to Aaron ConoleIt may even be possible to simply log the error and continue on letting the user check the logs and restart the application when things are failed.
--- lib/librte_eal/linuxapp/eal/eal.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
@@ -938,8 +938,11 @@ rte_eal_init(int argc, char **argv)rte_panic ("Cannot probe devices\n" );/* Probe & Initialize PCI devices */ -if (rte_eal_pci_probe ()) -rte_panic ("Cannot probe PCI\n" ); +if (rte_eal_pci_probe ()) { +RTE_LOG (ERR ,EAL ,"Cannot probe PCI\n" ); + rte_errno =ENOTSUP ; +return -1 ; + }if (rte_eal_dev_init () <0 )rte_panic ("Cannot init pmd devices\n" );
Signature · 1 line
2.9.3
PATCH v3 22/25 eal_common_dev: continue initializing vdevs
In reply to Aaron ConoleEven if one vdev should fail, there's no need to prevent further processing. Log the error, and reflect it to the higher levels to decide.
--- lib/librte_eal/common/eal_common_dev.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
lib/librte_eal/common/eal_common_dev.c +3 -2
@@ -80,6 +80,7 @@ intrte_eal_dev_init (void) {struct rte_devargs *devargs; +int ret =0 ;/* * Note that the dev_driver_list is populated here
@@ -97,11 +98,11 @@ rte_eal_dev_init(void) devargs->args)) {RTE_LOG (ERR ,EAL ,"failed to initialize %s device\n" , devargs->virt.drv_name); -return -1 ; + ret =-1 ; } } -return 0 ; +return ret; }int rte_eal_dev_attach (const char *name,const char *devargs)
Signature · 1 line
2.9.3
PATCH v3 23/25 eal: do not panic (or abort) if vdev init fails
In reply to Aaron ConoleSeems like it's possible to continue. At least, the error is reflected properly in the logs. A user could then go and correct or investigate the situation.
--- lib/librte_eal/linuxapp/eal/eal.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal.c +1 -1
@@ -945,7 +945,7 @@ rte_eal_init(int argc, char **argv) }if (rte_eal_dev_init () <0 ) -rte_panic ("Cannot init pmd devices\n" ); +RTE_LOG (ERR ,EAL ,"Cannot init pmd devices\n" );rte_eal_mcfg_complete ();
Signature · 1 line
2.9.3
PATCH v3 24/25 eal: do not panic when bus probe fails
In reply to Aaron Conole--- lib/librte_eal/linuxapp/eal/eal.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
@@ -934,8 +934,11 @@ rte_eal_init(int argc, char **argv)rte_eal_mp_wait_lcore ();/* Probe all the buses and devices/drivers on them */ -if (rte_bus_probe ()) -rte_panic ("Cannot probe devices\n" ); +if (rte_bus_probe ()) { +RTE_LOG (ERR ,EAL ,"Cannot probe devices\n" ); + rte_errno =ENOTSUP ; +return -1 ; + }/* Probe & Initialize PCI devices */ if (rte_eal_pci_probe ()) {
Signature · 1 line
2.9.3
PATCH v3 25/25 rte_eal_init: add info about rte_errno codes
In reply to Aaron ConoleThe rte_eal_init function will now pass failure reason hints to the application. To help app developers deciper this, add some brief information about what the codes are indicating.
--- lib/librte_eal/common/include/rte_eal.h | 27 ++++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-)
lib/librte_eal/common/include/rte_eal.h +26 -1
@@ -159,7 +159,32 @@ int rte_eal_iopl_init(void);* function call and should not be further interpreted by the * application. The EAL does not take any ownership of the memory used * for either the argv array, or its members. -* - On failure, a negative error value. +* - On failure, -1 and rte_errno is set to a value indicating the cause +* for failure. +* +* The error codes returned via rte_errno: +* EACCES indicates a permissions issue. +* +* EAGAIN indicates either a bus or system resource was not available, +* try again. +* +* EALREADY indicates that the rte_eal_init function has already been +* called, and cannot be called again. +* +* EFAULT indicates the tailq configuration name was not found in +* memory configuration. +* +* EINVAL indicates invalid parameters were passed as argv/argc. +* +* EIO indicates failure to setup the logging handlers. This is usually +* caused by an out-of-memory condition. +* +* ENODEV indicates memory setup issues. +* +* ENOTSUP indicates that the EAL cannot initialize on this system. +* +* EUNATCH indicates that the PCI bus is either not present, or is not +* readable by the eal. */ int rte_eal_init (int argc,char **argv);
Signature · 1 line
2.9.3
Re: [PATCH v3 25/25] rte_eal_init: add info about rte_errno codes
Quoted 16 lines · On Thu, 9 Feb 2017 09:29:53 -0500 Aaron Conole <[email protected]> wrote:
+ * The error codes returned via
rte_errno:
+ * EACCES indicates a permissions issue.
+ *
+ * EAGAIN indicates either a bus or system resource was not available,
+ * try again.
+ *
+ * EALREADY indicates that therte_eal_initfunction has already been
+ * called, and cannot be called again.
+ *
+ * EFAULT indicates the tailq configuration name was not found in
+ * memory configuration.
+ *
+ * EINVAL indicates invalid parameters were passed as argv/argc.
+ *
+ * EIO indicates failure to setup the logging handlers. This is usually
+ * caused by an out-of-memory condition.
Quoted 8 lines
+ *
+ * ENODEV indicates memory setup issues.
+ *
+ * ENOTSUP indicates that the EAL cannot initialize on this system.
+ *
+ * EUNATCH indicates that the PCI bus is either not present, or is not
+ * readable by the eal.
*/
You might want to be less restrictive about wording in the comment. In future more errors might be returned, and also for out of memory ENOMEM is better.
Re: [PATCH v3 25/25] rte_eal_init: add info about rte_errno codes
Quoted 28 lines · Stephen Hemminger
Quoted 16 lines · On Thu, 9 Feb 2017 09:29:53 -0500 Aaron Conole <[email protected]> wrote:
+ * The error codes returned via
rte_errno:
+ * EACCES indicates a permissions issue.
+ *
+ * EAGAIN indicates either a bus or system resource was not available,
+ * try again.
+ *
+ * EALREADY indicates that therte_eal_initfunction has already been
+ * called, and cannot be called again.
+ *
+ * EFAULT indicates the tailq configuration name was not found in
+ * memory configuration.
+ *
+ * EINVAL indicates invalid parameters were passed as argv/argc.
+ *
+ * EIO indicates failure to setup the logging handlers. This is usually
+ * caused by an out-of-memory condition.Quoted 8 lines
+ *
+ * ENODEV indicates memory setup issues.
+ *
+ * ENOTSUP indicates that the EAL cannot initialize on this system.
+ *
+ * EUNATCH indicates that the PCI bus is either not present, or is not
+ * readable by the eal.
*/
Quoted 3 lines
You might want to be less restrictive about wording in the comment. In future more errors might be returned, and also for out of memory ENOMEM is better.
Sure thing, I'll switch EIO and ENODEV to ENOMEM, does that make sense?
Also, which message do you refer to? Is it "The error codes returned via rte_errno" section? I assume that adding new error codes will also bring an update to the eal_init documentation, but perhaps I'm misunderstanding.
Thanks for your review, Stephen!
Quoted 48 lines · On Thu, 9 Feb 2017 09:29:28 -0500 Aaron Conole <[email protected]> wrote:
In many cases, it's enough to simply let the application know that the call to initialize DPDK has failed. A complete halt can then be decided by the application based on error returned (and the app could even attempt a possible re-attempt after some corrective action by the user or application).
Changes ->v2:
- Audited all "RTE_LOG(" calls that were introduced, and converted to "RTE_LOG("
- Added somefprintf(stderr, "")lines to indicate errors before logging is initialized
- Removed assignments to errno.
- Changed patch 14/25 to reflect EFAULT, and document in 25/25Changes ->v3:
- Checkpatch issues in patches 3 (spelling mistake), 9 (issue with leading spaces), and 19 (braces around single line statement if-condition)I kept the
rte_errnoreflection, since this is control-path code and the init function returns a sentinel value of -1.Aaron Conole (25):
eal: CPU init will no longer panic
eal: return error instead of panic for cpu init
eal: No panic on hugepages info init
eal: do not panic on failed hugepage query
eal: failure to parse args returns error
eal-common: introduce a way to query cpu support
eal: Signal error when CPU isn't supported
eal: do not panic on memzone initialization fails
eal: set errno when exiting for already called
eal: Do not panic on log failures
eal: Do not panic on pci-probe
eal: do not panic on vfio failure
eal: do not panic on memory init
eal: do not panic on tailq init
eal: do not panic on alarm init
eal: converttimer_initnot to call panic
eal: change the private pipe call to reflect errno
eal: Do not panic on interrupt thread init
eal: do not error if plugins fail to init
eal_pci: Continue probing even on failures
eal: do not panic on failed PCI probe
eal_common_dev: continue initializing vdevs
eal: do not panic (or abort) if vdev init fails
eal: do not panic when bus probe fails
rte_eal_init: add info aboutrte_errnocodes
Quoted 12 lines
lib/librte_eal/common/eal_common_cpuflags.c | 13 ++- lib/librte_eal/common/eal_common_dev.c | 5 +- lib/librte_eal/common/eal_common_lcore.c | 7 +- lib/librte_eal/common/eal_common_pci.c | 15 ++- lib/librte_eal/common/eal_common_tailqs.c | 3 +- .../common/include/generic/rte_cpuflags.h | 9 ++ lib/librte_eal/common/include/rte_eal.h | 27 ++++- lib/librte_eal/linuxapp/eal/eal.c | 122 +++++++++++++++------ lib/librte_eal/linuxapp/eal/eal_hugepage_info.c | 6 +- lib/librte_eal/linuxapp/eal/eal_interrupts.c | 5 +- 10 files changed, 161 insertions(+), 51 deletions(-)
I worry that some of these early failure messages may never be visible because the logging system has not been initialized. Might be safer to just use fprintf(stderr, ...) or define a new wrapper function.
Quoted 9 lines · Stephen Hemminger
Quoted 6 lines · On Thu, 9 Feb 2017 09:29:28 -0500 Aaron Conole <[email protected]> wrote:
In many cases, it's enough to simply let the application know that the call to initialize DPDK has failed. A complete halt can then be decided by the application based on error returned (and the app could even attempt a possible re-attempt after some corrective action by the user or application).
Quoted 6 lines
Quoted 2 lines
...
I worry that some of these early failure messages may never be visible because the logging system has not been initialized. Might be safer to just use
fprintf(stderr, ...)or define a new wrapper function.
Thanks for the suggestion, Stephen! I've folded it into my series.
-Aaron
v4 cover linux/eal: Remove most causes of panic on init
In reply to Aaron ConoleIn many cases, it's enough to simply let the application know that the call to initialize DPDK has failed. A complete halt can then be decided by the application based on error returned (and the app could even attempt a possible re-attempt after some corrective action by the user or application).
Changes ->v2:
- Audited all "RTE_LOG (" calls that were introduced, and converted to "RTE_LOG("
- Added some fprintf(stderr, "") lines to indicate errors before logging is initialized
- Removed assignments to errno.
- Changed patch 14/25 to reflect EFAULT, and document in 25/25
Changes ->v3:
- Checkpatch issues in patches 3 (spelling mistake), 9 (issue with leading spaces), and 19 (braces around single line statement if-condition)
Changes ->v4:
- Error text cleanup.
- Add a new check around rte_bus_scan(), added during the development of this series.
I kept the rte_errno reflection, since this is control-path code and the init function returns a sentinel value of -1.
Aaron Conole (26):
eal: CPU init will no longer panic
eal: return error instead of panic for cpu init
eal: No panic on hugepages info init
eal: do not panic on failed hugepage query
eal: failure to parse args returns error
eal-common: introduce a way to query cpu support
eal: Signal error when CPU isn't supported
eal: do not panic on memzone initialization fails
eal: set errno when exiting for already called
eal: Do not panic on log failures
eal: Do not panic on pci-probe
eal: do not panic on vfio failure
eal: do not panic on memory init
eal: do not panic on tailq init
eal: do not panic on alarm init
eal: convert timer_init not to call panic
eal: change the private pipe call to reflect errno
eal: Do not panic on interrupt thread init
eal: do not error if plugins fail to init
eal_pci: Continue probing even on failures
eal: do not panic on failed PCI probe
eal_common_dev: continue initializing vdevs
eal: do not panic (or abort) if vdev init fails
eal: do not panic when bus probe fails
eal: do not panic on failed bus scan
rte_eal_init: add info about rte_errno codes
lib/librte_eal/common/eal_common_cpuflags.c | 13 +- lib/librte_eal/common/eal_common_dev.c | 5 +- lib/librte_eal/common/eal_common_lcore.c | 7 +- lib/librte_eal/common/eal_common_pci.c | 15 ++- lib/librte_eal/common/eal_common_tailqs.c | 3 +- .../common/include/generic/rte_cpuflags.h | 9 ++ lib/librte_eal/common/include/rte_eal.h | 27 ++++- lib/librte_eal/linuxapp/eal/eal.c | 131 +++++++++++++++------ lib/librte_eal/linuxapp/eal/eal_hugepage_info.c | 6 +- lib/librte_eal/linuxapp/eal/eal_interrupts.c | 5 +- 10 files changed, 169 insertions(+), 52 deletions(-)
Signature · 1 line
2.9.3
PATCH v4 1/26 eal: CPU init will no longer panic
After this change, the EAL CPU NUMA node resolution step can no longer emit an rte_panic. This aligns with the code in rte_eal_init, which expects failures to return an error code.
--- lib/librte_eal/common/eal_common_lcore.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
lib/librte_eal/common/eal_common_lcore.c +4 -3
@@ -83,16 +83,17 @@ rte_eal_cpu_init(void) config->lcore_role[lcore_id] =ROLE_RTE ; lcore_config[lcore_id].core_id =eal_cpu_core_id (lcore_id); lcore_config[lcore_id].socket_id =eal_cpu_socket_id (lcore_id); - if (lcore_config[lcore_id].socket_id >= RTE_MAX_NUMA_NODES) +if (lcore_config[lcore_id].socket_id >=RTE_MAX_NUMA_NODES ) {#ifdef RTE_EAL_ALLOW_INV_SOCKET_ID lcore_config[lcore_id].socket_id =0 ;#else -rte_panic ("Socket ID (%u) is greater than " +RTE_LOG (ERR ,EAL ,"Socket ID (%u) is greater than " "RTE_MAX_NUMA_NODES (%d)\n" , lcore_config[lcore_id].socket_id,RTE_MAX_NUMA_NODES ); +return -1 ;#endif - + } RTE_LOG(DEBUG, EAL, "Detected lcore %u as " "core %u on socket %u\n", lcore_id, lcore_config[lcore_id].core_id,
Signature · 1 line
2.9.3
PATCH v4 2/26 eal: return error instead of panic for cpu init
In reply to Aaron ConoleThere may be no way to gracefully recover, but the application should be notified that a failure happened, rather than completely aborting. This allows the user to proceed with a "slow-path" type solution.
--- lib/librte_eal/linuxapp/eal/eal.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +11 -2
@@ -740,6 +740,12 @@ static int rte_eal_vfio_setup(void) }#endif +static void rte_eal_init_alert (const char *msg) +{ +fprintf (stderr,"EAL: FATAL: %s\n" , msg); +RTE_LOG (ERR ,EAL ,"%s\n" , msg); +} +/* Launch threads, called at application init(). */ int rte_eal_init (int argc,char **argv)
@@ -767,8 +773,11 @@ rte_eal_init(int argc, char **argv)/* set log level as early as possible */ rte_set_log_level (internal_config.log_level); -if (rte_eal_cpu_init () <0 ) -rte_panic ("Cannot detect lcores\n" ); +if (rte_eal_cpu_init () <0 ) { +rte_eal_init_alert ("Cannot detect lcores." ); + rte_errno =ENOTSUP ; +return -1 ; + } fctret =eal_parse_args (argc, argv);if (fctret <0 )
Signature · 1 line
2.9.3
Re: [PATCH v4 02/26] eal: return error instead of panic for cpu init
Quoted 32 lines · Aaron Conole
There may be no way to gracefully recover, but the application should be notified that a failure happened, rather than completely aborting. This allows the user to proceed with a "slow-path" type solution.
Signed-off-by Aaron Conole <[email protected]>--- lib/librte_eal/linuxapp/eal/eal.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-)lib/librte_eal/linuxapp/eal/eal.c +6
-0@@ -740,6 +740,12 @@ static int rte_eal_vfio_setup(void) }#endif +static void rte_eal_init_alert (const char *msg) +{ +fprintf (stderr,"EAL: FATAL: %s\n" , msg); +RTE_LOG (ERR ,EAL ,"%s\n" , msg); +} +/* Launch threads, called at application init(). */ int rte_eal_init (int argc,char **argv)@@ -767,8 +773,11 @@ rte_eal_init(int argc, char **argv)/* set log level as early as possible */ rte_set_log_level (internal_config.log_level);
lib/librte_eal/linuxapp/eal/eal.c rte_eal_init(int argc, char **argv) · line 783
Quoted 10 lines
lib/librte_eal/linuxapp/eal/eal.c +5
-2-if (rte_eal_cpu_init () <0 ) -rte_panic ("Cannot detect lcores\n" ); +if (rte_eal_cpu_init () <0 ) { +rte_eal_init_alert ("Cannot detect lcores." ); + rte_errno =ENOTSUP ; +return -1 ; + } fctret =eal_parse_args (argc, argv);if (fctret <0 )
eal.c needs to include rte_errno.h after this change, otherwise it won't compile.
/Bruce
Re: [PATCH v4 02/26] eal: return error instead of panic for cpu init
Quoted 40 lines · Bruce Richardson
Quoted 39 lines · Aaron Conole
There may be no way to gracefully recover, but the application should be notified that a failure happened, rather than completely aborting. This allows the user to proceed with a "slow-path" type solution.
Signed-off-by Aaron Conole <[email protected]>--- lib/librte_eal/linuxapp/eal/eal.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-)lib/librte_eal/linuxapp/eal/eal.c +11
-2@@ -740,6 +740,12 @@ static int rte_eal_vfio_setup(void) }#endif +static void rte_eal_init_alert (const char *msg) +{ +fprintf (stderr,"EAL: FATAL: %s\n" , msg); +RTE_LOG (ERR ,EAL ,"%s\n" , msg); +} +/* Launch threads, called at application init(). */ int rte_eal_init (int argc,char **argv)@@ -767,8 +773,11 @@ rte_eal_init(int argc, char **argv)/* set log level as early as possible */ rte_set_log_level (internal_config.log_level); -if (rte_eal_cpu_init () <0 ) -rte_panic ("Cannot detect lcores\n" ); +if (rte_eal_cpu_init () <0 ) { +rte_eal_init_alert ("Cannot detect lcores." ); + rte_errno =ENOTSUP ; +return -1 ; + }
Quoted 5 lines
lib/librte_eal/linuxapp/eal/eal.c rte_eal_init(int argc, char **argv) · line 783
Quoted 3 lines
lib/librte_eal/linuxapp/eal/eal.c +0
-0fctret =eal_parse_args (argc, argv);if (fctret <0 )
eal.cneeds to includerte_errno.hafter this change, otherwise it won't compile.
oops.. I reordered some of my original work, and the rte_errno.h change was introduced too late. Thanks for catching this!
Quoted 1 line
/Bruce
Re: [PATCH v4 02/26] eal: return error instead of panic for cpu init
In reply to Aaron ConoleQuoted 15 lines · Aaron Conole
There may be no way to gracefully recover, but the application should be notified that a failure happened, rather than completely aborting. This allows the user to proceed with a "slow-path" type solution.
Signed-off-by Aaron Conole <[email protected]>--- lib/librte_eal/linuxapp/eal/eal.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-)lib/librte_eal/linuxapp/eal/eal.c +0
-0
lib/librte_eal/linuxapp/eal/eal.c static int rte_eal_vfio_setup(void) · line 747
Quoted 8 lines
lib/librte_eal/linuxapp/eal/eal.c +5
-0@@ -740,6 +740,12 @@ static int rte_eal_vfio_setup(void) }#endif +static void rte_eal_init_alert (const char *msg) +{ +fprintf (stderr,"EAL: FATAL: %s\n" , msg); +RTE_LOG (ERR ,EAL ,"%s\n" , msg); +}
Checkpatch flags the use of spaces rather than tabs here.
/Bruce
Re: [PATCH v4 02/26] eal: return error instead of panic for cpu init
Quoted 20 lines · Bruce Richardson
Quoted 19 lines · Aaron Conole
There may be no way to gracefully recover, but the application should be notified that a failure happened, rather than completely aborting. This allows the user to proceed with a "slow-path" type solution.
Signed-off-by Aaron Conole <[email protected]>--- lib/librte_eal/linuxapp/eal/eal.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-)lib/librte_eal/linuxapp/eal/eal.c +0
-0@@ -740,6 +740,12 @@ static int rte_eal_vfio_setup(void) }#endif
Quoted 6 lines
lib/librte_eal/linuxapp/eal/eal.c static int rte_eal_vfio_setup(void) · line 747
Quoted 5 lines
lib/librte_eal/linuxapp/eal/eal.c +5
-0+static void rte_eal_init_alert (const char *msg) +{ +fprintf (stderr,"EAL: FATAL: %s\n" , msg); +RTE_LOG (ERR ,EAL ,"%s\n" , msg); +}Checkpatch flags the use of spaces rather than tabs here.
Yes, I caught it too late. Sorry, I'll fix it.
Quoted 1 line
/Bruce
PATCH v4 3/26 eal: No panic on hugepages info init
In reply to Aaron ConoleWhen attempting to scan hugepages, signal to the eal.c that an error has occurred, rather than performing a panic.
--- lib/librte_eal/linuxapp/eal/eal_hugepage_info.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal_hugepage_info.c +4 -2
@@ -283,9 +283,11 @@ eal_hugepage_info_init(void)struct dirent *dirent; dir =opendir (sys_dir_path); -if (dir ==NULL ) -rte_panic ("Cannot open directory %s to read system hugepage " +if (dir ==NULL ) { +RTE_LOG (ERR ,EAL ,"Cannot open directory %s to read system hugepage " "info\n" , sys_dir_path); +return -1 ; + }for (dirent =readdir (dir); dirent !=NULL ; dirent =readdir (dir)) {struct hugepage_info *hpi;
Signature · 1 line
2.9.3
PATCH v4 4/26 eal: do not panic on failed hugepage query
In reply to Aaron ConoleIf we fail to acquire hugepage information, simply signal an error to the application. This clears the run_once counter, allowing the user or application to take a corrective action and retry.
--- lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
@@ -786,8 +786,12 @@ rte_eal_init(int argc, char **argv)if (internal_config.no_hugetlbfs ==0 && internal_config.process_type !=RTE_PROC_SECONDARY && internal_config.xen_dom0_support ==0 && -eal_hugepage_info_init () <0 ) -rte_panic ("Cannot get hugepage information\n" ); +eal_hugepage_info_init () <0 ) { +rte_eal_init_alert ("Cannot get hugepage information." ); + rte_errno =EACCES ; +rte_atomic32_clear (&run_once); +return -1 ; + }if (internal_config.memory ==0 && internal_config.force_sockets ==0 ) {if (internal_config.no_hugetlbfs)
Signature · 1 line
2.9.3
PATCH v4 5/26 eal: failure to parse args returns error
In reply to Aaron ConoleIt's possible that the application could take a corrective action here, and either prompt the user for different arguments, or at least perform a better logging. Exiting this early prevents any useful information gathering from the application layer.
--- lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
@@ -780,8 +780,12 @@ rte_eal_init(int argc, char **argv) } fctret =eal_parse_args (argc, argv); -if (fctret <0 ) -exit (1 ); +if (fctret <0 ) { +rte_eal_init_alert ("Invalid 'command line' arguments." ); + rte_errno =EINVAL ; +rte_atomic32_clear (&run_once); +return -1 ; + } if (internal_config.no_hugetlbfs == 0 && internal_config.process_type != RTE_PROC_SECONDARY &&
Signature · 1 line
2.9.3
PATCH v4 6/26 eal-common: introduce a way to query cpu support
In reply to Aaron ConoleThis adds a new API to check for the eal cpu versions.
--- lib/librte_eal/common/eal_common_cpuflags.c | 13 +++++++++++-- lib/librte_eal/common/include/generic/rte_cpuflags.h | 9 +++++++++ 2 files changed, 20 insertions(+), 2 deletions(-)
lib/librte_eal/common/eal_common_cpuflags.c +11 -2
@@ -43,6 +43,13 @@void rte_cpu_check_supported (void ) { +if (!rte_cpu_is_supported ()) +exit (1 ); +} + +bool +rte_cpu_is_supported (void ) +{/* This is generated at compile-time by the build system */ static const enum rte_cpu_flag_t compile_time_flags[] = {RTE_COMPILE_TIME_CPUFLAGS
@@ -57,14 +64,16 @@ rte_cpu_check_supported(void)fprintf (stderr,"ERROR: CPU feature flag lookup failed with error %d\n" , ret); -exit (1 ); +return false; }if (!ret) {fprintf (stderr,"ERROR: This system does not support \"%s\".\n" "Please check that RTE_MACHINE is set correctly.\n" ,rte_cpu_get_flag_name (compile_time_flags[i])); -exit (1 ); +return false; } } + +return true; }
lib/librte_eal/common/include/generic/rte_cpuflags.h +9 -0
@@ -40,6 +40,7 @@*/ #include <errno.h> +#include <stdbool.h> /** * Enumeration of all CPU features supported
@@ -82,4 +83,12 @@ rte_cpu_get_flag_enabled(enum rte_cpu_flag_t feature);void rte_cpu_check_supported (void ); +/** +* This function checks that the currently used CPU supports the CPU features +* that were specified at compile time. It is called automatically within the +* EAL, so does not need to be used by applications. This version returns a +* result so that decisions may be made (for instance, graceful shutdowns). +*/ +bool +rte_cpu_is_supported (void );#endif /* _RTE_CPUFLAGS_H_ */
Signature · 1 line
2.9.3
Re: [PATCH v4 06/26] eal-common: introduce a way to query cpu support
Quoted 50 lines · Aaron Conole
This adds a new API to check for the eal cpu versions.
Signed-off-by Aaron Conole <[email protected]>--- lib/librte_eal/common/eal_common_cpuflags.c | 13 +++++++++++-- lib/librte_eal/common/include/generic/rte_cpuflags.h | 9 +++++++++ 2 files changed, 20 insertions(+), 2 deletions(-)lib/librte_eal/common/eal_common_cpuflags.c +11
-2@@ -43,6 +43,13 @@void rte_cpu_check_supported (void ) { +if (!rte_cpu_is_supported ()) +exit (1 ); +} + +bool +rte_cpu_is_supported (void ) +{/* This is generated at compile-time by the build system */ static const enum rte_cpu_flag_t compile_time_flags[] = {RTE_COMPILE_TIME_CPUFLAGS @@ -57,14 +64,16 @@ rte_cpu_check_supported(void)fprintf (stderr,"ERROR: CPU feature flag lookup failed with error %d\n" , ret); -exit (1 ); +return false; }if (!ret) {fprintf (stderr,"ERROR: This system does not support \"%s\".\n" "Please check that RTE_MACHINE is set correctly.\n" ,rte_cpu_get_flag_name (compile_time_flags[i])); -exit (1 ); +return false; } } + +return true; }lib/librte_eal/common/include/generic/rte_cpuflags.h +0
-0
lib/librte_eal/common/include/generic/rte_cpuflags.h line 44
Quoted 5 lines
lib/librte_eal/common/include/generic/rte_cpuflags.h +1
-0@@ -40,6 +40,7 @@*/ #include <errno.h> +#include <stdbool.h>
The addition of this include is causing all sorts of compilation errors inside the PMDs, as many of them seem to be defining their own bools types. :-(
For safety sake, probably best to have the function return int rather than bool.
/Bruce
Re: [PATCH v4 06/26] eal-common: introduce a way to query cpu support
Quoted 61 lines · Bruce Richardson
Quoted 50 lines · Aaron Conole
This adds a new API to check for the eal cpu versions.
Signed-off-by Aaron Conole <[email protected]>--- lib/librte_eal/common/eal_common_cpuflags.c | 13 +++++++++++-- lib/librte_eal/common/include/generic/rte_cpuflags.h | 9 +++++++++ 2 files changed, 20 insertions(+), 2 deletions(-)lib/librte_eal/common/eal_common_cpuflags.c +11
-2@@ -43,6 +43,13 @@void rte_cpu_check_supported (void ) { +if (!rte_cpu_is_supported ()) +exit (1 ); +} + +bool +rte_cpu_is_supported (void ) +{/* This is generated at compile-time by the build system */ static const enum rte_cpu_flag_t compile_time_flags[] = {RTE_COMPILE_TIME_CPUFLAGS @@ -57,14 +64,16 @@ rte_cpu_check_supported(void)fprintf (stderr,"ERROR: CPU feature flag lookup failed with error %d\n" , ret); -exit (1 ); +return false; }if (!ret) {fprintf (stderr,"ERROR: This system does not support \"%s\".\n" "Please check that RTE_MACHINE is set correctly.\n" ,rte_cpu_get_flag_name (compile_time_flags[i])); -exit (1 ); +return false; } } + +return true; }lib/librte_eal/common/include/generic/rte_cpuflags.h +0
-0lib/librte_eal/common/include/generic/rte_cpuflags.h line 44
Quoted 5 lines
lib/librte_eal/common/include/generic/rte_cpuflags.h +1
-0@@ -40,6 +40,7 @@*/ #include <errno.h> +#include <stdbool.h> The addition of this include is causing all sorts of compilation errors inside the PMDs, as many of them seem to be defining their own bools types. :-(
Quoted 2 lines
For safety sake, probably best to have the function return int rather than bool.
Will do - I never saw the issue, but perhaps I was excluding the PMDs in question.
Thanks for the review, Bruce!
Re: [PATCH v4 06/26] eal-common: introduce a way to query cpu support
Quoted 63 lines · Aaron Conole
Quoted 61 lines · Bruce Richardson
Quoted 50 lines · Aaron Conole
This adds a new API to check for the eal cpu versions.
Signed-off-by Aaron Conole <[email protected]>--- lib/librte_eal/common/eal_common_cpuflags.c | 13 +++++++++++-- lib/librte_eal/common/include/generic/rte_cpuflags.h | 9 +++++++++ 2 files changed, 20 insertions(+), 2 deletions(-)lib/librte_eal/common/eal_common_cpuflags.c +11
-2@@ -43,6 +43,13 @@void rte_cpu_check_supported (void ) { +if (!rte_cpu_is_supported ()) +exit (1 ); +} + +bool +rte_cpu_is_supported (void ) +{/* This is generated at compile-time by the build system */ static const enum rte_cpu_flag_t compile_time_flags[] = {RTE_COMPILE_TIME_CPUFLAGS @@ -57,14 +64,16 @@ rte_cpu_check_supported(void)fprintf (stderr,"ERROR: CPU feature flag lookup failed with error %d\n" , ret); -exit (1 ); +return false; }if (!ret) {fprintf (stderr,"ERROR: This system does not support \"%s\".\n" "Please check that RTE_MACHINE is set correctly.\n" ,rte_cpu_get_flag_name (compile_time_flags[i])); -exit (1 ); +return false; } } + +return true; }lib/librte_eal/common/include/generic/rte_cpuflags.h +0
-0lib/librte_eal/common/include/generic/rte_cpuflags.h line 44
Quoted 5 lines
lib/librte_eal/common/include/generic/rte_cpuflags.h +1
-0@@ -40,6 +40,7 @@*/ #include <errno.h> +#include <stdbool.h> The addition of this include is causing all sorts of compilation errors inside the PMDs, as many of them seem to be defining their own bools types. :-(
Quoted 7 lines
Quoted 2 lines
For safety sake, probably best to have the function return int rather than bool.
Will do - I never saw the issue, but perhaps I was excluding the PMDs in question.
Thanks for the review, Bruce!
No problem.
FYI, the drivers I saw the errors in with this patch are:
* qede
* i40e
* ixgbe
* cxgbe
Regards,
/Bruce
PATCH v4 7/26 eal: Signal error when CPU isn't supported
In reply to Aaron ConoleIt's now possible to gracefully exit the application, or for applications which support non-dpdk datapaths working in concert with DPDK datapaths, there no longer is the possibility of exiting for unsupported CPUs.
--- lib/librte_eal/linuxapp/eal/eal.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -1
@@ -758,7 +758,11 @@ rte_eal_init(int argc, char **argv)char thread_name[RTE_MAX_THREAD_NAME_LEN ];/* checks if the machine is adequate */ -rte_cpu_check_supported (); +if (!rte_cpu_is_supported ()) { +rte_eal_init_alert ("unsupported cpu type." ); + rte_errno =ENOTSUP ; +return -1 ; + }if (!rte_atomic32_test_and_set (&run_once))return -1 ;
Signature · 1 line
2.9.3
PATCH v4 8/26 eal: do not panic on memzone initialization fails
In reply to Aaron ConoleWhen memzone initialization fails, report the error to the calling application rather than panic(). Without a good way of detaching / releasing hugepages, at this point the application will have to restart.
--- lib/librte_eal/linuxapp/eal/eal.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
@@ -838,8 +838,11 @@ rte_eal_init(int argc, char **argv)/* the directories are locked during eal_hugepage_info_init */ eal_hugedirs_unlock (); -if (rte_eal_memzone_init () <0 ) -rte_panic ("Cannot init memzone\n" ); +if (rte_eal_memzone_init () <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot init memzone\n" ); + rte_errno =ENODEV ; +return -1 ; + }if (rte_eal_tailqs_init () <0 )rte_panic ("Cannot init tail queues for objects\n" );
Signature · 1 line
2.9.3
PATCH v4 9/26 eal: set errno when exiting for already called
In reply to Aaron Conole--- lib/librte_eal/linuxapp/eal/eal.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal.c +4 -1
@@ -764,8 +764,11 @@ rte_eal_init(int argc, char **argv)return -1 ; } -if (!rte_atomic32_test_and_set (&run_once)) +if (!rte_atomic32_test_and_set (&run_once)) { +rte_eal_init_alert ("already called initialization." ); + rte_errno =EALREADY ;return -1 ; + } logid =strrchr (argv[0 ],'/' ); logid =strdup (logid ? logid +1 : argv[0 ]);
Signature · 1 line
2.9.3
PATCH v4 10/26 eal: Do not panic on log failures
In reply to Aaron ConoleWhen log initialization fails, it's generally because the fopencookie failed. While this is rare in practice, it could happen, and it is likely because of memory pressure. So, flag the error, and allow the user to retry.
--- lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
@@ -824,8 +824,12 @@ rte_eal_init(int argc, char **argv)rte_config_init (); -if (rte_eal_log_init (logid, internal_config.syslog_facility) <0 ) -rte_panic ("Cannot init logs\n" ); +if (rte_eal_log_init (logid, internal_config.syslog_facility) <0 ) { +rte_eal_init_alert ("Cannot init logging." ); + rte_errno =ENOMEM ; +rte_atomic32_clear (&run_once); +return -1 ; + }if (rte_eal_pci_init () <0 )rte_panic ("Cannot init PCI\n" );
Signature · 1 line
2.9.3
PATCH v4 11/26 eal: Do not panic on pci-probe
In reply to Aaron ConoleThis will usually be an issue because of permissions. However, it could also be caused by OOM. In either case, errno will contain the underlying cause. It is safe to re-init the system here, so allow the application to take corrective action and reinit.
--- lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
@@ -831,8 +831,12 @@ rte_eal_init(int argc, char **argv)return -1 ; } -if (rte_eal_pci_init () <0 ) -rte_panic ("Cannot init PCI\n" ); +if (rte_eal_pci_init () <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot init PCI\n" ); + rte_errno =EUNATCH ; +rte_atomic32_clear (&run_once); +return -1 ; + }#ifdef VFIO_PRESENT if (rte_eal_vfio_setup () <0 )
Signature · 1 line
2.9.3
PATCH v4 12/26 eal: do not panic on vfio failure
In reply to Aaron Conole--- lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
@@ -839,8 +839,12 @@ rte_eal_init(int argc, char **argv) }#ifdef VFIO_PRESENT -if (rte_eal_vfio_setup () <0 ) -rte_panic ("Cannot init VFIO\n" ); +if (rte_eal_vfio_setup () <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot init VFIO\n" ); + rte_errno =EAGAIN ; +rte_atomic32_clear (&run_once); +return -1 ; + }#endif if (rte_eal_memory_init () <0 )
Signature · 1 line
2.9.3
PATCH v4 13/26 eal: do not panic on memory init
In reply to Aaron ConoleThis can only happen when access to hugepages (either as primary or secondary process) fails (and that is usually permissions). Since the manner of failure is not reversible, we cannot allow retry.
--- lib/librte_eal/linuxapp/eal/eal.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
@@ -847,8 +847,11 @@ rte_eal_init(int argc, char **argv) }#endif -if (rte_eal_memory_init () <0 ) -rte_panic ("Cannot init memory\n" ); +if (rte_eal_memory_init () <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot init memory\n" ); + rte_errno =ENOMEM ; +return -1 ; + }/* the directories are locked during eal_hugepage_info_init */ eal_hugedirs_unlock ();
Signature · 1 line
2.9.3
PATCH v4 14/26 eal: do not panic on tailq init
In reply to Aaron ConoleThere are some theoretical racy conditions in the system that _could_ cause early tailq init to fail; however, no need to panic the application. While it can't continue using DPDK, it could make better alerts to the user.
--- lib/librte_eal/common/eal_common_tailqs.c | 3 +-- lib/librte_eal/linuxapp/eal/eal.c | 7 +++++-- 2 files changed, 6 insertions(+), 4 deletions(-)
lib/librte_eal/common/eal_common_tailqs.c +1 -2
@@ -188,8 +188,7 @@ rte_eal_tailqs_init(void)if (t->head ==NULL ) {RTE_LOG (ERR ,EAL ,"Cannot initialize tailq: %s\n" , t->name); -/* no need to TAILQ_REMOVE, we are going to panic in -* rte_eal_init() */ +/* TAILQ_REMOVE not needed, error is already fatal */ goto fail; } }
lib/librte_eal/linuxapp/eal/eal.c +5 -2
@@ -862,8 +862,11 @@ rte_eal_init(int argc, char **argv)return -1 ; } -if (rte_eal_tailqs_init () <0 ) -rte_panic ("Cannot init tail queues for objects\n" ); +if (rte_eal_tailqs_init () <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot init tail queues for objects\n" ); + rte_errno =EFAULT ; +return -1 ; + }if (rte_eal_alarm_init () <0 )rte_panic ("Cannot init interrupt-handling thread\n" );
Signature · 1 line
2.9.3
PATCH v4 15/26 eal: do not panic on alarm init
In reply to Aaron Conolerte_eal_alarm_init() call uses the linux timerfd framework to create a poll()-able timer using standard posix file operations. This could fail for a few reasons given in the man-pages, but many could be corrected by the user application. No need to panic.
--- lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
@@ -61,6 +61,7 @@#include <rte_launch.h> #include <rte_eal.h> #include <rte_eal_memconfig.h> +#include <rte_errno.h> #include <rte_per_lcore.h> #include <rte_lcore.h> #include <rte_log.h>
@@ -868,8 +869,11 @@ rte_eal_init(int argc, char **argv)return -1 ; } -if (rte_eal_alarm_init () <0 ) -rte_panic ("Cannot init interrupt-handling thread\n" ); +if (rte_eal_alarm_init () <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot init interrupt-handling thread\n" ); +/* rte_eal_alarm_init sets rte_errno on failure. */ +return -1 ; + }if (rte_eal_timer_init () <0 )rte_panic ("Cannot init HPET or TSC timers\n" );
Signature · 1 line
2.9.3
PATCH v4 16/26 eal: convert timer_init not to call panic
In reply to Aaron ConoleAfter code inspection, there is no way for eal_timer_init() to fail. It simply returns 0 in all cases. As such, this test could either go-away or stay here as 'future-proofing'.
--- lib/librte_eal/linuxapp/eal/eal.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
@@ -875,8 +875,11 @@ rte_eal_init(int argc, char **argv)return -1 ; } -if (rte_eal_timer_init () <0 ) -rte_panic ("Cannot init HPET or TSC timers\n" ); +if (rte_eal_timer_init () <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot init HPET or TSC timers\n" ); + rte_errno =ENOTSUP ; +return -1 ; + }eal_check_mem_on_local_socket ();
Signature · 1 line
2.9.3
PATCH v4 17/26 eal: change the private pipe call to reflect errno
In reply to Aaron ConoleThere could be some confusion as to why the call failed - this change will always reflect the value of the error in rte_error.
--- lib/librte_eal/linuxapp/eal/eal_interrupts.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal_interrupts.c +4 -1
@@ -898,13 +898,16 @@ rte_eal_intr_init(void)* create a pipe which will be waited by epoll and notified to * rebuild the wait list of epoll. */ -if (pipe (intr_pipe.pipefd) <0 ) +if (pipe (intr_pipe.pipefd) <0 ) { + rte_errno = errno;return -1 ; + }/* create the host thread to wait/handle the interrupt */ ret =pthread_create (&intr_thread,NULL , eal_intr_thread_main,NULL );if (ret !=0 ) { + rte_errno = ret;RTE_LOG (ERR ,EAL ,"Failed to create thread for interrupt handling\n" ); }else {
Signature · 1 line
2.9.3
PATCH v4 18/26 eal: Do not panic on interrupt thread init
In reply to Aaron ConoleWhen initializing the interrupt thread, there are a number of possible reasons for failure - some of which are correctable by the application. Do not panic() needlessly, and give the application a change to reflect this information to the user.
--- lib/librte_eal/linuxapp/eal/eal.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +4 -2
@@ -894,8 +894,10 @@ rte_eal_init(int argc, char **argv) rte_config.master_lcore, (int )thread_id, cpuset, ret == 0 ? "" : "..."); -if (rte_eal_intr_init () <0 ) -rte_panic ("Cannot init interrupt-handling thread\n" ); +if (rte_eal_intr_init () <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot init interrupt-handling thread\n" ); +return -1 ; + }if (rte_bus_scan ())rte_panic ("Cannot scan the buses for devices\n" );
Signature · 1 line
2.9.3
PATCH v4 19/26 eal: do not error if plugins fail to init
In reply to Aaron ConolePlugins are useful and important. However, it seems crazy to abort everything just because they don't initialize properly.
--- lib/librte_eal/linuxapp/eal/eal.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal.c +1 -1
@@ -884,7 +884,7 @@ rte_eal_init(int argc, char **argv)eal_check_mem_on_local_socket ();if (eal_plugins_init () <0 ) -rte_panic ("Cannot init plugins\n" ); +RTE_LOG (ERR ,EAL ,"Cannot init plugins\n" );eal_thread_init_master (rte_config.master_lcore);
Signature · 1 line
2.9.3
PATCH v4 20/26 eal_pci: Continue probing even on failures
In reply to Aaron ConoleSome devices may be inaccessible for a variety of reasons, or the PCI-bus may be unavailable causing the whole thing to fail. Still, better to continue attempts at probes.
--- lib/librte_eal/common/eal_common_pci.c | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-)
lib/librte_eal/common/eal_common_pci.c +10 -5
@@ -69,6 +69,7 @@#include <sys/queue.h> #include <sys/mman.h> +#include <rte_errno.h> #include <rte_interrupts.h> #include <rte_log.h> #include <rte_pci.h>
@@ -416,6 +417,7 @@ rte_eal_pci_probe(void)struct rte_pci_device *dev =NULL ;struct rte_devargs *devargs;int probe_all =0 ; +int ret_1 =0 ;int ret =0 ;if (rte_eal_devargs_type_count (RTE_DEVTYPE_WHITELISTED_PCI ) ==0 )
@@ -430,17 +432,20 @@ rte_eal_pci_probe(void)/* probe all or only whitelisted devices */ if (probe_all) - ret =pci_probe_all_drivers (dev); + ret_1 =pci_probe_all_drivers (dev);else if (devargs !=NULL && devargs->type ==RTE_DEVTYPE_WHITELISTED_PCI ) - ret =pci_probe_all_drivers (dev); -if (ret <0 ) -rte_exit (EXIT_FAILURE ,"Requested device " PCI_PRI_FMT + ret_1 =pci_probe_all_drivers (dev); +if (ret_1 <0 ) { +RTE_LOG (ERR ,EAL ,"Requested device " PCI_PRI_FMT " cannot be used\n" , dev->addr.domain, dev->addr.bus, dev->addr.devid, dev->addr.function); + rte_errno = errno; + ret =1 ; + } } -return 0 ; +return -ret; }/* dump one device */
Signature · 1 line
2.9.3
PATCH v4 21/26 eal: do not panic on failed PCI probe
In reply to Aaron ConoleIt may even be possible to simply log the error and continue on letting the user check the logs and restart the application when things are failed.
--- lib/librte_eal/linuxapp/eal/eal.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
@@ -943,8 +943,11 @@ rte_eal_init(int argc, char **argv)rte_panic ("Cannot probe devices\n" );/* Probe & Initialize PCI devices */ -if (rte_eal_pci_probe ()) -rte_panic ("Cannot probe PCI\n" ); +if (rte_eal_pci_probe ()) { +RTE_LOG (ERR ,EAL ,"Cannot probe PCI\n" ); + rte_errno =ENOTSUP ; +return -1 ; + }if (rte_eal_dev_init () <0 )rte_panic ("Cannot init pmd devices\n" );
Signature · 1 line
2.9.3
PATCH v4 22/26 eal_common_dev: continue initializing vdevs
In reply to Aaron ConoleEven if one vdev should fail, there's no need to prevent further processing. Log the error, and reflect it to the higher levels to decide.
--- lib/librte_eal/common/eal_common_dev.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
lib/librte_eal/common/eal_common_dev.c +3 -2
@@ -80,6 +80,7 @@ intrte_eal_dev_init (void) {struct rte_devargs *devargs; +int ret =0 ;/* * Note that the dev_driver_list is populated here
@@ -97,11 +98,11 @@ rte_eal_dev_init(void) devargs->args)) {RTE_LOG (ERR ,EAL ,"failed to initialize %s device\n" , devargs->virt.drv_name); -return -1 ; + ret =-1 ; } } -return 0 ; +return ret; }int rte_eal_dev_attach (const char *name,const char *devargs)
Signature · 1 line
2.9.3
PATCH v4 23/26 eal: do not panic (or abort) if vdev init fails
In reply to Aaron ConoleSeems like it's possible to continue. At least, the error is reflected properly in the logs. A user could then go and correct or investigate the situation.
--- lib/librte_eal/linuxapp/eal/eal.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal.c +1 -1
@@ -950,7 +950,7 @@ rte_eal_init(int argc, char **argv) }if (rte_eal_dev_init () <0 ) -rte_panic ("Cannot init pmd devices\n" ); +RTE_LOG (ERR ,EAL ,"Cannot init pmd devices\n" );rte_eal_mcfg_complete ();
Signature · 1 line
2.9.3
PATCH v4 24/26 eal: do not panic when bus probe fails
In reply to Aaron Conole--- lib/librte_eal/linuxapp/eal/eal.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
@@ -939,8 +939,11 @@ rte_eal_init(int argc, char **argv)rte_eal_mp_wait_lcore ();/* Probe all the buses and devices/drivers on them */ -if (rte_bus_probe ()) -rte_panic ("Cannot probe devices\n" ); +if (rte_bus_probe ()) { +RTE_LOG (ERR ,EAL ,"Cannot probe devices\n" ); + rte_errno =ENOTSUP ; +return -1 ; + }/* Probe & Initialize PCI devices */ if (rte_eal_pci_probe ()) {
Signature · 1 line
2.9.3
PATCH v4 25/26 eal: do not panic on failed bus scan
In reply to Aaron ConoleFor now, do an abort. It's likely that even aborting the initialization is premature in this case, as it may be possible to proceed even if one bus or another is not available.
--- lib/librte_eal/linuxapp/eal/eal.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
@@ -899,8 +899,11 @@ rte_eal_init(int argc, char **argv)return -1 ; } -if (rte_bus_scan ()) -rte_panic ("Cannot scan the buses for devices\n" ); +if (rte_bus_scan ()) { +RTE_LOG (ERR ,EAL ,"Cannot scan the buses for devices\n" ); + rte_errno =ENODEV ; +return -1 ; + }RTE_LCORE_FOREACH_SLAVE (i) {
Signature · 1 line
2.9.3
PATCH v4 26/26 rte_eal_init: add info about rte_errno codes
In reply to Aaron ConoleThe rte_eal_init function will now pass failure reason hints to the application. To help app developers deciper this, add some brief information about what the codes are indicating.
--- lib/librte_eal/common/include/rte_eal.h | 27 ++++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-)
lib/librte_eal/common/include/rte_eal.h +26 -1
@@ -159,7 +159,32 @@ int rte_eal_iopl_init(void);* function call and should not be further interpreted by the * application. The EAL does not take any ownership of the memory used * for either the argv array, or its members. -* - On failure, a negative error value. +* - On failure, -1 and rte_errno is set to a value indicating the cause +* for failure. In some instances, the application will need to be +* restarted as part of clearing the issue. +* +* Error codes returned via rte_errno: +* EACCES indicates a permissions issue. +* +* EAGAIN indicates either a bus or system resource was not available, +* setup may be attempted again. +* +* EALREADY indicates that the rte_eal_init function has already been +* called, and cannot be called again. +* +* EFAULT indicates the tailq configuration name was not found in +* memory configuration. +* +* EINVAL indicates invalid parameters were passed as argv/argc. +* +* ENOMEM indicates failure likely caused by an out-of-memory condition. +* +* ENODEV indicates memory setup issues. +* +* ENOTSUP indicates that the EAL cannot initialize on this system. +* +* EUNATCH indicates that the PCI bus is either not present, or is not +* readable by the eal. */ int rte_eal_init (int argc,char **argv);
Signature · 1 line
2.9.3
Quoted 6 lines · Aaron Conole
In many cases, it's enough to simply let the application know that the call to initialize DPDK has failed. A complete halt can then be decided by the application based on error returned (and the app could even attempt a possible re-attempt after some corrective action by the user or application).
Spotted a few issues when I tried applying each of the patches and compile testing them; those I've flagged on the patches that I test applied. Otherwise, this looks a great set to have.
I assume the equivalent changes to the BSD EAL are "left as an exercise for the reader"? :-)
/Bruce
Quoted 7 lines · Bruce Richardson
Quoted 6 lines · Aaron Conole
In many cases, it's enough to simply let the application know that the call to initialize DPDK has failed. A complete halt can then be decided by the application based on error returned (and the app could even attempt a possible re-attempt after some corrective action by the user or application).
Quoted 6 lines
Spotted a few issues when I tried applying each of the patches and compile testing them; those I've flagged on the patches that I test applied. Otherwise, this looks a great set to have.
I assume the equivalent changes to the BSD EAL are "left as an exercise for the reader"? :-)
For now. In my copious free time, I might have a crack at it, although I don't have a working BSD DPDK setup.
Quoted 1 line
/Bruce
v5 cover linux/eal: Remove most causes of panic on init
In reply to Aaron ConoleIn many cases, it's enough to simply let the application know that the call to initialize DPDK has failed. A complete halt can then be decided by the application based on error returned (and the app could even attempt a possible re-attempt after some corrective action by the user or application).
Changes ->v2:
- Audited all "RTE_LOG (" calls that were introduced, and converted to "RTE_LOG("
- Added some fprintf(stderr, "") lines to indicate errors before logging is initialized
- Removed assignments to errno.
- Changed patch 14/25 to reflect EFAULT, and document in 25/25
Changes ->v3:
- Checkpatch issues in patches 3 (spelling mistake), 9 (issue with leading spaces), and 19 (braces around single line statement if-condition)
Changes ->v4:
- Error text cleanup.
- Add a new check around rte_bus_scan(), added during the development of this series.
Changes ->v5:
- checkpatch.pl cleanup in patch 02/26
- move rte_errno.h include from patch 15 to patch 02
- remove stdbool.h and use int as return type in patch 06/26
I kept the rte_errno reflection, since this is control-path code and the init function returns a sentinel value of -1.
Aaron Conole (26):
eal: CPU init will no longer panic
eal: return error instead of panic for cpu init
eal: No panic on hugepages info init
eal: do not panic on failed hugepage query
eal: failure to parse args returns error
eal-common: introduce a way to query cpu support
eal: Signal error when CPU isn't supported
eal: do not panic on memzone initialization fails
eal: set errno when exiting for already called
eal: Do not panic on log failures
eal: Do not panic on pci-probe
eal: do not panic on vfio failure
eal: do not panic on memory init
eal: do not panic on tailq init
eal: do not panic on alarm init
eal: convert timer_init not to call panic
eal: change the private pipe call to reflect errno
eal: Do not panic on interrupt thread init
eal: do not error if plugins fail to init
eal_pci: Continue probing even on failures
eal: do not panic on failed PCI probe
eal_common_dev: continue initializing vdevs
eal: do not panic (or abort) if vdev init fails
eal: do not panic when bus probe fails
eal: do not panic on failed bus scan
rte_eal_init: add info about rte_errno codes
lib/librte_eal/common/eal_common_cpuflags.c | 13 +- lib/librte_eal/common/eal_common_dev.c | 5 +- lib/librte_eal/common/eal_common_lcore.c | 7 +- lib/librte_eal/common/eal_common_pci.c | 15 ++- lib/librte_eal/common/eal_common_tailqs.c | 3 +- .../common/include/generic/rte_cpuflags.h | 8 ++ lib/librte_eal/common/include/rte_eal.h | 27 ++++- lib/librte_eal/linuxapp/eal/eal.c | 131 +++++++++++++++------ lib/librte_eal/linuxapp/eal/eal_hugepage_info.c | 6 +- lib/librte_eal/linuxapp/eal/eal_interrupts.c | 5 +- 10 files changed, 168 insertions(+), 52 deletions(-)
Signature · 1 line
2.9.3
PATCH v5 1/26 eal: CPU init will no longer panic
After this change, the EAL CPU NUMA node resolution step can no longer emit an rte_panic. This aligns with the code in rte_eal_init, which expects failures to return an error code.
--- lib/librte_eal/common/eal_common_lcore.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
lib/librte_eal/common/eal_common_lcore.c +4 -3
@@ -83,16 +83,17 @@ rte_eal_cpu_init(void) config->lcore_role[lcore_id] =ROLE_RTE ; lcore_config[lcore_id].core_id =eal_cpu_core_id (lcore_id); lcore_config[lcore_id].socket_id =eal_cpu_socket_id (lcore_id); - if (lcore_config[lcore_id].socket_id >= RTE_MAX_NUMA_NODES) +if (lcore_config[lcore_id].socket_id >=RTE_MAX_NUMA_NODES ) {#ifdef RTE_EAL_ALLOW_INV_SOCKET_ID lcore_config[lcore_id].socket_id =0 ;#else -rte_panic ("Socket ID (%u) is greater than " +RTE_LOG (ERR ,EAL ,"Socket ID (%u) is greater than " "RTE_MAX_NUMA_NODES (%d)\n" , lcore_config[lcore_id].socket_id,RTE_MAX_NUMA_NODES ); +return -1 ;#endif - + } RTE_LOG(DEBUG, EAL, "Detected lcore %u as " "core %u on socket %u\n", lcore_id, lcore_config[lcore_id].core_id,
Signature · 1 line
2.9.3
PATCH v5 2/26 eal: return error instead of panic for cpu init
In reply to Aaron ConoleThere may be no way to gracefully recover, but the application should be notified that a failure happened, rather than completely aborting. This allows the user to proceed with a "slow-path" type solution.
--- lib/librte_eal/linuxapp/eal/eal.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +12 -2
@@ -61,6 +61,7 @@#include <rte_launch.h> #include <rte_eal.h> #include <rte_eal_memconfig.h> +#include <rte_errno.h> #include <rte_per_lcore.h> #include <rte_lcore.h> #include <rte_log.h>
@@ -740,6 +741,12 @@ static int rte_eal_vfio_setup(void) }#endif +static void rte_eal_init_alert (const char *msg) +{ +fprintf (stderr,"EAL: FATAL: %s\n" , msg); +RTE_LOG (ERR ,EAL ,"%s\n" , msg); +} +/* Launch threads, called at application init(). */ int rte_eal_init (int argc,char **argv)
@@ -767,8 +774,11 @@ rte_eal_init(int argc, char **argv)/* set log level as early as possible */ rte_set_log_level (internal_config.log_level); -if (rte_eal_cpu_init () <0 ) -rte_panic ("Cannot detect lcores\n" ); +if (rte_eal_cpu_init () <0 ) { +rte_eal_init_alert ("Cannot detect lcores." ); + rte_errno =ENOTSUP ; +return -1 ; + } fctret =eal_parse_args (argc, argv);if (fctret <0 )
Signature · 1 line
2.9.3
PATCH v5 3/26 eal: No panic on hugepages info init
In reply to Aaron ConoleWhen attempting to scan hugepages, signal to the eal.c that an error has occurred, rather than performing a panic.
--- lib/librte_eal/linuxapp/eal/eal_hugepage_info.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal_hugepage_info.c +4 -2
@@ -283,9 +283,11 @@ eal_hugepage_info_init(void)struct dirent *dirent; dir =opendir (sys_dir_path); -if (dir ==NULL ) -rte_panic ("Cannot open directory %s to read system hugepage " +if (dir ==NULL ) { +RTE_LOG (ERR ,EAL ,"Cannot open directory %s to read system hugepage " "info\n" , sys_dir_path); +return -1 ; + }for (dirent =readdir (dir); dirent !=NULL ; dirent =readdir (dir)) {struct hugepage_info *hpi;
Signature · 1 line
2.9.3
Re: [PATCH v5 03/26] eal: No panic on hugepages info init
Quoted 13 lines · Aaron Conole
When attempting to scan hugepages, signal to the
eal.cthat an error has occurred, rather than performing a panic.Signed-off-by Aaron Conole <[email protected]>--- lib/librte_eal/linuxapp/eal/eal_hugepage_info.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)lib/librte_eal/linuxapp/eal/eal_hugepage_info.c +0
-0
lib/librte_eal/linuxapp/eal/eal_hugepage_info.c eal_hugepage_info_init(void) · line 290
Quoted 10 lines
lib/librte_eal/linuxapp/eal/eal_hugepage_info.c +4
-2@@ -283,9 +283,11 @@ eal_hugepage_info_init(void)struct dirent *dirent; dir =opendir (sys_dir_path); -if (dir ==NULL ) -rte_panic ("Cannot open directory %s to read system hugepage " +if (dir ==NULL ) { +RTE_LOG (ERR ,EAL ,"Cannot open directory %s to read system hugepage " "info\n" , sys_dir_path); +return -1 ; + }
Minor nit.
The error message should go on a line on its own, without any breaks to make it easy to "grep". This should also eliminate the checkpatch complaint about it being too long.
/Bruce
Re: [PATCH v5 03/26] eal: No panic on hugepages info init
Quoted 16 lines · Bruce Richardson
Quoted 15 lines · Aaron Conole
When attempting to scan hugepages, signal to the
eal.cthat an error has occurred, rather than performing a panic.Signed-off-by Aaron Conole <[email protected]>--- lib/librte_eal/linuxapp/eal/eal_hugepage_info.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)lib/librte_eal/linuxapp/eal/eal_hugepage_info.c +0
-0@@ -283,9 +283,11 @@ eal_hugepage_info_init(void)struct dirent *dirent;
Quoted 13 lines
lib/librte_eal/linuxapp/eal/eal_hugepage_info.c eal_hugepage_info_init(void) · line 290
Quoted 8 lines
lib/librte_eal/linuxapp/eal/eal_hugepage_info.c +4
-2dir =opendir (sys_dir_path); -if (dir ==NULL ) -rte_panic ("Cannot open directory %s to read system hugepage " +if (dir ==NULL ) { +RTE_LOG (ERR ,EAL ,"Cannot open directory %s to read system hugepage " "info\n" , sys_dir_path); +return -1 ; + }Minor nit.
The error message should go on a line on its own, without any breaks to make it easy to "grep". This should also eliminate the checkpatch complaint about it being too long.
Yes, will fix it for v6.
-Aaron
PATCH v5 4/26 eal: do not panic on failed hugepage query
In reply to Aaron ConoleIf we fail to acquire hugepage information, simply signal an error to the application. This clears the run_once counter, allowing the user or application to take a corrective action and retry.
--- lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
@@ -787,8 +787,12 @@ rte_eal_init(int argc, char **argv)if (internal_config.no_hugetlbfs ==0 && internal_config.process_type !=RTE_PROC_SECONDARY && internal_config.xen_dom0_support ==0 && -eal_hugepage_info_init () <0 ) -rte_panic ("Cannot get hugepage information\n" ); +eal_hugepage_info_init () <0 ) { +rte_eal_init_alert ("Cannot get hugepage information." ); + rte_errno =EACCES ; +rte_atomic32_clear (&run_once); +return -1 ; + }if (internal_config.memory ==0 && internal_config.force_sockets ==0 ) {if (internal_config.no_hugetlbfs)
Signature · 1 line
2.9.3
PATCH v5 5/26 eal: failure to parse args returns error
In reply to Aaron ConoleIt's possible that the application could take a corrective action here, and either prompt the user for different arguments, or at least perform a better logging. Exiting this early prevents any useful information gathering from the application layer.
--- lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
@@ -781,8 +781,12 @@ rte_eal_init(int argc, char **argv) } fctret =eal_parse_args (argc, argv); -if (fctret <0 ) -exit (1 ); +if (fctret <0 ) { +rte_eal_init_alert ("Invalid 'command line' arguments." ); + rte_errno =EINVAL ; +rte_atomic32_clear (&run_once); +return -1 ; + } if (internal_config.no_hugetlbfs == 0 && internal_config.process_type != RTE_PROC_SECONDARY &&
Signature · 1 line
2.9.3
PATCH v5 6/26 eal-common: introduce a way to query cpu support
In reply to Aaron ConoleThis adds a new API to check for the eal cpu versions.
--- lib/librte_eal/common/eal_common_cpuflags.c | 13 +++++++++++-- lib/librte_eal/common/include/generic/rte_cpuflags.h | 8 ++++++++ 2 files changed, 19 insertions(+), 2 deletions(-)
lib/librte_eal/common/eal_common_cpuflags.c +11 -2
@@ -43,6 +43,13 @@void rte_cpu_check_supported (void ) { +if (!rte_cpu_is_supported ()) +exit (1 ); +} + +int +rte_cpu_is_supported (void ) +{/* This is generated at compile-time by the build system */ static const enum rte_cpu_flag_t compile_time_flags[] = {RTE_COMPILE_TIME_CPUFLAGS
@@ -57,14 +64,16 @@ rte_cpu_check_supported(void)fprintf (stderr,"ERROR: CPU feature flag lookup failed with error %d\n" , ret); -exit (1 ); +return 0 ; }if (!ret) {fprintf (stderr,"ERROR: This system does not support \"%s\".\n" "Please check that RTE_MACHINE is set correctly.\n" ,rte_cpu_get_flag_name (compile_time_flags[i])); -exit (1 ); +return 0 ; } } + +return 1 ; }
lib/librte_eal/common/include/generic/rte_cpuflags.h +8 -0
@@ -82,4 +82,12 @@ rte_cpu_get_flag_enabled(enum rte_cpu_flag_t feature);void rte_cpu_check_supported (void ); +/** +* This function checks that the currently used CPU supports the CPU features +* that were specified at compile time. It is called automatically within the +* EAL, so does not need to be used by applications. This version returns a +* result so that decisions may be made (for instance, graceful shutdowns). +*/ +int +rte_cpu_is_supported (void );#endif /* _RTE_CPUFLAGS_H_ */
Signature · 1 line
2.9.3
PATCH v5 7/26 eal: Signal error when CPU isn't supported
In reply to Aaron ConoleIt's now possible to gracefully exit the application, or for applications which support non-dpdk datapaths working in concert with DPDK datapaths, there no longer is the possibility of exiting for unsupported CPUs.
--- lib/librte_eal/linuxapp/eal/eal.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -1
@@ -759,7 +759,11 @@ rte_eal_init(int argc, char **argv)char thread_name[RTE_MAX_THREAD_NAME_LEN ];/* checks if the machine is adequate */ -rte_cpu_check_supported (); +if (!rte_cpu_is_supported ()) { +rte_eal_init_alert ("unsupported cpu type." ); + rte_errno =ENOTSUP ; +return -1 ; + }if (!rte_atomic32_test_and_set (&run_once))return -1 ;
Signature · 1 line
2.9.3
PATCH v5 8/26 eal: do not panic on memzone initialization fails
In reply to Aaron ConoleWhen memzone initialization fails, report the error to the calling application rather than panic(). Without a good way of detaching / releasing hugepages, at this point the application will have to restart.
--- lib/librte_eal/linuxapp/eal/eal.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
@@ -839,8 +839,11 @@ rte_eal_init(int argc, char **argv)/* the directories are locked during eal_hugepage_info_init */ eal_hugedirs_unlock (); -if (rte_eal_memzone_init () <0 ) -rte_panic ("Cannot init memzone\n" ); +if (rte_eal_memzone_init () <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot init memzone\n" ); + rte_errno =ENODEV ; +return -1 ; + }if (rte_eal_tailqs_init () <0 )rte_panic ("Cannot init tail queues for objects\n" );
Signature · 1 line
2.9.3
Re: [PATCH v5 08/26] eal: do not panic on memzone initialization fails
Quoted 14 lines · Aaron Conole
When memzone initialization fails, report the error to the calling application rather than
panic(). Without a good way of detaching / releasing hugepages, at this point the application will have to restart.Signed-off-by Aaron Conole <[email protected]>--- lib/librte_eal/linuxapp/eal/eal.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)lib/librte_eal/linuxapp/eal/eal.c +0
-0
lib/librte_eal/linuxapp/eal/eal.c rte_eal_init(int argc, char **argv) · line 843
Quoted 7 lines
lib/librte_eal/linuxapp/eal/eal.c +2
-2@@ -839,8 +839,11 @@ rte_eal_init(int argc, char **argv)/* the directories are locked during eal_hugepage_info_init */ eal_hugedirs_unlock (); -if (rte_eal_memzone_init () <0 ) -rte_panic ("Cannot init memzone\n" ); +if (rte_eal_memzone_init () <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot init memzone\n" );
Any particular reason why not "rte_eal_init_alert" as with the other cases?
/Bruce
Re: [PATCH v5 08/26] eal: do not panic on memzone initialization fails
Quoted 18 lines · Bruce Richardson
Quoted 17 lines · Aaron Conole
When memzone initialization fails, report the error to the calling application rather than
panic(). Without a good way of detaching / releasing hugepages, at this point the application will have to restart.Signed-off-by Aaron Conole <[email protected]>--- lib/librte_eal/linuxapp/eal/eal.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)lib/librte_eal/linuxapp/eal/eal.c +0
-0@@ -839,8 +839,11 @@ rte_eal_init(int argc, char **argv)/* the directories are locked during eal_hugepage_info_init */ eal_hugedirs_unlock ();
Quoted 7 lines
lib/librte_eal/linuxapp/eal/eal.c rte_eal_init(int argc, char **argv) · line 843
Quoted 4 lines
lib/librte_eal/linuxapp/eal/eal.c +2
-2-if (rte_eal_memzone_init () <0 ) -rte_panic ("Cannot init memzone\n" ); +if (rte_eal_memzone_init () <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot init memzone\n" );Any particular reason why not "
rte_eal_init_alert" as with the other cases?
I only used rte_eal_init_alert() for cases which occur before logging happens, but I am not opposed to switching it for all the cases. I'll swap them all for v6.
Thanks again for the review, Bruce!
-Aaron
PATCH v5 9/26 eal: set errno when exiting for already called
In reply to Aaron Conole--- lib/librte_eal/linuxapp/eal/eal.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal.c +4 -1
@@ -765,8 +765,11 @@ rte_eal_init(int argc, char **argv)return -1 ; } -if (!rte_atomic32_test_and_set (&run_once)) +if (!rte_atomic32_test_and_set (&run_once)) { +rte_eal_init_alert ("already called initialization." ); + rte_errno =EALREADY ;return -1 ; + } logid =strrchr (argv[0 ],'/' ); logid =strdup (logid ? logid +1 : argv[0 ]);
Signature · 1 line
2.9.3
PATCH v5 10/26 eal: Do not panic on log failures
In reply to Aaron ConoleWhen log initialization fails, it's generally because the fopencookie failed. While this is rare in practice, it could happen, and it is likely because of memory pressure. So, flag the error, and allow the user to retry.
--- lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
@@ -825,8 +825,12 @@ rte_eal_init(int argc, char **argv)rte_config_init (); -if (rte_eal_log_init (logid, internal_config.syslog_facility) <0 ) -rte_panic ("Cannot init logs\n" ); +if (rte_eal_log_init (logid, internal_config.syslog_facility) <0 ) { +rte_eal_init_alert ("Cannot init logging." ); + rte_errno =ENOMEM ; +rte_atomic32_clear (&run_once); +return -1 ; + }if (rte_eal_pci_init () <0 )rte_panic ("Cannot init PCI\n" );
Signature · 1 line
2.9.3
PATCH v5 11/26 eal: Do not panic on pci-probe
In reply to Aaron ConoleThis will usually be an issue because of permissions. However, it could also be caused by OOM. In either case, errno will contain the underlying cause. It is safe to re-init the system here, so allow the application to take corrective action and reinit.
--- lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
@@ -832,8 +832,12 @@ rte_eal_init(int argc, char **argv)return -1 ; } -if (rte_eal_pci_init () <0 ) -rte_panic ("Cannot init PCI\n" ); +if (rte_eal_pci_init () <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot init PCI\n" ); + rte_errno =EUNATCH ; +rte_atomic32_clear (&run_once); +return -1 ; + }#ifdef VFIO_PRESENT if (rte_eal_vfio_setup () <0 )
Signature · 1 line
2.9.3
PATCH v5 12/26 eal: do not panic on vfio failure
In reply to Aaron Conole--- lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
@@ -840,8 +840,12 @@ rte_eal_init(int argc, char **argv) }#ifdef VFIO_PRESENT -if (rte_eal_vfio_setup () <0 ) -rte_panic ("Cannot init VFIO\n" ); +if (rte_eal_vfio_setup () <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot init VFIO\n" ); + rte_errno =EAGAIN ; +rte_atomic32_clear (&run_once); +return -1 ; + }#endif if (rte_eal_memory_init () <0 )
Signature · 1 line
2.9.3
PATCH v5 13/26 eal: do not panic on memory init
In reply to Aaron ConoleThis can only happen when access to hugepages (either as primary or secondary process) fails (and that is usually permissions). Since the manner of failure is not reversible, we cannot allow retry.
--- lib/librte_eal/linuxapp/eal/eal.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
@@ -848,8 +848,11 @@ rte_eal_init(int argc, char **argv) }#endif -if (rte_eal_memory_init () <0 ) -rte_panic ("Cannot init memory\n" ); +if (rte_eal_memory_init () <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot init memory\n" ); + rte_errno =ENOMEM ; +return -1 ; + }/* the directories are locked during eal_hugepage_info_init */ eal_hugedirs_unlock ();
Signature · 1 line
2.9.3
PATCH v5 14/26 eal: do not panic on tailq init
In reply to Aaron ConoleThere are some theoretical racy conditions in the system that _could_ cause early tailq init to fail; however, no need to panic the application. While it can't continue using DPDK, it could make better alerts to the user.
--- lib/librte_eal/common/eal_common_tailqs.c | 3 +-- lib/librte_eal/linuxapp/eal/eal.c | 7 +++++-- 2 files changed, 6 insertions(+), 4 deletions(-)
lib/librte_eal/common/eal_common_tailqs.c +1 -2
@@ -188,8 +188,7 @@ rte_eal_tailqs_init(void)if (t->head ==NULL ) {RTE_LOG (ERR ,EAL ,"Cannot initialize tailq: %s\n" , t->name); -/* no need to TAILQ_REMOVE, we are going to panic in -* rte_eal_init() */ +/* TAILQ_REMOVE not needed, error is already fatal */ goto fail; } }
lib/librte_eal/linuxapp/eal/eal.c +5 -2
@@ -863,8 +863,11 @@ rte_eal_init(int argc, char **argv)return -1 ; } -if (rte_eal_tailqs_init () <0 ) -rte_panic ("Cannot init tail queues for objects\n" ); +if (rte_eal_tailqs_init () <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot init tail queues for objects\n" ); + rte_errno =EFAULT ; +return -1 ; + }if (rte_eal_alarm_init () <0 )rte_panic ("Cannot init interrupt-handling thread\n" );
Signature · 1 line
2.9.3
PATCH v5 15/26 eal: do not panic on alarm init
In reply to Aaron Conolerte_eal_alarm_init() call uses the linux timerfd framework to create a poll()-able timer using standard posix file operations. This could fail for a few reasons given in the man-pages, but many could be corrected by the user application. No need to panic.
--- lib/librte_eal/linuxapp/eal/eal.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
@@ -869,8 +869,11 @@ rte_eal_init(int argc, char **argv)return -1 ; } -if (rte_eal_alarm_init () <0 ) -rte_panic ("Cannot init interrupt-handling thread\n" ); +if (rte_eal_alarm_init () <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot init interrupt-handling thread\n" ); +/* rte_eal_alarm_init sets rte_errno on failure. */ +return -1 ; + }if (rte_eal_timer_init () <0 )rte_panic ("Cannot init HPET or TSC timers\n" );
Signature · 1 line
2.9.3
PATCH v5 16/26 eal: convert timer_init not to call panic
In reply to Aaron ConoleAfter code inspection, there is no way for eal_timer_init() to fail. It simply returns 0 in all cases. As such, this test could either go-away or stay here as 'future-proofing'.
--- lib/librte_eal/linuxapp/eal/eal.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
@@ -875,8 +875,11 @@ rte_eal_init(int argc, char **argv)return -1 ; } -if (rte_eal_timer_init () <0 ) -rte_panic ("Cannot init HPET or TSC timers\n" ); +if (rte_eal_timer_init () <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot init HPET or TSC timers\n" ); + rte_errno =ENOTSUP ; +return -1 ; + }eal_check_mem_on_local_socket ();
Signature · 1 line
2.9.3
PATCH v5 17/26 eal: change the private pipe call to reflect errno
In reply to Aaron ConoleThere could be some confusion as to why the call failed - this change will always reflect the value of the error in rte_error.
--- lib/librte_eal/linuxapp/eal/eal_interrupts.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal_interrupts.c +4 -1
@@ -898,13 +898,16 @@ rte_eal_intr_init(void)* create a pipe which will be waited by epoll and notified to * rebuild the wait list of epoll. */ -if (pipe (intr_pipe.pipefd) <0 ) +if (pipe (intr_pipe.pipefd) <0 ) { + rte_errno = errno;return -1 ; + }/* create the host thread to wait/handle the interrupt */ ret =pthread_create (&intr_thread,NULL , eal_intr_thread_main,NULL );if (ret !=0 ) { + rte_errno = ret;RTE_LOG (ERR ,EAL ,"Failed to create thread for interrupt handling\n" ); }else {
Signature · 1 line
2.9.3
PATCH v5 18/26 eal: Do not panic on interrupt thread init
In reply to Aaron ConoleWhen initializing the interrupt thread, there are a number of possible reasons for failure - some of which are correctable by the application. Do not panic() needlessly, and give the application a change to reflect this information to the user.
--- lib/librte_eal/linuxapp/eal/eal.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +4 -2
@@ -894,8 +894,10 @@ rte_eal_init(int argc, char **argv) rte_config.master_lcore, (int )thread_id, cpuset, ret == 0 ? "" : "..."); -if (rte_eal_intr_init () <0 ) -rte_panic ("Cannot init interrupt-handling thread\n" ); +if (rte_eal_intr_init () <0 ) { +RTE_LOG (ERR ,EAL ,"Cannot init interrupt-handling thread\n" ); +return -1 ; + }if (rte_bus_scan ())rte_panic ("Cannot scan the buses for devices\n" );
Signature · 1 line
2.9.3
PATCH v5 19/26 eal: do not error if plugins fail to init
In reply to Aaron ConolePlugins are useful and important. However, it seems crazy to abort everything just because they don't initialize properly.
--- lib/librte_eal/linuxapp/eal/eal.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal.c +1 -1
@@ -884,7 +884,7 @@ rte_eal_init(int argc, char **argv)eal_check_mem_on_local_socket ();if (eal_plugins_init () <0 ) -rte_panic ("Cannot init plugins\n" ); +RTE_LOG (ERR ,EAL ,"Cannot init plugins\n" );eal_thread_init_master (rte_config.master_lcore);
Signature · 1 line
2.9.3
PATCH v5 20/26 eal_pci: Continue probing even on failures
In reply to Aaron ConoleSome devices may be inaccessible for a variety of reasons, or the PCI-bus may be unavailable causing the whole thing to fail. Still, better to continue attempts at probes.
--- lib/librte_eal/common/eal_common_pci.c | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-)
lib/librte_eal/common/eal_common_pci.c +10 -5
@@ -69,6 +69,7 @@#include <sys/queue.h> #include <sys/mman.h> +#include <rte_errno.h> #include <rte_interrupts.h> #include <rte_log.h> #include <rte_pci.h>
@@ -416,6 +417,7 @@ rte_eal_pci_probe(void)struct rte_pci_device *dev =NULL ;struct rte_devargs *devargs;int probe_all =0 ; +int ret_1 =0 ;int ret =0 ;if (rte_eal_devargs_type_count (RTE_DEVTYPE_WHITELISTED_PCI ) ==0 )
@@ -430,17 +432,20 @@ rte_eal_pci_probe(void)/* probe all or only whitelisted devices */ if (probe_all) - ret =pci_probe_all_drivers (dev); + ret_1 =pci_probe_all_drivers (dev);else if (devargs !=NULL && devargs->type ==RTE_DEVTYPE_WHITELISTED_PCI ) - ret =pci_probe_all_drivers (dev); -if (ret <0 ) -rte_exit (EXIT_FAILURE ,"Requested device " PCI_PRI_FMT + ret_1 =pci_probe_all_drivers (dev); +if (ret_1 <0 ) { +RTE_LOG (ERR ,EAL ,"Requested device " PCI_PRI_FMT " cannot be used\n" , dev->addr.domain, dev->addr.bus, dev->addr.devid, dev->addr.function); + rte_errno = errno; + ret =1 ; + } } -return 0 ; +return -ret; }/* dump one device */
Signature · 1 line
2.9.3
PATCH v5 21/26 eal: do not panic on failed PCI probe
In reply to Aaron ConoleIt may even be possible to simply log the error and continue on letting the user check the logs and restart the application when things are failed.
--- lib/librte_eal/linuxapp/eal/eal.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
@@ -943,8 +943,11 @@ rte_eal_init(int argc, char **argv)rte_panic ("Cannot probe devices\n" );/* Probe & Initialize PCI devices */ -if (rte_eal_pci_probe ()) -rte_panic ("Cannot probe PCI\n" ); +if (rte_eal_pci_probe ()) { +RTE_LOG (ERR ,EAL ,"Cannot probe PCI\n" ); + rte_errno =ENOTSUP ; +return -1 ; + }if (rte_eal_dev_init () <0 )rte_panic ("Cannot init pmd devices\n" );
Signature · 1 line
2.9.3
PATCH v5 22/26 eal_common_dev: continue initializing vdevs
In reply to Aaron ConoleEven if one vdev should fail, there's no need to prevent further processing. Log the error, and reflect it to the higher levels to decide.
--- lib/librte_eal/common/eal_common_dev.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
lib/librte_eal/common/eal_common_dev.c +3 -2
@@ -80,6 +80,7 @@ intrte_eal_dev_init (void) {struct rte_devargs *devargs; +int ret =0 ;/* * Note that the dev_driver_list is populated here
@@ -97,11 +98,11 @@ rte_eal_dev_init(void) devargs->args)) {RTE_LOG (ERR ,EAL ,"failed to initialize %s device\n" , devargs->virt.drv_name); -return -1 ; + ret =-1 ; } } -return 0 ; +return ret; }int rte_eal_dev_attach (const char *name,const char *devargs)
Signature · 1 line
2.9.3
PATCH v5 23/26 eal: do not panic (or abort) if vdev init fails
In reply to Aaron ConoleSeems like it's possible to continue. At least, the error is reflected properly in the logs. A user could then go and correct or investigate the situation.
--- lib/librte_eal/linuxapp/eal/eal.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal.c +1 -1
@@ -950,7 +950,7 @@ rte_eal_init(int argc, char **argv) }if (rte_eal_dev_init () <0 ) -rte_panic ("Cannot init pmd devices\n" ); +RTE_LOG (ERR ,EAL ,"Cannot init pmd devices\n" );rte_eal_mcfg_complete ();
Signature · 1 line
2.9.3
PATCH v5 24/26 eal: do not panic when bus probe fails
In reply to Aaron Conole--- lib/librte_eal/linuxapp/eal/eal.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
@@ -939,8 +939,11 @@ rte_eal_init(int argc, char **argv)rte_eal_mp_wait_lcore ();/* Probe all the buses and devices/drivers on them */ -if (rte_bus_probe ()) -rte_panic ("Cannot probe devices\n" ); +if (rte_bus_probe ()) { +RTE_LOG (ERR ,EAL ,"Cannot probe devices\n" ); + rte_errno =ENOTSUP ; +return -1 ; + }/* Probe & Initialize PCI devices */ if (rte_eal_pci_probe ()) {
Signature · 1 line
2.9.3
PATCH v5 25/26 eal: do not panic on failed bus scan
In reply to Aaron ConoleFor now, do an abort. It's likely that even aborting the initialization is premature in this case, as it may be possible to proceed even if one bus or another is not available.
--- lib/librte_eal/linuxapp/eal/eal.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
@@ -899,8 +899,11 @@ rte_eal_init(int argc, char **argv)return -1 ; } -if (rte_bus_scan ()) -rte_panic ("Cannot scan the buses for devices\n" ); +if (rte_bus_scan ()) { +RTE_LOG (ERR ,EAL ,"Cannot scan the buses for devices\n" ); + rte_errno =ENODEV ; +return -1 ; + }RTE_LCORE_FOREACH_SLAVE (i) {
Signature · 1 line
2.9.3
PATCH v5 26/26 rte_eal_init: add info about rte_errno codes
In reply to Aaron ConoleThe rte_eal_init function will now pass failure reason hints to the application. To help app developers deciper this, add some brief information about what the codes are indicating.
--- lib/librte_eal/common/include/rte_eal.h | 27 ++++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-)
lib/librte_eal/common/include/rte_eal.h +26 -1
@@ -159,7 +159,32 @@ int rte_eal_iopl_init(void);* function call and should not be further interpreted by the * application. The EAL does not take any ownership of the memory used * for either the argv array, or its members. -* - On failure, a negative error value. +* - On failure, -1 and rte_errno is set to a value indicating the cause +* for failure. In some instances, the application will need to be +* restarted as part of clearing the issue. +* +* Error codes returned via rte_errno: +* EACCES indicates a permissions issue. +* +* EAGAIN indicates either a bus or system resource was not available, +* setup may be attempted again. +* +* EALREADY indicates that the rte_eal_init function has already been +* called, and cannot be called again. +* +* EFAULT indicates the tailq configuration name was not found in +* memory configuration. +* +* EINVAL indicates invalid parameters were passed as argv/argc. +* +* ENOMEM indicates failure likely caused by an out-of-memory condition. +* +* ENODEV indicates memory setup issues. +* +* ENOTSUP indicates that the EAL cannot initialize on this system. +* +* EUNATCH indicates that the PCI bus is either not present, or is not +* readable by the eal. */ int rte_eal_init (int argc,char **argv);
Signature · 1 line
2.9.3
Quoted 6 lines · Aaron Conole
In many cases, it's enough to simply let the application know that the call to initialize DPDK has failed. A complete halt can then be decided by the application based on error returned (and the app could even attempt a possible re-attempt after some corrective action by the user or application).
Set looks pretty good. Just a couple of minor issues, apart from the one checkpatch issue I flagged:
* seems to be some inconsistency between using the "rte_eal_init_alert" function and RTE_LOG. If there is some reason why some cases use the function and others don't I think that might need to be called out somewhere.
* check-git-log flags a number of minor errors with commit titles and messages - most common is commit titles starting with a capital letter.
Otherwise:
v6 cover linux/eal: Remove most causes of panic on init
In reply to Aaron ConoleFrom: Aaron Conole <[email protected]>
In many cases, it's enough to simply let the application know that the call to initialize DPDK has failed. A complete halt can then be decided by the application based on error returned (and the app could even attempt a possible re-attempt after some corrective action by the user or application).
Changes ->v2:
- Audited all "RTE_LOG (" calls that were introduced, and converted to "RTE_LOG("
- Added some fprintf(stderr, "") lines to indicate errors before logging is initialized
- Removed assignments to errno.
- Changed patch 14/25 to reflect EFAULT, and document in 25/25
Changes ->v3:
- Checkpatch issues in patches 3 (spelling mistake), 9 (issue with leading spaces), and 19 (braces around single line statement if-condition)
Changes ->v4:
- Error text cleanup.
- Add a new check around rte_bus_scan(), added during the development of this series.
Changes ->v5:
- checkpatch.pl cleanup in patch 02/26
- move rte_errno.h include from patch 15 to patch 02
- remove stdbool.h and use int as return type in patch 06/26
Changes ->v6:
- convert all of the initialization calls to RTE_LOG() to rte_eal_init_alert()
- run through check-git-log and checkpatches
- add Bruce's ack to the series
I kept the rte_errno reflection, since this is control-path code and the init function returns a sentinel value of -1.
Aaron Conole (26):
eal: cpu init will no longer panic
eal: return error instead of panic for cpu init
eal: do not panic on hugepage info init
eal: do not panic on failed hugepage query
eal: do not panic if parsing args returns error
eal-common: introduce a way to query cpu support
eal: do not panic when CPU isn't supported
eal: do not panic on memzone initialization fails
eal: set errno when exiting for already called
eal: do not panic on log failures
eal: do not panic on PCI-probe
eal: do not panic on vfio failure
eal: do not panic on memory init
eal: do not panic on tailq init
eal: do not panic on alarm init
eal: convert timer init not to call panic
eal: change the private pipe call to reflect errno
eal: do not panic on interrupt thread init
eal: do not error if plugins fail to init
eal_pci: continue probing even on failures
eal: do not panic on failed PCI-probe
eal_common_dev: continue initializing vdevs
eal: do not panic (or abort) if vdev init fails
eal: do not panic when bus probe fails
eal: do not panic on failed bus scan
rte_eal_init: add info about various error codes
lib/librte_eal/common/eal_common_cpuflags.c | 13 +- lib/librte_eal/common/eal_common_dev.c | 5 +- lib/librte_eal/common/eal_common_lcore.c | 7 +- lib/librte_eal/common/eal_common_pci.c | 15 ++- lib/librte_eal/common/eal_common_tailqs.c | 3 +- .../common/include/generic/rte_cpuflags.h | 8 ++ lib/librte_eal/common/include/rte_eal.h | 27 ++++- lib/librte_eal/linuxapp/eal/eal.c | 131 +++++++++++++++------ lib/librte_eal/linuxapp/eal/eal_hugepage_info.c | 9 +- lib/librte_eal/linuxapp/eal/eal_interrupts.c | 5 +- 10 files changed, 170 insertions(+), 53 deletions(-)
Signature · 1 line
2.9.3
PATCH v6 1/26 eal: cpu init will no longer panic
After this change, the EAL CPU NUMA node resolution step can no longer emit an rte_panic. This aligns with the code in rte_eal_init, which expects failures to return an error code.
--- lib/librte_eal/common/eal_common_lcore.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
lib/librte_eal/common/eal_common_lcore.c +4 -3
@@ -83,16 +83,17 @@ rte_eal_cpu_init(void) config->lcore_role[lcore_id] =ROLE_RTE ; lcore_config[lcore_id].core_id =eal_cpu_core_id (lcore_id); lcore_config[lcore_id].socket_id =eal_cpu_socket_id (lcore_id); - if (lcore_config[lcore_id].socket_id >= RTE_MAX_NUMA_NODES) +if (lcore_config[lcore_id].socket_id >=RTE_MAX_NUMA_NODES ) {#ifdef RTE_EAL_ALLOW_INV_SOCKET_ID lcore_config[lcore_id].socket_id =0 ;#else -rte_panic ("Socket ID (%u) is greater than " +RTE_LOG (ERR ,EAL ,"Socket ID (%u) is greater than " "RTE_MAX_NUMA_NODES (%d)\n" , lcore_config[lcore_id].socket_id,RTE_MAX_NUMA_NODES ); +return -1 ;#endif - + } RTE_LOG(DEBUG, EAL, "Detected lcore %u as " "core %u on socket %u\n", lcore_id, lcore_config[lcore_id].core_id,
Signature · 1 line
2.9.3
PATCH v6 2/26 eal: return error instead of panic for cpu init
In reply to Aaron ConoleThere may be no way to gracefully recover, but the application should be notified that a failure happened, rather than completely aborting. This allows the user to proceed with a "slow-path" type solution.
--- lib/librte_eal/linuxapp/eal/eal.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +12 -2
@@ -61,6 +61,7 @@#include <rte_launch.h> #include <rte_eal.h> #include <rte_eal_memconfig.h> +#include <rte_errno.h> #include <rte_per_lcore.h> #include <rte_lcore.h> #include <rte_log.h>
@@ -740,6 +741,12 @@ static int rte_eal_vfio_setup(void) }#endif +static void rte_eal_init_alert (const char *msg) +{ +fprintf (stderr,"EAL: FATAL: %s\n" , msg); +RTE_LOG (ERR ,EAL ,"%s\n" , msg); +} +/* Launch threads, called at application init(). */ int rte_eal_init (int argc,char **argv)
@@ -767,8 +774,11 @@ rte_eal_init(int argc, char **argv)/* set log level as early as possible */ rte_set_log_level (internal_config.log_level); -if (rte_eal_cpu_init () <0 ) -rte_panic ("Cannot detect lcores\n" ); +if (rte_eal_cpu_init () <0 ) { +rte_eal_init_alert ("Cannot detect lcores." ); + rte_errno =ENOTSUP ; +return -1 ; + } fctret =eal_parse_args (argc, argv);if (fctret <0 )
Signature · 1 line
2.9.3
PATCH v6 3/26 eal: do not panic on hugepage info init
In reply to Aaron ConoleWhen attempting to scan hugepages, signal to the eal that an error has occurred, rather than performing a panic.
--- lib/librte_eal/linuxapp/eal/eal_hugepage_info.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
lib/librte_eal/linuxapp/eal/eal_hugepage_info.c +6 -3
@@ -283,9 +283,12 @@ eal_hugepage_info_init(void)struct dirent *dirent; dir =opendir (sys_dir_path); -if (dir ==NULL ) -rte_panic ("Cannot open directory %s to read system hugepage " -"info\n" , sys_dir_path); +if (dir ==NULL ) { +RTE_LOG (ERR ,EAL , +"Cannot open directory %s to read system hugepage info\n" , + sys_dir_path); +return -1 ; + }for (dirent =readdir (dir); dirent !=NULL ; dirent =readdir (dir)) {struct hugepage_info *hpi;
Signature · 1 line
2.9.3
PATCH v6 4/26 eal: do not panic on failed hugepage query
In reply to Aaron ConoleIf we fail to acquire hugepage information, simply signal an error to the application. This clears the run_once counter, allowing the user or application to take a corrective action and retry.
--- lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
@@ -787,8 +787,12 @@ rte_eal_init(int argc, char **argv)if (internal_config.no_hugetlbfs ==0 && internal_config.process_type !=RTE_PROC_SECONDARY && internal_config.xen_dom0_support ==0 && -eal_hugepage_info_init () <0 ) -rte_panic ("Cannot get hugepage information\n" ); +eal_hugepage_info_init () <0 ) { +rte_eal_init_alert ("Cannot get hugepage information." ); + rte_errno =EACCES ; +rte_atomic32_clear (&run_once); +return -1 ; + }if (internal_config.memory ==0 && internal_config.force_sockets ==0 ) {if (internal_config.no_hugetlbfs)
Signature · 1 line
2.9.3
PATCH v6 5/26 eal: do not panic if parsing args returns error
In reply to Aaron ConoleIt's possible that the application could take a corrective action here, and either prompt the user for different arguments, or at least perform a better logging. Exiting this early prevents any useful information gathering from the application layer.
--- lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
@@ -781,8 +781,12 @@ rte_eal_init(int argc, char **argv) } fctret =eal_parse_args (argc, argv); -if (fctret <0 ) -exit (1 ); +if (fctret <0 ) { +rte_eal_init_alert ("Invalid 'command line' arguments." ); + rte_errno =EINVAL ; +rte_atomic32_clear (&run_once); +return -1 ; + } if (internal_config.no_hugetlbfs == 0 && internal_config.process_type != RTE_PROC_SECONDARY &&
Signature · 1 line
2.9.3
PATCH v6 6/26 eal-common: introduce a way to query cpu support
In reply to Aaron ConoleThis adds a new API to check for the eal cpu versions.
--- lib/librte_eal/common/eal_common_cpuflags.c | 13 +++++++++++-- lib/librte_eal/common/include/generic/rte_cpuflags.h | 8 ++++++++ 2 files changed, 19 insertions(+), 2 deletions(-)
lib/librte_eal/common/eal_common_cpuflags.c +11 -2
@@ -43,6 +43,13 @@void rte_cpu_check_supported (void ) { +if (!rte_cpu_is_supported ()) +exit (1 ); +} + +int +rte_cpu_is_supported (void ) +{/* This is generated at compile-time by the build system */ static const enum rte_cpu_flag_t compile_time_flags[] = {RTE_COMPILE_TIME_CPUFLAGS
@@ -57,14 +64,16 @@ rte_cpu_check_supported(void)fprintf (stderr,"ERROR: CPU feature flag lookup failed with error %d\n" , ret); -exit (1 ); +return 0 ; }if (!ret) {fprintf (stderr,"ERROR: This system does not support \"%s\".\n" "Please check that RTE_MACHINE is set correctly.\n" ,rte_cpu_get_flag_name (compile_time_flags[i])); -exit (1 ); +return 0 ; } } + +return 1 ; }
lib/librte_eal/common/include/generic/rte_cpuflags.h +8 -0
@@ -82,4 +82,12 @@ rte_cpu_get_flag_enabled(enum rte_cpu_flag_t feature);void rte_cpu_check_supported (void ); +/** +* This function checks that the currently used CPU supports the CPU features +* that were specified at compile time. It is called automatically within the +* EAL, so does not need to be used by applications. This version returns a +* result so that decisions may be made (for instance, graceful shutdowns). +*/ +int +rte_cpu_is_supported (void );#endif /* _RTE_CPUFLAGS_H_ */
Signature · 1 line
2.9.3
Re: [PATCH v6 06/26] eal-common: introduce a way to query cpu support
2017-02-28 13:52, Aaron Conole:
lib/librte_eal/common/include/generic/rte_cpuflags.h rte_cpu_get_flag_enabled(enum rte_cpu_flag_t feature); · line 93
Quoted 9 lines
lib/librte_eal/common/include/generic/rte_cpuflags.h +8
-0+/** +* This function checks that the currently used CPU supports the CPU features +* that were specified at compile time. It is called automatically within the +* EAL, so does not need to be used by applications. This version returns a +* result so that decisions may be made (for instance, graceful shutdowns). +*/ +int +rte_cpu_is_supported (void );#endif /* _RTE_CPUFLAGS_H_ */
A blank line is missing.
PATCH v6 7/26 eal: do not panic when CPU isn't supported
In reply to Aaron ConoleIt's now possible to gracefully exit the application, or for applications which support non-dpdk datapaths working in concert with DPDK datapaths, there no longer is the possibility of exiting for unsupported CPUs.
--- lib/librte_eal/linuxapp/eal/eal.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -1
@@ -759,7 +759,11 @@ rte_eal_init(int argc, char **argv)char thread_name[RTE_MAX_THREAD_NAME_LEN ];/* checks if the machine is adequate */ -rte_cpu_check_supported (); +if (!rte_cpu_is_supported ()) { +rte_eal_init_alert ("unsupported cpu type." ); + rte_errno =ENOTSUP ; +return -1 ; + }if (!rte_atomic32_test_and_set (&run_once))return -1 ;
Signature · 1 line
2.9.3
PATCH v6 8/26 eal: do not panic on memzone initialization fails
In reply to Aaron ConoleWhen memzone initialization fails, report the error to the calling application rather than panic(). Without a good way of detaching / releasing hugepages, at this point the application will have to restart.
--- lib/librte_eal/linuxapp/eal/eal.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
@@ -839,8 +839,11 @@ rte_eal_init(int argc, char **argv)/* the directories are locked during eal_hugepage_info_init */ eal_hugedirs_unlock (); -if (rte_eal_memzone_init () <0 ) -rte_panic ("Cannot init memzone\n" ); +if (rte_eal_memzone_init () <0 ) { +rte_eal_init_alert ("Cannot init memzone\n" ); + rte_errno =ENODEV ; +return -1 ; + }if (rte_eal_tailqs_init () <0 )rte_panic ("Cannot init tail queues for objects\n" );
Signature · 1 line
2.9.3
PATCH v6 9/26 eal: set errno when exiting for already called
In reply to Aaron Conole--- lib/librte_eal/linuxapp/eal/eal.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal.c +4 -1
@@ -765,8 +765,11 @@ rte_eal_init(int argc, char **argv)return -1 ; } -if (!rte_atomic32_test_and_set (&run_once)) +if (!rte_atomic32_test_and_set (&run_once)) { +rte_eal_init_alert ("already called initialization." ); + rte_errno =EALREADY ;return -1 ; + } logid =strrchr (argv[0 ],'/' ); logid =strdup (logid ? logid +1 : argv[0 ]);
Signature · 1 line
2.9.3
PATCH v6 10/26 eal: do not panic on log failures
In reply to Aaron ConoleWhen log initialization fails, it's generally because the fopencookie failed. While this is rare in practice, it could happen, and it is likely because of memory pressure. So, flag the error, and allow the user to retry.
--- lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
@@ -825,8 +825,12 @@ rte_eal_init(int argc, char **argv)rte_config_init (); -if (rte_eal_log_init (logid, internal_config.syslog_facility) <0 ) -rte_panic ("Cannot init logs\n" ); +if (rte_eal_log_init (logid, internal_config.syslog_facility) <0 ) { +rte_eal_init_alert ("Cannot init logging." ); + rte_errno =ENOMEM ; +rte_atomic32_clear (&run_once); +return -1 ; + }if (rte_eal_pci_init () <0 )rte_panic ("Cannot init PCI\n" );
Signature · 1 line
2.9.3
PATCH v6 11/26 eal: do not panic on PCI-probe
In reply to Aaron ConoleThis will usually be an issue because of permissions. However, it could also be caused by OOM. In either case, errno will contain the underlying cause. It is safe to re-init the system here, so allow the application to take corrective action and reinit.
--- lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
@@ -832,8 +832,12 @@ rte_eal_init(int argc, char **argv)return -1 ; } -if (rte_eal_pci_init () <0 ) -rte_panic ("Cannot init PCI\n" ); +if (rte_eal_pci_init () <0 ) { +rte_eal_init_alert ("Cannot init PCI\n" ); + rte_errno =EUNATCH ; +rte_atomic32_clear (&run_once); +return -1 ; + }#ifdef VFIO_PRESENT if (rte_eal_vfio_setup () <0 )
Signature · 1 line
2.9.3
PATCH v6 12/26 eal: do not panic on vfio failure
In reply to Aaron Conole--- lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
@@ -840,8 +840,12 @@ rte_eal_init(int argc, char **argv) }#ifdef VFIO_PRESENT -if (rte_eal_vfio_setup () <0 ) -rte_panic ("Cannot init VFIO\n" ); +if (rte_eal_vfio_setup () <0 ) { +rte_eal_init_alert ("Cannot init VFIO\n" ); + rte_errno =EAGAIN ; +rte_atomic32_clear (&run_once); +return -1 ; + }#endif if (rte_eal_memory_init () <0 )
Signature · 1 line
2.9.3
PATCH v6 13/26 eal: do not panic on memory init
In reply to Aaron ConoleThis can only happen when access to hugepages (either as primary or secondary process) fails (and that is usually permissions). Since the manner of failure is not reversible, we cannot allow retry.
--- lib/librte_eal/linuxapp/eal/eal.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
@@ -848,8 +848,11 @@ rte_eal_init(int argc, char **argv) }#endif -if (rte_eal_memory_init () <0 ) -rte_panic ("Cannot init memory\n" ); +if (rte_eal_memory_init () <0 ) { +rte_eal_init_alert ("Cannot init memory\n" ); + rte_errno =ENOMEM ; +return -1 ; + }/* the directories are locked during eal_hugepage_info_init */ eal_hugedirs_unlock ();
Signature · 1 line
2.9.3
PATCH v6 14/26 eal: do not panic on tailq init
In reply to Aaron ConoleThere are some theoretical racy conditions in the system that _could_ cause early tailq init to fail; however, no need to panic the application. While it can't continue using DPDK, it could make better alerts to the user.
--- lib/librte_eal/common/eal_common_tailqs.c | 3 +-- lib/librte_eal/linuxapp/eal/eal.c | 7 +++++-- 2 files changed, 6 insertions(+), 4 deletions(-)
lib/librte_eal/common/eal_common_tailqs.c +1 -2
@@ -188,8 +188,7 @@ rte_eal_tailqs_init(void)if (t->head ==NULL ) {RTE_LOG (ERR ,EAL ,"Cannot initialize tailq: %s\n" , t->name); -/* no need to TAILQ_REMOVE, we are going to panic in -* rte_eal_init() */ +/* TAILQ_REMOVE not needed, error is already fatal */ goto fail; } }
lib/librte_eal/linuxapp/eal/eal.c +5 -2
@@ -863,8 +863,11 @@ rte_eal_init(int argc, char **argv)return -1 ; } -if (rte_eal_tailqs_init () <0 ) -rte_panic ("Cannot init tail queues for objects\n" ); +if (rte_eal_tailqs_init () <0 ) { +rte_eal_init_alert ("Cannot init tail queues for objects\n" ); + rte_errno =EFAULT ; +return -1 ; + }if (rte_eal_alarm_init () <0 )rte_panic ("Cannot init interrupt-handling thread\n" );
Signature · 1 line
2.9.3
PATCH v6 15/26 eal: do not panic on alarm init
In reply to Aaron Conolerte_eal_alarm_init() call uses the linux timerfd framework to create a poll()-able timer using standard posix file operations. This could fail for a few reasons given in the man-pages, but many could be corrected by the user application. No need to panic.
--- lib/librte_eal/linuxapp/eal/eal.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
@@ -869,8 +869,11 @@ rte_eal_init(int argc, char **argv)return -1 ; } -if (rte_eal_alarm_init () <0 ) -rte_panic ("Cannot init interrupt-handling thread\n" ); +if (rte_eal_alarm_init () <0 ) { +rte_eal_init_alert ("Cannot init interrupt-handling thread\n" ); +/* rte_eal_alarm_init sets rte_errno on failure. */ +return -1 ; + }if (rte_eal_timer_init () <0 )rte_panic ("Cannot init HPET or TSC timers\n" );
Signature · 1 line
2.9.3
PATCH v6 16/26 eal: convert timer init not to call panic
In reply to Aaron ConoleAfter code inspection, there is no way for eal_timer_init() to fail. It simply returns 0 in all cases. As such, this test could either go-away or stay here as 'future-proofing'.
--- lib/librte_eal/linuxapp/eal/eal.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
@@ -875,8 +875,11 @@ rte_eal_init(int argc, char **argv)return -1 ; } -if (rte_eal_timer_init () <0 ) -rte_panic ("Cannot init HPET or TSC timers\n" ); +if (rte_eal_timer_init () <0 ) { +rte_eal_init_alert ("Cannot init HPET or TSC timers\n" ); + rte_errno =ENOTSUP ; +return -1 ; + }eal_check_mem_on_local_socket ();
Signature · 1 line
2.9.3
PATCH v6 17/26 eal: change the private pipe call to reflect errno
In reply to Aaron ConoleThere could be some confusion as to why the call failed - this change will always reflect the value of the error in rte_error.
--- lib/librte_eal/linuxapp/eal/eal_interrupts.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal_interrupts.c +4 -1
@@ -898,13 +898,16 @@ rte_eal_intr_init(void)* create a pipe which will be waited by epoll and notified to * rebuild the wait list of epoll. */ -if (pipe (intr_pipe.pipefd) <0 ) +if (pipe (intr_pipe.pipefd) <0 ) { + rte_errno = errno;return -1 ; + }/* create the host thread to wait/handle the interrupt */ ret =pthread_create (&intr_thread,NULL , eal_intr_thread_main,NULL );if (ret !=0 ) { + rte_errno = ret;RTE_LOG (ERR ,EAL ,"Failed to create thread for interrupt handling\n" ); }else {
Signature · 1 line
2.9.3
PATCH v6 18/26 eal: do not panic on interrupt thread init
In reply to Aaron ConoleWhen initializing the interrupt thread, there are a number of possible reasons for failure - some of which are correctable by the application. Do not panic() needlessly, and give the application a change to reflect this information to the user.
--- lib/librte_eal/linuxapp/eal/eal.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +4 -2
@@ -894,8 +894,10 @@ rte_eal_init(int argc, char **argv) rte_config.master_lcore, (int )thread_id, cpuset, ret == 0 ? "" : "..."); -if (rte_eal_intr_init () <0 ) -rte_panic ("Cannot init interrupt-handling thread\n" ); +if (rte_eal_intr_init () <0 ) { +rte_eal_init_alert ("Cannot init interrupt-handling thread\n" ); +return -1 ; + }if (rte_bus_scan ())rte_panic ("Cannot scan the buses for devices\n" );
Signature · 1 line
2.9.3
PATCH v6 19/26 eal: do not error if plugins fail to init
In reply to Aaron ConolePlugins are useful and important. However, it seems crazy to abort everything just because they don't initialize properly.
--- lib/librte_eal/linuxapp/eal/eal.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal.c +1 -1
@@ -884,7 +884,7 @@ rte_eal_init(int argc, char **argv)eal_check_mem_on_local_socket ();if (eal_plugins_init () <0 ) -rte_panic ("Cannot init plugins\n" ); +rte_eal_init_alert ("Cannot init plugins\n" );eal_thread_init_master (rte_config.master_lcore);
Signature · 1 line
2.9.3
PATCH v6 20/26 eal_pci: continue probing even on failures
In reply to Aaron ConoleSome devices may be inaccessible for a variety of reasons, or the PCI-bus may be unavailable causing the whole thing to fail. Still, better to continue attempts at probes.
--- lib/librte_eal/common/eal_common_pci.c | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-)
lib/librte_eal/common/eal_common_pci.c +10 -5
@@ -69,6 +69,7 @@#include <sys/queue.h> #include <sys/mman.h> +#include <rte_errno.h> #include <rte_interrupts.h> #include <rte_log.h> #include <rte_pci.h>
@@ -416,6 +417,7 @@ rte_eal_pci_probe(void)struct rte_pci_device *dev =NULL ;struct rte_devargs *devargs;int probe_all =0 ; +int ret_1 =0 ;int ret =0 ;if (rte_eal_devargs_type_count (RTE_DEVTYPE_WHITELISTED_PCI ) ==0 )
@@ -430,17 +432,20 @@ rte_eal_pci_probe(void)/* probe all or only whitelisted devices */ if (probe_all) - ret =pci_probe_all_drivers (dev); + ret_1 =pci_probe_all_drivers (dev);else if (devargs !=NULL && devargs->type ==RTE_DEVTYPE_WHITELISTED_PCI ) - ret =pci_probe_all_drivers (dev); -if (ret <0 ) -rte_exit (EXIT_FAILURE ,"Requested device " PCI_PRI_FMT + ret_1 =pci_probe_all_drivers (dev); +if (ret_1 <0 ) { +RTE_LOG (ERR ,EAL ,"Requested device " PCI_PRI_FMT " cannot be used\n" , dev->addr.domain, dev->addr.bus, dev->addr.devid, dev->addr.function); + rte_errno = errno; + ret =1 ; + } } -return 0 ; +return -ret; }/* dump one device */
Signature · 1 line
2.9.3
Re: [PATCH v6 20/26] eal_pci: continue probing even on failures
2017-02-28 13:53, Aaron Conole:
lib/librte_eal/common/eal_common_pci.c rte_eal_pci_probe(void) · line 420
Quoted 1 line
lib/librte_eal/common/eal_common_pci.c +1
-0+int ret_1 =0 ;
You do not need to add a new variable.
Quoted 16 lines
lib/librte_eal/common/eal_common_pci.c +3
-4int ret =0 ;if (rte_eal_devargs_type_count (RTE_DEVTYPE_WHITELISTED_PCI ) ==0 )@@ -430,17 +432,20 @@ rte_eal_pci_probe(void)/* probe all or only whitelisted devices */ if (probe_all) - ret =pci_probe_all_drivers (dev); + ret_1 =pci_probe_all_drivers (dev);else if (devargs !=NULL && devargs->type ==RTE_DEVTYPE_WHITELISTED_PCI ) - ret =pci_probe_all_drivers (dev); -if (ret <0 ) - rte_exit(EXIT_FAILURE, "Requested device " PCI_PRI_FMT + ret_1 =pci_probe_all_drivers (dev); +if (ret_1 <0 ) {
lib/librte_eal/common/eal_common_pci.c rte_eal_pci_probe(void) · line 448
Quoted 10 lines
lib/librte_eal/common/eal_common_pci.c +5
-1+RTE_LOG (ERR ,EAL ,"Requested device " PCI_PRI_FMT " cannot be used\n" , dev->addr.domain, dev->addr.bus, dev->addr.devid, dev->addr.function); + rte_errno = errno; + ret =1 ; + } } -return 0 ; +return -ret;
It may be more explicit to use only one variable ret and filter the positive values:
ret < 0 ? -1 : 0
PATCH v6 21/26 eal: do not panic on failed PCI-probe
In reply to Aaron ConoleSince PCI isn't neccessarily required, it may be possible to simply log the error and continue on letting the user check the logs and restart the application when things have failed.
--- lib/librte_eal/linuxapp/eal/eal.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
@@ -943,8 +943,11 @@ rte_eal_init(int argc, char **argv)rte_panic ("Cannot probe devices\n" );/* Probe & Initialize PCI devices */ -if (rte_eal_pci_probe ()) -rte_panic ("Cannot probe PCI\n" ); +if (rte_eal_pci_probe ()) { +rte_eal_init_alert ("Cannot probe PCI\n" ); + rte_errno =ENOTSUP ; +return -1 ; + }if (rte_eal_dev_init () <0 )rte_panic ("Cannot init pmd devices\n" );
Signature · 1 line
2.9.3
PATCH v6 22/26 eal_common_dev: continue initializing vdevs
In reply to Aaron ConoleEven if one vdev should fail, there's no need to prevent further processing. Log the error, and reflect it to the higher levels to decide.
--- lib/librte_eal/common/eal_common_dev.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
lib/librte_eal/common/eal_common_dev.c +3 -2
@@ -80,6 +80,7 @@ intrte_eal_dev_init (void) {struct rte_devargs *devargs; +int ret =0 ;/* * Note that the dev_driver_list is populated here
@@ -97,11 +98,11 @@ rte_eal_dev_init(void) devargs->args)) {RTE_LOG (ERR ,EAL ,"failed to initialize %s device\n" , devargs->virt.drv_name); -return -1 ; + ret =-1 ; } } -return 0 ; +return ret; }int rte_eal_dev_attach (const char *name,const char *devargs)
Signature · 1 line
2.9.3
PATCH v6 23/26 eal: do not panic (or abort) if vdev init fails
In reply to Aaron ConoleSeems like it's possible to continue. At least, the error is reflected properly in the logs. A user could then go and correct or investigate the situation.
--- lib/librte_eal/linuxapp/eal/eal.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal.c +1 -1
@@ -950,7 +950,7 @@ rte_eal_init(int argc, char **argv) }if (rte_eal_dev_init () <0 ) -rte_panic ("Cannot init pmd devices\n" ); +rte_eal_init_alert ("Cannot init pmd devices\n" );rte_eal_mcfg_complete ();
Signature · 1 line
2.9.3
PATCH v6 24/26 eal: do not panic when bus probe fails
In reply to Aaron Conole--- lib/librte_eal/linuxapp/eal/eal.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
@@ -939,8 +939,11 @@ rte_eal_init(int argc, char **argv)rte_eal_mp_wait_lcore ();/* Probe all the buses and devices/drivers on them */ -if (rte_bus_probe ()) -rte_panic ("Cannot probe devices\n" ); +if (rte_bus_probe ()) { +rte_eal_init_alert ("Cannot probe devices\n" ); + rte_errno =ENOTSUP ; +return -1 ; + }/* Probe & Initialize PCI devices */ if (rte_eal_pci_probe ()) {
Signature · 1 line
2.9.3
PATCH v6 25/26 eal: do not panic on failed bus scan
In reply to Aaron ConoleFor now, do an abort. It's likely that even aborting the initialization is premature in this case, as it may be possible to proceed even if one bus or another is not available.
--- lib/librte_eal/linuxapp/eal/eal.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
@@ -899,8 +899,11 @@ rte_eal_init(int argc, char **argv)return -1 ; } -if (rte_bus_scan ()) -rte_panic ("Cannot scan the buses for devices\n" ); +if (rte_bus_scan ()) { +rte_eal_init_alert ("Cannot scan the buses for devices\n" ); + rte_errno =ENODEV ; +return -1 ; + }RTE_LCORE_FOREACH_SLAVE (i) {
Signature · 1 line
2.9.3
PATCH v6 26/26 rte_eal_init: add info about various error codes
In reply to Aaron ConoleThe rte_eal_init function will now pass failure reason hints to the application. To help app developers deciper this, add some brief information about what the codes are indicating.
--- lib/librte_eal/common/include/rte_eal.h | 27 ++++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-)
lib/librte_eal/common/include/rte_eal.h +26 -1
@@ -159,7 +159,32 @@ int rte_eal_iopl_init(void);* function call and should not be further interpreted by the * application. The EAL does not take any ownership of the memory used * for either the argv array, or its members. -* - On failure, a negative error value. +* - On failure, -1 and rte_errno is set to a value indicating the cause +* for failure. In some instances, the application will need to be +* restarted as part of clearing the issue. +* +* Error codes returned via rte_errno: +* EACCES indicates a permissions issue. +* +* EAGAIN indicates either a bus or system resource was not available, +* setup may be attempted again. +* +* EALREADY indicates that the rte_eal_init function has already been +* called, and cannot be called again. +* +* EFAULT indicates the tailq configuration name was not found in +* memory configuration. +* +* EINVAL indicates invalid parameters were passed as argv/argc. +* +* ENOMEM indicates failure likely caused by an out-of-memory condition. +* +* ENODEV indicates memory setup issues. +* +* ENOTSUP indicates that the EAL cannot initialize on this system. +* +* EUNATCH indicates that the PCI bus is either not present, or is not +* readable by the eal. */ int rte_eal_init (int argc,char **argv);
Signature · 1 line
2.9.3
Hi,
Thanks for the work.
I think it needs to be completed to have the same behaviour on bsdapp.
As a another version is required, I add some small comments about the formatting.
I think you should use the form "do not panic on <step>" for most of the commits.
Some commits may be squashed (see below):
2017-02-28 13:52, Aaron Conole:
Quoted 3 lines
Aaron Conole (26):
eal: cpu init will no longer panic
eal: return error instead of panic for cpu init
squashed?
Quoted 2 lines
eal: do not panic on hugepage info init
eal: do not panic on failed hugepage query
squashed?
Quoted 3 lines
eal: do not panic if parsing args returns error
eal-common: introduce a way to query cpu support
eal: do not panic when CPU isn't supported
squashed?
Quoted 4 lines
eal: do not panic on memzone initialization fails
eal: set errno when exiting for already called
eal: do not panic on log failures
eal: do not panic on PCI-probe
It is not really the probe here
Quoted 10 lines
eal: do not panic on vfio failure
eal: do not panic on memory init
eal: do not panic on tailq init
eal: do not panic on alarm init
eal: convert timer init not to call panic
eal: change the private pipe call to reflect errno
eal: do not panic on interrupt thread init
eal: do not error if plugins fail to init
eal_pci: continue probing even on failures
eal: do not panic on failed PCI-probe
squashed?
Quoted 2 lines
eal_common_dev: continue initializing vdevs
eal: do not panic (or abort) if vdev init fails
squashed?
Quoted 3 lines
eal: do not panic when bus probe fails
eal: do not panic on failed bus scan
rte_eal_init: add info about various error codes
Quoted 4 lines · Thomas Monjalon
Hi,
Thanks for the work.
I think it needs to be completed to have the same behaviour on bsdapp.
Ideally, yes, but I also don't think the lack of BSD changes should block the inclusion of this set. In terms of application writers, the apps don't need to be written differently for BSD compared to Linux because of this change. All that is different is that the BSD version will panic rather than return the error code.
Regards,
/Bruce
Quoted 1 line
2017-03-09 09:11, Bruce Richardson:
Quoted 11 lines · Thomas Monjalon
Quoted 4 lines
Hi,
Thanks for the work.
I think it needs to be completed to have the same behaviour on bsdapp.Ideally, yes, but I also don't think the lack of BSD changes should block the inclusion of this set. In terms of application writers, the apps don't need to be written differently for BSD compared to Linux because of this change. All that is different is that the BSD version will panic rather than return the error code.
So you do not have any issue about having a different behaviour on Linux and BSD?
You are the bsdapp maintainer, so it is your call.
Quoted 24 lines
-----Original Message-----
From: Thomas Monjalon [mailto:[email protected]]
Sent: Thursday, March 9, 2017 9:26 AM
To: Richardson, Bruce <[email protected]>
Cc: Aaron Conole <[email protected]>; [email protected]; Stephen Hemminger <[email protected]>
Subject: Re: [dpdk-dev] [PATCH v6 00/26] linux/eal: Remove most causes of panic on init2017-03-09 09:11, Bruce Richardson:
Quoted 11 lines · Thomas Monjalon
Quoted 4 lines
Hi,
Thanks for the work.
I think it needs to be completed to have the same behaviour on bsdapp.Ideally, yes, but I also don't think the lack of BSD changes should block the inclusion of this set. In terms of application writers, the apps don't need to be written differently for BSD compared to Linux because of this change. All that is different is that the BSD version will panic rather than return the error code.
Quoted 3 lines
So you do not have any issue about having a different behaviour on Linux and BSD?
You are the bsdapp maintainer, so it is your call.
I would infinitely prefer to have the same behavior. However, so long as this does not require a user to change their app to be different on BSD, I don't think lack of BSD support should block improving Linux.
Aaron - will you be able to do equivalent changes to BSD within the 17.05 timeframe?
/Bruce
Quoted 26 lines · Bruce
Quoted 22 lines
-----Original Message-----
From: Thomas Monjalon [mailto:[email protected]]
Sent: Thursday, March 9, 2017 9:26 AM
To: Richardson, Bruce <[email protected]>
Cc: Aaron Conole <[email protected]>; [email protected]; Stephen Hemminger <[email protected]>
Subject: Re: [dpdk-dev] [PATCH v6 00/26] linux/eal: Remove most causes of panic on init2017-03-09 09:11, Bruce Richardson:
Quoted 11 lines · Thomas Monjalon
Quoted 4 lines
Hi,
Thanks for the work.
I think it needs to be completed to have the same behaviour on bsdapp.Ideally, yes, but I also don't think the lack of BSD changes should block the inclusion of this set. In terms of application writers, the apps don't need to be written differently for BSD compared to Linux because of this change. All that is different is that the BSD version will panic rather than return the error code.
Quoted 3 lines
So you do not have any issue about having a different behaviour on Linux and BSD?
You are the bsdapp maintainer, so it is your call.
Quoted 5 lines
I would infinitely prefer to have the same behavior. However, so long as this does not require a user to change their app to be different on BSD, I don't think lack of BSD support should block improving Linux.
Aaron - will you be able to do equivalent changes to BSD within the 17.05 timeframe?
I'm going to spend some time next week looking into it. If it looks like it's akin to s/linuxapp/bsdapp/ and everything works, then I'll do it. If I get hung up on anything, I'll probably update with my status.
-Aaron