Skip to contentSkip to conversation

linux/eal: Remove most causes of panic on init

PATCH Superseded Aaron Conole 4 participants 159 messages Feb 8, 2017 – Mar 10, 2017

v2 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
#0 lore raw

v2 cover linux/eal: Remove most causes of panic on init

In many cases, it's enough to simply let the application know that the call to initialize DPDK has failed. A complete halt can then be decided by the application based on error returned (and the app could even attempt a possible re-attempt after some corrective action by the user or application).

Changes ->v2:
- Audited all "RTE_LOG (" calls that were introduced, and converted to "RTE_LOG("
- Added some fprintf(stderr, "") lines to indicate errors before logging is initialized
- Removed assignments to errno.
- Changed patch 14/25 to reflect EFAULT, and document in 25/25

I kept the rte_errno reflection, since this is control-path code and the init function returns a sentinel value of -1.

Aaron Conole (25):
eal: CPU init will no longer panic
eal: return error instead of panic for cpu init
eal: No panic on hugepages info init
eal: do not panic on failed hugepage query
eal: failure to parse args returns error
eal-common: introduce a way to query cpu support
eal: Signal error when CPU isn't supported
eal: do not panic on memzone initialization fails
eal: set errno when exiting for already called
eal: Do not panic on log failures
eal: Do not panic on pci-probe
eal: do not panic on vfio failure
eal: do not panic on memory init
eal: do not panic on tailq init
eal: do not panic on alarm init
eal: convert timer_init not to call panic
eal: change the private pipe call to reflect errno
eal: Do not panic on interrupt thread init
eal: do not error if plugins fail to init
eal_pci: Continue probing even on failures
eal: do not panic on failed PCI probe
eal_common_dev: continue initializing vdevs
eal: do not panic (or abort) if vdev init fails
eal: do not panic when bus probe fails
rte_eal_init: add info about rte_errno codes

 lib/librte_eal/common/eal_common_cpuflags.c        |  13 ++-
 lib/librte_eal/common/eal_common_dev.c             |   5 +-
 lib/librte_eal/common/eal_common_lcore.c           |   7 +-
 lib/librte_eal/common/eal_common_pci.c             |  15 ++-
 lib/librte_eal/common/eal_common_tailqs.c          |   3 +-
 .../common/include/generic/rte_cpuflags.h          |   9 ++
 lib/librte_eal/common/include/rte_eal.h            |  27 ++++-
 lib/librte_eal/linuxapp/eal/eal.c                  | 122 +++++++++++++++------
 lib/librte_eal/linuxapp/eal/eal_hugepage_info.c    |   6 +-
 lib/librte_eal/linuxapp/eal/eal_interrupts.c       |   5 +-
 10 files changed, 161 insertions(+), 51 deletions(-)
Signature · 1 line
2.9.3
#1 lore raw

PATCH v2 1/25 eal: CPU init will no longer panic

After this change, the EAL CPU NUMA node resolution step can no longer emit an rte_panic. This aligns with the code in rte_eal_init, which expects failures to return an error code.

---
 lib/librte_eal/common/eal_common_lcore.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
lib/librte_eal/common/eal_common_lcore.c +4 -3
index 2cd4132..84fa0cb 100644
@@ -83,16 +83,17 @@ rte_eal_cpu_init(void)
 		config->lcore_role[lcore_id] = ROLE_RTE;
 		lcore_config[lcore_id].core_id = eal_cpu_core_id(lcore_id);
 		lcore_config[lcore_id].socket_id = eal_cpu_socket_id(lcore_id);
-		if (lcore_config[lcore_id].socket_id >= RTE_MAX_NUMA_NODES)
+		if (lcore_config[lcore_id].socket_id >= RTE_MAX_NUMA_NODES) {
 #ifdef RTE_EAL_ALLOW_INV_SOCKET_ID
 			lcore_config[lcore_id].socket_id = 0;
 #else
-			rte_panic("Socket ID (%u) is greater than "
+			RTE_LOG(ERR, EAL, "Socket ID (%u) is greater than "
 				"RTE_MAX_NUMA_NODES (%d)\n",
 				lcore_config[lcore_id].socket_id,
 				RTE_MAX_NUMA_NODES);
+			return -1;
 #endif
-
+		}
 		RTE_LOG(DEBUG, EAL, "Detected lcore %u as "
 				"core %u on socket %u\n",
 				lcore_id, lcore_config[lcore_id].core_id,
Signature · 1 line
2.9.3
#2 lore raw

PATCH v2 2/25 eal: return error instead of panic for cpu init

In reply to Aaron Conole

There may be no way to gracefully recover, but the application should be notified that a failure happened, rather than completely aborting. This allows the user to proceed with a "slow-path" type solution.

---
 lib/librte_eal/linuxapp/eal/eal.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
index bf6b818..d8e00f5 100644
@@ -767,8 +767,11 @@ rte_eal_init(int argc, char **argv)
 	/* set log level as early as possible */
 	rte_set_log_level(internal_config.log_level);
 
-	if (rte_eal_cpu_init() < 0)
-		rte_panic("Cannot detect lcores\n");
+	if (rte_eal_cpu_init() < 0) {
+		RTE_LOG(ERR, EAL, "Cannot detect lcores\n");
+		rte_errno = ENOTSUP;
+		return -1;
+	}
 
 	fctret = eal_parse_args(argc, argv);
 	if (fctret < 0)
Signature · 1 line
2.9.3
#3 lore raw

PATCH v2 3/25 eal: No panic on hugepages info init

In reply to Aaron Conole

When attempting to scan hugepages, signal to the eal.c that an error has occured, rather than performing a panic.

---
 lib/librte_eal/linuxapp/eal/eal_hugepage_info.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal_hugepage_info.c +4 -2
index 18858e2..4d47eaf 100644
@@ -283,9 +283,11 @@ eal_hugepage_info_init(void)
 	struct dirent *dirent;
 
 	dir = opendir(sys_dir_path);
-	if (dir == NULL)
-		rte_panic("Cannot open directory %s to read system hugepage "
+	if (dir == NULL) {
+		RTE_LOG(ERR, EAL, "Cannot open directory %s to read system hugepage "
 			  "info\n", sys_dir_path);
+		return -1;
+	}
 
 	for (dirent = readdir(dir); dirent != NULL; dirent = readdir(dir)) {
 		struct hugepage_info *hpi;
Signature · 1 line
2.9.3
#4 lore raw

PATCH v2 4/25 eal: do not panic on failed hugepage query

In reply to Aaron Conole

If we fail to acquire hugepage information, simply signal an error to the application. This clears the run_once counter, allowing the user or application to take a corrective action and retry.

---
 lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
index d8e00f5..8daa2be 100644
@@ -780,8 +780,12 @@ rte_eal_init(int argc, char **argv)
 	if (internal_config.no_hugetlbfs == 0 &&
 			internal_config.process_type != RTE_PROC_SECONDARY &&
 			internal_config.xen_dom0_support == 0 &&
-			eal_hugepage_info_init() < 0)
-		rte_panic("Cannot get hugepage information\n");
+			eal_hugepage_info_init() < 0) {
+		RTE_LOG(ERR, EAL, "Cannot get hugepage information\n");
+		rte_errno = EACCES;
+		rte_atomic32_clear(&run_once);
+		return -1;
+	}
 
 	if (internal_config.memory == 0 && internal_config.force_sockets == 0) {
 		if (internal_config.no_hugetlbfs)
Signature · 1 line
2.9.3
#5 lore raw

PATCH v2 5/25 eal: failure to parse args returns error

In reply to Aaron Conole

It's possible that the application could take a corrective action here, and either prompt the user for different arguments, or at least perform a better logging. Exiting this early prevents any useful information gathering from the application layer.

---
 lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
index 8daa2be..a626774 100644
@@ -774,8 +774,12 @@ rte_eal_init(int argc, char **argv)
 	}
 
 	fctret = eal_parse_args(argc, argv);
-	if (fctret < 0)
-		exit(1);
+	if (fctret < 0) {
+		RTE_LOG(ERR, EAL, "Invalid 'command line' arguments\n");
+		rte_errno = EINVAL;
+		rte_atomic32_clear(&run_once);
+		return -1;
+	}
 
 	if (internal_config.no_hugetlbfs == 0 &&
 			internal_config.process_type != RTE_PROC_SECONDARY &&
Signature · 1 line
2.9.3
#6 lore raw

PATCH v2 6/25 eal-common: introduce a way to query cpu support

In reply to Aaron Conole

This adds a new API to check for the eal cpu versions.

---
 lib/librte_eal/common/eal_common_cpuflags.c          | 13 +++++++++++--
 lib/librte_eal/common/include/generic/rte_cpuflags.h |  9 +++++++++
 2 files changed, 20 insertions(+), 2 deletions(-)
lib/librte_eal/common/eal_common_cpuflags.c +11 -2
index b5f76f7..2c2127b 100644
@@ -43,6 +43,13 @@
 void
 rte_cpu_check_supported(void)
 {
+	if (!rte_cpu_is_supported())
+		exit(1);
+}
+
+bool
+rte_cpu_is_supported(void)
+{
 	/* This is generated at compile-time by the build system */
 	static const enum rte_cpu_flag_t compile_time_flags[] = {
 			RTE_COMPILE_TIME_CPUFLAGS
@@ -57,14 +64,16 @@ rte_cpu_check_supported(void)
 			fprintf(stderr,
 				"ERROR: CPU feature flag lookup failed with error %d\n",
 				ret);
-			exit(1);
+			return false;
 		}
 		if (!ret) {
 			fprintf(stderr,
 			        "ERROR: This system does not support \"%s\".\n"
 			        "Please check that RTE_MACHINE is set correctly.\n",
 			        rte_cpu_get_flag_name(compile_time_flags[i]));
-			exit(1);
+			return false;
 		}
 	}
+
+	return true;
 }
lib/librte_eal/common/include/generic/rte_cpuflags.h +9 -0
index 71321f3..e4342ad 100644
@@ -40,6 +40,7 @@
  */
 
 #include <errno.h>
+#include <stdbool.h>
 
 /**
  * Enumeration of all CPU features supported
@@ -82,4 +83,12 @@ rte_cpu_get_flag_enabled(enum rte_cpu_flag_t feature);
 void
 rte_cpu_check_supported(void);
 
+/**
+ * This function checks that the currently used CPU supports the CPU features
+ * that were specified at compile time. It is called automatically within the
+ * EAL, so does not need to be used by applications.  This version returns a
+ * result so that decisions may be made (for instance, graceful shutdowns).
+ */
+bool
+rte_cpu_is_supported(void);
 #endif /* _RTE_CPUFLAGS_H_ */
Signature · 1 line
2.9.3
#7 lore raw

PATCH v2 7/25 eal: Signal error when CPU isn't supported

In reply to Aaron Conole

It's now possible to gracefully exit the application, or for applications which support non-dpdk datapaths working in concert with DPDK datapaths, there no longer is the possibility of exiting for unsupported CPUs.

---
 lib/librte_eal/linuxapp/eal/eal.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -1
index a626774..88d59a2 100644
@@ -752,7 +752,11 @@ rte_eal_init(int argc, char **argv)
 	char thread_name[RTE_MAX_THREAD_NAME_LEN];
 
 	/* checks if the machine is adequate */
-	rte_cpu_check_supported();
+	if (!rte_cpu_is_supported()) {
+		fprintf(stderr, "EAL: FATAL - unsupported cpu type.\n");
+		rte_errno = ENOTSUP;
+		return -1;
+	}
 
 	if (!rte_atomic32_test_and_set(&run_once))
 		return -1;
Signature · 1 line
2.9.3
#8 lore raw

PATCH v2 8/25 eal: do not panic on memzone initialization fails

In reply to Aaron Conole

When memzone initialization fails, report the error to the calling application rather than panic(). Without a good way of detaching / releasing hugepages, at this point the application will have to restart.

---
 lib/librte_eal/linuxapp/eal/eal.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
index 88d59a2..8f9bce1 100644
@@ -832,8 +832,11 @@ rte_eal_init(int argc, char **argv)
 	/* the directories are locked during eal_hugepage_info_init */
 	eal_hugedirs_unlock();
 
-	if (rte_eal_memzone_init() < 0)
-		rte_panic("Cannot init memzone\n");
+	if (rte_eal_memzone_init() < 0) {
+		RTE_LOG(ERR, EAL, "Cannot init memzone\n");
+		rte_errno = ENODEV;
+		return -1;
+	}
 
 	if (rte_eal_tailqs_init() < 0)
 		rte_panic("Cannot init tail queues for objects\n");
Signature · 1 line
2.9.3
#9 lore raw

PATCH v2 9/25 eal: set errno when exiting for already called

In reply to Aaron Conole
---
 lib/librte_eal/linuxapp/eal/eal.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal.c +4 -1
index 8f9bce1..debb083 100644
@@ -758,8 +758,11 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 
-	if (!rte_atomic32_test_and_set(&run_once))
+	if (!rte_atomic32_test_and_set(&run_once)) {
+        fprintf(stderr, "EAL: ERROR - already called initialization.\n");
+		rte_errno = EALREADY;
 		return -1;
+	}
 
 	logid = strrchr(argv[0], '/');
 	logid = strdup(logid ? logid + 1: argv[0]);
Signature · 1 line
2.9.3
#10 lore raw

PATCH v2 10/25 eal: Do not panic on log failures

In reply to Aaron Conole

When log initialization fails, it's generally because the fopencookie failed. While this is rare in practice, it could happen, and it is likely because of memory pressure. So, flag the error, and allow the user to retry.

---
 lib/librte_eal/linuxapp/eal/eal.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +7 -2
index debb083..1d828bf 100644
@@ -818,8 +818,13 @@ rte_eal_init(int argc, char **argv)
 
 	rte_config_init();
 
-	if (rte_eal_log_init(logid, internal_config.syslog_facility) < 0)
-		rte_panic("Cannot init logs\n");
+	if (rte_eal_log_init(logid, internal_config.syslog_facility) < 0) {
+		RTE_LOG(ERR, EAL, "Cannot init logging\n");
+		fprintf(stderr, "EAL: ERROR - cannot init logging.\n");
+		rte_errno = EIO;
+		rte_atomic32_clear(&run_once);
+		return -1;
+	}
 
 	if (rte_eal_pci_init() < 0)
 		rte_panic("Cannot init PCI\n");
Signature · 1 line
2.9.3
#11 lore raw

PATCH v2 11/25 eal: Do not panic on pci-probe

In reply to Aaron Conole

This will usually be an issue because of permissions. However, it could also be caused by OOM. In either case, errno will contain the underlying cause. It is safe to re-init the system here, so allow the application to take corrective action and reinit.

---
 lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
index 1d828bf..ab1aeef 100644
@@ -826,8 +826,12 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 
-	if (rte_eal_pci_init() < 0)
-		rte_panic("Cannot init PCI\n");
+	if (rte_eal_pci_init() < 0) {
+		RTE_LOG(ERR, EAL, "Cannot init PCI\n");
+		rte_errno = EUNATCH;
+		rte_atomic32_clear(&run_once);
+		return -1;
+	}
 
 #ifdef VFIO_PRESENT
 	if (rte_eal_vfio_setup() < 0)
Signature · 1 line
2.9.3
#12 lore raw

PATCH v2 12/25 eal: do not panic on vfio failure

In reply to Aaron Conole
---
 lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
index ab1aeef..ec26153 100644
@@ -834,8 +834,12 @@ rte_eal_init(int argc, char **argv)
 	}
 
 #ifdef VFIO_PRESENT
-	if (rte_eal_vfio_setup() < 0)
-		rte_panic("Cannot init VFIO\n");
+	if (rte_eal_vfio_setup() < 0) {
+		RTE_LOG(ERR, EAL, "Cannot init VFIO\n");
+		rte_errno = EAGAIN;
+		rte_atomic32_clear(&run_once);
+		return -1;
+	}
 #endif
 
 	if (rte_eal_memory_init() < 0)
Signature · 1 line
2.9.3
#13 lore raw

PATCH v2 13/25 eal: do not panic on memory init

In reply to Aaron Conole

This can only happen when access to hugepages (either as primary or secondary process) fails (and that is usually permissions). Since the manner of failure is not reversible, we cannot allow retry.

---
 lib/librte_eal/linuxapp/eal/eal.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
index ec26153..f5f0ad4 100644
@@ -842,8 +842,11 @@ rte_eal_init(int argc, char **argv)
 	}
 #endif
 
-	if (rte_eal_memory_init() < 0)
-		rte_panic("Cannot init memory\n");
+	if (rte_eal_memory_init() < 0) {
+		RTE_LOG(ERR, EAL, "Cannot init memory\n");
+		rte_errno = EACCES;
+		return -1;
+	}
 
 	/* the directories are locked during eal_hugepage_info_init */
 	eal_hugedirs_unlock();
Signature · 1 line
2.9.3
#14 lore raw

PATCH v2 14/25 eal: do not panic on tailq init

In reply to Aaron Conole

There are some theoretical racy conditions in the system that _could_ cause early tailq init to fail; however, no need to panic the application. While it can't continue using DPDK, it could make better alerts to the user.

---
 lib/librte_eal/common/eal_common_tailqs.c | 3 +--
 lib/librte_eal/linuxapp/eal/eal.c         | 7 +++++--
 2 files changed, 6 insertions(+), 4 deletions(-)
lib/librte_eal/common/eal_common_tailqs.c +1 -2
index bb08ec8..4f69828 100644
@@ -188,8 +188,7 @@ rte_eal_tailqs_init(void)
 		if (t->head == NULL) {
 			RTE_LOG(ERR, EAL,
 				"Cannot initialize tailq: %s\n", t->name);
-			/* no need to TAILQ_REMOVE, we are going to panic in
-			 * rte_eal_init() */
+			/* TAILQ_REMOVE not needed, error is already fatal */
 			goto fail;
 		}
 	}
lib/librte_eal/linuxapp/eal/eal.c +5 -2
index f5f0ad4..adcebc8 100644
@@ -857,8 +857,11 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 
-	if (rte_eal_tailqs_init() < 0)
-		rte_panic("Cannot init tail queues for objects\n");
+	if (rte_eal_tailqs_init() < 0) {
+		RTE_LOG(ERR, EAL, "Cannot init tail queues for objects\n");
+		rte_errno = EFAULT;
+		return -1;
+	}
 
 	if (rte_eal_alarm_init() < 0)
 		rte_panic("Cannot init interrupt-handling thread\n");
Signature · 1 line
2.9.3
#15 lore raw

PATCH v2 15/25 eal: do not panic on alarm init

In reply to Aaron Conole

rte_eal_alarm_init() call uses the linux timerfd framework to create a poll()-able timer using standard posix file operations. This could fail for a few reasons given in the man-pages, but many could be corrected by the user application. No need to panic.

---
 lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
index adcebc8..df8c794 100644
@@ -61,6 +61,7 @@
 #include <rte_launch.h>
 #include <rte_eal.h>
 #include <rte_eal_memconfig.h>
+#include <rte_errno.h>
 #include <rte_per_lcore.h>
 #include <rte_lcore.h>
 #include <rte_log.h>
@@ -863,8 +864,11 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 
-	if (rte_eal_alarm_init() < 0)
-		rte_panic("Cannot init interrupt-handling thread\n");
+	if (rte_eal_alarm_init() < 0) {
+		RTE_LOG(ERR, EAL, "Cannot init interrupt-handling thread\n");
+		/* rte_eal_alarm_init sets rte_errno on failure. */
+		return -1;
+	}
 
 	if (rte_eal_timer_init() < 0)
 		rte_panic("Cannot init HPET or TSC timers\n");
Signature · 1 line
2.9.3
#16 lore raw

PATCH v2 16/25 eal: convert timer_init not to call panic

In reply to Aaron Conole

After code inspection, there is no way for eal_timer_init() to fail. It simply returns 0 in all cases. As such, this test could either go-away or stay here as 'future-proofing'.

---
 lib/librte_eal/linuxapp/eal/eal.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
index df8c794..a9f1385 100644
@@ -870,8 +870,11 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 
-	if (rte_eal_timer_init() < 0)
-		rte_panic("Cannot init HPET or TSC timers\n");
+	if (rte_eal_timer_init() < 0) {
+		RTE_LOG(ERR, EAL, "Cannot init HPET or TSC timers\n");
+		rte_errno = ENOTSUP;
+		return -1;
+	}
 
 	eal_check_mem_on_local_socket();
 
Signature · 1 line
2.9.3
#17 lore raw

PATCH v2 17/25 eal: change the private pipe call to reflect errno

In reply to Aaron Conole

There could be some confusion as to why the call failed - this change will always reflect the value of the error in rte_error.

---
 lib/librte_eal/linuxapp/eal/eal_interrupts.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal_interrupts.c +4 -1
index b5b3f2b..b1a287c 100644
@@ -896,13 +896,16 @@ rte_eal_intr_init(void)
 	 * create a pipe which will be waited by epoll and notified to
 	 * rebuild the wait list of epoll.
 	 */
-	if (pipe(intr_pipe.pipefd) < 0)
+	if (pipe(intr_pipe.pipefd) < 0) {
+		rte_errno = errno;
 		return -1;
+	}
 
 	/* create the host thread to wait/handle the interrupt */
 	ret = pthread_create(&intr_thread, NULL,
 			eal_intr_thread_main, NULL);
 	if (ret != 0) {
+		rte_errno = ret;
 		RTE_LOG(ERR, EAL,
 			"Failed to create thread for interrupt handling\n");
 	} else {
Signature · 1 line
2.9.3
#18 lore raw

PATCH v2 18/25 eal: Do not panic on interrupt thread init

In reply to Aaron Conole

When initializing the interrupt thread, there are a number of possible reasons for failure - some of which are correctable by the application. Do not panic() needlessly, and give the application a change to reflect this information to the user.

---
 lib/librte_eal/linuxapp/eal/eal.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +4 -2
index a9f1385..14ec2e0 100644
@@ -889,8 +889,10 @@ rte_eal_init(int argc, char **argv)
 		rte_config.master_lcore, (int)thread_id, cpuset,
 		ret == 0 ? "" : "...");
 
-	if (rte_eal_intr_init() < 0)
-		rte_panic("Cannot init interrupt-handling thread\n");
+	if (rte_eal_intr_init() < 0) {
+		RTE_LOG(ERR, EAL, "Cannot init interrupt-handling thread\n");
+		return -1;
+	}
 
 	if (rte_bus_scan())
 		rte_panic("Cannot scan the buses for devices\n");
Signature · 1 line
2.9.3
#19 lore raw

PATCH v2 19/25 eal: do not error if plugins fail to init

In reply to Aaron Conole

Plugins are useful and important. However, it seems crazy to abort everything just because they don't initialize properly.

---
 lib/librte_eal/linuxapp/eal/eal.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +3 -2
index 14ec2e0..2ca6c20 100644
@@ -878,8 +878,9 @@ rte_eal_init(int argc, char **argv)
 
 	eal_check_mem_on_local_socket();
 
-	if (eal_plugins_init() < 0)
-		rte_panic("Cannot init plugins\n");
+	if (eal_plugins_init() < 0) {
+		RTE_LOG(ERR, EAL, "Cannot init plugins\n");
+	}
 
 	eal_thread_init_master(rte_config.master_lcore);
 
Signature · 1 line
2.9.3
#20 lore raw

PATCH v2 20/25 eal_pci: Continue probing even on failures

In reply to Aaron Conole

Some devices may be inaccessible for a variety of reasons, or the PCI-bus may be unavailable causing the whole thing to fail. Still, better to continue attempts at probes.

---
 lib/librte_eal/common/eal_common_pci.c | 15 ++++++++++-----
 1 file changed, 10 insertions(+), 5 deletions(-)
lib/librte_eal/common/eal_common_pci.c +10 -5
index 72547bd..9416190 100644
@@ -69,6 +69,7 @@
 #include <sys/queue.h>
 #include <sys/mman.h>
 
+#include <rte_errno.h>
 #include <rte_interrupts.h>
 #include <rte_log.h>
 #include <rte_pci.h>
@@ -416,6 +417,7 @@ rte_eal_pci_probe(void)
 	struct rte_pci_device *dev = NULL;
 	struct rte_devargs *devargs;
 	int probe_all = 0;
+	int ret_1 = 0;
 	int ret = 0;
 
 	if (rte_eal_devargs_type_count(RTE_DEVTYPE_WHITELISTED_PCI) == 0)
@@ -430,17 +432,20 @@ rte_eal_pci_probe(void)
 
 		/* probe all or only whitelisted devices */
 		if (probe_all)
-			ret = pci_probe_all_drivers(dev);
+			ret_1 = pci_probe_all_drivers(dev);
 		else if (devargs != NULL &&
 			devargs->type == RTE_DEVTYPE_WHITELISTED_PCI)
-			ret = pci_probe_all_drivers(dev);
-		if (ret < 0)
-			rte_exit(EXIT_FAILURE, "Requested device " PCI_PRI_FMT
+			ret_1 = pci_probe_all_drivers(dev);
+		if (ret_1 < 0) {
+			RTE_LOG(ERR, EAL, "Requested device " PCI_PRI_FMT
 				 " cannot be used\n", dev->addr.domain, dev->addr.bus,
 				 dev->addr.devid, dev->addr.function);
+			rte_errno = errno;
+			ret = 1;
+		}
 	}
 
-	return 0;
+	return -ret;
 }
 
 /* dump one device */
Signature · 1 line
2.9.3
#21 lore raw

PATCH v2 21/25 eal: do not panic on failed PCI probe

In reply to Aaron Conole

It may even be possible to simply log the error and continue on letting the user check the logs and restart the application when things are failed.

---
 lib/librte_eal/linuxapp/eal/eal.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
index 2ca6c20..a692307 100644
@@ -939,8 +939,11 @@ rte_eal_init(int argc, char **argv)
 		rte_panic("Cannot probe devices\n");
 
 	/* Probe & Initialize PCI devices */
-	if (rte_eal_pci_probe())
-		rte_panic("Cannot probe PCI\n");
+	if (rte_eal_pci_probe()) {
+		RTE_LOG(ERR, EAL, "Cannot probe PCI\n");
+		rte_errno = ENOTSUP;
+		return -1;
+	}
 
 	if (rte_eal_dev_init() < 0)
 		rte_panic("Cannot init pmd devices\n");
Signature · 1 line
2.9.3
#22 lore raw

PATCH v2 22/25 eal_common_dev: continue initializing vdevs

In reply to Aaron Conole

Even if one vdev should fail, there's no need to prevent further processing. Log the error, and reflect it to the higher levels to decide.

---
 lib/librte_eal/common/eal_common_dev.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
lib/librte_eal/common/eal_common_dev.c +3 -2
index 4f3b493..9889997 100644
@@ -80,6 +80,7 @@ int
 rte_eal_dev_init(void)
 {
 	struct rte_devargs *devargs;
+	int ret = 0;
 
 	/*
 	 * Note that the dev_driver_list is populated here
@@ -97,11 +98,11 @@ rte_eal_dev_init(void)
 					devargs->args)) {
 			RTE_LOG(ERR, EAL, "failed to initialize %s device\n",
 					devargs->virt.drv_name);
-			return -1;
+			ret = -1;
 		}
 	}
 
-	return 0;
+	return ret;
 }
 
 int rte_eal_dev_attach(const char *name, const char *devargs)
Signature · 1 line
2.9.3
#23 lore raw

PATCH v2 23/25 eal: do not panic (or abort) if vdev init fails

In reply to Aaron Conole

Seems like it's possible to continue. At least, the error is reflected properly in the logs. A user could then go and correct or investigate the situation.

---
 lib/librte_eal/linuxapp/eal/eal.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal.c +1 -1
index a692307..d338e61 100644
@@ -946,7 +946,7 @@ rte_eal_init(int argc, char **argv)
 	}
 
 	if (rte_eal_dev_init() < 0)
-		rte_panic("Cannot init pmd devices\n");
+		RTE_LOG(ERR, EAL, "Cannot init pmd devices\n");
 
 	rte_eal_mcfg_complete();
 
Signature · 1 line
2.9.3
#24 lore raw

PATCH v2 24/25 eal: do not panic when bus probe fails

In reply to Aaron Conole
---
 lib/librte_eal/linuxapp/eal/eal.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
index d338e61..5018796 100644
@@ -935,8 +935,11 @@ rte_eal_init(int argc, char **argv)
 	rte_eal_mp_wait_lcore();
 
 	/* Probe all the buses and devices/drivers on them */
-	if (rte_bus_probe())
-		rte_panic("Cannot probe devices\n");
+	if (rte_bus_probe()) {
+		RTE_LOG(ERR, EAL, "Cannot probe devices\n");
+		rte_errno = ENOTSUP;
+		return -1;
+	}
 
 	/* Probe & Initialize PCI devices */
 	if (rte_eal_pci_probe()) {
Signature · 1 line
2.9.3
#25 lore raw

PATCH v2 25/25 rte_eal_init: add info about rte_errno codes

In reply to Aaron Conole

The rte_eal_init function will now pass failure reason hints to the application. To help app developers deciper this, add some brief information about what the codes are indicating.

---
 lib/librte_eal/common/include/rte_eal.h | 27 ++++++++++++++++++++++++++-
 1 file changed, 26 insertions(+), 1 deletion(-)
lib/librte_eal/common/include/rte_eal.h +26 -1
index 03fee50..5f184c9 100644
@@ -159,7 +159,32 @@ int rte_eal_iopl_init(void);
  *     function call and should not be further interpreted by the
  *     application.  The EAL does not take any ownership of the memory used
  *     for either the argv array, or its members.
- *   - On failure, a negative error value.
+ *   - On failure, -1 and rte_errno is set to a value indicating the cause
+ *     for failure.
+ *
+ *   The error codes returned via rte_errno:
+ *     EACCES indicates a permissions issue.
+ *
+ *     EAGAIN indicates either a bus or system resource was not available,
+ *            try again.
+ *
+ *     EALREADY indicates that the rte_eal_init function has already been
+ *              called, and cannot be called again.
+ *
+ *     EFAULT indicates the tailq configuration name was not found in
+ *            memory configuration.
+ *
+ *     EINVAL indicates invalid parameters were passed as argv/argc.
+ *
+ *     EIO indicates failure to setup the logging handlers.  This is usually
+ *         caused by an out-of-memory condition.
+ *
+ *     ENODEV indicates memory setup issues.
+ *
+ *     ENOTSUP indicates that the EAL cannot initialize on this system.
+ *
+ *     EUNATCH indicates that the PCI bus is either not present, or is not
+ *             readable by the eal.
  */
 int rte_eal_init(int argc, char **argv);
 
Signature · 1 line
2.9.3
#26 lore raw
In reply to Aaron Conole
Quoted 14 lines · Aaron Conole

In many cases, it's enough to simply let the application know that the call to initialize DPDK has failed. A complete halt can then be decided by the application based on error returned (and the app could even attempt a possible re-attempt after some corrective action by the user or application).

Changes ->v2:
- Audited all "RTE_LOG (" calls that were introduced, and converted to "RTE_LOG("
- Added some fprintf(stderr, "") lines to indicate errors before logging is initialized
- Removed assignments to errno.
- Changed patch 14/25 to reflect EFAULT, and document in 25/25

Quoted 3 lines

I kept the rte_errno reflection, since this is control-path code and the init function returns a sentinel value of -1.

I got 3 new checkpatch warnings that seem to have missed my local commit hook. I'll fix them up and send a new series in-reply to this one.

Sorry for the noise.

#27 lore raw

v3 cover linux/eal: Remove most causes of panic on init

In reply to Aaron Conole

In many cases, it's enough to simply let the application know that the call to initialize DPDK has failed. A complete halt can then be decided by the application based on error returned (and the app could even attempt a possible re-attempt after some corrective action by the user or application).

Changes ->v2:
- Audited all "RTE_LOG (" calls that were introduced, and converted to "RTE_LOG("
- Added some fprintf(stderr, "") lines to indicate errors before logging is initialized
- Removed assignments to errno.
- Changed patch 14/25 to reflect EFAULT, and document in 25/25

Changes ->v3:
- Checkpatch issues in patches 3 (spelling mistake), 9 (issue with leading spaces), and 19 (braces around single line statement if-condition)

I kept the rte_errno reflection, since this is control-path code and the init function returns a sentinel value of -1.

Aaron Conole (25):
eal: CPU init will no longer panic
eal: return error instead of panic for cpu init
eal: No panic on hugepages info init
eal: do not panic on failed hugepage query
eal: failure to parse args returns error
eal-common: introduce a way to query cpu support
eal: Signal error when CPU isn't supported
eal: do not panic on memzone initialization fails
eal: set errno when exiting for already called
eal: Do not panic on log failures
eal: Do not panic on pci-probe
eal: do not panic on vfio failure
eal: do not panic on memory init
eal: do not panic on tailq init
eal: do not panic on alarm init
eal: convert timer_init not to call panic
eal: change the private pipe call to reflect errno
eal: Do not panic on interrupt thread init
eal: do not error if plugins fail to init
eal_pci: Continue probing even on failures
eal: do not panic on failed PCI probe
eal_common_dev: continue initializing vdevs
eal: do not panic (or abort) if vdev init fails
eal: do not panic when bus probe fails
rte_eal_init: add info about rte_errno codes

 lib/librte_eal/common/eal_common_cpuflags.c        |  13 ++-
 lib/librte_eal/common/eal_common_dev.c             |   5 +-
 lib/librte_eal/common/eal_common_lcore.c           |   7 +-
 lib/librte_eal/common/eal_common_pci.c             |  15 ++-
 lib/librte_eal/common/eal_common_tailqs.c          |   3 +-
 .../common/include/generic/rte_cpuflags.h          |   9 ++
 lib/librte_eal/common/include/rte_eal.h            |  27 ++++-
 lib/librte_eal/linuxapp/eal/eal.c                  | 122 +++++++++++++++------
 lib/librte_eal/linuxapp/eal/eal_hugepage_info.c    |   6 +-
 lib/librte_eal/linuxapp/eal/eal_interrupts.c       |   5 +-
 10 files changed, 161 insertions(+), 51 deletions(-)
Signature · 1 line
2.9.3
#28 lore raw

PATCH v3 1/25 eal: CPU init will no longer panic

After this change, the EAL CPU NUMA node resolution step can no longer emit an rte_panic. This aligns with the code in rte_eal_init, which expects failures to return an error code.

---
 lib/librte_eal/common/eal_common_lcore.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
lib/librte_eal/common/eal_common_lcore.c +4 -3
index 2cd4132..84fa0cb 100644
@@ -83,16 +83,17 @@ rte_eal_cpu_init(void)
 		config->lcore_role[lcore_id] = ROLE_RTE;
 		lcore_config[lcore_id].core_id = eal_cpu_core_id(lcore_id);
 		lcore_config[lcore_id].socket_id = eal_cpu_socket_id(lcore_id);
-		if (lcore_config[lcore_id].socket_id >= RTE_MAX_NUMA_NODES)
+		if (lcore_config[lcore_id].socket_id >= RTE_MAX_NUMA_NODES) {
 #ifdef RTE_EAL_ALLOW_INV_SOCKET_ID
 			lcore_config[lcore_id].socket_id = 0;
 #else
-			rte_panic("Socket ID (%u) is greater than "
+			RTE_LOG(ERR, EAL, "Socket ID (%u) is greater than "
 				"RTE_MAX_NUMA_NODES (%d)\n",
 				lcore_config[lcore_id].socket_id,
 				RTE_MAX_NUMA_NODES);
+			return -1;
 #endif
-
+		}
 		RTE_LOG(DEBUG, EAL, "Detected lcore %u as "
 				"core %u on socket %u\n",
 				lcore_id, lcore_config[lcore_id].core_id,
Signature · 1 line
2.9.3
#29 lore raw

PATCH v3 2/25 eal: return error instead of panic for cpu init

In reply to Aaron Conole

There may be no way to gracefully recover, but the application should be notified that a failure happened, rather than completely aborting. This allows the user to proceed with a "slow-path" type solution.

---
 lib/librte_eal/linuxapp/eal/eal.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
index bf6b818..d8e00f5 100644
@@ -767,8 +767,11 @@ rte_eal_init(int argc, char **argv)
 	/* set log level as early as possible */
 	rte_set_log_level(internal_config.log_level);
 
-	if (rte_eal_cpu_init() < 0)
-		rte_panic("Cannot detect lcores\n");
+	if (rte_eal_cpu_init() < 0) {
+		RTE_LOG(ERR, EAL, "Cannot detect lcores\n");
+		rte_errno = ENOTSUP;
+		return -1;
+	}
 
 	fctret = eal_parse_args(argc, argv);
 	if (fctret < 0)
Signature · 1 line
2.9.3
#30 lore raw

PATCH v3 3/25 eal: No panic on hugepages info init

In reply to Aaron Conole

When attempting to scan hugepages, signal to the eal.c that an error has occurred, rather than performing a panic.

---
 lib/librte_eal/linuxapp/eal/eal_hugepage_info.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal_hugepage_info.c +4 -2
index 18858e2..4d47eaf 100644
@@ -283,9 +283,11 @@ eal_hugepage_info_init(void)
 	struct dirent *dirent;
 
 	dir = opendir(sys_dir_path);
-	if (dir == NULL)
-		rte_panic("Cannot open directory %s to read system hugepage "
+	if (dir == NULL) {
+		RTE_LOG(ERR, EAL, "Cannot open directory %s to read system hugepage "
 			  "info\n", sys_dir_path);
+		return -1;
+	}
 
 	for (dirent = readdir(dir); dirent != NULL; dirent = readdir(dir)) {
 		struct hugepage_info *hpi;
Signature · 1 line
2.9.3
#31 lore raw

PATCH v3 4/25 eal: do not panic on failed hugepage query

In reply to Aaron Conole

If we fail to acquire hugepage information, simply signal an error to the application. This clears the run_once counter, allowing the user or application to take a corrective action and retry.

---
 lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
index d8e00f5..8daa2be 100644
@@ -780,8 +780,12 @@ rte_eal_init(int argc, char **argv)
 	if (internal_config.no_hugetlbfs == 0 &&
 			internal_config.process_type != RTE_PROC_SECONDARY &&
 			internal_config.xen_dom0_support == 0 &&
-			eal_hugepage_info_init() < 0)
-		rte_panic("Cannot get hugepage information\n");
+			eal_hugepage_info_init() < 0) {
+		RTE_LOG(ERR, EAL, "Cannot get hugepage information\n");
+		rte_errno = EACCES;
+		rte_atomic32_clear(&run_once);
+		return -1;
+	}
 
 	if (internal_config.memory == 0 && internal_config.force_sockets == 0) {
 		if (internal_config.no_hugetlbfs)
Signature · 1 line
2.9.3
#32 lore raw

PATCH v3 5/25 eal: failure to parse args returns error

In reply to Aaron Conole

It's possible that the application could take a corrective action here, and either prompt the user for different arguments, or at least perform a better logging. Exiting this early prevents any useful information gathering from the application layer.

---
 lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
index 8daa2be..a626774 100644
@@ -774,8 +774,12 @@ rte_eal_init(int argc, char **argv)
 	}
 
 	fctret = eal_parse_args(argc, argv);
-	if (fctret < 0)
-		exit(1);
+	if (fctret < 0) {
+		RTE_LOG(ERR, EAL, "Invalid 'command line' arguments\n");
+		rte_errno = EINVAL;
+		rte_atomic32_clear(&run_once);
+		return -1;
+	}
 
 	if (internal_config.no_hugetlbfs == 0 &&
 			internal_config.process_type != RTE_PROC_SECONDARY &&
Signature · 1 line
2.9.3
#33 lore raw

PATCH v3 6/25 eal-common: introduce a way to query cpu support

In reply to Aaron Conole

This adds a new API to check for the eal cpu versions.

---
 lib/librte_eal/common/eal_common_cpuflags.c          | 13 +++++++++++--
 lib/librte_eal/common/include/generic/rte_cpuflags.h |  9 +++++++++
 2 files changed, 20 insertions(+), 2 deletions(-)
lib/librte_eal/common/eal_common_cpuflags.c +11 -2
index b5f76f7..2c2127b 100644
@@ -43,6 +43,13 @@
 void
 rte_cpu_check_supported(void)
 {
+	if (!rte_cpu_is_supported())
+		exit(1);
+}
+
+bool
+rte_cpu_is_supported(void)
+{
 	/* This is generated at compile-time by the build system */
 	static const enum rte_cpu_flag_t compile_time_flags[] = {
 			RTE_COMPILE_TIME_CPUFLAGS
@@ -57,14 +64,16 @@ rte_cpu_check_supported(void)
 			fprintf(stderr,
 				"ERROR: CPU feature flag lookup failed with error %d\n",
 				ret);
-			exit(1);
+			return false;
 		}
 		if (!ret) {
 			fprintf(stderr,
 			        "ERROR: This system does not support \"%s\".\n"
 			        "Please check that RTE_MACHINE is set correctly.\n",
 			        rte_cpu_get_flag_name(compile_time_flags[i]));
-			exit(1);
+			return false;
 		}
 	}
+
+	return true;
 }
lib/librte_eal/common/include/generic/rte_cpuflags.h +9 -0
index 71321f3..e4342ad 100644
@@ -40,6 +40,7 @@
  */
 
 #include <errno.h>
+#include <stdbool.h>
 
 /**
  * Enumeration of all CPU features supported
@@ -82,4 +83,12 @@ rte_cpu_get_flag_enabled(enum rte_cpu_flag_t feature);
 void
 rte_cpu_check_supported(void);
 
+/**
+ * This function checks that the currently used CPU supports the CPU features
+ * that were specified at compile time. It is called automatically within the
+ * EAL, so does not need to be used by applications.  This version returns a
+ * result so that decisions may be made (for instance, graceful shutdowns).
+ */
+bool
+rte_cpu_is_supported(void);
 #endif /* _RTE_CPUFLAGS_H_ */
Signature · 1 line
2.9.3
#34 lore raw

PATCH v3 7/25 eal: Signal error when CPU isn't supported

In reply to Aaron Conole

It's now possible to gracefully exit the application, or for applications which support non-dpdk datapaths working in concert with DPDK datapaths, there no longer is the possibility of exiting for unsupported CPUs.

---
 lib/librte_eal/linuxapp/eal/eal.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -1
index a626774..88d59a2 100644
@@ -752,7 +752,11 @@ rte_eal_init(int argc, char **argv)
 	char thread_name[RTE_MAX_THREAD_NAME_LEN];
 
 	/* checks if the machine is adequate */
-	rte_cpu_check_supported();
+	if (!rte_cpu_is_supported()) {
+		fprintf(stderr, "EAL: FATAL - unsupported cpu type.\n");
+		rte_errno = ENOTSUP;
+		return -1;
+	}
 
 	if (!rte_atomic32_test_and_set(&run_once))
 		return -1;
Signature · 1 line
2.9.3
#35 lore raw

PATCH v3 8/25 eal: do not panic on memzone initialization fails

In reply to Aaron Conole

When memzone initialization fails, report the error to the calling application rather than panic(). Without a good way of detaching / releasing hugepages, at this point the application will have to restart.

---
 lib/librte_eal/linuxapp/eal/eal.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
index 88d59a2..8f9bce1 100644
@@ -832,8 +832,11 @@ rte_eal_init(int argc, char **argv)
 	/* the directories are locked during eal_hugepage_info_init */
 	eal_hugedirs_unlock();
 
-	if (rte_eal_memzone_init() < 0)
-		rte_panic("Cannot init memzone\n");
+	if (rte_eal_memzone_init() < 0) {
+		RTE_LOG(ERR, EAL, "Cannot init memzone\n");
+		rte_errno = ENODEV;
+		return -1;
+	}
 
 	if (rte_eal_tailqs_init() < 0)
 		rte_panic("Cannot init tail queues for objects\n");
Signature · 1 line
2.9.3
#36 lore raw

PATCH v3 9/25 eal: set errno when exiting for already called

In reply to Aaron Conole
---
 lib/librte_eal/linuxapp/eal/eal.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal.c +4 -1
index 8f9bce1..e0dff6e 100644
@@ -758,8 +758,11 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 
-	if (!rte_atomic32_test_and_set(&run_once))
+	if (!rte_atomic32_test_and_set(&run_once)) {
+		fprintf(stderr, "EAL: ERROR - already called initialization.\n");
+		rte_errno = EALREADY;
 		return -1;
+	}
 
 	logid = strrchr(argv[0], '/');
 	logid = strdup(logid ? logid + 1: argv[0]);
Signature · 1 line
2.9.3
#37 lore raw

PATCH v3 10/25 eal: Do not panic on log failures

In reply to Aaron Conole

When log initialization fails, it's generally because the fopencookie failed. While this is rare in practice, it could happen, and it is likely because of memory pressure. So, flag the error, and allow the user to retry.

---
 lib/librte_eal/linuxapp/eal/eal.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +7 -2
index e0dff6e..9bb00d5 100644
@@ -818,8 +818,13 @@ rte_eal_init(int argc, char **argv)
 
 	rte_config_init();
 
-	if (rte_eal_log_init(logid, internal_config.syslog_facility) < 0)
-		rte_panic("Cannot init logs\n");
+	if (rte_eal_log_init(logid, internal_config.syslog_facility) < 0) {
+		RTE_LOG(ERR, EAL, "Cannot init logging\n");
+		fprintf(stderr, "EAL: ERROR - cannot init logging.\n");
+		rte_errno = EIO;
+		rte_atomic32_clear(&run_once);
+		return -1;
+	}
 
 	if (rte_eal_pci_init() < 0)
 		rte_panic("Cannot init PCI\n");
Signature · 1 line
2.9.3
#38 lore raw

PATCH v3 11/25 eal: Do not panic on pci-probe

In reply to Aaron Conole

This will usually be an issue because of permissions. However, it could also be caused by OOM. In either case, errno will contain the underlying cause. It is safe to re-init the system here, so allow the application to take corrective action and reinit.

---
 lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
index 9bb00d5..2a3d2f6 100644
@@ -826,8 +826,12 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 
-	if (rte_eal_pci_init() < 0)
-		rte_panic("Cannot init PCI\n");
+	if (rte_eal_pci_init() < 0) {
+		RTE_LOG(ERR, EAL, "Cannot init PCI\n");
+		rte_errno = EUNATCH;
+		rte_atomic32_clear(&run_once);
+		return -1;
+	}
 
 #ifdef VFIO_PRESENT
 	if (rte_eal_vfio_setup() < 0)
Signature · 1 line
2.9.3
#39 lore raw

PATCH v3 12/25 eal: do not panic on vfio failure

In reply to Aaron Conole
---
 lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
index 2a3d2f6..b21d715 100644
@@ -834,8 +834,12 @@ rte_eal_init(int argc, char **argv)
 	}
 
 #ifdef VFIO_PRESENT
-	if (rte_eal_vfio_setup() < 0)
-		rte_panic("Cannot init VFIO\n");
+	if (rte_eal_vfio_setup() < 0) {
+		RTE_LOG(ERR, EAL, "Cannot init VFIO\n");
+		rte_errno = EAGAIN;
+		rte_atomic32_clear(&run_once);
+		return -1;
+	}
 #endif
 
 	if (rte_eal_memory_init() < 0)
Signature · 1 line
2.9.3
#40 lore raw

PATCH v3 13/25 eal: do not panic on memory init

In reply to Aaron Conole

This can only happen when access to hugepages (either as primary or secondary process) fails (and that is usually permissions). Since the manner of failure is not reversible, we cannot allow retry.

---
 lib/librte_eal/linuxapp/eal/eal.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
index b21d715..97ee409 100644
@@ -842,8 +842,11 @@ rte_eal_init(int argc, char **argv)
 	}
 #endif
 
-	if (rte_eal_memory_init() < 0)
-		rte_panic("Cannot init memory\n");
+	if (rte_eal_memory_init() < 0) {
+		RTE_LOG(ERR, EAL, "Cannot init memory\n");
+		rte_errno = EACCES;
+		return -1;
+	}
 
 	/* the directories are locked during eal_hugepage_info_init */
 	eal_hugedirs_unlock();
Signature · 1 line
2.9.3
#41 lore raw

PATCH v3 14/25 eal: do not panic on tailq init

In reply to Aaron Conole

There are some theoretical racy conditions in the system that _could_ cause early tailq init to fail; however, no need to panic the application. While it can't continue using DPDK, it could make better alerts to the user.

---
 lib/librte_eal/common/eal_common_tailqs.c | 3 +--
 lib/librte_eal/linuxapp/eal/eal.c         | 7 +++++--
 2 files changed, 6 insertions(+), 4 deletions(-)
lib/librte_eal/common/eal_common_tailqs.c +1 -2
index bb08ec8..4f69828 100644
@@ -188,8 +188,7 @@ rte_eal_tailqs_init(void)
 		if (t->head == NULL) {
 			RTE_LOG(ERR, EAL,
 				"Cannot initialize tailq: %s\n", t->name);
-			/* no need to TAILQ_REMOVE, we are going to panic in
-			 * rte_eal_init() */
+			/* TAILQ_REMOVE not needed, error is already fatal */
 			goto fail;
 		}
 	}
lib/librte_eal/linuxapp/eal/eal.c +5 -2
index 97ee409..9e2daca 100644
@@ -857,8 +857,11 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 
-	if (rte_eal_tailqs_init() < 0)
-		rte_panic("Cannot init tail queues for objects\n");
+	if (rte_eal_tailqs_init() < 0) {
+		RTE_LOG(ERR, EAL, "Cannot init tail queues for objects\n");
+		rte_errno = EFAULT;
+		return -1;
+	}
 
 	if (rte_eal_alarm_init() < 0)
 		rte_panic("Cannot init interrupt-handling thread\n");
Signature · 1 line
2.9.3
#42 lore raw

PATCH v3 15/25 eal: do not panic on alarm init

In reply to Aaron Conole

rte_eal_alarm_init() call uses the linux timerfd framework to create a poll()-able timer using standard posix file operations. This could fail for a few reasons given in the man-pages, but many could be corrected by the user application. No need to panic.

---
 lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
index 9e2daca..5dcd2d5 100644
@@ -61,6 +61,7 @@
 #include <rte_launch.h>
 #include <rte_eal.h>
 #include <rte_eal_memconfig.h>
+#include <rte_errno.h>
 #include <rte_per_lcore.h>
 #include <rte_lcore.h>
 #include <rte_log.h>
@@ -863,8 +864,11 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 
-	if (rte_eal_alarm_init() < 0)
-		rte_panic("Cannot init interrupt-handling thread\n");
+	if (rte_eal_alarm_init() < 0) {
+		RTE_LOG(ERR, EAL, "Cannot init interrupt-handling thread\n");
+		/* rte_eal_alarm_init sets rte_errno on failure. */
+		return -1;
+	}
 
 	if (rte_eal_timer_init() < 0)
 		rte_panic("Cannot init HPET or TSC timers\n");
Signature · 1 line
2.9.3
#43 lore raw

PATCH v3 16/25 eal: convert timer_init not to call panic

In reply to Aaron Conole

After code inspection, there is no way for eal_timer_init() to fail. It simply returns 0 in all cases. As such, this test could either go-away or stay here as 'future-proofing'.

---
 lib/librte_eal/linuxapp/eal/eal.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
index 5dcd2d5..805f284 100644
@@ -870,8 +870,11 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 
-	if (rte_eal_timer_init() < 0)
-		rte_panic("Cannot init HPET or TSC timers\n");
+	if (rte_eal_timer_init() < 0) {
+		RTE_LOG(ERR, EAL, "Cannot init HPET or TSC timers\n");
+		rte_errno = ENOTSUP;
+		return -1;
+	}
 
 	eal_check_mem_on_local_socket();
 
Signature · 1 line
2.9.3
#44 lore raw

PATCH v3 17/25 eal: change the private pipe call to reflect errno

In reply to Aaron Conole

There could be some confusion as to why the call failed - this change will always reflect the value of the error in rte_error.

---
 lib/librte_eal/linuxapp/eal/eal_interrupts.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal_interrupts.c +4 -1
index b5b3f2b..b1a287c 100644
@@ -896,13 +896,16 @@ rte_eal_intr_init(void)
 	 * create a pipe which will be waited by epoll and notified to
 	 * rebuild the wait list of epoll.
 	 */
-	if (pipe(intr_pipe.pipefd) < 0)
+	if (pipe(intr_pipe.pipefd) < 0) {
+		rte_errno = errno;
 		return -1;
+	}
 
 	/* create the host thread to wait/handle the interrupt */
 	ret = pthread_create(&intr_thread, NULL,
 			eal_intr_thread_main, NULL);
 	if (ret != 0) {
+		rte_errno = ret;
 		RTE_LOG(ERR, EAL,
 			"Failed to create thread for interrupt handling\n");
 	} else {
Signature · 1 line
2.9.3
#45 lore raw

PATCH v3 18/25 eal: Do not panic on interrupt thread init

In reply to Aaron Conole

When initializing the interrupt thread, there are a number of possible reasons for failure - some of which are correctable by the application. Do not panic() needlessly, and give the application a change to reflect this information to the user.

---
 lib/librte_eal/linuxapp/eal/eal.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +4 -2
index 805f284..93b83aa 100644
@@ -889,8 +889,10 @@ rte_eal_init(int argc, char **argv)
 		rte_config.master_lcore, (int)thread_id, cpuset,
 		ret == 0 ? "" : "...");
 
-	if (rte_eal_intr_init() < 0)
-		rte_panic("Cannot init interrupt-handling thread\n");
+	if (rte_eal_intr_init() < 0) {
+		RTE_LOG(ERR, EAL, "Cannot init interrupt-handling thread\n");
+		return -1;
+	}
 
 	if (rte_bus_scan())
 		rte_panic("Cannot scan the buses for devices\n");
Signature · 1 line
2.9.3
#46 lore raw

PATCH v3 19/25 eal: do not error if plugins fail to init

In reply to Aaron Conole

Plugins are useful and important. However, it seems crazy to abort everything just because they don't initialize properly.

---
 lib/librte_eal/linuxapp/eal/eal.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal.c +1 -1
index 93b83aa..31a5986 100644
@@ -879,7 +879,7 @@ rte_eal_init(int argc, char **argv)
 	eal_check_mem_on_local_socket();
 
 	if (eal_plugins_init() < 0)
-		rte_panic("Cannot init plugins\n");
+		RTE_LOG(ERR, EAL, "Cannot init plugins\n");
 
 	eal_thread_init_master(rte_config.master_lcore);
 
Signature · 1 line
2.9.3
#47 lore raw

PATCH v3 20/25 eal_pci: Continue probing even on failures

In reply to Aaron Conole

Some devices may be inaccessible for a variety of reasons, or the PCI-bus may be unavailable causing the whole thing to fail. Still, better to continue attempts at probes.

---
 lib/librte_eal/common/eal_common_pci.c | 15 ++++++++++-----
 1 file changed, 10 insertions(+), 5 deletions(-)
lib/librte_eal/common/eal_common_pci.c +10 -5
index 72547bd..9416190 100644
@@ -69,6 +69,7 @@
 #include <sys/queue.h>
 #include <sys/mman.h>
 
+#include <rte_errno.h>
 #include <rte_interrupts.h>
 #include <rte_log.h>
 #include <rte_pci.h>
@@ -416,6 +417,7 @@ rte_eal_pci_probe(void)
 	struct rte_pci_device *dev = NULL;
 	struct rte_devargs *devargs;
 	int probe_all = 0;
+	int ret_1 = 0;
 	int ret = 0;
 
 	if (rte_eal_devargs_type_count(RTE_DEVTYPE_WHITELISTED_PCI) == 0)
@@ -430,17 +432,20 @@ rte_eal_pci_probe(void)
 
 		/* probe all or only whitelisted devices */
 		if (probe_all)
-			ret = pci_probe_all_drivers(dev);
+			ret_1 = pci_probe_all_drivers(dev);
 		else if (devargs != NULL &&
 			devargs->type == RTE_DEVTYPE_WHITELISTED_PCI)
-			ret = pci_probe_all_drivers(dev);
-		if (ret < 0)
-			rte_exit(EXIT_FAILURE, "Requested device " PCI_PRI_FMT
+			ret_1 = pci_probe_all_drivers(dev);
+		if (ret_1 < 0) {
+			RTE_LOG(ERR, EAL, "Requested device " PCI_PRI_FMT
 				 " cannot be used\n", dev->addr.domain, dev->addr.bus,
 				 dev->addr.devid, dev->addr.function);
+			rte_errno = errno;
+			ret = 1;
+		}
 	}
 
-	return 0;
+	return -ret;
 }
 
 /* dump one device */
Signature · 1 line
2.9.3
#48 lore raw

PATCH v3 21/25 eal: do not panic on failed PCI probe

In reply to Aaron Conole

It may even be possible to simply log the error and continue on letting the user check the logs and restart the application when things are failed.

---
 lib/librte_eal/linuxapp/eal/eal.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
index 31a5986..3458d41 100644
@@ -938,8 +938,11 @@ rte_eal_init(int argc, char **argv)
 		rte_panic("Cannot probe devices\n");
 
 	/* Probe & Initialize PCI devices */
-	if (rte_eal_pci_probe())
-		rte_panic("Cannot probe PCI\n");
+	if (rte_eal_pci_probe()) {
+		RTE_LOG(ERR, EAL, "Cannot probe PCI\n");
+		rte_errno = ENOTSUP;
+		return -1;
+	}
 
 	if (rte_eal_dev_init() < 0)
 		rte_panic("Cannot init pmd devices\n");
Signature · 1 line
2.9.3
#49 lore raw

PATCH v3 22/25 eal_common_dev: continue initializing vdevs

In reply to Aaron Conole

Even if one vdev should fail, there's no need to prevent further processing. Log the error, and reflect it to the higher levels to decide.

---
 lib/librte_eal/common/eal_common_dev.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
lib/librte_eal/common/eal_common_dev.c +3 -2
index 4f3b493..9889997 100644
@@ -80,6 +80,7 @@ int
 rte_eal_dev_init(void)
 {
 	struct rte_devargs *devargs;
+	int ret = 0;
 
 	/*
 	 * Note that the dev_driver_list is populated here
@@ -97,11 +98,11 @@ rte_eal_dev_init(void)
 					devargs->args)) {
 			RTE_LOG(ERR, EAL, "failed to initialize %s device\n",
 					devargs->virt.drv_name);
-			return -1;
+			ret = -1;
 		}
 	}
 
-	return 0;
+	return ret;
 }
 
 int rte_eal_dev_attach(const char *name, const char *devargs)
Signature · 1 line
2.9.3
#50 lore raw

PATCH v3 23/25 eal: do not panic (or abort) if vdev init fails

In reply to Aaron Conole

Seems like it's possible to continue. At least, the error is reflected properly in the logs. A user could then go and correct or investigate the situation.

---
 lib/librte_eal/linuxapp/eal/eal.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal.c +1 -1
index 3458d41..be03b63 100644
@@ -945,7 +945,7 @@ rte_eal_init(int argc, char **argv)
 	}
 
 	if (rte_eal_dev_init() < 0)
-		rte_panic("Cannot init pmd devices\n");
+		RTE_LOG(ERR, EAL, "Cannot init pmd devices\n");
 
 	rte_eal_mcfg_complete();
 
Signature · 1 line
2.9.3
#51 lore raw

PATCH v3 24/25 eal: do not panic when bus probe fails

In reply to Aaron Conole
---
 lib/librte_eal/linuxapp/eal/eal.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
index be03b63..57c41e0 100644
@@ -934,8 +934,11 @@ rte_eal_init(int argc, char **argv)
 	rte_eal_mp_wait_lcore();
 
 	/* Probe all the buses and devices/drivers on them */
-	if (rte_bus_probe())
-		rte_panic("Cannot probe devices\n");
+	if (rte_bus_probe()) {
+		RTE_LOG(ERR, EAL, "Cannot probe devices\n");
+		rte_errno = ENOTSUP;
+		return -1;
+	}
 
 	/* Probe & Initialize PCI devices */
 	if (rte_eal_pci_probe()) {
Signature · 1 line
2.9.3
#52 lore raw

PATCH v3 25/25 rte_eal_init: add info about rte_errno codes

In reply to Aaron Conole

The rte_eal_init function will now pass failure reason hints to the application. To help app developers deciper this, add some brief information about what the codes are indicating.

---
 lib/librte_eal/common/include/rte_eal.h | 27 ++++++++++++++++++++++++++-
 1 file changed, 26 insertions(+), 1 deletion(-)
lib/librte_eal/common/include/rte_eal.h +26 -1
index 03fee50..5f184c9 100644
@@ -159,7 +159,32 @@ int rte_eal_iopl_init(void);
  *     function call and should not be further interpreted by the
  *     application.  The EAL does not take any ownership of the memory used
  *     for either the argv array, or its members.
- *   - On failure, a negative error value.
+ *   - On failure, -1 and rte_errno is set to a value indicating the cause
+ *     for failure.
+ *
+ *   The error codes returned via rte_errno:
+ *     EACCES indicates a permissions issue.
+ *
+ *     EAGAIN indicates either a bus or system resource was not available,
+ *            try again.
+ *
+ *     EALREADY indicates that the rte_eal_init function has already been
+ *              called, and cannot be called again.
+ *
+ *     EFAULT indicates the tailq configuration name was not found in
+ *            memory configuration.
+ *
+ *     EINVAL indicates invalid parameters were passed as argv/argc.
+ *
+ *     EIO indicates failure to setup the logging handlers.  This is usually
+ *         caused by an out-of-memory condition.
+ *
+ *     ENODEV indicates memory setup issues.
+ *
+ *     ENOTSUP indicates that the EAL cannot initialize on this system.
+ *
+ *     EUNATCH indicates that the PCI bus is either not present, or is not
+ *             readable by the eal.
  */
 int rte_eal_init(int argc, char **argv);
 
Signature · 1 line
2.9.3
#53 lore raw

Re: [PATCH v3 25/25] rte_eal_init: add info about rte_errno codes

Quoted 16 lines · On Thu, 9 Feb 2017 09:29:53 -0500 Aaron Conole <[email protected]> wrote:

+ * The error codes returned via rte_errno:
+ * EACCES indicates a permissions issue.
+ *
+ * EAGAIN indicates either a bus or system resource was not available,
+ * try again.
+ *
+ * EALREADY indicates that the rte_eal_init function has already been
+ * called, and cannot be called again.
+ *
+ * EFAULT indicates the tailq configuration name was not found in
+ * memory configuration.
+ *
+ * EINVAL indicates invalid parameters were passed as argv/argc.
+ *
+ * EIO indicates failure to setup the logging handlers. This is usually
+ * caused by an out-of-memory condition.

Quoted 8 lines

+ *
+ * ENODEV indicates memory setup issues.
+ *
+ * ENOTSUP indicates that the EAL cannot initialize on this system.
+ *
+ * EUNATCH indicates that the PCI bus is either not present, or is not
+ * readable by the eal.
*/

You might want to be less restrictive about wording in the comment. In future more errors might be returned, and also for out of memory ENOMEM is better.

#54 lore raw

Re: [PATCH v3 25/25] rte_eal_init: add info about rte_errno codes

Quoted 28 lines · Stephen Hemminger
Quoted 16 lines · On Thu, 9 Feb 2017 09:29:53 -0500 Aaron Conole <[email protected]> wrote:

+ * The error codes returned via rte_errno:
+ * EACCES indicates a permissions issue.
+ *
+ * EAGAIN indicates either a bus or system resource was not available,
+ * try again.
+ *
+ * EALREADY indicates that the rte_eal_init function has already been
+ * called, and cannot be called again.
+ *
+ * EFAULT indicates the tailq configuration name was not found in
+ * memory configuration.
+ *
+ * EINVAL indicates invalid parameters were passed as argv/argc.
+ *
+ * EIO indicates failure to setup the logging handlers. This is usually
+ * caused by an out-of-memory condition.

Quoted 8 lines

+ *
+ * ENODEV indicates memory setup issues.
+ *
+ * ENOTSUP indicates that the EAL cannot initialize on this system.
+ *
+ * EUNATCH indicates that the PCI bus is either not present, or is not
+ * readable by the eal.
*/

Quoted 3 lines

You might want to be less restrictive about wording in the comment. In future more errors might be returned, and also for out of memory ENOMEM is better.

Sure thing, I'll switch EIO and ENODEV to ENOMEM, does that make sense?

Also, which message do you refer to? Is it "The error codes returned via rte_errno" section? I assume that adding new error codes will also bring an update to the eal_init documentation, but perhaps I'm misunderstanding.

Thanks for your review, Stephen!

#55 lore raw
In reply to Aaron Conole
Quoted 48 lines · On Thu, 9 Feb 2017 09:29:28 -0500 Aaron Conole <[email protected]> wrote:

In many cases, it's enough to simply let the application know that the call to initialize DPDK has failed. A complete halt can then be decided by the application based on error returned (and the app could even attempt a possible re-attempt after some corrective action by the user or application).

Changes ->v2:
- Audited all "RTE_LOG (" calls that were introduced, and converted to "RTE_LOG("
- Added some fprintf(stderr, "") lines to indicate errors before logging is initialized
- Removed assignments to errno.
- Changed patch 14/25 to reflect EFAULT, and document in 25/25

Changes ->v3:
- Checkpatch issues in patches 3 (spelling mistake), 9 (issue with leading spaces), and 19 (braces around single line statement if-condition)

I kept the rte_errno reflection, since this is control-path code and the init function returns a sentinel value of -1.

Aaron Conole (25):
eal: CPU init will no longer panic
eal: return error instead of panic for cpu init
eal: No panic on hugepages info init
eal: do not panic on failed hugepage query
eal: failure to parse args returns error
eal-common: introduce a way to query cpu support
eal: Signal error when CPU isn't supported
eal: do not panic on memzone initialization fails
eal: set errno when exiting for already called
eal: Do not panic on log failures
eal: Do not panic on pci-probe
eal: do not panic on vfio failure
eal: do not panic on memory init
eal: do not panic on tailq init
eal: do not panic on alarm init
eal: convert timer_init not to call panic
eal: change the private pipe call to reflect errno
eal: Do not panic on interrupt thread init
eal: do not error if plugins fail to init
eal_pci: Continue probing even on failures
eal: do not panic on failed PCI probe
eal_common_dev: continue initializing vdevs
eal: do not panic (or abort) if vdev init fails
eal: do not panic when bus probe fails
rte_eal_init: add info about rte_errno codes

Quoted 12 lines
 lib/librte_eal/common/eal_common_cpuflags.c        |  13 ++-
 lib/librte_eal/common/eal_common_dev.c             |   5 +-
 lib/librte_eal/common/eal_common_lcore.c           |   7 +-
 lib/librte_eal/common/eal_common_pci.c             |  15 ++-
 lib/librte_eal/common/eal_common_tailqs.c          |   3 +-
 .../common/include/generic/rte_cpuflags.h          |   9 ++
 lib/librte_eal/common/include/rte_eal.h            |  27 ++++-
 lib/librte_eal/linuxapp/eal/eal.c                  | 122 +++++++++++++++------
 lib/librte_eal/linuxapp/eal/eal_hugepage_info.c    |   6 +-
 lib/librte_eal/linuxapp/eal/eal_interrupts.c       |   5 +-
 10 files changed, 161 insertions(+), 51 deletions(-)

I worry that some of these early failure messages may never be visible because the logging system has not been initialized. Might be safer to just use fprintf(stderr, ...) or define a new wrapper function.

#56 lore raw
Quoted 9 lines · Stephen Hemminger
Quoted 6 lines · On Thu, 9 Feb 2017 09:29:28 -0500 Aaron Conole <[email protected]> wrote:

In many cases, it's enough to simply let the application know that the call to initialize DPDK has failed. A complete halt can then be decided by the application based on error returned (and the app could even attempt a possible re-attempt after some corrective action by the user or application).

Quoted 6 lines
Quoted 2 lines

...

I worry that some of these early failure messages may never be visible because the logging system has not been initialized. Might be safer to just use fprintf(stderr, ...) or define a new wrapper function.

Thanks for the suggestion, Stephen! I've folded it into my series.

-Aaron

#57 lore raw

v4 cover linux/eal: Remove most causes of panic on init

In reply to Aaron Conole

In many cases, it's enough to simply let the application know that the call to initialize DPDK has failed. A complete halt can then be decided by the application based on error returned (and the app could even attempt a possible re-attempt after some corrective action by the user or application).

Changes ->v2:
- Audited all "RTE_LOG (" calls that were introduced, and converted to "RTE_LOG("
- Added some fprintf(stderr, "") lines to indicate errors before logging is initialized
- Removed assignments to errno.
- Changed patch 14/25 to reflect EFAULT, and document in 25/25

Changes ->v3:
- Checkpatch issues in patches 3 (spelling mistake), 9 (issue with leading spaces), and 19 (braces around single line statement if-condition)

Changes ->v4:
- Error text cleanup.
- Add a new check around rte_bus_scan(), added during the development of this series.

I kept the rte_errno reflection, since this is control-path code and the init function returns a sentinel value of -1.

Aaron Conole (26):
eal: CPU init will no longer panic
eal: return error instead of panic for cpu init
eal: No panic on hugepages info init
eal: do not panic on failed hugepage query
eal: failure to parse args returns error
eal-common: introduce a way to query cpu support
eal: Signal error when CPU isn't supported
eal: do not panic on memzone initialization fails
eal: set errno when exiting for already called
eal: Do not panic on log failures
eal: Do not panic on pci-probe
eal: do not panic on vfio failure
eal: do not panic on memory init
eal: do not panic on tailq init
eal: do not panic on alarm init
eal: convert timer_init not to call panic
eal: change the private pipe call to reflect errno
eal: Do not panic on interrupt thread init
eal: do not error if plugins fail to init
eal_pci: Continue probing even on failures
eal: do not panic on failed PCI probe
eal_common_dev: continue initializing vdevs
eal: do not panic (or abort) if vdev init fails
eal: do not panic when bus probe fails
eal: do not panic on failed bus scan
rte_eal_init: add info about rte_errno codes

 lib/librte_eal/common/eal_common_cpuflags.c        |  13 +-
 lib/librte_eal/common/eal_common_dev.c             |   5 +-
 lib/librte_eal/common/eal_common_lcore.c           |   7 +-
 lib/librte_eal/common/eal_common_pci.c             |  15 ++-
 lib/librte_eal/common/eal_common_tailqs.c          |   3 +-
 .../common/include/generic/rte_cpuflags.h          |   9 ++
 lib/librte_eal/common/include/rte_eal.h            |  27 ++++-
 lib/librte_eal/linuxapp/eal/eal.c                  | 131 +++++++++++++++------
 lib/librte_eal/linuxapp/eal/eal_hugepage_info.c    |   6 +-
 lib/librte_eal/linuxapp/eal/eal_interrupts.c       |   5 +-
 10 files changed, 169 insertions(+), 52 deletions(-)
Signature · 1 line
2.9.3
#58 lore raw

PATCH v4 1/26 eal: CPU init will no longer panic

After this change, the EAL CPU NUMA node resolution step can no longer emit an rte_panic. This aligns with the code in rte_eal_init, which expects failures to return an error code.

---
 lib/librte_eal/common/eal_common_lcore.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
lib/librte_eal/common/eal_common_lcore.c +4 -3
index 2cd4132..84fa0cb 100644
@@ -83,16 +83,17 @@ rte_eal_cpu_init(void)
 		config->lcore_role[lcore_id] = ROLE_RTE;
 		lcore_config[lcore_id].core_id = eal_cpu_core_id(lcore_id);
 		lcore_config[lcore_id].socket_id = eal_cpu_socket_id(lcore_id);
-		if (lcore_config[lcore_id].socket_id >= RTE_MAX_NUMA_NODES)
+		if (lcore_config[lcore_id].socket_id >= RTE_MAX_NUMA_NODES) {
 #ifdef RTE_EAL_ALLOW_INV_SOCKET_ID
 			lcore_config[lcore_id].socket_id = 0;
 #else
-			rte_panic("Socket ID (%u) is greater than "
+			RTE_LOG(ERR, EAL, "Socket ID (%u) is greater than "
 				"RTE_MAX_NUMA_NODES (%d)\n",
 				lcore_config[lcore_id].socket_id,
 				RTE_MAX_NUMA_NODES);
+			return -1;
 #endif
-
+		}
 		RTE_LOG(DEBUG, EAL, "Detected lcore %u as "
 				"core %u on socket %u\n",
 				lcore_id, lcore_config[lcore_id].core_id,
Signature · 1 line
2.9.3
#59 lore raw

PATCH v4 2/26 eal: return error instead of panic for cpu init

In reply to Aaron Conole

There may be no way to gracefully recover, but the application should be notified that a failure happened, rather than completely aborting. This allows the user to proceed with a "slow-path" type solution.

---
 lib/librte_eal/linuxapp/eal/eal.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +11 -2
index bf6b818..5023d0d 100644
@@ -740,6 +740,12 @@ static int rte_eal_vfio_setup(void)
 }
 #endif
 
+static void rte_eal_init_alert(const char *msg)
+{
+    fprintf(stderr, "EAL: FATAL: %s\n", msg);
+    RTE_LOG(ERR, EAL, "%s\n", msg);
+}
+
 /* Launch threads, called at application init(). */
 int
 rte_eal_init(int argc, char **argv)
@@ -767,8 +773,11 @@ rte_eal_init(int argc, char **argv)
 	/* set log level as early as possible */
 	rte_set_log_level(internal_config.log_level);
 
-	if (rte_eal_cpu_init() < 0)
-		rte_panic("Cannot detect lcores\n");
+	if (rte_eal_cpu_init() < 0) {
+		rte_eal_init_alert("Cannot detect lcores.");
+		rte_errno = ENOTSUP;
+		return -1;
+	}
 
 	fctret = eal_parse_args(argc, argv);
 	if (fctret < 0)
Signature · 1 line
2.9.3
#60 lore raw

Re: [PATCH v4 02/26] eal: return error instead of panic for cpu init

Quoted 32 lines · Aaron Conole

There may be no way to gracefully recover, but the application should be notified that a failure happened, rather than completely aborting. This allows the user to proceed with a "slow-path" type solution.

---
 lib/librte_eal/linuxapp/eal/eal.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -0
index bf6b818..5023d0d 100644
@@ -740,6 +740,12 @@ static int rte_eal_vfio_setup(void)
 }
 #endif
 
+static void rte_eal_init_alert(const char *msg)
+{
+    fprintf(stderr, "EAL: FATAL: %s\n", msg);
+    RTE_LOG(ERR, EAL, "%s\n", msg);
+}
+
 /* Launch threads, called at application init(). */
 int
 rte_eal_init(int argc, char **argv)
@@ -767,8 +773,11 @@ rte_eal_init(int argc, char **argv)
 	/* set log level as early as possible */
 	rte_set_log_level(internal_config.log_level);
 

lib/librte_eal/linuxapp/eal/eal.c rte_eal_init(int argc, char **argv) · line 783

Quoted 10 lines
lib/librte_eal/linuxapp/eal/eal.c +5 -2
-	if (rte_eal_cpu_init() < 0)
-		rte_panic("Cannot detect lcores\n");
+	if (rte_eal_cpu_init() < 0) {
+		rte_eal_init_alert("Cannot detect lcores.");
+		rte_errno = ENOTSUP;
+		return -1;
+	}
 
 	fctret = eal_parse_args(argc, argv);
 	if (fctret < 0)

eal.c needs to include rte_errno.h after this change, otherwise it won't compile.

/Bruce

#61 lore raw

Re: [PATCH v4 02/26] eal: return error instead of panic for cpu init

Quoted 40 lines · Bruce Richardson
Quoted 39 lines · Aaron Conole

There may be no way to gracefully recover, but the application should be notified that a failure happened, rather than completely aborting. This allows the user to proceed with a "slow-path" type solution.

---
 lib/librte_eal/linuxapp/eal/eal.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +11 -2
index bf6b818..5023d0d 100644
@@ -740,6 +740,12 @@ static int rte_eal_vfio_setup(void)
 }
 #endif
 
+static void rte_eal_init_alert(const char *msg)
+{
+    fprintf(stderr, "EAL: FATAL: %s\n", msg);
+    RTE_LOG(ERR, EAL, "%s\n", msg);
+}
+
 /* Launch threads, called at application init(). */
 int
 rte_eal_init(int argc, char **argv)
@@ -767,8 +773,11 @@ rte_eal_init(int argc, char **argv)
 	/* set log level as early as possible */
 	rte_set_log_level(internal_config.log_level);
 
-	if (rte_eal_cpu_init() < 0)
-		rte_panic("Cannot detect lcores\n");
+	if (rte_eal_cpu_init() < 0) {
+		rte_eal_init_alert("Cannot detect lcores.");
+		rte_errno = ENOTSUP;
+		return -1;
+	}
 
Quoted 5 lines

lib/librte_eal/linuxapp/eal/eal.c rte_eal_init(int argc, char **argv) · line 783

Quoted 3 lines
lib/librte_eal/linuxapp/eal/eal.c +0 -0
 	fctret = eal_parse_args(argc, argv);
 	if (fctret < 0)

eal.c needs to include rte_errno.h after this change, otherwise it won't compile.

oops.. I reordered some of my original work, and the rte_errno.h change was introduced too late. Thanks for catching this!

Quoted 1 line

/Bruce

#62 lore raw

Re: [PATCH v4 02/26] eal: return error instead of panic for cpu init

In reply to Aaron Conole
Quoted 15 lines · Aaron Conole

There may be no way to gracefully recover, but the application should be notified that a failure happened, rather than completely aborting. This allows the user to proceed with a "slow-path" type solution.

---
 lib/librte_eal/linuxapp/eal/eal.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +0 -0
index bf6b818..5023d0d 100644

lib/librte_eal/linuxapp/eal/eal.c static int rte_eal_vfio_setup(void) · line 747

Quoted 8 lines
lib/librte_eal/linuxapp/eal/eal.c +5 -0
@@ -740,6 +740,12 @@ static int rte_eal_vfio_setup(void)
 }
 #endif
 
+static void rte_eal_init_alert(const char *msg)
+{
+    fprintf(stderr, "EAL: FATAL: %s\n", msg);
+    RTE_LOG(ERR, EAL, "%s\n", msg);
+}

Checkpatch flags the use of spaces rather than tabs here.

/Bruce

#63 lore raw

Re: [PATCH v4 02/26] eal: return error instead of panic for cpu init

Quoted 20 lines · Bruce Richardson
Quoted 19 lines · Aaron Conole

There may be no way to gracefully recover, but the application should be notified that a failure happened, rather than completely aborting. This allows the user to proceed with a "slow-path" type solution.

---
 lib/librte_eal/linuxapp/eal/eal.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +0 -0
index bf6b818..5023d0d 100644
@@ -740,6 +740,12 @@ static int rte_eal_vfio_setup(void)
 }
 #endif
 
Quoted 6 lines

lib/librte_eal/linuxapp/eal/eal.c static int rte_eal_vfio_setup(void) · line 747

Quoted 5 lines
lib/librte_eal/linuxapp/eal/eal.c +5 -0
+static void rte_eal_init_alert(const char *msg)
+{
+    fprintf(stderr, "EAL: FATAL: %s\n", msg);
+    RTE_LOG(ERR, EAL, "%s\n", msg);
+}

Checkpatch flags the use of spaces rather than tabs here.

Yes, I caught it too late. Sorry, I'll fix it.

Quoted 1 line

/Bruce

#64 lore raw

PATCH v4 3/26 eal: No panic on hugepages info init

In reply to Aaron Conole

When attempting to scan hugepages, signal to the eal.c that an error has occurred, rather than performing a panic.

---
 lib/librte_eal/linuxapp/eal/eal_hugepage_info.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal_hugepage_info.c +4 -2
index 18858e2..4d47eaf 100644
@@ -283,9 +283,11 @@ eal_hugepage_info_init(void)
 	struct dirent *dirent;
 
 	dir = opendir(sys_dir_path);
-	if (dir == NULL)
-		rte_panic("Cannot open directory %s to read system hugepage "
+	if (dir == NULL) {
+		RTE_LOG(ERR, EAL, "Cannot open directory %s to read system hugepage "
 			  "info\n", sys_dir_path);
+		return -1;
+	}
 
 	for (dirent = readdir(dir); dirent != NULL; dirent = readdir(dir)) {
 		struct hugepage_info *hpi;
Signature · 1 line
2.9.3
#65 lore raw

PATCH v4 4/26 eal: do not panic on failed hugepage query

In reply to Aaron Conole

If we fail to acquire hugepage information, simply signal an error to the application. This clears the run_once counter, allowing the user or application to take a corrective action and retry.

---
 lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
index 5023d0d..c76ed94 100644
@@ -786,8 +786,12 @@ rte_eal_init(int argc, char **argv)
 	if (internal_config.no_hugetlbfs == 0 &&
 			internal_config.process_type != RTE_PROC_SECONDARY &&
 			internal_config.xen_dom0_support == 0 &&
-			eal_hugepage_info_init() < 0)
-		rte_panic("Cannot get hugepage information\n");
+			eal_hugepage_info_init() < 0) {
+		rte_eal_init_alert("Cannot get hugepage information.");
+		rte_errno = EACCES;
+		rte_atomic32_clear(&run_once);
+		return -1;
+	}
 
 	if (internal_config.memory == 0 && internal_config.force_sockets == 0) {
 		if (internal_config.no_hugetlbfs)
Signature · 1 line
2.9.3
#66 lore raw

PATCH v4 5/26 eal: failure to parse args returns error

In reply to Aaron Conole

It's possible that the application could take a corrective action here, and either prompt the user for different arguments, or at least perform a better logging. Exiting this early prevents any useful information gathering from the application layer.

---
 lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
index c76ed94..0f682e1 100644
@@ -780,8 +780,12 @@ rte_eal_init(int argc, char **argv)
 	}
 
 	fctret = eal_parse_args(argc, argv);
-	if (fctret < 0)
-		exit(1);
+	if (fctret < 0) {
+		rte_eal_init_alert("Invalid 'command line' arguments.");
+		rte_errno = EINVAL;
+		rte_atomic32_clear(&run_once);
+		return -1;
+	}
 
 	if (internal_config.no_hugetlbfs == 0 &&
 			internal_config.process_type != RTE_PROC_SECONDARY &&
Signature · 1 line
2.9.3
#67 lore raw

PATCH v4 6/26 eal-common: introduce a way to query cpu support

In reply to Aaron Conole

This adds a new API to check for the eal cpu versions.

---
 lib/librte_eal/common/eal_common_cpuflags.c          | 13 +++++++++++--
 lib/librte_eal/common/include/generic/rte_cpuflags.h |  9 +++++++++
 2 files changed, 20 insertions(+), 2 deletions(-)
lib/librte_eal/common/eal_common_cpuflags.c +11 -2
index b5f76f7..2c2127b 100644
@@ -43,6 +43,13 @@
 void
 rte_cpu_check_supported(void)
 {
+	if (!rte_cpu_is_supported())
+		exit(1);
+}
+
+bool
+rte_cpu_is_supported(void)
+{
 	/* This is generated at compile-time by the build system */
 	static const enum rte_cpu_flag_t compile_time_flags[] = {
 			RTE_COMPILE_TIME_CPUFLAGS
@@ -57,14 +64,16 @@ rte_cpu_check_supported(void)
 			fprintf(stderr,
 				"ERROR: CPU feature flag lookup failed with error %d\n",
 				ret);
-			exit(1);
+			return false;
 		}
 		if (!ret) {
 			fprintf(stderr,
 			        "ERROR: This system does not support \"%s\".\n"
 			        "Please check that RTE_MACHINE is set correctly.\n",
 			        rte_cpu_get_flag_name(compile_time_flags[i]));
-			exit(1);
+			return false;
 		}
 	}
+
+	return true;
 }
lib/librte_eal/common/include/generic/rte_cpuflags.h +9 -0
index 71321f3..e4342ad 100644
@@ -40,6 +40,7 @@
  */
 
 #include <errno.h>
+#include <stdbool.h>
 
 /**
  * Enumeration of all CPU features supported
@@ -82,4 +83,12 @@ rte_cpu_get_flag_enabled(enum rte_cpu_flag_t feature);
 void
 rte_cpu_check_supported(void);
 
+/**
+ * This function checks that the currently used CPU supports the CPU features
+ * that were specified at compile time. It is called automatically within the
+ * EAL, so does not need to be used by applications.  This version returns a
+ * result so that decisions may be made (for instance, graceful shutdowns).
+ */
+bool
+rte_cpu_is_supported(void);
 #endif /* _RTE_CPUFLAGS_H_ */
Signature · 1 line
2.9.3
#68 lore raw

Re: [PATCH v4 06/26] eal-common: introduce a way to query cpu support

Quoted 50 lines · Aaron Conole

This adds a new API to check for the eal cpu versions.

---
 lib/librte_eal/common/eal_common_cpuflags.c          | 13 +++++++++++--
 lib/librte_eal/common/include/generic/rte_cpuflags.h |  9 +++++++++
 2 files changed, 20 insertions(+), 2 deletions(-)
lib/librte_eal/common/eal_common_cpuflags.c +11 -2
index b5f76f7..2c2127b 100644
@@ -43,6 +43,13 @@
 void
 rte_cpu_check_supported(void)
 {
+	if (!rte_cpu_is_supported())
+		exit(1);
+}
+
+bool
+rte_cpu_is_supported(void)
+{
 	/* This is generated at compile-time by the build system */
 	static const enum rte_cpu_flag_t compile_time_flags[] = {
 			RTE_COMPILE_TIME_CPUFLAGS
@@ -57,14 +64,16 @@ rte_cpu_check_supported(void)
 			fprintf(stderr,
 				"ERROR: CPU feature flag lookup failed with error %d\n",
 				ret);
-			exit(1);
+			return false;
 		}
 		if (!ret) {
 			fprintf(stderr,
 			        "ERROR: This system does not support \"%s\".\n"
 			        "Please check that RTE_MACHINE is set correctly.\n",
 			        rte_cpu_get_flag_name(compile_time_flags[i]));
-			exit(1);
+			return false;
 		}
 	}
+
+	return true;
 }
lib/librte_eal/common/include/generic/rte_cpuflags.h +0 -0
index 71321f3..e4342ad 100644

lib/librte_eal/common/include/generic/rte_cpuflags.h line 44

Quoted 5 lines
lib/librte_eal/common/include/generic/rte_cpuflags.h +1 -0
@@ -40,6 +40,7 @@
  */
 
 #include <errno.h>
+#include <stdbool.h>
 

The addition of this include is causing all sorts of compilation errors inside the PMDs, as many of them seem to be defining their own bools types. :-(

For safety sake, probably best to have the function return int rather than bool.

/Bruce

#69 lore raw

Re: [PATCH v4 06/26] eal-common: introduce a way to query cpu support

Quoted 61 lines · Bruce Richardson
Quoted 50 lines · Aaron Conole

This adds a new API to check for the eal cpu versions.

---
 lib/librte_eal/common/eal_common_cpuflags.c          | 13 +++++++++++--
 lib/librte_eal/common/include/generic/rte_cpuflags.h |  9 +++++++++
 2 files changed, 20 insertions(+), 2 deletions(-)
lib/librte_eal/common/eal_common_cpuflags.c +11 -2
index b5f76f7..2c2127b 100644
@@ -43,6 +43,13 @@
 void
 rte_cpu_check_supported(void)
 {
+	if (!rte_cpu_is_supported())
+		exit(1);
+}
+
+bool
+rte_cpu_is_supported(void)
+{
 	/* This is generated at compile-time by the build system */
 	static const enum rte_cpu_flag_t compile_time_flags[] = {
 			RTE_COMPILE_TIME_CPUFLAGS
@@ -57,14 +64,16 @@ rte_cpu_check_supported(void)
 			fprintf(stderr,
 				"ERROR: CPU feature flag lookup failed with error %d\n",
 				ret);
-			exit(1);
+			return false;
 		}
 		if (!ret) {
 			fprintf(stderr,
 			        "ERROR: This system does not support \"%s\".\n"
 			        "Please check that RTE_MACHINE is set correctly.\n",
 			        rte_cpu_get_flag_name(compile_time_flags[i]));
-			exit(1);
+			return false;
 		}
 	}
+
+	return true;
 }
lib/librte_eal/common/include/generic/rte_cpuflags.h +0 -0
index 71321f3..e4342ad 100644

lib/librte_eal/common/include/generic/rte_cpuflags.h line 44

Quoted 5 lines
lib/librte_eal/common/include/generic/rte_cpuflags.h +1 -0
@@ -40,6 +40,7 @@
  */
 
 #include <errno.h>
+#include <stdbool.h>
 

The addition of this include is causing all sorts of compilation errors inside the PMDs, as many of them seem to be defining their own bools types. :-(

Quoted 2 lines

For safety sake, probably best to have the function return int rather than bool.

Will do - I never saw the issue, but perhaps I was excluding the PMDs in question.

Thanks for the review, Bruce!

#70 lore raw

Re: [PATCH v4 06/26] eal-common: introduce a way to query cpu support

Quoted 63 lines · Aaron Conole
Quoted 61 lines · Bruce Richardson
Quoted 50 lines · Aaron Conole

This adds a new API to check for the eal cpu versions.

---
 lib/librte_eal/common/eal_common_cpuflags.c          | 13 +++++++++++--
 lib/librte_eal/common/include/generic/rte_cpuflags.h |  9 +++++++++
 2 files changed, 20 insertions(+), 2 deletions(-)
lib/librte_eal/common/eal_common_cpuflags.c +11 -2
index b5f76f7..2c2127b 100644
@@ -43,6 +43,13 @@
 void
 rte_cpu_check_supported(void)
 {
+	if (!rte_cpu_is_supported())
+		exit(1);
+}
+
+bool
+rte_cpu_is_supported(void)
+{
 	/* This is generated at compile-time by the build system */
 	static const enum rte_cpu_flag_t compile_time_flags[] = {
 			RTE_COMPILE_TIME_CPUFLAGS
@@ -57,14 +64,16 @@ rte_cpu_check_supported(void)
 			fprintf(stderr,
 				"ERROR: CPU feature flag lookup failed with error %d\n",
 				ret);
-			exit(1);
+			return false;
 		}
 		if (!ret) {
 			fprintf(stderr,
 			        "ERROR: This system does not support \"%s\".\n"
 			        "Please check that RTE_MACHINE is set correctly.\n",
 			        rte_cpu_get_flag_name(compile_time_flags[i]));
-			exit(1);
+			return false;
 		}
 	}
+
+	return true;
 }
lib/librte_eal/common/include/generic/rte_cpuflags.h +0 -0
index 71321f3..e4342ad 100644

lib/librte_eal/common/include/generic/rte_cpuflags.h line 44

Quoted 5 lines
lib/librte_eal/common/include/generic/rte_cpuflags.h +1 -0
@@ -40,6 +40,7 @@
  */
 
 #include <errno.h>
+#include <stdbool.h>
 

The addition of this include is causing all sorts of compilation errors inside the PMDs, as many of them seem to be defining their own bools types. :-(

Quoted 7 lines
Quoted 2 lines

For safety sake, probably best to have the function return int rather than bool.

Will do - I never saw the issue, but perhaps I was excluding the PMDs in question.

Thanks for the review, Bruce!

No problem.
FYI, the drivers I saw the errors in with this patch are:
* qede
* i40e
* ixgbe
* cxgbe

Regards,
/Bruce

#71 lore raw

PATCH v4 7/26 eal: Signal error when CPU isn't supported

In reply to Aaron Conole

It's now possible to gracefully exit the application, or for applications which support non-dpdk datapaths working in concert with DPDK datapaths, there no longer is the possibility of exiting for unsupported CPUs.

---
 lib/librte_eal/linuxapp/eal/eal.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -1
index 0f682e1..0adaaa2 100644
@@ -758,7 +758,11 @@ rte_eal_init(int argc, char **argv)
 	char thread_name[RTE_MAX_THREAD_NAME_LEN];
 
 	/* checks if the machine is adequate */
-	rte_cpu_check_supported();
+	if (!rte_cpu_is_supported()) {
+		rte_eal_init_alert("unsupported cpu type.");
+		rte_errno = ENOTSUP;
+		return -1;
+	}
 
 	if (!rte_atomic32_test_and_set(&run_once))
 		return -1;
Signature · 1 line
2.9.3
#72 lore raw

PATCH v4 8/26 eal: do not panic on memzone initialization fails

In reply to Aaron Conole

When memzone initialization fails, report the error to the calling application rather than panic(). Without a good way of detaching / releasing hugepages, at this point the application will have to restart.

---
 lib/librte_eal/linuxapp/eal/eal.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
index 0adaaa2..0005ebc 100644
@@ -838,8 +838,11 @@ rte_eal_init(int argc, char **argv)
 	/* the directories are locked during eal_hugepage_info_init */
 	eal_hugedirs_unlock();
 
-	if (rte_eal_memzone_init() < 0)
-		rte_panic("Cannot init memzone\n");
+	if (rte_eal_memzone_init() < 0) {
+		RTE_LOG(ERR, EAL, "Cannot init memzone\n");
+		rte_errno = ENODEV;
+		return -1;
+	}
 
 	if (rte_eal_tailqs_init() < 0)
 		rte_panic("Cannot init tail queues for objects\n");
Signature · 1 line
2.9.3
#73 lore raw

PATCH v4 9/26 eal: set errno when exiting for already called

In reply to Aaron Conole
---
 lib/librte_eal/linuxapp/eal/eal.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal.c +4 -1
index 0005ebc..beb786e 100644
@@ -764,8 +764,11 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 
-	if (!rte_atomic32_test_and_set(&run_once))
+	if (!rte_atomic32_test_and_set(&run_once)) {
+		rte_eal_init_alert("already called initialization.");
+		rte_errno = EALREADY;
 		return -1;
+	}
 
 	logid = strrchr(argv[0], '/');
 	logid = strdup(logid ? logid + 1: argv[0]);
Signature · 1 line
2.9.3
#74 lore raw

PATCH v4 10/26 eal: Do not panic on log failures

In reply to Aaron Conole

When log initialization fails, it's generally because the fopencookie failed. While this is rare in practice, it could happen, and it is likely because of memory pressure. So, flag the error, and allow the user to retry.

---
 lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
index beb786e..25f8ae8 100644
@@ -824,8 +824,12 @@ rte_eal_init(int argc, char **argv)
 
 	rte_config_init();
 
-	if (rte_eal_log_init(logid, internal_config.syslog_facility) < 0)
-		rte_panic("Cannot init logs\n");
+	if (rte_eal_log_init(logid, internal_config.syslog_facility) < 0) {
+		rte_eal_init_alert("Cannot init logging.");
+		rte_errno = ENOMEM;
+		rte_atomic32_clear(&run_once);
+		return -1;
+	}
 
 	if (rte_eal_pci_init() < 0)
 		rte_panic("Cannot init PCI\n");
Signature · 1 line
2.9.3
#75 lore raw

PATCH v4 11/26 eal: Do not panic on pci-probe

In reply to Aaron Conole

This will usually be an issue because of permissions. However, it could also be caused by OOM. In either case, errno will contain the underlying cause. It is safe to re-init the system here, so allow the application to take corrective action and reinit.

---
 lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
index 25f8ae8..5534b4b 100644
@@ -831,8 +831,12 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 
-	if (rte_eal_pci_init() < 0)
-		rte_panic("Cannot init PCI\n");
+	if (rte_eal_pci_init() < 0) {
+		RTE_LOG(ERR, EAL, "Cannot init PCI\n");
+		rte_errno = EUNATCH;
+		rte_atomic32_clear(&run_once);
+		return -1;
+	}
 
 #ifdef VFIO_PRESENT
 	if (rte_eal_vfio_setup() < 0)
Signature · 1 line
2.9.3
#76 lore raw

PATCH v4 12/26 eal: do not panic on vfio failure

In reply to Aaron Conole
---
 lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
index 5534b4b..0e7e8c8 100644
@@ -839,8 +839,12 @@ rte_eal_init(int argc, char **argv)
 	}
 
 #ifdef VFIO_PRESENT
-	if (rte_eal_vfio_setup() < 0)
-		rte_panic("Cannot init VFIO\n");
+	if (rte_eal_vfio_setup() < 0) {
+		RTE_LOG(ERR, EAL, "Cannot init VFIO\n");
+		rte_errno = EAGAIN;
+		rte_atomic32_clear(&run_once);
+		return -1;
+	}
 #endif
 
 	if (rte_eal_memory_init() < 0)
Signature · 1 line
2.9.3
#77 lore raw

PATCH v4 13/26 eal: do not panic on memory init

In reply to Aaron Conole

This can only happen when access to hugepages (either as primary or secondary process) fails (and that is usually permissions). Since the manner of failure is not reversible, we cannot allow retry.

---
 lib/librte_eal/linuxapp/eal/eal.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
index 0e7e8c8..30d6a7d 100644
@@ -847,8 +847,11 @@ rte_eal_init(int argc, char **argv)
 	}
 #endif
 
-	if (rte_eal_memory_init() < 0)
-		rte_panic("Cannot init memory\n");
+	if (rte_eal_memory_init() < 0) {
+		RTE_LOG(ERR, EAL, "Cannot init memory\n");
+		rte_errno = ENOMEM;
+		return -1;
+	}
 
 	/* the directories are locked during eal_hugepage_info_init */
 	eal_hugedirs_unlock();
Signature · 1 line
2.9.3
#78 lore raw

PATCH v4 14/26 eal: do not panic on tailq init

In reply to Aaron Conole

There are some theoretical racy conditions in the system that _could_ cause early tailq init to fail; however, no need to panic the application. While it can't continue using DPDK, it could make better alerts to the user.

---
 lib/librte_eal/common/eal_common_tailqs.c | 3 +--
 lib/librte_eal/linuxapp/eal/eal.c         | 7 +++++--
 2 files changed, 6 insertions(+), 4 deletions(-)
lib/librte_eal/common/eal_common_tailqs.c +1 -2
index bb08ec8..4f69828 100644
@@ -188,8 +188,7 @@ rte_eal_tailqs_init(void)
 		if (t->head == NULL) {
 			RTE_LOG(ERR, EAL,
 				"Cannot initialize tailq: %s\n", t->name);
-			/* no need to TAILQ_REMOVE, we are going to panic in
-			 * rte_eal_init() */
+			/* TAILQ_REMOVE not needed, error is already fatal */
 			goto fail;
 		}
 	}
lib/librte_eal/linuxapp/eal/eal.c +5 -2
index 30d6a7d..008e804 100644
@@ -862,8 +862,11 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 
-	if (rte_eal_tailqs_init() < 0)
-		rte_panic("Cannot init tail queues for objects\n");
+	if (rte_eal_tailqs_init() < 0) {
+		RTE_LOG(ERR, EAL, "Cannot init tail queues for objects\n");
+		rte_errno = EFAULT;
+		return -1;
+	}
 
 	if (rte_eal_alarm_init() < 0)
 		rte_panic("Cannot init interrupt-handling thread\n");
Signature · 1 line
2.9.3
#79 lore raw

PATCH v4 15/26 eal: do not panic on alarm init

In reply to Aaron Conole

rte_eal_alarm_init() call uses the linux timerfd framework to create a poll()-able timer using standard posix file operations. This could fail for a few reasons given in the man-pages, but many could be corrected by the user application. No need to panic.

---
 lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
index 008e804..aa64f69 100644
@@ -61,6 +61,7 @@
 #include <rte_launch.h>
 #include <rte_eal.h>
 #include <rte_eal_memconfig.h>
+#include <rte_errno.h>
 #include <rte_per_lcore.h>
 #include <rte_lcore.h>
 #include <rte_log.h>
@@ -868,8 +869,11 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 
-	if (rte_eal_alarm_init() < 0)
-		rte_panic("Cannot init interrupt-handling thread\n");
+	if (rte_eal_alarm_init() < 0) {
+		RTE_LOG(ERR, EAL, "Cannot init interrupt-handling thread\n");
+		/* rte_eal_alarm_init sets rte_errno on failure. */
+		return -1;
+	}
 
 	if (rte_eal_timer_init() < 0)
 		rte_panic("Cannot init HPET or TSC timers\n");
Signature · 1 line
2.9.3
#80 lore raw

PATCH v4 16/26 eal: convert timer_init not to call panic

In reply to Aaron Conole

After code inspection, there is no way for eal_timer_init() to fail. It simply returns 0 in all cases. As such, this test could either go-away or stay here as 'future-proofing'.

---
 lib/librte_eal/linuxapp/eal/eal.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
index aa64f69..bf20245 100644
@@ -875,8 +875,11 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 
-	if (rte_eal_timer_init() < 0)
-		rte_panic("Cannot init HPET or TSC timers\n");
+	if (rte_eal_timer_init() < 0) {
+		RTE_LOG(ERR, EAL, "Cannot init HPET or TSC timers\n");
+		rte_errno = ENOTSUP;
+		return -1;
+	}
 
 	eal_check_mem_on_local_socket();
 
Signature · 1 line
2.9.3
#81 lore raw

PATCH v4 17/26 eal: change the private pipe call to reflect errno

In reply to Aaron Conole

There could be some confusion as to why the call failed - this change will always reflect the value of the error in rte_error.

---
 lib/librte_eal/linuxapp/eal/eal_interrupts.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal_interrupts.c +4 -1
index 92a19cb..5bb833e 100644
@@ -898,13 +898,16 @@ rte_eal_intr_init(void)
 	 * create a pipe which will be waited by epoll and notified to
 	 * rebuild the wait list of epoll.
 	 */
-	if (pipe(intr_pipe.pipefd) < 0)
+	if (pipe(intr_pipe.pipefd) < 0) {
+		rte_errno = errno;
 		return -1;
+	}
 
 	/* create the host thread to wait/handle the interrupt */
 	ret = pthread_create(&intr_thread, NULL,
 			eal_intr_thread_main, NULL);
 	if (ret != 0) {
+		rte_errno = ret;
 		RTE_LOG(ERR, EAL,
 			"Failed to create thread for interrupt handling\n");
 	} else {
Signature · 1 line
2.9.3
#82 lore raw

PATCH v4 18/26 eal: Do not panic on interrupt thread init

In reply to Aaron Conole

When initializing the interrupt thread, there are a number of possible reasons for failure - some of which are correctable by the application. Do not panic() needlessly, and give the application a change to reflect this information to the user.

---
 lib/librte_eal/linuxapp/eal/eal.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +4 -2
index bf20245..02b075c 100644
@@ -894,8 +894,10 @@ rte_eal_init(int argc, char **argv)
 		rte_config.master_lcore, (int)thread_id, cpuset,
 		ret == 0 ? "" : "...");
 
-	if (rte_eal_intr_init() < 0)
-		rte_panic("Cannot init interrupt-handling thread\n");
+	if (rte_eal_intr_init() < 0) {
+		RTE_LOG(ERR, EAL, "Cannot init interrupt-handling thread\n");
+		return -1;
+	}
 
 	if (rte_bus_scan())
 		rte_panic("Cannot scan the buses for devices\n");
Signature · 1 line
2.9.3
#83 lore raw

PATCH v4 19/26 eal: do not error if plugins fail to init

In reply to Aaron Conole

Plugins are useful and important. However, it seems crazy to abort everything just because they don't initialize properly.

---
 lib/librte_eal/linuxapp/eal/eal.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal.c +1 -1
index 02b075c..2069b6f 100644
@@ -884,7 +884,7 @@ rte_eal_init(int argc, char **argv)
 	eal_check_mem_on_local_socket();
 
 	if (eal_plugins_init() < 0)
-		rte_panic("Cannot init plugins\n");
+		RTE_LOG(ERR, EAL, "Cannot init plugins\n");
 
 	eal_thread_init_master(rte_config.master_lcore);
 
Signature · 1 line
2.9.3
#84 lore raw

PATCH v4 20/26 eal_pci: Continue probing even on failures

In reply to Aaron Conole

Some devices may be inaccessible for a variety of reasons, or the PCI-bus may be unavailable causing the whole thing to fail. Still, better to continue attempts at probes.

---
 lib/librte_eal/common/eal_common_pci.c | 15 ++++++++++-----
 1 file changed, 10 insertions(+), 5 deletions(-)
lib/librte_eal/common/eal_common_pci.c +10 -5
index 72547bd..9416190 100644
@@ -69,6 +69,7 @@
 #include <sys/queue.h>
 #include <sys/mman.h>
 
+#include <rte_errno.h>
 #include <rte_interrupts.h>
 #include <rte_log.h>
 #include <rte_pci.h>
@@ -416,6 +417,7 @@ rte_eal_pci_probe(void)
 	struct rte_pci_device *dev = NULL;
 	struct rte_devargs *devargs;
 	int probe_all = 0;
+	int ret_1 = 0;
 	int ret = 0;
 
 	if (rte_eal_devargs_type_count(RTE_DEVTYPE_WHITELISTED_PCI) == 0)
@@ -430,17 +432,20 @@ rte_eal_pci_probe(void)
 
 		/* probe all or only whitelisted devices */
 		if (probe_all)
-			ret = pci_probe_all_drivers(dev);
+			ret_1 = pci_probe_all_drivers(dev);
 		else if (devargs != NULL &&
 			devargs->type == RTE_DEVTYPE_WHITELISTED_PCI)
-			ret = pci_probe_all_drivers(dev);
-		if (ret < 0)
-			rte_exit(EXIT_FAILURE, "Requested device " PCI_PRI_FMT
+			ret_1 = pci_probe_all_drivers(dev);
+		if (ret_1 < 0) {
+			RTE_LOG(ERR, EAL, "Requested device " PCI_PRI_FMT
 				 " cannot be used\n", dev->addr.domain, dev->addr.bus,
 				 dev->addr.devid, dev->addr.function);
+			rte_errno = errno;
+			ret = 1;
+		}
 	}
 
-	return 0;
+	return -ret;
 }
 
 /* dump one device */
Signature · 1 line
2.9.3
#85 lore raw

PATCH v4 21/26 eal: do not panic on failed PCI probe

In reply to Aaron Conole

It may even be possible to simply log the error and continue on letting the user check the logs and restart the application when things are failed.

---
 lib/librte_eal/linuxapp/eal/eal.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
index 2069b6f..481b53b 100644
@@ -943,8 +943,11 @@ rte_eal_init(int argc, char **argv)
 		rte_panic("Cannot probe devices\n");
 
 	/* Probe & Initialize PCI devices */
-	if (rte_eal_pci_probe())
-		rte_panic("Cannot probe PCI\n");
+	if (rte_eal_pci_probe()) {
+		RTE_LOG(ERR, EAL, "Cannot probe PCI\n");
+		rte_errno = ENOTSUP;
+		return -1;
+	}
 
 	if (rte_eal_dev_init() < 0)
 		rte_panic("Cannot init pmd devices\n");
Signature · 1 line
2.9.3
#86 lore raw

PATCH v4 22/26 eal_common_dev: continue initializing vdevs

In reply to Aaron Conole

Even if one vdev should fail, there's no need to prevent further processing. Log the error, and reflect it to the higher levels to decide.

---
 lib/librte_eal/common/eal_common_dev.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
lib/librte_eal/common/eal_common_dev.c +3 -2
index 4f3b493..9889997 100644
@@ -80,6 +80,7 @@ int
 rte_eal_dev_init(void)
 {
 	struct rte_devargs *devargs;
+	int ret = 0;
 
 	/*
 	 * Note that the dev_driver_list is populated here
@@ -97,11 +98,11 @@ rte_eal_dev_init(void)
 					devargs->args)) {
 			RTE_LOG(ERR, EAL, "failed to initialize %s device\n",
 					devargs->virt.drv_name);
-			return -1;
+			ret = -1;
 		}
 	}
 
-	return 0;
+	return ret;
 }
 
 int rte_eal_dev_attach(const char *name, const char *devargs)
Signature · 1 line
2.9.3
#87 lore raw

PATCH v4 23/26 eal: do not panic (or abort) if vdev init fails

In reply to Aaron Conole

Seems like it's possible to continue. At least, the error is reflected properly in the logs. A user could then go and correct or investigate the situation.

---
 lib/librte_eal/linuxapp/eal/eal.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal.c +1 -1
index 481b53b..803cdd2 100644
@@ -950,7 +950,7 @@ rte_eal_init(int argc, char **argv)
 	}
 
 	if (rte_eal_dev_init() < 0)
-		rte_panic("Cannot init pmd devices\n");
+		RTE_LOG(ERR, EAL, "Cannot init pmd devices\n");
 
 	rte_eal_mcfg_complete();
 
Signature · 1 line
2.9.3
#88 lore raw

PATCH v4 24/26 eal: do not panic when bus probe fails

In reply to Aaron Conole
---
 lib/librte_eal/linuxapp/eal/eal.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
index 803cdd2..c94ac9b 100644
@@ -939,8 +939,11 @@ rte_eal_init(int argc, char **argv)
 	rte_eal_mp_wait_lcore();
 
 	/* Probe all the buses and devices/drivers on them */
-	if (rte_bus_probe())
-		rte_panic("Cannot probe devices\n");
+	if (rte_bus_probe()) {
+		RTE_LOG(ERR, EAL, "Cannot probe devices\n");
+		rte_errno = ENOTSUP;
+		return -1;
+	}
 
 	/* Probe & Initialize PCI devices */
 	if (rte_eal_pci_probe()) {
Signature · 1 line
2.9.3
#89 lore raw

PATCH v4 25/26 eal: do not panic on failed bus scan

In reply to Aaron Conole

For now, do an abort. It's likely that even aborting the initialization is premature in this case, as it may be possible to proceed even if one bus or another is not available.

---
 lib/librte_eal/linuxapp/eal/eal.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
index c94ac9b..41050a0 100644
@@ -899,8 +899,11 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 
-	if (rte_bus_scan())
-		rte_panic("Cannot scan the buses for devices\n");
+	if (rte_bus_scan()) {
+		RTE_LOG(ERR, EAL, "Cannot scan the buses for devices\n");
+		rte_errno = ENODEV;
+		return -1;
+	}
 
 	RTE_LCORE_FOREACH_SLAVE(i) {
 
Signature · 1 line
2.9.3
#90 lore raw

PATCH v4 26/26 rte_eal_init: add info about rte_errno codes

In reply to Aaron Conole

The rte_eal_init function will now pass failure reason hints to the application. To help app developers deciper this, add some brief information about what the codes are indicating.

---
 lib/librte_eal/common/include/rte_eal.h | 27 ++++++++++++++++++++++++++-
 1 file changed, 26 insertions(+), 1 deletion(-)
lib/librte_eal/common/include/rte_eal.h +26 -1
index 03fee50..9251244 100644
@@ -159,7 +159,32 @@ int rte_eal_iopl_init(void);
  *     function call and should not be further interpreted by the
  *     application.  The EAL does not take any ownership of the memory used
  *     for either the argv array, or its members.
- *   - On failure, a negative error value.
+ *   - On failure, -1 and rte_errno is set to a value indicating the cause
+ *     for failure.  In some instances, the application will need to be
+ *     restarted as part of clearing the issue.
+ *
+ *   Error codes returned via rte_errno:
+ *     EACCES indicates a permissions issue.
+ *
+ *     EAGAIN indicates either a bus or system resource was not available,
+ *            setup may be attempted again.
+ *
+ *     EALREADY indicates that the rte_eal_init function has already been
+ *              called, and cannot be called again.
+ *
+ *     EFAULT indicates the tailq configuration name was not found in
+ *            memory configuration.
+ *
+ *     EINVAL indicates invalid parameters were passed as argv/argc.
+ *
+ *     ENOMEM indicates failure likely caused by an out-of-memory condition.
+ *
+ *     ENODEV indicates memory setup issues.
+ *
+ *     ENOTSUP indicates that the EAL cannot initialize on this system.
+ *
+ *     EUNATCH indicates that the PCI bus is either not present, or is not
+ *             readable by the eal.
  */
 int rte_eal_init(int argc, char **argv);
 
Signature · 1 line
2.9.3
#91 lore raw
In reply to Aaron Conole
Quoted 6 lines · Aaron Conole

In many cases, it's enough to simply let the application know that the call to initialize DPDK has failed. A complete halt can then be decided by the application based on error returned (and the app could even attempt a possible re-attempt after some corrective action by the user or application).

Spotted a few issues when I tried applying each of the patches and compile testing them; those I've flagged on the patches that I test applied. Otherwise, this looks a great set to have.

I assume the equivalent changes to the BSD EAL are "left as an exercise for the reader"? :-)

/Bruce

#92 lore raw
Quoted 7 lines · Bruce Richardson
Quoted 6 lines · Aaron Conole

In many cases, it's enough to simply let the application know that the call to initialize DPDK has failed. A complete halt can then be decided by the application based on error returned (and the app could even attempt a possible re-attempt after some corrective action by the user or application).

Quoted 6 lines

Spotted a few issues when I tried applying each of the patches and compile testing them; those I've flagged on the patches that I test applied. Otherwise, this looks a great set to have.

I assume the equivalent changes to the BSD EAL are "left as an exercise for the reader"? :-)

For now. In my copious free time, I might have a crack at it, although I don't have a working BSD DPDK setup.

Quoted 1 line

/Bruce

#93 lore raw

v5 cover linux/eal: Remove most causes of panic on init

In reply to Aaron Conole

In many cases, it's enough to simply let the application know that the call to initialize DPDK has failed. A complete halt can then be decided by the application based on error returned (and the app could even attempt a possible re-attempt after some corrective action by the user or application).

Changes ->v2:
- Audited all "RTE_LOG (" calls that were introduced, and converted to "RTE_LOG("
- Added some fprintf(stderr, "") lines to indicate errors before logging is initialized
- Removed assignments to errno.
- Changed patch 14/25 to reflect EFAULT, and document in 25/25

Changes ->v3:
- Checkpatch issues in patches 3 (spelling mistake), 9 (issue with leading spaces), and 19 (braces around single line statement if-condition)

Changes ->v4:
- Error text cleanup.
- Add a new check around rte_bus_scan(), added during the development of this series.

Changes ->v5:
- checkpatch.pl cleanup in patch 02/26
- move rte_errno.h include from patch 15 to patch 02
- remove stdbool.h and use int as return type in patch 06/26

I kept the rte_errno reflection, since this is control-path code and the init function returns a sentinel value of -1.

Aaron Conole (26):
eal: CPU init will no longer panic
eal: return error instead of panic for cpu init
eal: No panic on hugepages info init
eal: do not panic on failed hugepage query
eal: failure to parse args returns error
eal-common: introduce a way to query cpu support
eal: Signal error when CPU isn't supported
eal: do not panic on memzone initialization fails
eal: set errno when exiting for already called
eal: Do not panic on log failures
eal: Do not panic on pci-probe
eal: do not panic on vfio failure
eal: do not panic on memory init
eal: do not panic on tailq init
eal: do not panic on alarm init
eal: convert timer_init not to call panic
eal: change the private pipe call to reflect errno
eal: Do not panic on interrupt thread init
eal: do not error if plugins fail to init
eal_pci: Continue probing even on failures
eal: do not panic on failed PCI probe
eal_common_dev: continue initializing vdevs
eal: do not panic (or abort) if vdev init fails
eal: do not panic when bus probe fails
eal: do not panic on failed bus scan
rte_eal_init: add info about rte_errno codes

 lib/librte_eal/common/eal_common_cpuflags.c        |  13 +-
 lib/librte_eal/common/eal_common_dev.c             |   5 +-
 lib/librte_eal/common/eal_common_lcore.c           |   7 +-
 lib/librte_eal/common/eal_common_pci.c             |  15 ++-
 lib/librte_eal/common/eal_common_tailqs.c          |   3 +-
 .../common/include/generic/rte_cpuflags.h          |   8 ++
 lib/librte_eal/common/include/rte_eal.h            |  27 ++++-
 lib/librte_eal/linuxapp/eal/eal.c                  | 131 +++++++++++++++------
 lib/librte_eal/linuxapp/eal/eal_hugepage_info.c    |   6 +-
 lib/librte_eal/linuxapp/eal/eal_interrupts.c       |   5 +-
 10 files changed, 168 insertions(+), 52 deletions(-)
Signature · 1 line
2.9.3
#94 lore raw

PATCH v5 1/26 eal: CPU init will no longer panic

After this change, the EAL CPU NUMA node resolution step can no longer emit an rte_panic. This aligns with the code in rte_eal_init, which expects failures to return an error code.

---
 lib/librte_eal/common/eal_common_lcore.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
lib/librte_eal/common/eal_common_lcore.c +4 -3
index 2cd4132..84fa0cb 100644
@@ -83,16 +83,17 @@ rte_eal_cpu_init(void)
 		config->lcore_role[lcore_id] = ROLE_RTE;
 		lcore_config[lcore_id].core_id = eal_cpu_core_id(lcore_id);
 		lcore_config[lcore_id].socket_id = eal_cpu_socket_id(lcore_id);
-		if (lcore_config[lcore_id].socket_id >= RTE_MAX_NUMA_NODES)
+		if (lcore_config[lcore_id].socket_id >= RTE_MAX_NUMA_NODES) {
 #ifdef RTE_EAL_ALLOW_INV_SOCKET_ID
 			lcore_config[lcore_id].socket_id = 0;
 #else
-			rte_panic("Socket ID (%u) is greater than "
+			RTE_LOG(ERR, EAL, "Socket ID (%u) is greater than "
 				"RTE_MAX_NUMA_NODES (%d)\n",
 				lcore_config[lcore_id].socket_id,
 				RTE_MAX_NUMA_NODES);
+			return -1;
 #endif
-
+		}
 		RTE_LOG(DEBUG, EAL, "Detected lcore %u as "
 				"core %u on socket %u\n",
 				lcore_id, lcore_config[lcore_id].core_id,
Signature · 1 line
2.9.3
#95 lore raw

PATCH v5 2/26 eal: return error instead of panic for cpu init

In reply to Aaron Conole

There may be no way to gracefully recover, but the application should be notified that a failure happened, rather than completely aborting. This allows the user to proceed with a "slow-path" type solution.

---
 lib/librte_eal/linuxapp/eal/eal.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +12 -2
index bf6b818..81692e7 100644
@@ -61,6 +61,7 @@
 #include <rte_launch.h>
 #include <rte_eal.h>
 #include <rte_eal_memconfig.h>
+#include <rte_errno.h>
 #include <rte_per_lcore.h>
 #include <rte_lcore.h>
 #include <rte_log.h>
@@ -740,6 +741,12 @@ static int rte_eal_vfio_setup(void)
 }
 #endif
 
+static void rte_eal_init_alert(const char *msg)
+{
+	fprintf(stderr, "EAL: FATAL: %s\n", msg);
+	RTE_LOG(ERR, EAL, "%s\n", msg);
+}
+
 /* Launch threads, called at application init(). */
 int
 rte_eal_init(int argc, char **argv)
@@ -767,8 +774,11 @@ rte_eal_init(int argc, char **argv)
 	/* set log level as early as possible */
 	rte_set_log_level(internal_config.log_level);
 
-	if (rte_eal_cpu_init() < 0)
-		rte_panic("Cannot detect lcores\n");
+	if (rte_eal_cpu_init() < 0) {
+		rte_eal_init_alert("Cannot detect lcores.");
+		rte_errno = ENOTSUP;
+		return -1;
+	}
 
 	fctret = eal_parse_args(argc, argv);
 	if (fctret < 0)
Signature · 1 line
2.9.3
#96 lore raw

PATCH v5 3/26 eal: No panic on hugepages info init

In reply to Aaron Conole

When attempting to scan hugepages, signal to the eal.c that an error has occurred, rather than performing a panic.

---
 lib/librte_eal/linuxapp/eal/eal_hugepage_info.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal_hugepage_info.c +4 -2
index 18858e2..4d47eaf 100644
@@ -283,9 +283,11 @@ eal_hugepage_info_init(void)
 	struct dirent *dirent;
 
 	dir = opendir(sys_dir_path);
-	if (dir == NULL)
-		rte_panic("Cannot open directory %s to read system hugepage "
+	if (dir == NULL) {
+		RTE_LOG(ERR, EAL, "Cannot open directory %s to read system hugepage "
 			  "info\n", sys_dir_path);
+		return -1;
+	}
 
 	for (dirent = readdir(dir); dirent != NULL; dirent = readdir(dir)) {
 		struct hugepage_info *hpi;
Signature · 1 line
2.9.3
#97 lore raw

Re: [PATCH v5 03/26] eal: No panic on hugepages info init

Quoted 13 lines · Aaron Conole

When attempting to scan hugepages, signal to the eal.c that an error has occurred, rather than performing a panic.

---
 lib/librte_eal/linuxapp/eal/eal_hugepage_info.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal_hugepage_info.c +0 -0
index 18858e2..4d47eaf 100644

lib/librte_eal/linuxapp/eal/eal_hugepage_info.c eal_hugepage_info_init(void) · line 290

Quoted 10 lines
lib/librte_eal/linuxapp/eal/eal_hugepage_info.c +4 -2
@@ -283,9 +283,11 @@ eal_hugepage_info_init(void)
 	struct dirent *dirent;
 
 	dir = opendir(sys_dir_path);
-	if (dir == NULL)
-		rte_panic("Cannot open directory %s to read system hugepage "
+	if (dir == NULL) {
+		RTE_LOG(ERR, EAL, "Cannot open directory %s to read system hugepage "
 			  "info\n", sys_dir_path);
+		return -1;
+	}

Minor nit.
The error message should go on a line on its own, without any breaks to make it easy to "grep". This should also eliminate the checkpatch complaint about it being too long.

/Bruce

#98 lore raw

Re: [PATCH v5 03/26] eal: No panic on hugepages info init

Quoted 16 lines · Bruce Richardson
Quoted 15 lines · Aaron Conole

When attempting to scan hugepages, signal to the eal.c that an error has occurred, rather than performing a panic.

---
 lib/librte_eal/linuxapp/eal/eal_hugepage_info.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal_hugepage_info.c +0 -0
index 18858e2..4d47eaf 100644
@@ -283,9 +283,11 @@ eal_hugepage_info_init(void)
 	struct dirent *dirent;
 
Quoted 13 lines

lib/librte_eal/linuxapp/eal/eal_hugepage_info.c eal_hugepage_info_init(void) · line 290

Quoted 8 lines
lib/librte_eal/linuxapp/eal/eal_hugepage_info.c +4 -2
 	dir = opendir(sys_dir_path);
-	if (dir == NULL)
-		rte_panic("Cannot open directory %s to read system hugepage "
+	if (dir == NULL) {
+		RTE_LOG(ERR, EAL, "Cannot open directory %s to read system hugepage "
 			  "info\n", sys_dir_path);
+		return -1;
+	}

Minor nit.
The error message should go on a line on its own, without any breaks to make it easy to "grep". This should also eliminate the checkpatch complaint about it being too long.

Yes, will fix it for v6.

-Aaron

#99 lore raw

PATCH v5 4/26 eal: do not panic on failed hugepage query

In reply to Aaron Conole

If we fail to acquire hugepage information, simply signal an error to the application. This clears the run_once counter, allowing the user or application to take a corrective action and retry.

---
 lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
index 81692e7..12bd941 100644
@@ -787,8 +787,12 @@ rte_eal_init(int argc, char **argv)
 	if (internal_config.no_hugetlbfs == 0 &&
 			internal_config.process_type != RTE_PROC_SECONDARY &&
 			internal_config.xen_dom0_support == 0 &&
-			eal_hugepage_info_init() < 0)
-		rte_panic("Cannot get hugepage information\n");
+			eal_hugepage_info_init() < 0) {
+		rte_eal_init_alert("Cannot get hugepage information.");
+		rte_errno = EACCES;
+		rte_atomic32_clear(&run_once);
+		return -1;
+	}
 
 	if (internal_config.memory == 0 && internal_config.force_sockets == 0) {
 		if (internal_config.no_hugetlbfs)
Signature · 1 line
2.9.3
#100 lore raw

PATCH v5 5/26 eal: failure to parse args returns error

In reply to Aaron Conole

It's possible that the application could take a corrective action here, and either prompt the user for different arguments, or at least perform a better logging. Exiting this early prevents any useful information gathering from the application layer.

---
 lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
index 12bd941..f7511ab 100644
@@ -781,8 +781,12 @@ rte_eal_init(int argc, char **argv)
 	}
 
 	fctret = eal_parse_args(argc, argv);
-	if (fctret < 0)
-		exit(1);
+	if (fctret < 0) {
+		rte_eal_init_alert("Invalid 'command line' arguments.");
+		rte_errno = EINVAL;
+		rte_atomic32_clear(&run_once);
+		return -1;
+	}
 
 	if (internal_config.no_hugetlbfs == 0 &&
 			internal_config.process_type != RTE_PROC_SECONDARY &&
Signature · 1 line
2.9.3
#101 lore raw

PATCH v5 6/26 eal-common: introduce a way to query cpu support

In reply to Aaron Conole

This adds a new API to check for the eal cpu versions.

---
 lib/librte_eal/common/eal_common_cpuflags.c          | 13 +++++++++++--
 lib/librte_eal/common/include/generic/rte_cpuflags.h |  8 ++++++++
 2 files changed, 19 insertions(+), 2 deletions(-)
lib/librte_eal/common/eal_common_cpuflags.c +11 -2
index b5f76f7..9a2d080 100644
@@ -43,6 +43,13 @@
 void
 rte_cpu_check_supported(void)
 {
+	if (!rte_cpu_is_supported())
+		exit(1);
+}
+
+int
+rte_cpu_is_supported(void)
+{
 	/* This is generated at compile-time by the build system */
 	static const enum rte_cpu_flag_t compile_time_flags[] = {
 			RTE_COMPILE_TIME_CPUFLAGS
@@ -57,14 +64,16 @@ rte_cpu_check_supported(void)
 			fprintf(stderr,
 				"ERROR: CPU feature flag lookup failed with error %d\n",
 				ret);
-			exit(1);
+			return 0;
 		}
 		if (!ret) {
 			fprintf(stderr,
 			        "ERROR: This system does not support \"%s\".\n"
 			        "Please check that RTE_MACHINE is set correctly.\n",
 			        rte_cpu_get_flag_name(compile_time_flags[i]));
-			exit(1);
+			return 0;
 		}
 	}
+
+	return 1;
 }
lib/librte_eal/common/include/generic/rte_cpuflags.h +8 -0
index 71321f3..8d27031 100644
@@ -82,4 +82,12 @@ rte_cpu_get_flag_enabled(enum rte_cpu_flag_t feature);
 void
 rte_cpu_check_supported(void);
 
+/**
+ * This function checks that the currently used CPU supports the CPU features
+ * that were specified at compile time. It is called automatically within the
+ * EAL, so does not need to be used by applications.  This version returns a
+ * result so that decisions may be made (for instance, graceful shutdowns).
+ */
+int
+rte_cpu_is_supported(void);
 #endif /* _RTE_CPUFLAGS_H_ */
Signature · 1 line
2.9.3
#102 lore raw

PATCH v5 7/26 eal: Signal error when CPU isn't supported

In reply to Aaron Conole

It's now possible to gracefully exit the application, or for applications which support non-dpdk datapaths working in concert with DPDK datapaths, there no longer is the possibility of exiting for unsupported CPUs.

---
 lib/librte_eal/linuxapp/eal/eal.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -1
index f7511ab..a671ed4 100644
@@ -759,7 +759,11 @@ rte_eal_init(int argc, char **argv)
 	char thread_name[RTE_MAX_THREAD_NAME_LEN];
 
 	/* checks if the machine is adequate */
-	rte_cpu_check_supported();
+	if (!rte_cpu_is_supported()) {
+		rte_eal_init_alert("unsupported cpu type.");
+		rte_errno = ENOTSUP;
+		return -1;
+	}
 
 	if (!rte_atomic32_test_and_set(&run_once))
 		return -1;
Signature · 1 line
2.9.3
#103 lore raw

PATCH v5 8/26 eal: do not panic on memzone initialization fails

In reply to Aaron Conole

When memzone initialization fails, report the error to the calling application rather than panic(). Without a good way of detaching / releasing hugepages, at this point the application will have to restart.

---
 lib/librte_eal/linuxapp/eal/eal.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
index a671ed4..1e54ca1 100644
@@ -839,8 +839,11 @@ rte_eal_init(int argc, char **argv)
 	/* the directories are locked during eal_hugepage_info_init */
 	eal_hugedirs_unlock();
 
-	if (rte_eal_memzone_init() < 0)
-		rte_panic("Cannot init memzone\n");
+	if (rte_eal_memzone_init() < 0) {
+		RTE_LOG(ERR, EAL, "Cannot init memzone\n");
+		rte_errno = ENODEV;
+		return -1;
+	}
 
 	if (rte_eal_tailqs_init() < 0)
 		rte_panic("Cannot init tail queues for objects\n");
Signature · 1 line
2.9.3
#104 lore raw

Re: [PATCH v5 08/26] eal: do not panic on memzone initialization fails

Quoted 14 lines · Aaron Conole

When memzone initialization fails, report the error to the calling application rather than panic(). Without a good way of detaching / releasing hugepages, at this point the application will have to restart.

---
 lib/librte_eal/linuxapp/eal/eal.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +0 -0
index a671ed4..1e54ca1 100644

lib/librte_eal/linuxapp/eal/eal.c rte_eal_init(int argc, char **argv) · line 843

Quoted 7 lines
lib/librte_eal/linuxapp/eal/eal.c +2 -2
@@ -839,8 +839,11 @@ rte_eal_init(int argc, char **argv)
 	/* the directories are locked during eal_hugepage_info_init */
 	eal_hugedirs_unlock();
 
-	if (rte_eal_memzone_init() < 0)
-		rte_panic("Cannot init memzone\n");
+	if (rte_eal_memzone_init() < 0) {
+		RTE_LOG(ERR, EAL, "Cannot init memzone\n");

Any particular reason why not "rte_eal_init_alert" as with the other cases?

/Bruce

#105 lore raw

Re: [PATCH v5 08/26] eal: do not panic on memzone initialization fails

Quoted 18 lines · Bruce Richardson
Quoted 17 lines · Aaron Conole

When memzone initialization fails, report the error to the calling application rather than panic(). Without a good way of detaching / releasing hugepages, at this point the application will have to restart.

---
 lib/librte_eal/linuxapp/eal/eal.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +0 -0
index a671ed4..1e54ca1 100644
@@ -839,8 +839,11 @@ rte_eal_init(int argc, char **argv)
 	/* the directories are locked during eal_hugepage_info_init */
 	eal_hugedirs_unlock();
 
Quoted 7 lines

lib/librte_eal/linuxapp/eal/eal.c rte_eal_init(int argc, char **argv) · line 843

Quoted 4 lines
lib/librte_eal/linuxapp/eal/eal.c +2 -2
-	if (rte_eal_memzone_init() < 0)
-		rte_panic("Cannot init memzone\n");
+	if (rte_eal_memzone_init() < 0) {
+		RTE_LOG(ERR, EAL, "Cannot init memzone\n");

Any particular reason why not "rte_eal_init_alert" as with the other cases?

I only used rte_eal_init_alert() for cases which occur before logging happens, but I am not opposed to switching it for all the cases. I'll swap them all for v6.

Thanks again for the review, Bruce!

-Aaron

#106 lore raw

PATCH v5 9/26 eal: set errno when exiting for already called

In reply to Aaron Conole
---
 lib/librte_eal/linuxapp/eal/eal.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal.c +4 -1
index 1e54ca1..4b6c7b8 100644
@@ -765,8 +765,11 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 
-	if (!rte_atomic32_test_and_set(&run_once))
+	if (!rte_atomic32_test_and_set(&run_once)) {
+		rte_eal_init_alert("already called initialization.");
+		rte_errno = EALREADY;
 		return -1;
+	}
 
 	logid = strrchr(argv[0], '/');
 	logid = strdup(logid ? logid + 1: argv[0]);
Signature · 1 line
2.9.3
#107 lore raw

PATCH v5 10/26 eal: Do not panic on log failures

In reply to Aaron Conole

When log initialization fails, it's generally because the fopencookie failed. While this is rare in practice, it could happen, and it is likely because of memory pressure. So, flag the error, and allow the user to retry.

---
 lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
index 4b6c7b8..46bbaa7 100644
@@ -825,8 +825,12 @@ rte_eal_init(int argc, char **argv)
 
 	rte_config_init();
 
-	if (rte_eal_log_init(logid, internal_config.syslog_facility) < 0)
-		rte_panic("Cannot init logs\n");
+	if (rte_eal_log_init(logid, internal_config.syslog_facility) < 0) {
+		rte_eal_init_alert("Cannot init logging.");
+		rte_errno = ENOMEM;
+		rte_atomic32_clear(&run_once);
+		return -1;
+	}
 
 	if (rte_eal_pci_init() < 0)
 		rte_panic("Cannot init PCI\n");
Signature · 1 line
2.9.3
#108 lore raw

PATCH v5 11/26 eal: Do not panic on pci-probe

In reply to Aaron Conole

This will usually be an issue because of permissions. However, it could also be caused by OOM. In either case, errno will contain the underlying cause. It is safe to re-init the system here, so allow the application to take corrective action and reinit.

---
 lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
index 46bbaa7..c9f8c11 100644
@@ -832,8 +832,12 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 
-	if (rte_eal_pci_init() < 0)
-		rte_panic("Cannot init PCI\n");
+	if (rte_eal_pci_init() < 0) {
+		RTE_LOG(ERR, EAL, "Cannot init PCI\n");
+		rte_errno = EUNATCH;
+		rte_atomic32_clear(&run_once);
+		return -1;
+	}
 
 #ifdef VFIO_PRESENT
 	if (rte_eal_vfio_setup() < 0)
Signature · 1 line
2.9.3
#109 lore raw

PATCH v5 12/26 eal: do not panic on vfio failure

In reply to Aaron Conole
---
 lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
index c9f8c11..2e7faa8 100644
@@ -840,8 +840,12 @@ rte_eal_init(int argc, char **argv)
 	}
 
 #ifdef VFIO_PRESENT
-	if (rte_eal_vfio_setup() < 0)
-		rte_panic("Cannot init VFIO\n");
+	if (rte_eal_vfio_setup() < 0) {
+		RTE_LOG(ERR, EAL, "Cannot init VFIO\n");
+		rte_errno = EAGAIN;
+		rte_atomic32_clear(&run_once);
+		return -1;
+	}
 #endif
 
 	if (rte_eal_memory_init() < 0)
Signature · 1 line
2.9.3
#110 lore raw

PATCH v5 13/26 eal: do not panic on memory init

In reply to Aaron Conole

This can only happen when access to hugepages (either as primary or secondary process) fails (and that is usually permissions). Since the manner of failure is not reversible, we cannot allow retry.

---
 lib/librte_eal/linuxapp/eal/eal.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
index 2e7faa8..799f5f6 100644
@@ -848,8 +848,11 @@ rte_eal_init(int argc, char **argv)
 	}
 #endif
 
-	if (rte_eal_memory_init() < 0)
-		rte_panic("Cannot init memory\n");
+	if (rte_eal_memory_init() < 0) {
+		RTE_LOG(ERR, EAL, "Cannot init memory\n");
+		rte_errno = ENOMEM;
+		return -1;
+	}
 
 	/* the directories are locked during eal_hugepage_info_init */
 	eal_hugedirs_unlock();
Signature · 1 line
2.9.3
#111 lore raw

PATCH v5 14/26 eal: do not panic on tailq init

In reply to Aaron Conole

There are some theoretical racy conditions in the system that _could_ cause early tailq init to fail; however, no need to panic the application. While it can't continue using DPDK, it could make better alerts to the user.

---
 lib/librte_eal/common/eal_common_tailqs.c | 3 +--
 lib/librte_eal/linuxapp/eal/eal.c         | 7 +++++--
 2 files changed, 6 insertions(+), 4 deletions(-)
lib/librte_eal/common/eal_common_tailqs.c +1 -2
index bb08ec8..4f69828 100644
@@ -188,8 +188,7 @@ rte_eal_tailqs_init(void)
 		if (t->head == NULL) {
 			RTE_LOG(ERR, EAL,
 				"Cannot initialize tailq: %s\n", t->name);
-			/* no need to TAILQ_REMOVE, we are going to panic in
-			 * rte_eal_init() */
+			/* TAILQ_REMOVE not needed, error is already fatal */
 			goto fail;
 		}
 	}
lib/librte_eal/linuxapp/eal/eal.c +5 -2
index 799f5f6..e0767c0 100644
@@ -863,8 +863,11 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 
-	if (rte_eal_tailqs_init() < 0)
-		rte_panic("Cannot init tail queues for objects\n");
+	if (rte_eal_tailqs_init() < 0) {
+		RTE_LOG(ERR, EAL, "Cannot init tail queues for objects\n");
+		rte_errno = EFAULT;
+		return -1;
+	}
 
 	if (rte_eal_alarm_init() < 0)
 		rte_panic("Cannot init interrupt-handling thread\n");
Signature · 1 line
2.9.3
#112 lore raw

PATCH v5 15/26 eal: do not panic on alarm init

In reply to Aaron Conole

rte_eal_alarm_init() call uses the linux timerfd framework to create a poll()-able timer using standard posix file operations. This could fail for a few reasons given in the man-pages, but many could be corrected by the user application. No need to panic.

---
 lib/librte_eal/linuxapp/eal/eal.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
index e0767c0..23811bf 100644
@@ -869,8 +869,11 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 
-	if (rte_eal_alarm_init() < 0)
-		rte_panic("Cannot init interrupt-handling thread\n");
+	if (rte_eal_alarm_init() < 0) {
+		RTE_LOG(ERR, EAL, "Cannot init interrupt-handling thread\n");
+		/* rte_eal_alarm_init sets rte_errno on failure. */
+		return -1;
+	}
 
 	if (rte_eal_timer_init() < 0)
 		rte_panic("Cannot init HPET or TSC timers\n");
Signature · 1 line
2.9.3
#113 lore raw

PATCH v5 16/26 eal: convert timer_init not to call panic

In reply to Aaron Conole

After code inspection, there is no way for eal_timer_init() to fail. It simply returns 0 in all cases. As such, this test could either go-away or stay here as 'future-proofing'.

---
 lib/librte_eal/linuxapp/eal/eal.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
index 23811bf..81085d5 100644
@@ -875,8 +875,11 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 
-	if (rte_eal_timer_init() < 0)
-		rte_panic("Cannot init HPET or TSC timers\n");
+	if (rte_eal_timer_init() < 0) {
+		RTE_LOG(ERR, EAL, "Cannot init HPET or TSC timers\n");
+		rte_errno = ENOTSUP;
+		return -1;
+	}
 
 	eal_check_mem_on_local_socket();
 
Signature · 1 line
2.9.3
#114 lore raw

PATCH v5 17/26 eal: change the private pipe call to reflect errno

In reply to Aaron Conole

There could be some confusion as to why the call failed - this change will always reflect the value of the error in rte_error.

---
 lib/librte_eal/linuxapp/eal/eal_interrupts.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal_interrupts.c +4 -1
index 92a19cb..5bb833e 100644
@@ -898,13 +898,16 @@ rte_eal_intr_init(void)
 	 * create a pipe which will be waited by epoll and notified to
 	 * rebuild the wait list of epoll.
 	 */
-	if (pipe(intr_pipe.pipefd) < 0)
+	if (pipe(intr_pipe.pipefd) < 0) {
+		rte_errno = errno;
 		return -1;
+	}
 
 	/* create the host thread to wait/handle the interrupt */
 	ret = pthread_create(&intr_thread, NULL,
 			eal_intr_thread_main, NULL);
 	if (ret != 0) {
+		rte_errno = ret;
 		RTE_LOG(ERR, EAL,
 			"Failed to create thread for interrupt handling\n");
 	} else {
Signature · 1 line
2.9.3
#115 lore raw

PATCH v5 18/26 eal: Do not panic on interrupt thread init

In reply to Aaron Conole

When initializing the interrupt thread, there are a number of possible reasons for failure - some of which are correctable by the application. Do not panic() needlessly, and give the application a change to reflect this information to the user.

---
 lib/librte_eal/linuxapp/eal/eal.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +4 -2
index 81085d5..b4ae845 100644
@@ -894,8 +894,10 @@ rte_eal_init(int argc, char **argv)
 		rte_config.master_lcore, (int)thread_id, cpuset,
 		ret == 0 ? "" : "...");
 
-	if (rte_eal_intr_init() < 0)
-		rte_panic("Cannot init interrupt-handling thread\n");
+	if (rte_eal_intr_init() < 0) {
+		RTE_LOG(ERR, EAL, "Cannot init interrupt-handling thread\n");
+		return -1;
+	}
 
 	if (rte_bus_scan())
 		rte_panic("Cannot scan the buses for devices\n");
Signature · 1 line
2.9.3
#116 lore raw

PATCH v5 19/26 eal: do not error if plugins fail to init

In reply to Aaron Conole

Plugins are useful and important. However, it seems crazy to abort everything just because they don't initialize properly.

---
 lib/librte_eal/linuxapp/eal/eal.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal.c +1 -1
index b4ae845..291fa53 100644
@@ -884,7 +884,7 @@ rte_eal_init(int argc, char **argv)
 	eal_check_mem_on_local_socket();
 
 	if (eal_plugins_init() < 0)
-		rte_panic("Cannot init plugins\n");
+		RTE_LOG(ERR, EAL, "Cannot init plugins\n");
 
 	eal_thread_init_master(rte_config.master_lcore);
 
Signature · 1 line
2.9.3
#117 lore raw

PATCH v5 20/26 eal_pci: Continue probing even on failures

In reply to Aaron Conole

Some devices may be inaccessible for a variety of reasons, or the PCI-bus may be unavailable causing the whole thing to fail. Still, better to continue attempts at probes.

---
 lib/librte_eal/common/eal_common_pci.c | 15 ++++++++++-----
 1 file changed, 10 insertions(+), 5 deletions(-)
lib/librte_eal/common/eal_common_pci.c +10 -5
index 72547bd..9416190 100644
@@ -69,6 +69,7 @@
 #include <sys/queue.h>
 #include <sys/mman.h>
 
+#include <rte_errno.h>
 #include <rte_interrupts.h>
 #include <rte_log.h>
 #include <rte_pci.h>
@@ -416,6 +417,7 @@ rte_eal_pci_probe(void)
 	struct rte_pci_device *dev = NULL;
 	struct rte_devargs *devargs;
 	int probe_all = 0;
+	int ret_1 = 0;
 	int ret = 0;
 
 	if (rte_eal_devargs_type_count(RTE_DEVTYPE_WHITELISTED_PCI) == 0)
@@ -430,17 +432,20 @@ rte_eal_pci_probe(void)
 
 		/* probe all or only whitelisted devices */
 		if (probe_all)
-			ret = pci_probe_all_drivers(dev);
+			ret_1 = pci_probe_all_drivers(dev);
 		else if (devargs != NULL &&
 			devargs->type == RTE_DEVTYPE_WHITELISTED_PCI)
-			ret = pci_probe_all_drivers(dev);
-		if (ret < 0)
-			rte_exit(EXIT_FAILURE, "Requested device " PCI_PRI_FMT
+			ret_1 = pci_probe_all_drivers(dev);
+		if (ret_1 < 0) {
+			RTE_LOG(ERR, EAL, "Requested device " PCI_PRI_FMT
 				 " cannot be used\n", dev->addr.domain, dev->addr.bus,
 				 dev->addr.devid, dev->addr.function);
+			rte_errno = errno;
+			ret = 1;
+		}
 	}
 
-	return 0;
+	return -ret;
 }
 
 /* dump one device */
Signature · 1 line
2.9.3
#118 lore raw

PATCH v5 21/26 eal: do not panic on failed PCI probe

In reply to Aaron Conole

It may even be possible to simply log the error and continue on letting the user check the logs and restart the application when things are failed.

---
 lib/librte_eal/linuxapp/eal/eal.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
index 291fa53..fbc1dcc 100644
@@ -943,8 +943,11 @@ rte_eal_init(int argc, char **argv)
 		rte_panic("Cannot probe devices\n");
 
 	/* Probe & Initialize PCI devices */
-	if (rte_eal_pci_probe())
-		rte_panic("Cannot probe PCI\n");
+	if (rte_eal_pci_probe()) {
+		RTE_LOG(ERR, EAL, "Cannot probe PCI\n");
+		rte_errno = ENOTSUP;
+		return -1;
+	}
 
 	if (rte_eal_dev_init() < 0)
 		rte_panic("Cannot init pmd devices\n");
Signature · 1 line
2.9.3
#119 lore raw

PATCH v5 22/26 eal_common_dev: continue initializing vdevs

In reply to Aaron Conole

Even if one vdev should fail, there's no need to prevent further processing. Log the error, and reflect it to the higher levels to decide.

---
 lib/librte_eal/common/eal_common_dev.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
lib/librte_eal/common/eal_common_dev.c +3 -2
index 4f3b493..9889997 100644
@@ -80,6 +80,7 @@ int
 rte_eal_dev_init(void)
 {
 	struct rte_devargs *devargs;
+	int ret = 0;
 
 	/*
 	 * Note that the dev_driver_list is populated here
@@ -97,11 +98,11 @@ rte_eal_dev_init(void)
 					devargs->args)) {
 			RTE_LOG(ERR, EAL, "failed to initialize %s device\n",
 					devargs->virt.drv_name);
-			return -1;
+			ret = -1;
 		}
 	}
 
-	return 0;
+	return ret;
 }
 
 int rte_eal_dev_attach(const char *name, const char *devargs)
Signature · 1 line
2.9.3
#120 lore raw

PATCH v5 23/26 eal: do not panic (or abort) if vdev init fails

In reply to Aaron Conole

Seems like it's possible to continue. At least, the error is reflected properly in the logs. A user could then go and correct or investigate the situation.

---
 lib/librte_eal/linuxapp/eal/eal.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal.c +1 -1
index fbc1dcc..77a1950 100644
@@ -950,7 +950,7 @@ rte_eal_init(int argc, char **argv)
 	}
 
 	if (rte_eal_dev_init() < 0)
-		rte_panic("Cannot init pmd devices\n");
+		RTE_LOG(ERR, EAL, "Cannot init pmd devices\n");
 
 	rte_eal_mcfg_complete();
 
Signature · 1 line
2.9.3
#121 lore raw

PATCH v5 24/26 eal: do not panic when bus probe fails

In reply to Aaron Conole
---
 lib/librte_eal/linuxapp/eal/eal.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
index 77a1950..361256f 100644
@@ -939,8 +939,11 @@ rte_eal_init(int argc, char **argv)
 	rte_eal_mp_wait_lcore();
 
 	/* Probe all the buses and devices/drivers on them */
-	if (rte_bus_probe())
-		rte_panic("Cannot probe devices\n");
+	if (rte_bus_probe()) {
+		RTE_LOG(ERR, EAL, "Cannot probe devices\n");
+		rte_errno = ENOTSUP;
+		return -1;
+	}
 
 	/* Probe & Initialize PCI devices */
 	if (rte_eal_pci_probe()) {
Signature · 1 line
2.9.3
#122 lore raw

PATCH v5 25/26 eal: do not panic on failed bus scan

In reply to Aaron Conole

For now, do an abort. It's likely that even aborting the initialization is premature in this case, as it may be possible to proceed even if one bus or another is not available.

---
 lib/librte_eal/linuxapp/eal/eal.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
index 361256f..77f0d24 100644
@@ -899,8 +899,11 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 
-	if (rte_bus_scan())
-		rte_panic("Cannot scan the buses for devices\n");
+	if (rte_bus_scan()) {
+		RTE_LOG(ERR, EAL, "Cannot scan the buses for devices\n");
+		rte_errno = ENODEV;
+		return -1;
+	}
 
 	RTE_LCORE_FOREACH_SLAVE(i) {
 
Signature · 1 line
2.9.3
#123 lore raw

PATCH v5 26/26 rte_eal_init: add info about rte_errno codes

In reply to Aaron Conole

The rte_eal_init function will now pass failure reason hints to the application. To help app developers deciper this, add some brief information about what the codes are indicating.

---
 lib/librte_eal/common/include/rte_eal.h | 27 ++++++++++++++++++++++++++-
 1 file changed, 26 insertions(+), 1 deletion(-)
lib/librte_eal/common/include/rte_eal.h +26 -1
index 03fee50..9251244 100644
@@ -159,7 +159,32 @@ int rte_eal_iopl_init(void);
  *     function call and should not be further interpreted by the
  *     application.  The EAL does not take any ownership of the memory used
  *     for either the argv array, or its members.
- *   - On failure, a negative error value.
+ *   - On failure, -1 and rte_errno is set to a value indicating the cause
+ *     for failure.  In some instances, the application will need to be
+ *     restarted as part of clearing the issue.
+ *
+ *   Error codes returned via rte_errno:
+ *     EACCES indicates a permissions issue.
+ *
+ *     EAGAIN indicates either a bus or system resource was not available,
+ *            setup may be attempted again.
+ *
+ *     EALREADY indicates that the rte_eal_init function has already been
+ *              called, and cannot be called again.
+ *
+ *     EFAULT indicates the tailq configuration name was not found in
+ *            memory configuration.
+ *
+ *     EINVAL indicates invalid parameters were passed as argv/argc.
+ *
+ *     ENOMEM indicates failure likely caused by an out-of-memory condition.
+ *
+ *     ENODEV indicates memory setup issues.
+ *
+ *     ENOTSUP indicates that the EAL cannot initialize on this system.
+ *
+ *     EUNATCH indicates that the PCI bus is either not present, or is not
+ *             readable by the eal.
  */
 int rte_eal_init(int argc, char **argv);
 
Signature · 1 line
2.9.3
#124 lore raw
In reply to Aaron Conole
Quoted 6 lines · Aaron Conole

In many cases, it's enough to simply let the application know that the call to initialize DPDK has failed. A complete halt can then be decided by the application based on error returned (and the app could even attempt a possible re-attempt after some corrective action by the user or application).

Set looks pretty good. Just a couple of minor issues, apart from the one checkpatch issue I flagged:

* seems to be some inconsistency between using the "rte_eal_init_alert" function and RTE_LOG. If there is some reason why some cases use the function and others don't I think that might need to be called out somewhere.
* check-git-log flags a number of minor errors with commit titles and messages - most common is commit titles starting with a capital letter.

Otherwise:

#125 lore raw

v6 cover linux/eal: Remove most causes of panic on init

In reply to Aaron Conole

From: Aaron Conole <[email protected]>

In many cases, it's enough to simply let the application know that the call to initialize DPDK has failed. A complete halt can then be decided by the application based on error returned (and the app could even attempt a possible re-attempt after some corrective action by the user or application).

Changes ->v2:
- Audited all "RTE_LOG (" calls that were introduced, and converted to "RTE_LOG("
- Added some fprintf(stderr, "") lines to indicate errors before logging is initialized
- Removed assignments to errno.
- Changed patch 14/25 to reflect EFAULT, and document in 25/25

Changes ->v3:
- Checkpatch issues in patches 3 (spelling mistake), 9 (issue with leading spaces), and 19 (braces around single line statement if-condition)

Changes ->v4:
- Error text cleanup.
- Add a new check around rte_bus_scan(), added during the development of this series.

Changes ->v5:
- checkpatch.pl cleanup in patch 02/26
- move rte_errno.h include from patch 15 to patch 02
- remove stdbool.h and use int as return type in patch 06/26

Changes ->v6:
- convert all of the initialization calls to RTE_LOG() to rte_eal_init_alert()
- run through check-git-log and checkpatches
- add Bruce's ack to the series

I kept the rte_errno reflection, since this is control-path code and the init function returns a sentinel value of -1.

Aaron Conole (26):
eal: cpu init will no longer panic
eal: return error instead of panic for cpu init
eal: do not panic on hugepage info init
eal: do not panic on failed hugepage query
eal: do not panic if parsing args returns error
eal-common: introduce a way to query cpu support
eal: do not panic when CPU isn't supported
eal: do not panic on memzone initialization fails
eal: set errno when exiting for already called
eal: do not panic on log failures
eal: do not panic on PCI-probe
eal: do not panic on vfio failure
eal: do not panic on memory init
eal: do not panic on tailq init
eal: do not panic on alarm init
eal: convert timer init not to call panic
eal: change the private pipe call to reflect errno
eal: do not panic on interrupt thread init
eal: do not error if plugins fail to init
eal_pci: continue probing even on failures
eal: do not panic on failed PCI-probe
eal_common_dev: continue initializing vdevs
eal: do not panic (or abort) if vdev init fails
eal: do not panic when bus probe fails
eal: do not panic on failed bus scan
rte_eal_init: add info about various error codes

 lib/librte_eal/common/eal_common_cpuflags.c        |  13 +-
 lib/librte_eal/common/eal_common_dev.c             |   5 +-
 lib/librte_eal/common/eal_common_lcore.c           |   7 +-
 lib/librte_eal/common/eal_common_pci.c             |  15 ++-
 lib/librte_eal/common/eal_common_tailqs.c          |   3 +-
 .../common/include/generic/rte_cpuflags.h          |   8 ++
 lib/librte_eal/common/include/rte_eal.h            |  27 ++++-
 lib/librte_eal/linuxapp/eal/eal.c                  | 131 +++++++++++++++------
 lib/librte_eal/linuxapp/eal/eal_hugepage_info.c    |   9 +-
 lib/librte_eal/linuxapp/eal/eal_interrupts.c       |   5 +-
 10 files changed, 170 insertions(+), 53 deletions(-)
Signature · 1 line
2.9.3
#126 lore raw

PATCH v6 1/26 eal: cpu init will no longer panic

After this change, the EAL CPU NUMA node resolution step can no longer emit an rte_panic. This aligns with the code in rte_eal_init, which expects failures to return an error code.

---
 lib/librte_eal/common/eal_common_lcore.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
lib/librte_eal/common/eal_common_lcore.c +4 -3
index 2cd4132..84fa0cb 100644
@@ -83,16 +83,17 @@ rte_eal_cpu_init(void)
 		config->lcore_role[lcore_id] = ROLE_RTE;
 		lcore_config[lcore_id].core_id = eal_cpu_core_id(lcore_id);
 		lcore_config[lcore_id].socket_id = eal_cpu_socket_id(lcore_id);
-		if (lcore_config[lcore_id].socket_id >= RTE_MAX_NUMA_NODES)
+		if (lcore_config[lcore_id].socket_id >= RTE_MAX_NUMA_NODES) {
 #ifdef RTE_EAL_ALLOW_INV_SOCKET_ID
 			lcore_config[lcore_id].socket_id = 0;
 #else
-			rte_panic("Socket ID (%u) is greater than "
+			RTE_LOG(ERR, EAL, "Socket ID (%u) is greater than "
 				"RTE_MAX_NUMA_NODES (%d)\n",
 				lcore_config[lcore_id].socket_id,
 				RTE_MAX_NUMA_NODES);
+			return -1;
 #endif
-
+		}
 		RTE_LOG(DEBUG, EAL, "Detected lcore %u as "
 				"core %u on socket %u\n",
 				lcore_id, lcore_config[lcore_id].core_id,
Signature · 1 line
2.9.3
#127 lore raw

PATCH v6 2/26 eal: return error instead of panic for cpu init

In reply to Aaron Conole

There may be no way to gracefully recover, but the application should be notified that a failure happened, rather than completely aborting. This allows the user to proceed with a "slow-path" type solution.

---
 lib/librte_eal/linuxapp/eal/eal.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +12 -2
index bf6b818..81692e7 100644
@@ -61,6 +61,7 @@
 #include <rte_launch.h>
 #include <rte_eal.h>
 #include <rte_eal_memconfig.h>
+#include <rte_errno.h>
 #include <rte_per_lcore.h>
 #include <rte_lcore.h>
 #include <rte_log.h>
@@ -740,6 +741,12 @@ static int rte_eal_vfio_setup(void)
 }
 #endif
 
+static void rte_eal_init_alert(const char *msg)
+{
+	fprintf(stderr, "EAL: FATAL: %s\n", msg);
+	RTE_LOG(ERR, EAL, "%s\n", msg);
+}
+
 /* Launch threads, called at application init(). */
 int
 rte_eal_init(int argc, char **argv)
@@ -767,8 +774,11 @@ rte_eal_init(int argc, char **argv)
 	/* set log level as early as possible */
 	rte_set_log_level(internal_config.log_level);
 
-	if (rte_eal_cpu_init() < 0)
-		rte_panic("Cannot detect lcores\n");
+	if (rte_eal_cpu_init() < 0) {
+		rte_eal_init_alert("Cannot detect lcores.");
+		rte_errno = ENOTSUP;
+		return -1;
+	}
 
 	fctret = eal_parse_args(argc, argv);
 	if (fctret < 0)
Signature · 1 line
2.9.3
#128 lore raw

PATCH v6 3/26 eal: do not panic on hugepage info init

In reply to Aaron Conole

When attempting to scan hugepages, signal to the eal that an error has occurred, rather than performing a panic.

---
 lib/librte_eal/linuxapp/eal/eal_hugepage_info.c | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)
lib/librte_eal/linuxapp/eal/eal_hugepage_info.c +6 -3
index 18858e2..7a21e8f 100644
@@ -283,9 +283,12 @@ eal_hugepage_info_init(void)
 	struct dirent *dirent;
 
 	dir = opendir(sys_dir_path);
-	if (dir == NULL)
-		rte_panic("Cannot open directory %s to read system hugepage "
-			  "info\n", sys_dir_path);
+	if (dir == NULL) {
+		RTE_LOG(ERR, EAL,
+			"Cannot open directory %s to read system hugepage info\n",
+			sys_dir_path);
+		return -1;
+	}
 
 	for (dirent = readdir(dir); dirent != NULL; dirent = readdir(dir)) {
 		struct hugepage_info *hpi;
Signature · 1 line
2.9.3
#129 lore raw

PATCH v6 4/26 eal: do not panic on failed hugepage query

In reply to Aaron Conole

If we fail to acquire hugepage information, simply signal an error to the application. This clears the run_once counter, allowing the user or application to take a corrective action and retry.

---
 lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
index 81692e7..12bd941 100644
@@ -787,8 +787,12 @@ rte_eal_init(int argc, char **argv)
 	if (internal_config.no_hugetlbfs == 0 &&
 			internal_config.process_type != RTE_PROC_SECONDARY &&
 			internal_config.xen_dom0_support == 0 &&
-			eal_hugepage_info_init() < 0)
-		rte_panic("Cannot get hugepage information\n");
+			eal_hugepage_info_init() < 0) {
+		rte_eal_init_alert("Cannot get hugepage information.");
+		rte_errno = EACCES;
+		rte_atomic32_clear(&run_once);
+		return -1;
+	}
 
 	if (internal_config.memory == 0 && internal_config.force_sockets == 0) {
 		if (internal_config.no_hugetlbfs)
Signature · 1 line
2.9.3
#130 lore raw

PATCH v6 5/26 eal: do not panic if parsing args returns error

In reply to Aaron Conole

It's possible that the application could take a corrective action here, and either prompt the user for different arguments, or at least perform a better logging. Exiting this early prevents any useful information gathering from the application layer.

---
 lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
index 12bd941..f7511ab 100644
@@ -781,8 +781,12 @@ rte_eal_init(int argc, char **argv)
 	}
 
 	fctret = eal_parse_args(argc, argv);
-	if (fctret < 0)
-		exit(1);
+	if (fctret < 0) {
+		rte_eal_init_alert("Invalid 'command line' arguments.");
+		rte_errno = EINVAL;
+		rte_atomic32_clear(&run_once);
+		return -1;
+	}
 
 	if (internal_config.no_hugetlbfs == 0 &&
 			internal_config.process_type != RTE_PROC_SECONDARY &&
Signature · 1 line
2.9.3
#131 lore raw

PATCH v6 6/26 eal-common: introduce a way to query cpu support

In reply to Aaron Conole

This adds a new API to check for the eal cpu versions.

---
 lib/librte_eal/common/eal_common_cpuflags.c          | 13 +++++++++++--
 lib/librte_eal/common/include/generic/rte_cpuflags.h |  8 ++++++++
 2 files changed, 19 insertions(+), 2 deletions(-)
lib/librte_eal/common/eal_common_cpuflags.c +11 -2
index b5f76f7..9a2d080 100644
@@ -43,6 +43,13 @@
 void
 rte_cpu_check_supported(void)
 {
+	if (!rte_cpu_is_supported())
+		exit(1);
+}
+
+int
+rte_cpu_is_supported(void)
+{
 	/* This is generated at compile-time by the build system */
 	static const enum rte_cpu_flag_t compile_time_flags[] = {
 			RTE_COMPILE_TIME_CPUFLAGS
@@ -57,14 +64,16 @@ rte_cpu_check_supported(void)
 			fprintf(stderr,
 				"ERROR: CPU feature flag lookup failed with error %d\n",
 				ret);
-			exit(1);
+			return 0;
 		}
 		if (!ret) {
 			fprintf(stderr,
 			        "ERROR: This system does not support \"%s\".\n"
 			        "Please check that RTE_MACHINE is set correctly.\n",
 			        rte_cpu_get_flag_name(compile_time_flags[i]));
-			exit(1);
+			return 0;
 		}
 	}
+
+	return 1;
 }
lib/librte_eal/common/include/generic/rte_cpuflags.h +8 -0
index 71321f3..8d27031 100644
@@ -82,4 +82,12 @@ rte_cpu_get_flag_enabled(enum rte_cpu_flag_t feature);
 void
 rte_cpu_check_supported(void);
 
+/**
+ * This function checks that the currently used CPU supports the CPU features
+ * that were specified at compile time. It is called automatically within the
+ * EAL, so does not need to be used by applications.  This version returns a
+ * result so that decisions may be made (for instance, graceful shutdowns).
+ */
+int
+rte_cpu_is_supported(void);
 #endif /* _RTE_CPUFLAGS_H_ */
Signature · 1 line
2.9.3
#132 lore raw

Re: [PATCH v6 06/26] eal-common: introduce a way to query cpu support

2017-02-28 13:52, Aaron Conole:

lib/librte_eal/common/include/generic/rte_cpuflags.h rte_cpu_get_flag_enabled(enum rte_cpu_flag_t feature); · line 93

Quoted 9 lines
lib/librte_eal/common/include/generic/rte_cpuflags.h +8 -0
+/**
+ * This function checks that the currently used CPU supports the CPU features
+ * that were specified at compile time. It is called automatically within the
+ * EAL, so does not need to be used by applications.  This version returns a
+ * result so that decisions may be made (for instance, graceful shutdowns).
+ */
+int
+rte_cpu_is_supported(void);
 #endif /* _RTE_CPUFLAGS_H_ */

A blank line is missing.

#133 lore raw

PATCH v6 7/26 eal: do not panic when CPU isn't supported

In reply to Aaron Conole

It's now possible to gracefully exit the application, or for applications which support non-dpdk datapaths working in concert with DPDK datapaths, there no longer is the possibility of exiting for unsupported CPUs.

---
 lib/librte_eal/linuxapp/eal/eal.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -1
index f7511ab..a671ed4 100644
@@ -759,7 +759,11 @@ rte_eal_init(int argc, char **argv)
 	char thread_name[RTE_MAX_THREAD_NAME_LEN];
 
 	/* checks if the machine is adequate */
-	rte_cpu_check_supported();
+	if (!rte_cpu_is_supported()) {
+		rte_eal_init_alert("unsupported cpu type.");
+		rte_errno = ENOTSUP;
+		return -1;
+	}
 
 	if (!rte_atomic32_test_and_set(&run_once))
 		return -1;
Signature · 1 line
2.9.3
#134 lore raw

PATCH v6 8/26 eal: do not panic on memzone initialization fails

In reply to Aaron Conole

When memzone initialization fails, report the error to the calling application rather than panic(). Without a good way of detaching / releasing hugepages, at this point the application will have to restart.

---
 lib/librte_eal/linuxapp/eal/eal.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
index a671ed4..5a92b28 100644
@@ -839,8 +839,11 @@ rte_eal_init(int argc, char **argv)
 	/* the directories are locked during eal_hugepage_info_init */
 	eal_hugedirs_unlock();
 
-	if (rte_eal_memzone_init() < 0)
-		rte_panic("Cannot init memzone\n");
+	if (rte_eal_memzone_init() < 0) {
+		rte_eal_init_alert("Cannot init memzone\n");
+		rte_errno = ENODEV;
+		return -1;
+	}
 
 	if (rte_eal_tailqs_init() < 0)
 		rte_panic("Cannot init tail queues for objects\n");
Signature · 1 line
2.9.3
#135 lore raw

PATCH v6 9/26 eal: set errno when exiting for already called

In reply to Aaron Conole
---
 lib/librte_eal/linuxapp/eal/eal.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal.c +4 -1
index 5a92b28..564cac3 100644
@@ -765,8 +765,11 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 
-	if (!rte_atomic32_test_and_set(&run_once))
+	if (!rte_atomic32_test_and_set(&run_once)) {
+		rte_eal_init_alert("already called initialization.");
+		rte_errno = EALREADY;
 		return -1;
+	}
 
 	logid = strrchr(argv[0], '/');
 	logid = strdup(logid ? logid + 1: argv[0]);
Signature · 1 line
2.9.3
#136 lore raw

PATCH v6 10/26 eal: do not panic on log failures

In reply to Aaron Conole

When log initialization fails, it's generally because the fopencookie failed. While this is rare in practice, it could happen, and it is likely because of memory pressure. So, flag the error, and allow the user to retry.

---
 lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
index 564cac3..e1740a6 100644
@@ -825,8 +825,12 @@ rte_eal_init(int argc, char **argv)
 
 	rte_config_init();
 
-	if (rte_eal_log_init(logid, internal_config.syslog_facility) < 0)
-		rte_panic("Cannot init logs\n");
+	if (rte_eal_log_init(logid, internal_config.syslog_facility) < 0) {
+		rte_eal_init_alert("Cannot init logging.");
+		rte_errno = ENOMEM;
+		rte_atomic32_clear(&run_once);
+		return -1;
+	}
 
 	if (rte_eal_pci_init() < 0)
 		rte_panic("Cannot init PCI\n");
Signature · 1 line
2.9.3
#137 lore raw

PATCH v6 11/26 eal: do not panic on PCI-probe

In reply to Aaron Conole

This will usually be an issue because of permissions. However, it could also be caused by OOM. In either case, errno will contain the underlying cause. It is safe to re-init the system here, so allow the application to take corrective action and reinit.

---
 lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
index e1740a6..d5ef7b5 100644
@@ -832,8 +832,12 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 
-	if (rte_eal_pci_init() < 0)
-		rte_panic("Cannot init PCI\n");
+	if (rte_eal_pci_init() < 0) {
+		rte_eal_init_alert("Cannot init PCI\n");
+		rte_errno = EUNATCH;
+		rte_atomic32_clear(&run_once);
+		return -1;
+	}
 
 #ifdef VFIO_PRESENT
 	if (rte_eal_vfio_setup() < 0)
Signature · 1 line
2.9.3
#138 lore raw

PATCH v6 12/26 eal: do not panic on vfio failure

In reply to Aaron Conole
---
 lib/librte_eal/linuxapp/eal/eal.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +6 -2
index d5ef7b5..10eefd3 100644
@@ -840,8 +840,12 @@ rte_eal_init(int argc, char **argv)
 	}
 
 #ifdef VFIO_PRESENT
-	if (rte_eal_vfio_setup() < 0)
-		rte_panic("Cannot init VFIO\n");
+	if (rte_eal_vfio_setup() < 0) {
+		rte_eal_init_alert("Cannot init VFIO\n");
+		rte_errno = EAGAIN;
+		rte_atomic32_clear(&run_once);
+		return -1;
+	}
 #endif
 
 	if (rte_eal_memory_init() < 0)
Signature · 1 line
2.9.3
#139 lore raw

PATCH v6 13/26 eal: do not panic on memory init

In reply to Aaron Conole

This can only happen when access to hugepages (either as primary or secondary process) fails (and that is usually permissions). Since the manner of failure is not reversible, we cannot allow retry.

---
 lib/librte_eal/linuxapp/eal/eal.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
index 10eefd3..ae0beed 100644
@@ -848,8 +848,11 @@ rte_eal_init(int argc, char **argv)
 	}
 #endif
 
-	if (rte_eal_memory_init() < 0)
-		rte_panic("Cannot init memory\n");
+	if (rte_eal_memory_init() < 0) {
+		rte_eal_init_alert("Cannot init memory\n");
+		rte_errno = ENOMEM;
+		return -1;
+	}
 
 	/* the directories are locked during eal_hugepage_info_init */
 	eal_hugedirs_unlock();
Signature · 1 line
2.9.3
#140 lore raw

PATCH v6 14/26 eal: do not panic on tailq init

In reply to Aaron Conole

There are some theoretical racy conditions in the system that _could_ cause early tailq init to fail; however, no need to panic the application. While it can't continue using DPDK, it could make better alerts to the user.

---
 lib/librte_eal/common/eal_common_tailqs.c | 3 +--
 lib/librte_eal/linuxapp/eal/eal.c         | 7 +++++--
 2 files changed, 6 insertions(+), 4 deletions(-)
lib/librte_eal/common/eal_common_tailqs.c +1 -2
index bb08ec8..4f69828 100644
@@ -188,8 +188,7 @@ rte_eal_tailqs_init(void)
 		if (t->head == NULL) {
 			RTE_LOG(ERR, EAL,
 				"Cannot initialize tailq: %s\n", t->name);
-			/* no need to TAILQ_REMOVE, we are going to panic in
-			 * rte_eal_init() */
+			/* TAILQ_REMOVE not needed, error is already fatal */
 			goto fail;
 		}
 	}
lib/librte_eal/linuxapp/eal/eal.c +5 -2
index ae0beed..aa10192 100644
@@ -863,8 +863,11 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 
-	if (rte_eal_tailqs_init() < 0)
-		rte_panic("Cannot init tail queues for objects\n");
+	if (rte_eal_tailqs_init() < 0) {
+		rte_eal_init_alert("Cannot init tail queues for objects\n");
+		rte_errno = EFAULT;
+		return -1;
+	}
 
 	if (rte_eal_alarm_init() < 0)
 		rte_panic("Cannot init interrupt-handling thread\n");
Signature · 1 line
2.9.3
#141 lore raw

PATCH v6 15/26 eal: do not panic on alarm init

In reply to Aaron Conole

rte_eal_alarm_init() call uses the linux timerfd framework to create a poll()-able timer using standard posix file operations. This could fail for a few reasons given in the man-pages, but many could be corrected by the user application. No need to panic.

---
 lib/librte_eal/linuxapp/eal/eal.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
index aa10192..7aa6b6e 100644
@@ -869,8 +869,11 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 
-	if (rte_eal_alarm_init() < 0)
-		rte_panic("Cannot init interrupt-handling thread\n");
+	if (rte_eal_alarm_init() < 0) {
+		rte_eal_init_alert("Cannot init interrupt-handling thread\n");
+		/* rte_eal_alarm_init sets rte_errno on failure. */
+		return -1;
+	}
 
 	if (rte_eal_timer_init() < 0)
 		rte_panic("Cannot init HPET or TSC timers\n");
Signature · 1 line
2.9.3
#142 lore raw

PATCH v6 16/26 eal: convert timer init not to call panic

In reply to Aaron Conole

After code inspection, there is no way for eal_timer_init() to fail. It simply returns 0 in all cases. As such, this test could either go-away or stay here as 'future-proofing'.

---
 lib/librte_eal/linuxapp/eal/eal.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
index 7aa6b6e..73a7d38 100644
@@ -875,8 +875,11 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 
-	if (rte_eal_timer_init() < 0)
-		rte_panic("Cannot init HPET or TSC timers\n");
+	if (rte_eal_timer_init() < 0) {
+		rte_eal_init_alert("Cannot init HPET or TSC timers\n");
+		rte_errno = ENOTSUP;
+		return -1;
+	}
 
 	eal_check_mem_on_local_socket();
 
Signature · 1 line
2.9.3
#143 lore raw

PATCH v6 17/26 eal: change the private pipe call to reflect errno

In reply to Aaron Conole

There could be some confusion as to why the call failed - this change will always reflect the value of the error in rte_error.

---
 lib/librte_eal/linuxapp/eal/eal_interrupts.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal_interrupts.c +4 -1
index 92a19cb..5bb833e 100644
@@ -898,13 +898,16 @@ rte_eal_intr_init(void)
 	 * create a pipe which will be waited by epoll and notified to
 	 * rebuild the wait list of epoll.
 	 */
-	if (pipe(intr_pipe.pipefd) < 0)
+	if (pipe(intr_pipe.pipefd) < 0) {
+		rte_errno = errno;
 		return -1;
+	}
 
 	/* create the host thread to wait/handle the interrupt */
 	ret = pthread_create(&intr_thread, NULL,
 			eal_intr_thread_main, NULL);
 	if (ret != 0) {
+		rte_errno = ret;
 		RTE_LOG(ERR, EAL,
 			"Failed to create thread for interrupt handling\n");
 	} else {
Signature · 1 line
2.9.3
#144 lore raw

PATCH v6 18/26 eal: do not panic on interrupt thread init

In reply to Aaron Conole

When initializing the interrupt thread, there are a number of possible reasons for failure - some of which are correctable by the application. Do not panic() needlessly, and give the application a change to reflect this information to the user.

---
 lib/librte_eal/linuxapp/eal/eal.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +4 -2
index 73a7d38..d174ad4 100644
@@ -894,8 +894,10 @@ rte_eal_init(int argc, char **argv)
 		rte_config.master_lcore, (int)thread_id, cpuset,
 		ret == 0 ? "" : "...");
 
-	if (rte_eal_intr_init() < 0)
-		rte_panic("Cannot init interrupt-handling thread\n");
+	if (rte_eal_intr_init() < 0) {
+		rte_eal_init_alert("Cannot init interrupt-handling thread\n");
+		return -1;
+	}
 
 	if (rte_bus_scan())
 		rte_panic("Cannot scan the buses for devices\n");
Signature · 1 line
2.9.3
#145 lore raw

PATCH v6 19/26 eal: do not error if plugins fail to init

In reply to Aaron Conole

Plugins are useful and important. However, it seems crazy to abort everything just because they don't initialize properly.

---
 lib/librte_eal/linuxapp/eal/eal.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal.c +1 -1
index d174ad4..0ba9766 100644
@@ -884,7 +884,7 @@ rte_eal_init(int argc, char **argv)
 	eal_check_mem_on_local_socket();
 
 	if (eal_plugins_init() < 0)
-		rte_panic("Cannot init plugins\n");
+		rte_eal_init_alert("Cannot init plugins\n");
 
 	eal_thread_init_master(rte_config.master_lcore);
 
Signature · 1 line
2.9.3
#146 lore raw

PATCH v6 20/26 eal_pci: continue probing even on failures

In reply to Aaron Conole

Some devices may be inaccessible for a variety of reasons, or the PCI-bus may be unavailable causing the whole thing to fail. Still, better to continue attempts at probes.

---
 lib/librte_eal/common/eal_common_pci.c | 15 ++++++++++-----
 1 file changed, 10 insertions(+), 5 deletions(-)
lib/librte_eal/common/eal_common_pci.c +10 -5
index 72547bd..9416190 100644
@@ -69,6 +69,7 @@
 #include <sys/queue.h>
 #include <sys/mman.h>
 
+#include <rte_errno.h>
 #include <rte_interrupts.h>
 #include <rte_log.h>
 #include <rte_pci.h>
@@ -416,6 +417,7 @@ rte_eal_pci_probe(void)
 	struct rte_pci_device *dev = NULL;
 	struct rte_devargs *devargs;
 	int probe_all = 0;
+	int ret_1 = 0;
 	int ret = 0;
 
 	if (rte_eal_devargs_type_count(RTE_DEVTYPE_WHITELISTED_PCI) == 0)
@@ -430,17 +432,20 @@ rte_eal_pci_probe(void)
 
 		/* probe all or only whitelisted devices */
 		if (probe_all)
-			ret = pci_probe_all_drivers(dev);
+			ret_1 = pci_probe_all_drivers(dev);
 		else if (devargs != NULL &&
 			devargs->type == RTE_DEVTYPE_WHITELISTED_PCI)
-			ret = pci_probe_all_drivers(dev);
-		if (ret < 0)
-			rte_exit(EXIT_FAILURE, "Requested device " PCI_PRI_FMT
+			ret_1 = pci_probe_all_drivers(dev);
+		if (ret_1 < 0) {
+			RTE_LOG(ERR, EAL, "Requested device " PCI_PRI_FMT
 				 " cannot be used\n", dev->addr.domain, dev->addr.bus,
 				 dev->addr.devid, dev->addr.function);
+			rte_errno = errno;
+			ret = 1;
+		}
 	}
 
-	return 0;
+	return -ret;
 }
 
 /* dump one device */
Signature · 1 line
2.9.3
#147 lore raw

Re: [PATCH v6 20/26] eal_pci: continue probing even on failures

2017-02-28 13:53, Aaron Conole:

lib/librte_eal/common/eal_common_pci.c rte_eal_pci_probe(void) · line 420

Quoted 1 line
lib/librte_eal/common/eal_common_pci.c +1 -0
+	int ret_1 = 0;

You do not need to add a new variable.

Quoted 16 lines
lib/librte_eal/common/eal_common_pci.c +3 -4
 	int ret = 0;
 
 	if (rte_eal_devargs_type_count(RTE_DEVTYPE_WHITELISTED_PCI) == 0)
@@ -430,17 +432,20 @@ rte_eal_pci_probe(void)
 
 		/* probe all or only whitelisted devices */
 		if (probe_all)
-			ret = pci_probe_all_drivers(dev);
+			ret_1 = pci_probe_all_drivers(dev);
 		else if (devargs != NULL &&
 			devargs->type == RTE_DEVTYPE_WHITELISTED_PCI)
-			ret = pci_probe_all_drivers(dev);
-		if (ret < 0)
-			rte_exit(EXIT_FAILURE, "Requested device " PCI_PRI_FMT
+			ret_1 = pci_probe_all_drivers(dev);
+		if (ret_1 < 0) {

lib/librte_eal/common/eal_common_pci.c rte_eal_pci_probe(void) · line 448

Quoted 10 lines
lib/librte_eal/common/eal_common_pci.c +5 -1
+			RTE_LOG(ERR, EAL, "Requested device " PCI_PRI_FMT
 				 " cannot be used\n", dev->addr.domain, dev->addr.bus,
 				 dev->addr.devid, dev->addr.function);
+			rte_errno = errno;
+			ret = 1;
+		}
 	}
 
-	return 0;
+	return -ret;

It may be more explicit to use only one variable ret and filter the positive values:

	ret < 0 ? -1 : 0
#148 lore raw

PATCH v6 21/26 eal: do not panic on failed PCI-probe

In reply to Aaron Conole

Since PCI isn't neccessarily required, it may be possible to simply log the error and continue on letting the user check the logs and restart the application when things have failed.

---
 lib/librte_eal/linuxapp/eal/eal.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
index 0ba9766..b13e1dd 100644
@@ -943,8 +943,11 @@ rte_eal_init(int argc, char **argv)
 		rte_panic("Cannot probe devices\n");
 
 	/* Probe & Initialize PCI devices */
-	if (rte_eal_pci_probe())
-		rte_panic("Cannot probe PCI\n");
+	if (rte_eal_pci_probe()) {
+		rte_eal_init_alert("Cannot probe PCI\n");
+		rte_errno = ENOTSUP;
+		return -1;
+	}
 
 	if (rte_eal_dev_init() < 0)
 		rte_panic("Cannot init pmd devices\n");
Signature · 1 line
2.9.3
#149 lore raw

PATCH v6 22/26 eal_common_dev: continue initializing vdevs

In reply to Aaron Conole

Even if one vdev should fail, there's no need to prevent further processing. Log the error, and reflect it to the higher levels to decide.

---
 lib/librte_eal/common/eal_common_dev.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
lib/librte_eal/common/eal_common_dev.c +3 -2
index 4f3b493..9889997 100644
@@ -80,6 +80,7 @@ int
 rte_eal_dev_init(void)
 {
 	struct rte_devargs *devargs;
+	int ret = 0;
 
 	/*
 	 * Note that the dev_driver_list is populated here
@@ -97,11 +98,11 @@ rte_eal_dev_init(void)
 					devargs->args)) {
 			RTE_LOG(ERR, EAL, "failed to initialize %s device\n",
 					devargs->virt.drv_name);
-			return -1;
+			ret = -1;
 		}
 	}
 
-	return 0;
+	return ret;
 }
 
 int rte_eal_dev_attach(const char *name, const char *devargs)
Signature · 1 line
2.9.3
#150 lore raw

PATCH v6 23/26 eal: do not panic (or abort) if vdev init fails

In reply to Aaron Conole

Seems like it's possible to continue. At least, the error is reflected properly in the logs. A user could then go and correct or investigate the situation.

---
 lib/librte_eal/linuxapp/eal/eal.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
lib/librte_eal/linuxapp/eal/eal.c +1 -1
index b13e1dd..ddc50f2 100644
@@ -950,7 +950,7 @@ rte_eal_init(int argc, char **argv)
 	}
 
 	if (rte_eal_dev_init() < 0)
-		rte_panic("Cannot init pmd devices\n");
+		rte_eal_init_alert("Cannot init pmd devices\n");
 
 	rte_eal_mcfg_complete();
 
Signature · 1 line
2.9.3
#151 lore raw

PATCH v6 24/26 eal: do not panic when bus probe fails

In reply to Aaron Conole
---
 lib/librte_eal/linuxapp/eal/eal.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
index ddc50f2..8274196 100644
@@ -939,8 +939,11 @@ rte_eal_init(int argc, char **argv)
 	rte_eal_mp_wait_lcore();
 
 	/* Probe all the buses and devices/drivers on them */
-	if (rte_bus_probe())
-		rte_panic("Cannot probe devices\n");
+	if (rte_bus_probe()) {
+		rte_eal_init_alert("Cannot probe devices\n");
+		rte_errno = ENOTSUP;
+		return -1;
+	}
 
 	/* Probe & Initialize PCI devices */
 	if (rte_eal_pci_probe()) {
Signature · 1 line
2.9.3
#152 lore raw

PATCH v6 25/26 eal: do not panic on failed bus scan

In reply to Aaron Conole

For now, do an abort. It's likely that even aborting the initialization is premature in this case, as it may be possible to proceed even if one bus or another is not available.

---
 lib/librte_eal/linuxapp/eal/eal.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
lib/librte_eal/linuxapp/eal/eal.c +5 -2
index 8274196..6d6b825 100644
@@ -899,8 +899,11 @@ rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 
-	if (rte_bus_scan())
-		rte_panic("Cannot scan the buses for devices\n");
+	if (rte_bus_scan()) {
+		rte_eal_init_alert("Cannot scan the buses for devices\n");
+		rte_errno = ENODEV;
+		return -1;
+	}
 
 	RTE_LCORE_FOREACH_SLAVE(i) {
 
Signature · 1 line
2.9.3
#153 lore raw

PATCH v6 26/26 rte_eal_init: add info about various error codes

In reply to Aaron Conole

The rte_eal_init function will now pass failure reason hints to the application. To help app developers deciper this, add some brief information about what the codes are indicating.

---
 lib/librte_eal/common/include/rte_eal.h | 27 ++++++++++++++++++++++++++-
 1 file changed, 26 insertions(+), 1 deletion(-)
lib/librte_eal/common/include/rte_eal.h +26 -1
index 03fee50..9251244 100644
@@ -159,7 +159,32 @@ int rte_eal_iopl_init(void);
  *     function call and should not be further interpreted by the
  *     application.  The EAL does not take any ownership of the memory used
  *     for either the argv array, or its members.
- *   - On failure, a negative error value.
+ *   - On failure, -1 and rte_errno is set to a value indicating the cause
+ *     for failure.  In some instances, the application will need to be
+ *     restarted as part of clearing the issue.
+ *
+ *   Error codes returned via rte_errno:
+ *     EACCES indicates a permissions issue.
+ *
+ *     EAGAIN indicates either a bus or system resource was not available,
+ *            setup may be attempted again.
+ *
+ *     EALREADY indicates that the rte_eal_init function has already been
+ *              called, and cannot be called again.
+ *
+ *     EFAULT indicates the tailq configuration name was not found in
+ *            memory configuration.
+ *
+ *     EINVAL indicates invalid parameters were passed as argv/argc.
+ *
+ *     ENOMEM indicates failure likely caused by an out-of-memory condition.
+ *
+ *     ENODEV indicates memory setup issues.
+ *
+ *     ENOTSUP indicates that the EAL cannot initialize on this system.
+ *
+ *     EUNATCH indicates that the PCI bus is either not present, or is not
+ *             readable by the eal.
  */
 int rte_eal_init(int argc, char **argv);
 
Signature · 1 line
2.9.3
#154 lore raw
In reply to Aaron Conole

Hi,

Thanks for the work.
I think it needs to be completed to have the same behaviour on bsdapp.

As a another version is required, I add some small comments about the formatting.

I think you should use the form "do not panic on <step>" for most of the commits.

Some commits may be squashed (see below):

2017-02-28 13:52, Aaron Conole:

Quoted 3 lines

Aaron Conole (26):
eal: cpu init will no longer panic
eal: return error instead of panic for cpu init

squashed?

Quoted 2 lines

eal: do not panic on hugepage info init
eal: do not panic on failed hugepage query

squashed?

Quoted 3 lines

eal: do not panic if parsing args returns error
eal-common: introduce a way to query cpu support
eal: do not panic when CPU isn't supported

squashed?

Quoted 4 lines

eal: do not panic on memzone initialization fails
eal: set errno when exiting for already called
eal: do not panic on log failures
eal: do not panic on PCI-probe

It is not really the probe here

Quoted 10 lines

eal: do not panic on vfio failure
eal: do not panic on memory init
eal: do not panic on tailq init
eal: do not panic on alarm init
eal: convert timer init not to call panic
eal: change the private pipe call to reflect errno
eal: do not panic on interrupt thread init
eal: do not error if plugins fail to init
eal_pci: continue probing even on failures
eal: do not panic on failed PCI-probe

squashed?

Quoted 2 lines

eal_common_dev: continue initializing vdevs
eal: do not panic (or abort) if vdev init fails

squashed?

Quoted 3 lines

eal: do not panic when bus probe fails
eal: do not panic on failed bus scan
rte_eal_init: add info about various error codes

#155 lore raw
Quoted 4 lines · Thomas Monjalon

Hi,

Thanks for the work.
I think it needs to be completed to have the same behaviour on bsdapp.

Ideally, yes, but I also don't think the lack of BSD changes should block the inclusion of this set. In terms of application writers, the apps don't need to be written differently for BSD compared to Linux because of this change. All that is different is that the BSD version will panic rather than return the error code.

Regards,
/Bruce

Quoted 1 line
#156 lore raw

2017-03-09 09:11, Bruce Richardson:

Quoted 11 lines · Thomas Monjalon
Quoted 4 lines

Hi,

Thanks for the work.
I think it needs to be completed to have the same behaviour on bsdapp.

Ideally, yes, but I also don't think the lack of BSD changes should block the inclusion of this set. In terms of application writers, the apps don't need to be written differently for BSD compared to Linux because of this change. All that is different is that the BSD version will panic rather than return the error code.

So you do not have any issue about having a different behaviour on Linux and BSD?
You are the bsdapp maintainer, so it is your call.

#157 lore raw
Quoted 24 lines

-----Original Message-----
From: Thomas Monjalon [mailto:[email protected]]
Sent: Thursday, March 9, 2017 9:26 AM
To: Richardson, Bruce <[email protected]>
Cc: Aaron Conole <[email protected]>; [email protected]; Stephen Hemminger <[email protected]>
Subject: Re: [dpdk-dev] [PATCH v6 00/26] linux/eal: Remove most causes of panic on init

2017-03-09 09:11, Bruce Richardson:

Quoted 11 lines · Thomas Monjalon
Quoted 4 lines

Hi,

Thanks for the work.
I think it needs to be completed to have the same behaviour on bsdapp.

Ideally, yes, but I also don't think the lack of BSD changes should block the inclusion of this set. In terms of application writers, the apps don't need to be written differently for BSD compared to Linux because of this change. All that is different is that the BSD version will panic rather than return the error code.

Quoted 3 lines

So you do not have any issue about having a different behaviour on Linux and BSD?
You are the bsdapp maintainer, so it is your call.

I would infinitely prefer to have the same behavior. However, so long as this does not require a user to change their app to be different on BSD, I don't think lack of BSD support should block improving Linux.

Aaron - will you be able to do equivalent changes to BSD within the 17.05 timeframe?

/Bruce

#158 lore raw
Quoted 26 lines · Bruce
Quoted 22 lines

-----Original Message-----
From: Thomas Monjalon [mailto:[email protected]]
Sent: Thursday, March 9, 2017 9:26 AM
To: Richardson, Bruce <[email protected]>
Cc: Aaron Conole <[email protected]>; [email protected]; Stephen Hemminger <[email protected]>
Subject: Re: [dpdk-dev] [PATCH v6 00/26] linux/eal: Remove most causes of panic on init

2017-03-09 09:11, Bruce Richardson:

Quoted 11 lines · Thomas Monjalon
Quoted 4 lines

Hi,

Thanks for the work.
I think it needs to be completed to have the same behaviour on bsdapp.

Ideally, yes, but I also don't think the lack of BSD changes should block the inclusion of this set. In terms of application writers, the apps don't need to be written differently for BSD compared to Linux because of this change. All that is different is that the BSD version will panic rather than return the error code.

Quoted 3 lines

So you do not have any issue about having a different behaviour on Linux and BSD?
You are the bsdapp maintainer, so it is your call.

Quoted 5 lines

I would infinitely prefer to have the same behavior. However, so long as this does not require a user to change their app to be different on BSD, I don't think lack of BSD support should block improving Linux.

Aaron - will you be able to do equivalent changes to BSD within the 17.05 timeframe?

I'm going to spend some time next week looking into it. If it looks like it's akin to s/linuxapp/bsdapp/ and everything works, then I'll do it. If I get hung up on anything, I'll probably update with my status.

-Aaron