Skip to contentSkip to conversation

eal/interrupts: add function to allow interruptible epoll

PATCH In review Stephen Hemminger 5 participants 8 messages Jul 24, 2019 – Oct 19, 2020

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 eal/interrupts: add function to allow interruptible epoll

The existing definition of rte_epoll_wait retries if interrupted by a signal. This behavior makes it hard to use rte_epoll_wait for applications that want to use signals do do things like exit polling loop and shutdown.

Since changing existing semantic might break applications, add a new rte_epoll_wait_interruptible() function that does the same thing as rte_epoll_wait but will return -1 and errno of EINTR if it receives a signal.

---
 .../common/include/rte_eal_interrupts.h       | 23 +++++++++
 lib/librte_eal/freebsd/eal/eal_interrupts.c   | 12 +++++
 lib/librte_eal/linux/eal/eal_interrupts.c     | 47 ++++++++++++-------
 lib/librte_eal/rte_eal_version.map            |  1 +
 4 files changed, 65 insertions(+), 18 deletions(-)
lib/librte_eal/common/include/rte_eal_interrupts.h +23 -0
index b370c0d26458..ac1c830f37a1 100644
@@ -87,6 +87,7 @@ struct rte_intr_handle {
 
 /**
  * It waits for events on the epoll instance.
+ * Retries if signal received.
  *
  * @param epfd
  *   Epoll instance fd on which the caller wait for events.
@@ -105,6 +106,28 @@ int
 rte_epoll_wait(int epfd, struct rte_epoll_event *events,
 	       int maxevents, int timeout);
 
+/**
+ * It waits for events on the epoll instance.
+ * Does not retry if signal received.
+ *
+ * @param epfd
+ *   Epoll instance fd on which the caller wait for events.
+ * @param events
+ *   Memory area contains the events that will be available for the caller.
+ * @param maxevents
+ *   Up to maxevents are returned, must greater than zero.
+ * @param timeout
+ *   Specifying a timeout of -1 causes a block indefinitely.
+ *   Specifying a timeout equal to zero cause to return immediately.
+ * @return
+ *   - On success, returns the number of available event.
+ *   - On failure, a negative value.
+ */
+__rte_experimental
+int
+rte_epoll_wait_interruptible(int epfd, struct rte_epoll_event *events,
+	       int maxevents, int timeout);
+
 /**
  * It performs control operations on epoll instance referred by the epfd.
  * It requests that the operation op be performed for the target fd.
lib/librte_eal/freebsd/eal/eal_interrupts.c +12 -0
index f6831b7902c5..d3f8fd78a1ee 100644
@@ -649,6 +649,18 @@ rte_epoll_wait(int epfd, struct rte_epoll_event *events,
 	return -ENOTSUP;
 }
 
+int
+rte_epoll_wait_interruptible(int epfd, struct rte_epoll_event *events,
+			     int maxevents, int timeout)
+{
+	RTE_SET_USED(epfd);
+	RTE_SET_USED(events);
+	RTE_SET_USED(maxevents);
+	RTE_SET_USED(timeout);
+
+	return -ENOTSUP;
+}
+
 int
 rte_epoll_ctl(int epfd, int op, int fd, struct rte_epoll_event *event)
 {
lib/librte_eal/linux/eal/eal_interrupts.c +29 -18

67 lines not on this page · raw message

lib/librte_eal/rte_eal_version.map +1 -0
index 234487787994..e1aeb99ba9c8 100644
@@ -331,6 +331,7 @@ EXPERIMENTAL {
 	rte_dev_hotplug_handle_enable;
 	rte_dev_iterator_init;
 	rte_dev_iterator_next;
+	rte_epoll_wait_interruptible;
 	rte_extmem_attach;
 	rte_extmem_detach;
 	rte_extmem_register;
Signature · 1 line
2.20.1
#1 lore raw
Quoted 11 lines · On Wed, 24 Jul 2019 10:20:37 -0700 Stephen Hemminger…

The existing definition of rte_epoll_wait retries if interrupted by a signal. This behavior makes it hard to use rte_epoll_wait for applications that want to use signals do do things like exit polling loop and shutdown.

Since changing existing semantic might break applications, add a new rte_epoll_wait_interruptible() function that does the same thing as rte_epoll_wait but will return -1 and errno of EINTR if it receives a signal.

This patch has been sitting unreviewed, uncommented, and unmerged for two releases. It seems only patches from HW vendors get the attention.

Reviewed-by
#2 lore raw
Quoted 8 lines · Stephen Hemminger
Quoted 5 lines · On Wed, 24 Jul 2019 10:20:37 -0700 Stephen Hemminger…

The existing definition of rte_epoll_wait retries if interrupted by a signal. This behavior makes it hard to use rte_epoll_wait for applications that want to use signals do do things like exit polling loop and shutdown.

Quoted 6 lines
Quoted 6 lines

Since changing existing semantic might break applications, add a new rte_epoll_wait_interruptible() function that does the same thing as rte_epoll_wait but will return -1 and errno of EINTR if it receives a signal.

Hi Stephen,

I have reviewed and tested the patch, looks good to me. Can you please rebase the patch to latest tree.

Thanks
Harman

Quoted 3 lines

This patch has been sitting unreviewed, uncommented, and unmerged for two releases. It seems only patches from HW vendors get the attention.

#3 lore raw
In reply to Stephen Hemminger

Hi Stephen,
Looks good overall, few nits inline.

Quoted 11 lines

-----Original Message-----
From: dev <[email protected]> On Behalf Of Stephen Hemminger
Sent: Wednesday, July 24, 2019 12:21 PM
To: [email protected]
Cc: Stephen Hemminger <[email protected]>
Subject: [dpdk-dev] [PATCH] eal/interrupts: add function to allow interruptible epoll

The existing definition of rte_epoll_wait retries if interrupted by a signal. This behavior makes it hard to use rte_epoll_wait for applications that want to use signals do do things like exit polling loop and shutdown.

nit ^^ to

Quoted 11 lines

Since changing existing semantic might break applications, add a new rte_epoll_wait_interruptible() function that does the same thing as rte_epoll_wait but will return -1 and errno of EINTR if it receives a signal.

---
 .../common/include/rte_eal_interrupts.h       | 23 +++++++++
 lib/librte_eal/freebsd/eal/eal_interrupts.c   | 12 +++++
 lib/librte_eal/linux/eal/eal_interrupts.c     | 47 ++++++++++++-------
 lib/librte_eal/rte_eal_version.map            |  1 +

Can you rebase with the master? It will help me apply this cleanly.

Quoted 20 lines
 4 files changed, 65 insertions(+), 18 deletions(-)
lib/librte_eal/common/include/rte_eal_interrupts.h +2 -0
index b370c0d26458..ac1c830f37a1 100644
@@ -87,6 +87,7 @@ struct rte_intr_handle {
 
 /**
  * It waits for events on the epoll instance.
+ * Retries if signal received.
  *
  * @param epfd
  *   Epoll instance fd on which the caller wait for events.
@@ -105,6 +106,28 @@ int
 rte_epoll_wait(int epfd, struct rte_epoll_event *events,
 	       int maxevents, int timeout);
 
+/**

lib/librte_eal/common/include/rte_eal_interrupts.h int · line 119

Quoted 10 lines
lib/librte_eal/common/include/rte_eal_interrupts.h +10 -0
+ * It waits for events on the epoll instance.
+ * Does not retry if signal received.
+ *
+ * @param epfd
+ *   Epoll instance fd on which the caller wait for events.
+ * @param events
+ *   Memory area contains the events that will be available for the caller.
+ * @param maxevents
+ *   Up to maxevents are returned, must greater than zero.
+ * @param timeout

It would be good to mention the units here, but might add complexities depending on the functionality supported by the underlying OS.

Quoted 61 lines
lib/librte_eal/common/include/rte_eal_interrupts.h +11 -0
+ *   Specifying a timeout of -1 causes a block indefinitely.
+ *   Specifying a timeout equal to zero cause to return immediately.
+ * @return
+ *   - On success, returns the number of available event.
+ *   - On failure, a negative value.
+ */
+__rte_experimental
+int
+rte_epoll_wait_interruptible(int epfd, struct rte_epoll_event *events,
+	       int maxevents, int timeout);
+
 /**
  * It performs control operations on epoll instance referred by the epfd.
  * It requests that the operation op be performed for the target fd.
lib/librte_eal/freebsd/eal/eal_interrupts.c +12 -0
index f6831b7902c5..d3f8fd78a1ee 100644
@@ -649,6 +649,18 @@ rte_epoll_wait(int epfd, struct rte_epoll_event
 *events,
 	return -ENOTSUP;
 }
 
+int
+rte_epoll_wait_interruptible(int epfd, struct rte_epoll_event *events,
+			     int maxevents, int timeout)
+{
+	RTE_SET_USED(epfd);
+	RTE_SET_USED(events);
+	RTE_SET_USED(maxevents);
+	RTE_SET_USED(timeout);
+
+	return -ENOTSUP;
+}
+
 int
 rte_epoll_ctl(int epfd, int op, int fd, struct rte_epoll_event *event)  { diff --

git a/lib/librte_eal/linux/eal/eal_interrupts.c
b/lib/librte_eal/linux/eal/eal_interrupts.c
index 1955324d3045..b6ae12a282e7 100644

lib/librte_eal/linux/eal/eal_interrupts.c +3 -3
@@ -1239,9 +1239,9 @@ rte_intr_tls_epfd(void)
 	return RTE_PER_LCORE(_epfd);
 }
 
-int
-rte_epoll_wait(int epfd, struct rte_epoll_event *events,
-	       int maxevents, int timeout)
+static int
+eal_epoll_wait(int epfd, struct rte_epoll_event *events,
+	       int maxevents, int timeout, bool interruptible)
 {
 	struct epoll_event evs[maxevents];
 	int rc;
@@ -1255,29 +1255,40 @@ rte_epoll_wait(int epfd, struct rte_epoll_event
 *events,
 	if (epfd == RTE_EPOLL_PER_THREAD)
 		epfd = rte_intr_tls_epfd();
 

lib/librte_eal/linux/eal/eal_interrupts.c rte_epoll_wait(int epfd, struct rte_epoll_event · line 1259

Quoted 10 lines
lib/librte_eal/linux/eal/eal_interrupts.c +1 -9
-	while (1) {
-		rc = epoll_wait(epfd, evs, maxevents, timeout);
-		if (likely(rc > 0)) {
-			/* epoll_wait has at least one fd ready to read */
-			rc = eal_epoll_process_event(evs, rc, events);
-			break;
-		} else if (rc < 0) {
-			if (errno == EINTR)
-				continue;
+retry:

I guess it is a personal choice. I personally prefer using goto for error handling.

Quoted 54 lines
lib/librte_eal/linux/eal/eal_interrupts.c +23 -6
+	rc = epoll_wait(epfd, evs, maxevents, timeout);
+	if (likely(rc > 0)) {
+		/* epoll_wait has at least one fd ready to read */
+		rc = eal_epoll_process_event(evs, rc, events);
+	} else if (rc < 0) {
+		if (errno == EINTR) {
+			if (!interruptible)
+				goto retry;
+		} else {
 			/* epoll_wait fail */
-			RTE_LOG(ERR, EAL, "epoll_wait returns with
 fail %s\n",
+			RTE_LOG(ERR, EAL,
+				"epoll_wait returns with fail %s\n",
 				strerror(errno));
-			rc = -1;
-			break;
-		} else {
-			/* rc == 0, epoll_wait timed out */
-			break;
 		}
 	}
 
 	return rc;
 }
 
+int
+rte_epoll_wait(int epfd, struct rte_epoll_event *events,
+	       int maxevents, int timeout)
+{
+	return eal_epoll_wait(epfd, events, maxevents, timeout, false); }
+
+int
+rte_epoll_wait_interruptible(int epfd, struct rte_epoll_event *events,
+			     int maxevents, int timeout)
+{
+	return eal_epoll_wait(epfd, events, maxevents, timeout, true); }
+
 static inline void
 eal_epoll_data_safe_free(struct rte_epoll_event *ev)  { diff --git

a/lib/librte_eal/rte_eal_version.map b/lib/librte_eal/rte_eal_version.map index 234487787994..e1aeb99ba9c8 100644

lib/librte_eal/rte_eal_version.map +1 -1
@@ -331,6 +331,7 @@ EXPERIMENTAL {
 	rte_dev_hotplug_handle_enable;
 	rte_dev_iterator_init;
 	rte_dev_iterator_next;
+	rte_epoll_wait_interruptible;
 	rte_extmem_attach;
 	rte_extmem_detach;
 	rte_extmem_register;
--

2.20.1

#4 lore raw

PATCH v2 1/1 eal/interrupts: add function to allow interruptible epoll

In reply to Stephen Hemminger

The existing definition of rte_epoll_wait retries if interrupted by a signal. This behavior makes it hard to use rte_epoll_wait for applications that want to use signals do do things like exit polling loop and shutdown.

Since changing existing semantic might break applications, add a new rte_epoll_wait_interruptible() function that does the same thing as rte_epoll_wait but will return -1 and errno of EINTR if it receives a signal.

---
v2 -- rebase onto current code organization
simplify logic to make it clear how common the code is

Note: the documentation of the new version is intentionally the the same as the original one.

 lib/librte_eal/freebsd/eal_interrupts.c     | 12 +++++++++
 lib/librte_eal/include/rte_eal_interrupts.h | 23 +++++++++++++++++
 lib/librte_eal/linux/eal_interrupts.c       | 28 +++++++++++++++++----
 lib/librte_eal/rte_eal_version.map          |  3 +++
 4 files changed, 61 insertions(+), 5 deletions(-)
lib/librte_eal/freebsd/eal_interrupts.c +12 -0
index 6d53d33c81a9..b07cff639318 100644
@@ -684,6 +684,18 @@ rte_epoll_wait(int epfd, struct rte_epoll_event *events,
 	return -ENOTSUP;
 }
 
+int
+rte_epoll_wait_interruptible(int epfd, struct rte_epoll_event *events,
+			     int maxevents, int timeout)
+{
+	RTE_SET_USED(epfd);
+	RTE_SET_USED(events);
+	RTE_SET_USED(maxevents);
+	RTE_SET_USED(timeout);
+
+	return -ENOTSUP;
+}
+
 int
 rte_epoll_ctl(int epfd, int op, int fd, struct rte_epoll_event *event)
 {
lib/librte_eal/include/rte_eal_interrupts.h +23 -0
index b1e8a2934e78..7d808a22624a 100644
@@ -87,6 +87,7 @@ struct rte_intr_handle {
 
 /**
  * It waits for events on the epoll instance.
+ * Retries if signal received.
  *
  * @param epfd
  *   Epoll instance fd on which the caller wait for events.
@@ -105,6 +106,28 @@ int
 rte_epoll_wait(int epfd, struct rte_epoll_event *events,
 	       int maxevents, int timeout);
 
+/**
+ * It waits for events on the epoll instance.
+ * Does not retry if signal received.
+ *
+ * @param epfd
+ *   Epoll instance fd on which the caller wait for events.
+ * @param events
+ *   Memory area contains the events that will be available for the caller.
+ * @param maxevents
+ *   Up to maxevents are returned, must greater than zero.
+ * @param timeout
+ *   Specifying a timeout of -1 causes a block indefinitely.
+ *   Specifying a timeout equal to zero cause to return immediately.
+ * @return
+ *   - On success, returns the number of available event.
+ *   - On failure, a negative value.
+ */
+__rte_experimental
+int
+rte_epoll_wait_interruptible(int epfd, struct rte_epoll_event *events,
+	       int maxevents, int timeout);
+
 /**
  * It performs control operations on epoll instance referred by the epfd.
  * It requests that the operation op be performed for the target fd.
lib/librte_eal/linux/eal_interrupts.c +23 -5
index 13db5c4e8aee..26b5dd4ca86b 100644
@@ -1275,9 +1275,9 @@ rte_intr_tls_epfd(void)
 	return RTE_PER_LCORE(_epfd);
 }
 
-int
-rte_epoll_wait(int epfd, struct rte_epoll_event *events,
-	       int maxevents, int timeout)
+static int
+eal_epoll_wait(int epfd, struct rte_epoll_event *events,
+	       int maxevents, int timeout, bool interruptible)
 {
 	struct epoll_event evs[maxevents];
 	int rc;
@@ -1298,8 +1298,12 @@ rte_epoll_wait(int epfd, struct rte_epoll_event *events,
 			rc = eal_epoll_process_event(evs, rc, events);
 			break;
 		} else if (rc < 0) {
-			if (errno == EINTR)
-				continue;
+			if (errno == EINTR) {
+				if (interruptible)
+					return -1;
+				else
+					continue;
+			}
 			/* epoll_wait fail */
 			RTE_LOG(ERR, EAL, "epoll_wait returns with fail %s\n",
 				strerror(errno));
@@ -1314,6 +1318,20 @@ rte_epoll_wait(int epfd, struct rte_epoll_event *events,
 	return rc;
 }
 
+int
+rte_epoll_wait(int epfd, struct rte_epoll_event *events,
+	       int maxevents, int timeout)
+{
+	return eal_epoll_wait(epfd, events, maxevents, timeout, false);
+}
+
+int
+rte_epoll_wait_interruptible(int epfd, struct rte_epoll_event *events,
+			     int maxevents, int timeout)
+{
+	return eal_epoll_wait(epfd, events, maxevents, timeout, true);
+}
+
 static inline void
 eal_epoll_data_safe_free(struct rte_epoll_event *ev)
 {
lib/librte_eal/rte_eal_version.map +3 -0
index 0b18e2ef85f9..a31a7cf5791a 100644
@@ -397,6 +397,9 @@ EXPERIMENTAL {
 	rte_mp_disable;
 	rte_thread_register;
 	rte_thread_unregister;
+
+	# added in 20.11
+	rte_epoll_wait_interruptible;
 };
 
 INTERNAL {
Signature · 1 line
2.27.0
#5 lore raw
Quoted 6 lines · Stephen Hemminger

The existing definition of rte_epoll_wait retries if interrupted by a signal. This behavior makes it hard to use rte_epoll_wait for applications that want to use signals do do things like exit polling loop and shutdown.

Quoted 7 lines

Since changing existing semantic might break applications, add a new rte_epoll_wait_interruptible() function that does the same thing as rte_epoll_wait but will return -1 and errno of EINTR if it receives a signal.

You will certainly argue that the existing function had no unit test but we want to fix this at some point.
Can a unit test be added?

Thanks.

Signature · 1 line
David Marchand
#6 lore raw
Quoted 13 lines · On Mon, 14 Sep 2020 10:56:33 +0200 David Marchand <[email protected]>…
Quoted 11 lines · Stephen Hemminger

The existing definition of rte_epoll_wait retries if interrupted by a signal. This behavior makes it hard to use rte_epoll_wait for applications that want to use signals do do things like exit polling loop and shutdown.

Since changing existing semantic might break applications, add a new rte_epoll_wait_interruptible() function that does the same thing as rte_epoll_wait but will return -1 and errno of EINTR if it receives a signal.

Quoted 6 lines
Quoted 2 lines

You will certainly argue that the existing function had no unit test but we want to fix this at some point.
Can a unit test be added?

The whole interrupt system really has no test, if it did then epoll would be part of that.

#7 lore raw
In reply to Stephen Hemminger

04/09/2020 01:28, Stephen Hemminger:

Quoted 5 lines

The existing definition of rte_epoll_wait retries if interrupted by a signal. This behavior makes it hard to use rte_epoll_wait for applications that want to use signals do do things like exit polling loop and shutdown.

Quoted 7 lines

Since changing existing semantic might break applications, add a new rte_epoll_wait_interruptible() function that does the same thing as rte_epoll_wait but will return -1 and errno of EINTR if it receives a signal.

Applied, thanks