Skip to contentSkip to conversation

doc/eal: add signal safety warning

PATCH Superseded Stephen Hemminger 6 participants 14 messages Jun 10, 2022 – Nov 15, 2022

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 doc/eal: add signal safety warning

The DPDK is not designed to be used from a signal handler. Add a notice in the documentation describing this limitation, similar to Linux signal-safety manual page.

Bugzilla ID: 1030

---
 doc/guides/prog_guide/env_abstraction_layer.rst | 13 +++++++++++++
 1 file changed, 13 insertions(+)
doc/guides/prog_guide/env_abstraction_layer.rst +13 -0
index 5f0748fba1c0..36ab4b5ba9b6 100644
@@ -732,6 +732,19 @@ controlled with tools like taskset (Linux) or cpuset (FreeBSD),
 - with affinity restricted to 2-3, the Control Threads will end up on
   CPU 2 (main lcore, which is the default when no CPU is available).
 
+Signal Safety
+~~~~~~~~~~~~~
+
+The DPDK functions in general can not be safely called from a signal handler.
+Most functions are not async-signal-safe because they can acquire locks
+and other resources that make them nonrentrant.
+
+To avoid problems with unsafe functions, can be avoided if required
+signals are blocked and a mechanism such as signalfd (Linux) is used
+to convert the asynchronous signals into messages that are processed
+by a EAL thread.
+
+
 .. _known_issue_label:
 
 Known Issues
Signature · 1 line
2.35.1
#1 lore raw
Quoted 16 lines · Stephen Hemminger

The DPDK is not designed to be used from a signal handler. Add a notice in the documentation describing this limitation, similar to Linux signal-safety manual page.

Bugzilla ID: 1030

---
 doc/guides/prog_guide/env_abstraction_layer.rst | 13 +++++++++++++
 1 file changed, 13 insertions(+)
doc/guides/prog_guide/env_abstraction_layer.rst +0 -0
index 5f0748fba1c0..36ab4b5ba9b6 100644
@@ -732,6 +732,19 @@ controlled with tools like taskset (Linux) or cpuset (FreeBSD),
 - with affinity restricted to 2-3, the Control Threads will end up on

doc/guides/prog_guide/env_abstraction_layer.rst controlled with tools like taskset (Linux) or cpuset (FreeBSD), · line 742

Quoted 10 lines
doc/guides/prog_guide/env_abstraction_layer.rst +8 -0
   CPU 2 (main lcore, which is the default when no CPU is available).
 
+Signal Safety
+~~~~~~~~~~~~~
+
+The DPDK functions in general can not be safely called from a signal handler.
+Most functions are not async-signal-safe because they can acquire locks
+and other resources that make them nonrentrant.
+
+To avoid problems with unsafe functions, can be avoided if required
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^

above doesn't quite read right for me, maybe a missing word / needs re-wording?

Quoted 10 lines
doc/guides/prog_guide/env_abstraction_layer.rst +5 -0
+signals are blocked and a mechanism such as signalfd (Linux) is used
+to convert the asynchronous signals into messages that are processed
+by a EAL thread.
+
+
 .. _known_issue_label:
 
 Known Issues
Signature · 1 line
2.35.1
#2 lore raw
Quoted 29 lines · On Fri, 10 Jun 2022 15:53:34 -0700 Tyler Retzlaff…
Quoted 16 lines · Stephen Hemminger

The DPDK is not designed to be used from a signal handler. Add a notice in the documentation describing this limitation, similar to Linux signal-safety manual page.

Bugzilla ID: 1030

---
 doc/guides/prog_guide/env_abstraction_layer.rst | 13 +++++++++++++
 1 file changed, 13 insertions(+)
doc/guides/prog_guide/env_abstraction_layer.rst +0 -0
index 5f0748fba1c0..36ab4b5ba9b6 100644
@@ -732,6 +732,19 @@ controlled with tools like taskset (Linux) or cpuset (FreeBSD),
 - with affinity restricted to 2-3, the Control Threads will end up on

doc/guides/prog_guide/env_abstraction_layer.rst controlled with tools like taskset (Linux) or cpuset (FreeBSD), · line 742

Quoted 10 lines
doc/guides/prog_guide/env_abstraction_layer.rst +8 -0
   CPU 2 (main lcore, which is the default when no CPU is available).
 
+Signal Safety
+~~~~~~~~~~~~~
+
+The DPDK functions in general can not be safely called from a signal handler.
+Most functions are not async-signal-safe because they can acquire locks
+and other resources that make them nonrentrant.
+
+To avoid problems with unsafe functions, can be avoided if required
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^
Quoted 2 lines

above doesn't quite read right for me, maybe a missing word / needs re-wording?

Yes, will reword that

Acked-by on v1 1/1
#3 lore raw
In reply to Stephen Hemminger
Quoted 6 lines · Stephen Hemminger

The DPDK is not designed to be used from a signal handler. Add a notice in the documentation describing this limitation, similar to Linux signal-safety manual page.

Bugzilla ID: 1030

Quoted 30 lines
---
 doc/guides/prog_guide/env_abstraction_layer.rst | 13 +++++++++++++
 1 file changed, 13 insertions(+)
doc/guides/prog_guide/env_abstraction_layer.rst +13 -0
index 5f0748fba1c0..36ab4b5ba9b6 100644
@@ -732,6 +732,19 @@ controlled with tools like taskset (Linux) or cpuset (FreeBSD),
 - with affinity restricted to 2-3, the Control Threads will end up on
   CPU 2 (main lcore, which is the default when no CPU is available).
 
+Signal Safety
+~~~~~~~~~~~~~
+
+The DPDK functions in general can not be safely called from a signal handler.
+Most functions are not async-signal-safe because they can acquire locks
+and other resources that make them nonrentrant.
+
+To avoid problems with unsafe functions, can be avoided if required
+signals are blocked and a mechanism such as signalfd (Linux) is used
+to convert the asynchronous signals into messages that are processed
+by a EAL thread.
+
+
 .. _known_issue_label:
 
 Known Issues
Signature · 1 line
2.35.1
Acked-by
#4 lore raw
Quoted 9 lines · Tyler Retzlaff
Quoted 6 lines · Stephen Hemminger

The DPDK is not designed to be used from a signal handler. Add a notice in the documentation describing this limitation, similar to Linux signal-safety manual page.

Bugzilla ID: 1030

Quoted 34 lines
Quoted 31 lines
---
 doc/guides/prog_guide/env_abstraction_layer.rst | 13 +++++++++++++
 1 file changed, 13 insertions(+)
doc/guides/prog_guide/env_abstraction_layer.rst +13 -0
index 5f0748fba1c0..36ab4b5ba9b6 100644
@@ -732,6 +732,19 @@ controlled with tools like taskset (Linux) or cpuset (FreeBSD),
 - with affinity restricted to 2-3, the Control Threads will end up on
   CPU 2 (main lcore, which is the default when no CPU is available).
 
+Signal Safety
+~~~~~~~~~~~~~
+
+The DPDK functions in general can not be safely called from a signal handler.
+Most functions are not async-signal-safe because they can acquire locks
+and other resources that make them nonrentrant.
+
+To avoid problems with unsafe functions, can be avoided if required
+signals are blocked and a mechanism such as signalfd (Linux) is used
+to convert the asynchronous signals into messages that are processed
+by a EAL thread.
+
+
 .. _known_issue_label:
 
 Known Issues
Signature · 1 line
2.35.1

.

#5 lore raw
In reply to Stephen Hemminger
Quoted 20 lines · Stephen Hemminger

The DPDK is not designed to be used from a signal handler. Add a notice in the documentation describing this limitation, similar to Linux signal-safety manual page.

Bugzilla ID: 1030

---
  doc/guides/prog_guide/env_abstraction_layer.rst | 13 +++++++++++++
  1 file changed, 13 insertions(+)
doc/guides/prog_guide/env_abstraction_layer.rst +2 -0
index 5f0748fba1c0..36ab4b5ba9b6 100644
@@ -732,6 +732,19 @@ controlled with tools like taskset (Linux) or cpuset (FreeBSD),
  - with affinity restricted to 2-3, the Control Threads will end up on
    CPU 2 (main lcore, which is the default when no CPU is available).
 
+Signal Safety
+~~~~~~~~~~~~~

doc/guides/prog_guide/env_abstraction_layer.rst controlled with tools like taskset (Linux) or cpuset (FreeBSD), · line 746

Quoted 10 lines
doc/guides/prog_guide/env_abstraction_layer.rst +10 -0
+
+The DPDK functions in general can not be safely called from a signal handler.
+Most functions are not async-signal-safe because they can acquire locks
+and other resources that make them nonrentrant.
+
+To avoid problems with unsafe functions, can be avoided if required
+signals are blocked and a mechanism such as signalfd (Linux) is used
+to convert the asynchronous signals into messages that are processed
+by a EAL thread.
+

Should we instead actually try to figure out what part of the API is and should remain async-signal-safe? And then say "nothing else is".

Without an exhaustive list, we will leave the user to guessing, or going into the current implementation to find out if a particular function is currently async-signal-safe. When that code changes in a future supposed-to-be-backward-compatible DPDK release, the application will break.

Quoted 4 lines
doc/guides/prog_guide/env_abstraction_layer.rst +1 -0
+
  .. _known_issue_label:
 
  Known Issues
#6 lore raw

PATCH v2 1/1 doc/eal: add signal safety warning

In reply to Stephen Hemminger

The DPDK is not designed to be used from a signal handler. Add a notice in the documentation describing this limitation, similar to Linux signal-safety manual page.

Bugzilla ID: 1030

---
 doc/guides/prog_guide/env_abstraction_layer.rst | 15 +++++++++++++++
 1 file changed, 15 insertions(+)
doc/guides/prog_guide/env_abstraction_layer.rst +15 -0
index 67842ae27207..de7ee92bba39 100644
@@ -818,6 +818,21 @@ Known Issues
 
   The debug statistics of rte_ring, rte_mempool and rte_timer are not supported in an unregistered non-EAL pthread.
 
++ signal safety
+
+  The DPDK library is not designed to be async-signal-safe.
+  Except where explicitly stated otherwise [#]_, the DPDK functions are nonreentrant and are unsafe to call from a signal handler.
+
+.. [#] Only the function ``rte_dump_stack()`` can safely be called from signal handler in this version of DPDK.
+
+.. note::
+  The kinds of issues that make DPDK functions unsafe can be understood when
+  one considers that much of the code in DPDK uses locks and other shared
+  resources. If a device driver holding a ``rte_spinlock`` is interrupted
+  by a signal and control operation is then performed that would acquire
+  the same lock, a deadlock would result.
+
+
 cgroup control
 ~~~~~~~~~~~~~~
 
Signature · 1 line
2.35.1
#7 lore raw

05/07/2022 22:44, Stephen Hemminger:

Quoted 17 lines

The DPDK is not designed to be used from a signal handler. Add a notice in the documentation describing this limitation, similar to Linux signal-safety manual page.

Bugzilla ID: 1030

---
 doc/guides/prog_guide/env_abstraction_layer.rst | 15 +++++++++++++++
 1 file changed, 15 insertions(+)
doc/guides/prog_guide/env_abstraction_layer.rst +0 -0
index 67842ae27207..de7ee92bba39 100644

doc/guides/prog_guide/env_abstraction_layer.rst Known Issues · line 826

Quoted 9 lines
doc/guides/prog_guide/env_abstraction_layer.rst +6 -0
@@ -818,6 +818,21 @@ Known Issues
 
   The debug statistics of rte_ring, rte_mempool and rte_timer are not supported in an unregistered non-EAL pthread.
 
++ signal safety
+
+  The DPDK library is not designed to be async-signal-safe.
+  Except where explicitly stated otherwise [#]_, the DPDK functions are nonreentrant and are unsafe to call from a signal handler.
+
+.. [#] Only the function ``rte_dump_stack()`` can safely be called from signal handler in this version of DPDK.

Really? Are you sure?

Note: the use of [#] is probably limited to a single usage in the page?

doc/guides/prog_guide/env_abstraction_layer.rst Known Issues · line 833

Quoted 7 lines
doc/guides/prog_guide/env_abstraction_layer.rst +7 -0
+
+.. note::
+  The kinds of issues that make DPDK functions unsafe can be understood when
+  one considers that much of the code in DPDK uses locks and other shared
+  resources. If a device driver holding a ``rte_spinlock`` is interrupted
+  by a signal and control operation is then performed that would acquire
+  the same lock, a deadlock would result.

I find this note quite confusing.

#8 lore raw
Quoted 21 lines · On Mon, 11 Jul 2022 23:15:26 +0200 Thomas Monjalon <[email protected]> wrote:

05/07/2022 22:44, Stephen Hemminger:

Quoted 20 lines

The DPDK is not designed to be used from a signal handler. Add a notice in the documentation describing this limitation, similar to Linux signal-safety manual page.

Bugzilla ID: 1030

---
 doc/guides/prog_guide/env_abstraction_layer.rst | 15 +++++++++++++++
 1 file changed, 15 insertions(+)
doc/guides/prog_guide/env_abstraction_layer.rst +0 -0
index 67842ae27207..de7ee92bba39 100644
@@ -818,6 +818,21 @@ Known Issues
 
   The debug statistics of rte_ring, rte_mempool and rte_timer are not supported in an unregistered non-EAL pthread.
 
Quoted 8 lines

doc/guides/prog_guide/env_abstraction_layer.rst Known Issues · line 826

Quoted 6 lines
doc/guides/prog_guide/env_abstraction_layer.rst +6 -0
++ signal safety
+
+  The DPDK library is not designed to be async-signal-safe.
+  Except where explicitly stated otherwise [#]_, the DPDK functions are nonreentrant and are unsafe to call from a signal handler.
+
+.. [#] Only the function ``rte_dump_stack()`` can safely be called from signal handler in this version of DPDK.  

Really? Are you sure?

There are some trivial ones that are signal safe that are pure functions. Like the bitfield and string functions.

But any function that matters such as rte_log, rte_mbuf, rte_mempool, and any driver function are going to be problematic.

Quoted 2 lines

Note: the use of [#] is probably limited to a single usage in the page?

Isn't it supposed to autonumber?

Quoted 10 lines

doc/guides/prog_guide/env_abstraction_layer.rst Known Issues · line 833

Quoted 8 lines
doc/guides/prog_guide/env_abstraction_layer.rst +7 -0
+
+.. note::
+  The kinds of issues that make DPDK functions unsafe can be understood when
+  one considers that much of the code in DPDK uses locks and other shared
+  resources. If a device driver holding a ``rte_spinlock`` is interrupted
+  by a signal and control operation is then performed that would acquire
+  the same lock, a deadlock would result.  

I find this note quite confusing.

I based it off what signal-safety says. Yes it really is that bad.

#9 lore raw
In reply to Thomas Monjalon
Quoted 30 lines · On Mon, 11 Jul 2022 23:15:26 +0200 Thomas Monjalon <[email protected]> wrote:

05/07/2022 22:44, Stephen Hemminger:

Quoted 17 lines

The DPDK is not designed to be used from a signal handler. Add a notice in the documentation describing this limitation, similar to Linux signal-safety manual page.

Bugzilla ID: 1030

---
 doc/guides/prog_guide/env_abstraction_layer.rst | 15 +++++++++++++++
 1 file changed, 15 insertions(+)
doc/guides/prog_guide/env_abstraction_layer.rst +0 -0
index 67842ae27207..de7ee92bba39 100644

doc/guides/prog_guide/env_abstraction_layer.rst Known Issues · line 826

Quoted 9 lines
doc/guides/prog_guide/env_abstraction_layer.rst +6 -0
@@ -818,6 +818,21 @@ Known Issues
 
   The debug statistics of rte_ring, rte_mempool and rte_timer are not supported in an unregistered non-EAL pthread.
 
++ signal safety
+
+  The DPDK library is not designed to be async-signal-safe.
+  Except where explicitly stated otherwise [#]_, the DPDK functions are nonreentrant and are unsafe to call from a signal handler.
+
+.. [#] Only the function ``rte_dump_stack()`` can safely be called from signal handler in this version of DPDK.  

Really? Are you sure?

Quoted 1 line

Note: the use of [#] is probably limited to a single usage in the page?

Yes. this is true, please apply this patch.

It is not safe to call functions that do any of the following:
printf
malloc
according to Linux async-signal-safe documentation

And because these all have locking or equivalent atomic the following in DPDK are unsafe.

rte_ring
rte_timer
rte_spinlock, rte_rwlock, ...
any PMD operations

#10 lore raw

PATCH v3 1/1 doc/eal: add signal safety warning

In reply to Stephen Hemminger

The DPDK is not designed to be used from a signal handler. Add a notice in the documentation describing this limitation, similar to Linux signal-safety manual page.

Bugzilla ID: 1030

---
 doc/guides/prog_guide/env_abstraction_layer.rst | 15 +++++++++++++++
 1 file changed, 15 insertions(+)
doc/guides/prog_guide/env_abstraction_layer.rst +15 -0
index 67842ae27207..de7ee92bba39 100644
@@ -818,6 +818,21 @@ Known Issues
 
   The debug statistics of rte_ring, rte_mempool and rte_timer are not supported in an unregistered non-EAL pthread.
 
++ signal safety
+
+  The DPDK library is not designed to be async-signal-safe.
+  Except where explicitly stated otherwise [#]_, the DPDK functions are nonreentrant and are unsafe to call from a signal handler.
+
+.. [#] Only the function ``rte_dump_stack()`` can safely be called from signal handler in this version of DPDK.
+
+.. note::
+  The kinds of issues that make DPDK functions unsafe can be understood when
+  one considers that much of the code in DPDK uses locks and other shared
+  resources. If a device driver holding a ``rte_spinlock`` is interrupted
+  by a signal and control operation is then performed that would acquire
+  the same lock, a deadlock would result.
+
+
 cgroup control
 ~~~~~~~~~~~~~~
 
Signature · 1 line
2.35.1
#11 lore raw

PATCH v4 1/1 doc/eal: add signal safety warning

In reply to Stephen Hemminger

The DPDK is not designed to be used from a signal handler. Add a notice in the documentation describing this limitation, similar to Linux signal-safety manual page.

Bugzilla ID: 1030

---
v4 - add more functions and clarify

v3 - mistake (ignore it)

 .../prog_guide/env_abstraction_layer.rst      | 70 +++++++++++++++++++
 1 file changed, 70 insertions(+)
doc/guides/prog_guide/env_abstraction_layer.rst +70 -0
index 67842ae27207..35fbebe1be04 100644
@@ -818,6 +818,76 @@ Known Issues
 
   The debug statistics of rte_ring, rte_mempool and rte_timer are not supported in an unregistered non-EAL pthread.
 
+Signal Safety
+~~~~~~~~~~~~~
+
+  The Posix API defines an async-signal-safe function as one that can be safely
+  called from with a signal handler. Many DPDK functions are non-reentrant and
+  therefore are unsafe to call from a signal handler.
+
+  The kinds of issues that make DPDK functions unsafe can be understood when
+  one considers that much of the code in DPDK uses locks and other shared
+  resources. For example, calling ``rte_mempool_lookup()`` from a signal
+  would deadlock if the signal happened during previous call ``rte_mempool``
+  routines.
+
+  Other functions are not signal safe because they use one or more
+  library routines that are not themselves signal safe.
+  For example, calling ``rte_panic()`` is not safe in a signal handler
+  because it uses ``rte_log()`` and ``rte_log()`` calls the
+  ``syslog()`` library function which is in the list of
+  signal safe functions in
+  `Signal-Safety manual page <https://man7.org/linux/man-pages/man7/signal-safety.7.html>`_.
+
+  The set of functions that are expected to be async-signal-safe in DPDK
+  is shown in the following table. The functions not otherwise noted
+  are not async-signal-safe.
+
+.. csv-table:: **Signal Safe Functions**
+   :header: "Function"
+   :widths: 32
+
+   rte_dump_stack
+   rte_eal_get_lcore_state
+   rte_eal_get_runtime_dir
+   rte_eal_has_hugepages
+   rte_eal_has_pci
+   rte_eal_lcore_role
+   rte_eal_process_type
+   rte_eal_using_phys_addrs
+   rte_get_hpet_cycles
+   rte_get_hpet_hz
+   rte_get_main_lcore
+   rte_get_next_lcore
+   rte_get_tsc_hz
+   rte_hypervisor_get
+   rte_hypervisor_get_name
+   rte_lcore_count
+   rte_lcore_cpuset
+   rte_lcore_has_role
+   rte_lcore_index
+   rte_lcore_is_enabled
+   rte_lcore_to_cpu_id
+   rte_lcore_to_socket_id
+   rte_log_get_global_level
+   rte_log_get_level
+   rte_memory_get_nchannel
+   rte_memory_get_nrank
+   rte_reciprocal_value
+   rte_reciprocal_value_u64
+   rte_socket_count
+   rte_socket_id
+   rte_socket_id_by_idx
+   rte_strerror
+   rte_strscpy
+   rte_strsplit
+   rte_sys_gettid
+   rte_uuid_compare
+   rte_uuid_is_null
+   rte_uuid_parse
+   rte_uuid_unparse
+
+
 cgroup control
 ~~~~~~~~~~~~~~
 
Signature · 1 line
2.35.1
#12 lore raw
Quoted 8 lines · On Mon, 11 Jul 2022 16:04:48 -0700 Stephen Hemminger…

The DPDK is not designed to be used from a signal handler. Add a notice in the documentation describing this limitation, similar to Linux signal-safety manual page.

Bugzilla ID: 1030

There have been reports of this problem, multiple ack's of this patch but it is still not merged. It documents a real problem that will trap users.