doc/eal: add caveat about spinlocks from non-pinned threads
PATCH 1/1 doc/eal: add caveat about spinlocks from non-pinned threads
Need to warn users of DPDK spinlocks from non-pinned threads. This is similar wording to Linux documentation in pthread_spin_init.
--- doc/guides/prog_guide/env_abstraction_layer.rst | 10 ++++++++++ 1 file changed, 10 insertions(+)
doc/guides/prog_guide/env_abstraction_layer.rst +10 -0
@@ -797,6 +797,16 @@ Known Issues The debug statistics of rte_ring, rte_mempool and rte_timer are not supported in an unregistered non-EAL pthread. ++ locking + + If a pthread, that is not pinned to an lcore acquires a lock such as a + DPDK based lock (rte_spinlock, rte_rwlock, rte_ticketlock, rte_mcslock) + then there is a possibility of large application delays. + The problem is that if a thread is scheduled off the CPU while it holds + a lock, then other threads will waste time spinning on the lock until + the lock holder is once more rescheduled and releases the lock. + + cgroup control ~~~~~~~~~~~~~~
Signature · 1 line
2.35.1
Quoted 4 lines · Stephen Hemminger
Need to warn users of DPDK spinlocks from non-pinned threads. This is similar wording to Linux documentation in
pthread_spin_init.Signed-off-by Stephen Hemminger <[email protected]>
Quoted 27 lines
--- doc/guides/prog_guide/env_abstraction_layer.rst | 10 ++++++++++ 1 file changed, 10 insertions(+)doc/guides/prog_guide/env_abstraction_layer.rst +10
-0@@ -797,6 +797,16 @@ Known Issues The debug statistics of rte_ring, rte_mempool and rte_timer are not supported in an unregistered non-EAL pthread. ++ locking + + If a pthread, that is not pinned to an lcore acquires a lock such as a + DPDK based lock (rte_spinlock, rte_rwlock, rte_ticketlock, rte_mcslock) + then there is a possibility of large application delays. + The problem is that if a thread is scheduled off the CPU while it holds + a lock, then other threads will waste time spinning on the lock until + the lock holder is once more rescheduled and releases the lock. + + cgroup control ~~~~~~~~~~~~~~Signature · 1 line
2.35.1
Quoted 13 lines · Stephen Hemminger
Need to warn users of DPDK spinlocks from non-pinned threads. This is similar wording to Linux documentation in
pthread_spin_init.Signed-off-by Stephen Hemminger <[email protected]>--- doc/guides/prog_guide/env_abstraction_layer.rst | 10 ++++++++++ 1 file changed, 10 insertions(+)doc/guides/prog_guide/env_abstraction_layer.rst +0
-0
doc/guides/prog_guide/env_abstraction_layer.rst Known Issues · line 802
Quoted 6 lines
doc/guides/prog_guide/env_abstraction_layer.rst +3
-0@@ -797,6 +797,16 @@ Known Issues The debug statistics of rte_ring, rte_mempool and rte_timer are not supported in an unregistered non-EAL pthread. ++ locking + + If a pthread, that is not pinned to an lcore acquires a lock such as a
nit: suggest not using term pthread but instead just say thread as not to imply a specific platform/implementation.
doc/guides/prog_guide/env_abstraction_layer.rst Known Issues · line 806
Quoted 4 lines
doc/guides/prog_guide/env_abstraction_layer.rst +4
-0+ DPDK based lock (rte_spinlock, rte_rwlock, rte_ticketlock, rte_mcslock) + then there is a possibility of large application delays. + The problem is that if a thread is scheduled off the CPU while it holds + a lock, then other threads will waste time spinning on the lock until
'until the lock holder' -> 'until the thread holding the lock'
but i'm not really fussed, just a suggestion.
Quoted 8 lines
doc/guides/prog_guide/env_abstraction_layer.rst +3
-0+ the lock holder is once more rescheduled and releases the lock. + + cgroup control ~~~~~~~~~~~~~~Signature · 1 line
2.35.1
Quoted 8 lines · On Fri, 10 Jun 2022 16:48:15 -0700 Tyler Retzlaff…
doc/guides/prog_guide/env_abstraction_layer.rst Known Issues · line 802
Quoted 4 lines
++ locking
doc/guides/prog_guide/env_abstraction_layer.rst +2
-0+ + If a pthread, that is not pinned to an lcore acquires a lock such as anit: suggest not using term pthread but instead just say thread as not to imply a specific platform/implementation.
Quoted 8 lines
doc/guides/prog_guide/env_abstraction_layer.rst Known Issues · line 806
Quoted 4 lines
doc/guides/prog_guide/env_abstraction_layer.rst +4
-0+ DPDK based lock (rte_spinlock, rte_rwlock, rte_ticketlock, rte_mcslock) + then there is a possibility of large application delays. + The problem is that if a thread is scheduled off the CPU while it holds + a lock, then other threads will waste time spinning on the lock until'until the lock holder' -> 'until the thread holding the lock'
but i'm not really fussed, just a suggestion.
Sure, that wording was from existing pthread_spin_init() man page
Quoted 13 lines · Stephen Hemminger
Need to warn users of DPDK spinlocks from non-pinned threads. This is similar wording to Linux documentation in
pthread_spin_init.Signed-off-by Stephen Hemminger <[email protected]>--- doc/guides/prog_guide/env_abstraction_layer.rst | 10 ++++++++++ 1 file changed, 10 insertions(+)doc/guides/prog_guide/env_abstraction_layer.rst +0
-0
doc/guides/prog_guide/env_abstraction_layer.rst Known Issues · line 803
Quoted 7 lines
doc/guides/prog_guide/env_abstraction_layer.rst +4
-0@@ -797,6 +797,16 @@ Known Issues The debug statistics of rte_ring, rte_mempool and rte_timer are not supported in an unregistered non-EAL pthread. ++ locking + + If a pthread, that is not pinned to an lcore acquires a lock such as a + DPDK based lock (rte_spinlock, rte_rwlock, rte_ticketlock, rte_mcslock)
Some APIs inherently use rte_spinlock, just like rte_malloc/rte_eal_alarm_set, Because DPDK API mainly use rte_spinlock to support thread-safty.
Suggest declare DPDK API mainly use rte_spinlock to support thread-safty, so if the caller thread is not pinned to an lcore may encount a possibility of large application delays.
Quoted 10 lines
doc/guides/prog_guide/env_abstraction_layer.rst +6
-0+ then there is a possibility of large application delays. + The problem is that if a thread is scheduled off the CPU while it holds + a lock, then other threads will waste time spinning on the lock until + the lock holder is once more rescheduled and releases the lock. + + cgroup control ~~~~~~~~~~~~~~
Quoted 25 lines · On Sat, 11 Jun 2022 09:55:00 +0800 fengchengwen <[email protected]> wrote:
Quoted 13 lines · Stephen Hemminger
Need to warn users of DPDK spinlocks from non-pinned threads. This is similar wording to Linux documentation in
pthread_spin_init.Signed-off-by Stephen Hemminger <[email protected]>--- doc/guides/prog_guide/env_abstraction_layer.rst | 10 ++++++++++ 1 file changed, 10 insertions(+)doc/guides/prog_guide/env_abstraction_layer.rst +0
-0doc/guides/prog_guide/env_abstraction_layer.rst Known Issues · line 803
Quoted 7 lines
doc/guides/prog_guide/env_abstraction_layer.rst +4
-0@@ -797,6 +797,16 @@ Known Issues The debug statistics of rte_ring, rte_mempool and rte_timer are not supported in an unregistered non-EAL pthread. ++ locking + + If a pthread, that is not pinned to an lcore acquires a lock such as a + DPDK based lock (rte_spinlock, rte_rwlock, rte_ticketlock, rte_mcslock)Some APIs inherently use
rte_spinlock, just like rte_malloc/rte_eal_alarm_set, Because DPDK API mainly userte_spinlockto support thread-safty.
Quoted 3 lines
Suggest declare DPDK API mainly use
rte_spinlockto support thread-safty, so if the caller thread is not pinned to an lcore may encount a possibility of large application delays.
I copied text from pthread_spinlock man page. The same caveats apply to pthread_spinlocks as DPDK; therefore using same wording seemed appropriate.
But it is worth mentioning that other API's may depend on spinlocks internally.
Quoted 13 lines · Stephen Hemminger
Need to warn users of DPDK spinlocks from non-pinned threads. This is similar wording to Linux documentation in
pthread_spin_init.Signed-off-by Stephen Hemminger <[email protected]>--- doc/guides/prog_guide/env_abstraction_layer.rst | 10 ++++++++++ 1 file changed, 10 insertions(+)doc/guides/prog_guide/env_abstraction_layer.rst +0
-0
doc/guides/prog_guide/env_abstraction_layer.rst Known Issues · line 801
Quoted 5 lines
doc/guides/prog_guide/env_abstraction_layer.rst +2
-0@@ -797,6 +797,16 @@ Known Issues The debug statistics of rte_ring, rte_mempool and rte_timer are not supported in an unregistered non-EAL pthread. ++ locking +
Isn't this problem more general than locks? The use of any non-preemption safe data structures potentially causes such delays. Regular DPDK rings for sure. The lock-less stack? The hash library?
Both actual and open-coded spinlocks internal to the APIs are also very common.
doc/guides/prog_guide/env_abstraction_layer.rst Known Issues · line 804
Quoted 3 lines
doc/guides/prog_guide/env_abstraction_layer.rst +3
-0+ If a pthread, that is not pinned to an lcore acquires a lock such as a + DPDK based lock (rte_spinlock, rte_rwlock, rte_ticketlock, rte_mcslock) + then there is a possibility of large application delays.
Pinning or not doesn't matter. What matters is if the thread is preempted and thus is prevented from making progress for a long time.
Quoted 8 lines
doc/guides/prog_guide/env_abstraction_layer.rst +5
-0+ The problem is that if a thread is scheduled off the CPU while it holds + a lock, then other threads will waste time spinning on the lock until + the lock holder is once more rescheduled and releases the lock. + + cgroup control ~~~~~~~~~~~~~~