Skip to contentSkip to conversation

doc: add multi-proc shared lib mempool note

PATCH Superseded Gage Eads 3 participants 6 messages Jun 18, 2019 – Jul 4, 2019

lore
Share & tools
lore

Gage Eads

author
#0 lore raw

PATCH 1/1 doc: add multi-proc shared lib mempool note

The mempool library assigns handler ops indexes based on the dynamic load order of mempool handlers. Indexes are used so a mempool can be used by multiple processes, but this only works if all processes agree on the mapping from index to mempool handler.

When using the '-d' argument, it's possible for different processes to load mempool handlers in different orders, and thus have different index->handler mappings. Using a mempool in multiple of such processes will result in undefined behavior.

This commit adds a note to the mempool library programmer's guide warning users against this.

Fixes 449c49b93a6b ("mempool: support handler operations")
---
 doc/guides/prog_guide/mempool_lib.rst | 7 +++++++
 1 file changed, 7 insertions(+)
doc/guides/prog_guide/mempool_lib.rst +7 -0
index 52a569f57..4470f6b38 100644
@@ -133,6 +133,13 @@ For applications that use ``rte_pktmbuf_create()``, there is a config setting
 (``RTE_MBUF_DEFAULT_MEMPOOL_OPS``) that allows the application to make use of
 an alternative mempool handler.
 
+  .. note::
+
+    When running a DPDK application with shared libraries, mempool handler
+    shared objects specified with the '-d' EAL command-line parameter are
+    dynamically loaded. When running a multi-process application with shared
+    libraries, the -d arguments for mempool handlers *must be specified in the
+    same order for all processes* to ensure correct operation.
 
 Use Cases
 ---------
Signature · 1 line
2.13.6
Acked-by on v1 1/1
#1 lore raw
Quoted 27 lines · Gage Eads

The mempool library assigns handler ops indexes based on the dynamic load order of mempool handlers. Indexes are used so a mempool can be used by multiple processes, but this only works if all processes agree on the mapping from index to mempool handler.

When using the '-d' argument, it's possible for different processes to load mempool handlers in different orders, and thus have different index->handler mappings. Using a mempool in multiple of such processes will result in undefined behavior.

This commit adds a note to the mempool library programmer's guide warning users against this.

Fixes 449c49b93a6b ("mempool: support handler operations")
---
  doc/guides/prog_guide/mempool_lib.rst | 7 +++++++
  1 file changed, 7 insertions(+)
doc/guides/prog_guide/mempool_lib.rst +0 -0
index 52a569f57..4470f6b38 100644
@@ -133,6 +133,13 @@ For applications that use ``rte_pktmbuf_create()``, there is a config setting
  (``RTE_MBUF_DEFAULT_MEMPOOL_OPS``) that allows the application to make use of

doc/guides/prog_guide/mempool_lib.rst For applications that use ``rte_pktmbuf_create()``, there is a config setting · line 143

Quoted 10 lines
doc/guides/prog_guide/mempool_lib.rst +7 -0
  an alternative mempool handler.
 
+  .. note::
+
+    When running a DPDK application with shared libraries, mempool handler
+    shared objects specified with the '-d' EAL command-line parameter are
+    dynamically loaded. When running a multi-process application with shared
+    libraries, the -d arguments for mempool handlers *must be specified in the
+    same order for all processes* to ensure correct operation.
 

One more empty line is required here, other than that:

I think it is OK as a fix for stable branches.

In theory I think it is still technically possible to guarantee ops indices correctness using dedicated memzone (as Olivier suggested), but with sync up on rte_mempool_{get,set}_ops() (which should be called when EAL is initialized) and reordering. However, it requires API breakage to avoid returning of ops_index on register (since it may change in secondary process on resync when EAL is already initialized).

author
#2 lore raw
Quoted 49 lines

-----Original Message-----
From: Andrew Rybchenko [mailto:[email protected]]
Sent: Thursday, June 20, 2019 1:01 PM
To: Eads, Gage <[email protected]>; [email protected]
Cc: [email protected]; [email protected]
Subject: Re: [PATCH] doc: add multi-proc shared lib mempool note

doc/guides/prog_guide/mempool_lib.rst For applications that use ``rte_pktmbuf_create()``,

Quoted 28 lines · Gage Eads

The mempool library assigns handler ops indexes based on the dynamic load order of mempool handlers. Indexes are used so a mempool can be used by multiple processes, but this only works if all processes agree on the mapping from index to mempool handler.

When using the '-d' argument, it's possible for different processes to load mempool handlers in different orders, and thus have different index->handler mappings. Using a mempool in multiple of such processes index->will
result in undefined behavior.

This commit adds a note to the mempool library programmer's guide warning users against this.

Fixes 449c49b93a6b ("mempool: support handler operations")
---
  doc/guides/prog_guide/mempool_lib.rst | 7 +++++++
  1 file changed, 7 insertions(+)
doc/guides/prog_guide/mempool_lib.rst +0 -0
index 52a569f57..4470f6b38 100644
@@ -133,6 +133,13 @@ For applications that use ``rte_pktmbuf_create()``,

there is a config setting

doc/guides/prog_guide/mempool_lib.rst For applications that use ``rte_pktmbuf_create()``, · line 133

Quoted 1 line
doc/guides/prog_guide/mempool_lib.rst +0 -0
  (``RTE_MBUF_DEFAULT_MEMPOOL_OPS``) that allows the application to

make use of

doc/guides/prog_guide/mempool_lib.rst For applications that use ``rte_pktmbuf_create()``, there is a config setting

Quoted 5 lines
doc/guides/prog_guide/mempool_lib.rst +3 -0
  an alternative mempool handler.
 
+  .. note::
+
+    When running a DPDK application with shared libraries, mempool

handler

doc/guides/prog_guide/mempool_lib.rst For applications that use ``rte_pktmbuf_create()``, · line 140

Quoted 2 lines
doc/guides/prog_guide/mempool_lib.rst +2 -0
+    shared objects specified with the '-d' EAL command-line parameter are
+    dynamically loaded. When running a multi-process application with
Quoted 8 lines

shared

doc/guides/prog_guide/mempool_lib.rst For applications that use ``rte_pktmbuf_create()``, · line 141

Quoted 1 line
doc/guides/prog_guide/mempool_lib.rst +1 -0
+    libraries, the -d arguments for mempool handlers *must be specified in

the

doc/guides/prog_guide/mempool_lib.rst For applications that use ``rte_pktmbuf_create()``, there is a config setting · line 143

Quoted 2 lines
doc/guides/prog_guide/mempool_lib.rst +1 -0
+    same order for all processes* to ensure correct operation.
 

One more empty line is required here, other than that:

Can do. Just for my understanding, why is the extra empty line required?

Quoted 11 lines

I think it is OK as a fix for stable branches.

In theory I think it is still technically possible to guarantee ops indices correctness using dedicated memzone (as Olivier suggested), but with sync up on
rte_mempool_{get,set}_ops() (which should be called when EAL is initialized) and
reordering. However, it requires API breakage to avoid returning of ops_index on register (since it may change in secondary process on resync when EAL is already initialized).

#3 lore raw
Quoted 47 lines · Gage
Quoted 47 lines

-----Original Message-----
From: Andrew Rybchenko [mailto:[email protected]]
Sent: Thursday, June 20, 2019 1:01 PM
To: Eads, Gage <[email protected]>; [email protected]
Cc: [email protected]; [email protected]
Subject: Re: [PATCH] doc: add multi-proc shared lib mempool note

doc/guides/prog_guide/mempool_lib.rst For applications that use ``rte_pktmbuf_create()``,

Quoted 28 lines · Gage Eads

The mempool library assigns handler ops indexes based on the dynamic load order of mempool handlers. Indexes are used so a mempool can be used by multiple processes, but this only works if all processes agree on the mapping from index to mempool handler.

When using the '-d' argument, it's possible for different processes to load mempool handlers in different orders, and thus have different index->handler mappings. Using a mempool in multiple of such processes index->will
result in undefined behavior.

This commit adds a note to the mempool library programmer's guide warning users against this.

Fixes 449c49b93a6b ("mempool: support handler operations")
---
   doc/guides/prog_guide/mempool_lib.rst | 7 +++++++
   1 file changed, 7 insertions(+)
doc/guides/prog_guide/mempool_lib.rst +0 -0
index 52a569f57..4470f6b38 100644
@@ -133,6 +133,13 @@ For applications that use ``rte_pktmbuf_create()``,

there is a config setting

doc/guides/prog_guide/mempool_lib.rst For applications that use ``rte_pktmbuf_create()``, · line 133

Quoted 1 line
doc/guides/prog_guide/mempool_lib.rst +0 -0
   (``RTE_MBUF_DEFAULT_MEMPOOL_OPS``) that allows the application to

make use of

doc/guides/prog_guide/mempool_lib.rst For applications that use ``rte_pktmbuf_create()``, there is a config setting

Quoted 5 lines
doc/guides/prog_guide/mempool_lib.rst +3 -0
   an alternative mempool handler.
 
+  .. note::
+
+    When running a DPDK application with shared libraries, mempool

handler

Quoted 2 lines
doc/guides/prog_guide/mempool_lib.rst +2 -0
+    shared objects specified with the '-d' EAL command-line parameter are
+    dynamically loaded. When running a multi-process application with
Quoted 8 lines
Quoted 7 lines

shared

doc/guides/prog_guide/mempool_lib.rst For applications that use ``rte_pktmbuf_create()``, · line 141

Quoted 1 line
doc/guides/prog_guide/mempool_lib.rst +1 -0
+    libraries, the -d arguments for mempool handlers *must be specified in

the

doc/guides/prog_guide/mempool_lib.rst For applications that use ``rte_pktmbuf_create()``, there is a config setting · line 143

Quoted 2 lines
doc/guides/prog_guide/mempool_lib.rst +1 -0
+    same order for all processes* to ensure correct operation.
 

One more empty line is required here, other than that:

Can do. Just for my understanding, why is the extra empty line required?

https://doc.dpdk.org/guides/contributing/documentation.html#whitespace
   * Add 2 blank lines before each section header.

author
#4 lore raw

PATCH v2 1/1 doc: add multi-proc shared lib mempool note

In reply to Gage Eads

The mempool library assigns handler ops indexes based on the dynamic load order of mempool handlers. Indexes are used so a mempool can be used by multiple processes, but this only works if all processes agree on the mapping from index to mempool handler.

When using the '-d' argument, it's possible for different processes to load mempool handlers in different orders, and thus have different index->handler mappings. Using a mempool in multiple of such processes will result in undefined behavior.

This commit adds a note to the mempool library programmer's guide warning users against this.

Fixes 449c49b93a6b ("mempool: support handler operations")
---
 doc/guides/prog_guide/mempool_lib.rst | 8 ++++++++
 1 file changed, 8 insertions(+)

v2: add another empty line

doc/guides/prog_guide/mempool_lib.rst +8 -0
index 52a569f57..3bb84b0a6 100644
@@ -133,6 +133,14 @@ For applications that use ``rte_pktmbuf_create()``, there is a config setting
 (``RTE_MBUF_DEFAULT_MEMPOOL_OPS``) that allows the application to make use of
 an alternative mempool handler.
 
+  .. note::
+
+    When running a DPDK application with shared libraries, mempool handler
+    shared objects specified with the '-d' EAL command-line parameter are
+    dynamically loaded. When running a multi-process application with shared
+    libraries, the -d arguments for mempool handlers *must be specified in the
+    same order for all processes* to ensure correct operation.
+
 
 Use Cases
 ---------
Signature · 1 line
2.13.6
#5 lore raw

21/06/2019 00:07, Gage Eads:

Quoted 10 lines

The mempool library assigns handler ops indexes based on the dynamic load order of mempool handlers. Indexes are used so a mempool can be used by multiple processes, but this only works if all processes agree on the mapping from index to mempool handler.

When using the '-d' argument, it's possible for different processes to load mempool handlers in different orders, and thus have different index->handler mappings. Using a mempool in multiple of such processes will result in undefined behavior.

Quoted 8 lines

This commit adds a note to the mempool library programmer's guide warning users against this.

Fixes 449c49b93a6b ("mempool: support handler operations")

Applied, thanks