doc: add multi-proc shared lib mempool note
PATCH 1/1 doc: add multi-proc shared lib mempool note
The mempool library assigns handler ops indexes based on the dynamic load order of mempool handlers. Indexes are used so a mempool can be used by multiple processes, but this only works if all processes agree on the mapping from index to mempool handler.
When using the '-d' argument, it's possible for different processes to load mempool handlers in different orders, and thus have different index->handler mappings. Using a mempool in multiple of such processes will result in undefined behavior.
This commit adds a note to the mempool library programmer's guide warning users against this.
--- doc/guides/prog_guide/mempool_lib.rst | 7 +++++++ 1 file changed, 7 insertions(+)
doc/guides/prog_guide/mempool_lib.rst +7 -0
@@ -133,6 +133,13 @@ For applications that use ``rte_pktmbuf_create()``, there is a config setting (``RTE_MBUF_DEFAULT_MEMPOOL_OPS``) that allows the application to make use of an alternative mempool handler. + .. note:: + + When running a DPDK application with shared libraries, mempool handler + shared objects specified with the '-d' EAL command-line parameter are + dynamically loaded. When running a multi-process application with shared + libraries, the -d arguments for mempool handlers *must be specified in the + same order for all processes* to ensure correct operation. Use Cases ---------
Signature · 1 line
2.13.6
Quoted 27 lines · Gage Eads
The mempool library assigns handler ops indexes based on the dynamic load order of mempool handlers. Indexes are used so a mempool can be used by multiple processes, but this only works if all processes agree on the mapping from index to mempool handler.
When using the '-d' argument, it's possible for different processes to load mempool handlers in different orders, and thus have different
index->handlermappings. Using a mempool in multiple of such processes will result in undefined behavior.This commit adds a note to the mempool library programmer's guide warning users against this.
Fixes 449c49b93a6b ("mempool: support handler operations")Signed-off-by Gage Eads <[email protected]>--- doc/guides/prog_guide/mempool_lib.rst | 7 +++++++ 1 file changed, 7 insertions(+)doc/guides/prog_guide/mempool_lib.rst +0
-0@@ -133,6 +133,13 @@ For applications that use ``rte_pktmbuf_create()``, there is a config setting (``RTE_MBUF_DEFAULT_MEMPOOL_OPS``) that allows the application to make use of
doc/guides/prog_guide/mempool_lib.rst For applications that use ``rte_pktmbuf_create()``, there is a config setting · line 143
Quoted 10 lines
doc/guides/prog_guide/mempool_lib.rst +7
-0an alternative mempool handler. + .. note:: + + When running a DPDK application with shared libraries, mempool handler + shared objects specified with the '-d' EAL command-line parameter are + dynamically loaded. When running a multi-process application with shared + libraries, the -d arguments for mempool handlers *must be specified in the + same order for all processes* to ensure correct operation.
One more empty line is required here, other than that:
I think it is OK as a fix for stable branches.
In theory I think it is still technically possible to guarantee ops indices correctness using dedicated memzone (as Olivier suggested), but with sync up on rte_mempool_{get,set}_ops() (which should be called when EAL is initialized) and reordering. However, it requires API breakage to avoid returning of ops_index on register (since it may change in secondary process on resync when EAL is already initialized).
Quoted 49 lines
-----Original Message-----
From: Andrew Rybchenko [mailto:[email protected]]
Sent: Thursday, June 20, 2019 1:01 PM
To: Eads, Gage <[email protected]>; [email protected]
Cc: [email protected]; [email protected]
Subject: Re: [PATCH] doc: add multi-proc shared lib mempool notedoc/guides/prog_guide/mempool_lib.rst For applications that use ``rte_pktmbuf_create()``,
Quoted 28 lines · Gage Eads
The mempool library assigns handler ops indexes based on the dynamic load order of mempool handlers. Indexes are used so a mempool can be used by multiple processes, but this only works if all processes agree on the mapping from index to mempool handler.
When using the '-d' argument, it's possible for different processes to load mempool handlers in different orders, and thus have different
index->handlermappings. Using a mempool in multiple of such processesindex->will
result in undefined behavior.This commit adds a note to the mempool library programmer's guide warning users against this.
Fixes 449c49b93a6b ("mempool: support handler operations")Signed-off-by Gage Eads <[email protected]>--- doc/guides/prog_guide/mempool_lib.rst | 7 +++++++ 1 file changed, 7 insertions(+)doc/guides/prog_guide/mempool_lib.rst +0
-0@@ -133,6 +133,13 @@ For applications that use ``rte_pktmbuf_create()``,there is a config setting
doc/guides/prog_guide/mempool_lib.rst For applications that use ``rte_pktmbuf_create()``, · line 133
Quoted 1 line
doc/guides/prog_guide/mempool_lib.rst +0
-0(``RTE_MBUF_DEFAULT_MEMPOOL_OPS``) that allows the application tomake use of
doc/guides/prog_guide/mempool_lib.rst For applications that use ``rte_pktmbuf_create()``, there is a config setting
Quoted 5 lines
doc/guides/prog_guide/mempool_lib.rst +3
-0an alternative mempool handler. + .. note:: + + When running a DPDK application with shared libraries, mempoolhandler
doc/guides/prog_guide/mempool_lib.rst For applications that use ``rte_pktmbuf_create()``, · line 140
Quoted 2 lines
doc/guides/prog_guide/mempool_lib.rst +2
-0+ shared objects specified with the '-d' EAL command-line parameter are + dynamically loaded. When running a multi-process application with
Quoted 8 lines
shared
doc/guides/prog_guide/mempool_lib.rst For applications that use ``rte_pktmbuf_create()``, · line 141
Quoted 1 line
doc/guides/prog_guide/mempool_lib.rst +1
-0+ libraries, the -d arguments for mempool handlers *must be specified inthe
doc/guides/prog_guide/mempool_lib.rst For applications that use ``rte_pktmbuf_create()``, there is a config setting · line 143
Quoted 2 lines
doc/guides/prog_guide/mempool_lib.rst +1
-0+ same order for all processes* to ensure correct operation.One more empty line is required here, other than that:
Acked-by Andrew Rybchenko <[email protected]>
Can do. Just for my understanding, why is the extra empty line required?
Quoted 11 lines
I think it is OK as a fix for stable branches.
In theory I think it is still technically possible to guarantee ops indices correctness using dedicated memzone (as Olivier suggested), but with sync up on
rte_mempool_{get,set}_ops() (which should be called when EAL is initialized) and
reordering. However, it requires API breakage to avoid returning ofops_indexon register (since it may change in secondary process on resync when EAL is already initialized).
Quoted 47 lines · Gage
Quoted 47 lines
-----Original Message-----
From: Andrew Rybchenko [mailto:[email protected]]
Sent: Thursday, June 20, 2019 1:01 PM
To: Eads, Gage <[email protected]>; [email protected]
Cc: [email protected]; [email protected]
Subject: Re: [PATCH] doc: add multi-proc shared lib mempool notedoc/guides/prog_guide/mempool_lib.rst For applications that use ``rte_pktmbuf_create()``,
Quoted 28 lines · Gage Eads
The mempool library assigns handler ops indexes based on the dynamic load order of mempool handlers. Indexes are used so a mempool can be used by multiple processes, but this only works if all processes agree on the mapping from index to mempool handler.
When using the '-d' argument, it's possible for different processes to load mempool handlers in different orders, and thus have different
index->handlermappings. Using a mempool in multiple of such processesindex->will
result in undefined behavior.This commit adds a note to the mempool library programmer's guide warning users against this.
Fixes 449c49b93a6b ("mempool: support handler operations")Signed-off-by Gage Eads <[email protected]>--- doc/guides/prog_guide/mempool_lib.rst | 7 +++++++ 1 file changed, 7 insertions(+)doc/guides/prog_guide/mempool_lib.rst +0
-0@@ -133,6 +133,13 @@ For applications that use ``rte_pktmbuf_create()``,there is a config setting
doc/guides/prog_guide/mempool_lib.rst For applications that use ``rte_pktmbuf_create()``, · line 133
Quoted 1 line
doc/guides/prog_guide/mempool_lib.rst +0
-0(``RTE_MBUF_DEFAULT_MEMPOOL_OPS``) that allows the application tomake use of
doc/guides/prog_guide/mempool_lib.rst For applications that use ``rte_pktmbuf_create()``, there is a config setting
Quoted 5 lines
doc/guides/prog_guide/mempool_lib.rst +3
-0an alternative mempool handler. + .. note:: + + When running a DPDK application with shared libraries, mempoolhandler
Quoted 2 lines
doc/guides/prog_guide/mempool_lib.rst +2
-0+ shared objects specified with the '-d' EAL command-line parameter are + dynamically loaded. When running a multi-process application with
Quoted 8 lines
Quoted 7 lines
shared
doc/guides/prog_guide/mempool_lib.rst For applications that use ``rte_pktmbuf_create()``, · line 141
Quoted 1 line
doc/guides/prog_guide/mempool_lib.rst +1
-0+ libraries, the -d arguments for mempool handlers *must be specified inthe
doc/guides/prog_guide/mempool_lib.rst For applications that use ``rte_pktmbuf_create()``, there is a config setting · line 143
Quoted 2 lines
doc/guides/prog_guide/mempool_lib.rst +1
-0+ same order for all processes* to ensure correct operation.One more empty line is required here, other than that:
Acked-by Andrew Rybchenko <[email protected]>Can do. Just for my understanding, why is the extra empty line required?
https://doc.dpdk.org/guides/contributing/documentation.html#whitespace
* Add 2 blank lines before each section header.
PATCH v2 1/1 doc: add multi-proc shared lib mempool note
In reply to Gage EadsThe mempool library assigns handler ops indexes based on the dynamic load order of mempool handlers. Indexes are used so a mempool can be used by multiple processes, but this only works if all processes agree on the mapping from index to mempool handler.
When using the '-d' argument, it's possible for different processes to load mempool handlers in different orders, and thus have different index->handler mappings. Using a mempool in multiple of such processes will result in undefined behavior.
This commit adds a note to the mempool library programmer's guide warning users against this.
--- doc/guides/prog_guide/mempool_lib.rst | 8 ++++++++ 1 file changed, 8 insertions(+)
v2: add another empty line
doc/guides/prog_guide/mempool_lib.rst +8 -0
@@ -133,6 +133,14 @@ For applications that use ``rte_pktmbuf_create()``, there is a config setting (``RTE_MBUF_DEFAULT_MEMPOOL_OPS``) that allows the application to make use of an alternative mempool handler. + .. note:: + + When running a DPDK application with shared libraries, mempool handler + shared objects specified with the '-d' EAL command-line parameter are + dynamically loaded. When running a multi-process application with shared + libraries, the -d arguments for mempool handlers *must be specified in the + same order for all processes* to ensure correct operation. + Use Cases ---------
Signature · 1 line
2.13.6
Thomas Monjalon
Applied21/06/2019 00:07, Gage Eads:
Quoted 10 lines
The mempool library assigns handler ops indexes based on the dynamic load order of mempool handlers. Indexes are used so a mempool can be used by multiple processes, but this only works if all processes agree on the mapping from index to mempool handler.
When using the '-d' argument, it's possible for different processes to load mempool handlers in different orders, and thus have different
index->handlermappings. Using a mempool in multiple of such processes will result in undefined behavior.
Quoted 8 lines
This commit adds a note to the mempool library programmer's guide warning users against this.
Fixes 449c49b93a6b ("mempool: support handler operations")Signed-off-by Gage Eads <[email protected]>Acked-by Andrew Rybchenko <[email protected]>
Applied, thanks