dumpcap: fix mbuf pool ring type
PATCH 1/1 dumpcap: fix mbuf pool ring type
The ring used to store mbufs needs to be multiple producer, multiple consumer because multiple queues might on multiple cores might be allocating and same time (consume) and in case of ring full, the mbufs will be returned (multiple producer).
Bugzilla ID: 1271
--- app/dumpcap/main.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-)
app/dumpcap/main.c +3 -4
@@ -691,10 +691,9 @@ static struct rte_mempool *create_mempool(void) data_size = mbuf_size; } - mp =rte_pktmbuf_pool_create_by_ops (pool_name, num_mbufs, -MBUF_POOL_CACHE_SIZE ,0 , - data_size, -rte_socket_id (),"ring_mp_sc" ); + mp =rte_pktmbuf_pool_create (pool_name, num_mbufs, +MBUF_POOL_CACHE_SIZE ,0 , + data_size,rte_socket_id ());if (mp ==NULL ) rte_exit(EXIT_FAILURE, "Mempool (%s) creation failed: %s\n", pool_name,
Signature · 1 line
2.39.2
Quoted 8 lines
From: Stephen Hemminger [mailto:[email protected]]
Sent: Friday, 4 August 2023 18.16The ring used to store mbufs needs to be multiple producer, multiple consumer because multiple queues might on multiple cores might be allocating and same time (consume) and in case of ring full, the mbufs will be returned (multiple producer).
Quoted 4 lines
Bugzilla ID: 1271
Fixes cb2440fd77af ("dumpcap: fix mbuf pool ring type")Signed-off-by Stephen Hemminger <[email protected]>---
Quoted 5 lines · Stephen Hemminger
The ring used to store mbufs needs to be multiple producer, multiple consumer because multiple queues might on multiple cores might be allocating and same time (consume) and in case of ring full, the mbufs will be returned (multiple producer).
I think I get the idea, but can you rephrase please?
Quoted 3 lines
Bugzilla ID: 1271
Fixes cb2440fd77af ("dumpcap: fix mbuf pool ring type")
This Fixes: tag looks wrong.
Quoted 10 lines
Signed-off-by Stephen Hemminger <[email protected]>--- app/dumpcap/main.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-)app/dumpcap/main.c +0
-0
app/dumpcap/main.c static struct rte_mempool *create_mempool(void) · line 696
Quoted 10 lines
app/dumpcap/main.c +3
-4@@ -691,10 +691,9 @@ static struct rte_mempool *create_mempool(void) data_size = mbuf_size; } - mp =rte_pktmbuf_pool_create_by_ops (pool_name, num_mbufs, -MBUF_POOL_CACHE_SIZE ,0 , - data_size, -rte_socket_id (),"ring_mp_sc" ); + mp =rte_pktmbuf_pool_create (pool_name, num_mbufs, +MBUF_POOL_CACHE_SIZE ,0 , + data_size,rte_socket_id ());
Switching to rte_pktmbuf_pool_create() still leaves the user with the possibility to shoot himself in the foot (I was thinking of setting some --mbuf-pool-ops-name EAL option).
This application has explicit requirements in terms of concurrent access (and I don't think the mempool library exposes per driver capabilities in that regard).
The application was enforcing the use of mempool/ring so far.
I think it is safer to go with an explicit
rte_pktmbuf_pool_create_by_ops(... "ring_mp_mc").
WDYT?
Quoted 6 lines
app/dumpcap/main.c +0
-1if (mp ==NULL ) rte_exit(EXIT_FAILURE, "Mempool (%s) creation failed: %s\n", pool_name, --2.39.2
Thanks.
Signature · 1 line
David Marchand
Quoted 46 lines
From: David Marchand [mailto:[email protected]]
Sent: Monday, 2 October 2023 09.34Quoted 5 lines · Stephen Hemminger
The ring used to store mbufs needs to be multiple producer, multiple consumer because multiple queues might on multiple cores might be allocating and same time (consume) and in case of ring full, the mbufs will be returned (multiple producer).
I think I get the idea, but can you rephrase please?
Quoted 3 lines
Bugzilla ID: 1271
Fixes cb2440fd77af ("dumpcap: fix mbuf pool ring type")This Fixes: tag looks wrong.
Quoted 10 lines
Signed-off-by Stephen Hemminger <[email protected]>--- app/dumpcap/main.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-)app/dumpcap/main.c +0
-0app/dumpcap/main.c static struct rte_mempool *create_mempool(void) · line 696
Quoted 10 lines
app/dumpcap/main.c +3
-4@@ -691,10 +691,9 @@ static struct rte_mempool *create_mempool(void) data_size = mbuf_size; } - mp =rte_pktmbuf_pool_create_by_ops (pool_name, num_mbufs, -MBUF_POOL_CACHE_SIZE ,0 , - data_size, -rte_socket_id (),"ring_mp_sc" ); + mp =rte_pktmbuf_pool_create (pool_name, num_mbufs, +MBUF_POOL_CACHE_SIZE ,0 , + data_size,rte_socket_id ());Switching to
rte_pktmbuf_pool_create()still leaves the user with the possibility to shoot himself in the foot (I was thinking of setting some --mbuf-pool-ops-name EAL option).
Quoted 8 lines
This application has explicit requirements in terms of concurrent access (and I don't think the mempool library exposes per driver capabilities in that regard).
The application was enforcing the use of mempool/ring so far.I think it is safer to go with an explicit
rte_pktmbuf_pool_create_by_ops(... "ring_mp_mc").
WDYT?
<feature creep>
Or perhaps one of "ring_mt_rts" or "ring_mt_hts", if any of those mbuf pool drivers are specified on the command line; otherwise fall back to "ring_mp_mc".
Actually, I prefer Stephen's suggestion of using the default mbuf pool driver. The option is there for a reason.
However, David is right: We want to prevent the user from using a thread-unsafe mempool driver in this use case.
And I guess there might be other use cases than this one, where a thread-safe mempool driver is required. So adding a generalized function to get the "upgraded" (i.e. thread safe) variant of a mempool driver would be nice. </feature creep>
Feel free to ignore my suggested feature creep, and go ahead with David's suggestion instead.
Quoted 13 lines
Quoted 8 lines
app/dumpcap/main.c +0
-1if (mp ==NULL ) rte_exit(EXIT_FAILURE, "Mempool (%s) creation failed: %s\n", pool_name, --2.39.2
Thanks.
--
David Marchand
Quoted 16 lines · On Mon, 2 Oct 2023 10:42:53 +0200 Morten Brørup <[email protected]>…
Quoted 4 lines
Switching to
rte_pktmbuf_pool_create()still leaves the user with the possibility to shoot himself in the foot (I was thinking of setting some --mbuf-pool-ops-name EAL option).Quoted 8 lines
This application has explicit requirements in terms of concurrent access (and I don't think the mempool library exposes per driver capabilities in that regard).
The application was enforcing the use of mempool/ring so far.I think it is safer to go with an explicit
rte_pktmbuf_pool_create_by_ops(... "ring_mp_mc").
WDYT?<feature creep>
Or perhaps one of "ring_mt_rts" or "ring_mt_hts", if any of those mbuf pool drivers are specified on the command line; otherwise fall back to "ring_mp_mc".
Quoted 6 lines
Actually, I prefer Stephen's suggestion of using the default mbuf pool driver. The option is there for a reason.
However, David is right: We want to prevent the user from using a thread-unsafe mempool driver in this use case.
And I guess there might be other use cases than this one, where a thread-safe mempool driver is required. So adding a generalized function to get the "upgraded" (i.e. thread safe) variant of a mempool driver would be nice. </feature creep>
If the user overrides the default mbuf pool type, then it will need to be thread safe for the general case of driver as well (or they are on single cpu).
I think the patch should be applied as is.
Quoted 32 lines
From: Stephen Hemminger [mailto:[email protected]]
Sent: Monday, 6 November 2023 20.24Quoted 1 line · On Mon, 2 Oct 2023 10:42:53 +0200 Morten Brørup <[email protected]>…
Quoted 1 line
Switching to
rte_pktmbuf_pool_create()still leaves the user withthe
Quoted 8 lines
Quoted 8 lines
possibility to shoot himself in the foot (I was thinking of setting some --mbuf-pool-ops-name EAL option).
This application has explicit requirements in terms of concurrent access (and I don't think the mempool library exposes per driver capabilities in that regard).
The application was enforcing the use of mempool/ring so far.Quoted 6 lines
Quoted 3 lines
I think it is safer to go with an explicit
rte_pktmbuf_pool_create_by_ops(... "ring_mp_mc").
WDYT?<feature creep>
Or perhaps one of "ring_mt_rts" or "ring_mt_hts", if any of thosembuf pool drivers are specified on the command line; otherwise fall back to "
ring_mp_mc".Quoted 2 lines
Actually, I prefer Stephen's suggestion of using the default mbuf
pool driver. The option is there for a reason.
Quoted 2 lines
However, David is right: We want to prevent the user from using a
thread-unsafe mempool driver in this use case.
Quoted 2 lines
And I guess there might be other use cases than this one, where a
Quoted 8 lines
thread-safe mempool driver is required. So adding a generalized function to get the "upgraded" (i.e. thread safe) variant of a mempool driver would be nice.
Quoted 1 line
</feature creep>
If the user overrides the default mbuf pool type, then it will need to be thread safe for
the general case of driver as well (or they are on single cpu).
If they have chosen a thread safe pool type, using the chosen type will work for dumpcap too. I think we all agree on that.
I'm not sure I understand your single-cpu argument, so let me try to paraphrase:
If their application only uses one EAL thread, and they have chosen "ring_sc_sp", dumpcap also work, because mempool accesses are still single-threaded. Is this correctly understood?
Or are you saying that if they want to use dumpcap, they must choose a thread safe pool type for their application (regardless if the application is single-threaded or not)?
Quoted 3 lines
I think the patch should be applied as is.
Quoted 10 lines · On Mon, 6 Nov 2023 22:50:50 +0100 Morten Brørup <[email protected]>…
Quoted 9 lines
Quoted 1 line
And I guess there might be other use cases than this one, where a
thread-safe mempool driver is required. So adding a generalized function to get the "upgraded" (i.e. thread safe) variant of a mempool driver would be nice.
Quoted 1 line
</feature creep>
If the user overrides the default mbuf pool type, then it will need to be thread safe for
the general case of driver as well (or they are on single cpu).
Quoted 7 lines
If they have chosen a thread safe pool type, using the chosen type will work for dumpcap too. I think we all agree on that.
I'm not sure I understand your single-cpu argument, so let me try to paraphrase:
If their application only uses one EAL thread, and they have chosen "
ring_sc_sp", dumpcap also work, because mempool accesses are still single-threaded. Is this correctly understood?Or are you saying that if they want to use dumpcap, they must choose a thread safe pool type for their application (regardless if the application is single-threaded or not)?
There is no command line of EAL nature in dumpcap. This is intentional.
QED: overriding default pool type is not going to be a possible
Quoted 28 lines
From: Stephen Hemminger [mailto:[email protected]]
Sent: Tuesday, 7 November 2023 03.36Quoted 3 lines · On Mon, 6 Nov 2023 22:50:50 +0100 Morten Brørup <[email protected]>…
Quoted 3 lines
Quoted 1 line
And I guess there might be other use cases than this one, where a
thread-safe mempool driver is required. So adding a generalized function to get the "upgraded" (i.e. thread safe) variant of a
mempool
Quoted 4 lines
Quoted 4 lines
driver would be nice.
Quoted 1 line
</feature creep>
If the user overrides the default mbuf pool type, then it will need
to
Quoted 4 lines
Quoted 2 lines
be thread safe for
the general case of driver as well (or they are on single cpu).If they have chosen a thread safe pool type, using the chosen type
will work for dumpcap too. I think we all agree on that.
Quoted 2 lines
I'm not sure I understand your single-cpu argument, so let me try to
paraphrase:
Quoted 2 lines
If their application only uses one EAL thread, and they have chosen
"
ring_sc_sp", dumpcap also work, because mempool accesses are still single-threaded. Is this correctly understood?Quoted 1 line
Quoted 7 lines
Quoted 1 line
Or are you saying that if they want to use dumpcap, they must choose
a thread safe pool type for their application (regardless if the application is single-threaded or not)?
There is no command line of EAL nature in dumpcap. This is intentional.
QED: overriding default pool type is not going to be a possible
The preferred mbuf pool type can configured in the primary process by EAL params. If so configured, it is stored in a memzone named "mbuf_user_pool_ops". And if it is set there, the secondary process will also use it as its preferred mbuf pool type.
Quoted 11 lines · On Tue, 7 Nov 2023 08:22:37 +0100 Morten Brørup <[email protected]>…
Quoted 8 lines
Quoted 2 lines
Or are you saying that if they want to use dumpcap, they must choose
a thread safe pool type for their application (regardless if the application is single-threaded or not)?
There is no command line of EAL nature in dumpcap. This is intentional.
QED: overriding default pool type is not going to be a possibleThe preferred mbuf pool type can configured in the primary process by EAL params. If so configured, it is stored in a memzone named "
mbuf_user_pool_ops". And if it is set there, the secondary process will also use it as its preferred mbuf pool type.
If user decides to use a thread unsafe mempool, wouldn't it break in every PMD as well.
Quoted 16 lines
From: Stephen Hemminger [mailto:[email protected]]
Sent: Tuesday, 7 November 2023 17.41Quoted 2 lines · On Tue, 7 Nov 2023 08:22:37 +0100 Morten Brørup <[email protected]>…
Quoted 2 lines
Quoted 2 lines
Or are you saying that if they want to use dumpcap, they must
choose
Quoted 7 lines
Quoted 6 lines
a thread safe pool type for their application (regardless if the application is single-threaded or not)?
There is no command line of EAL nature in dumpcap. This is intentional.
QED: overriding default pool type is not going to be a possible
Quoted 8 lines
Quoted 1 line
The preferred mbuf pool type can configured in the primary process by
EAL params. If so configured, it is stored in a memzone named "
mbuf_user_pool_ops".Quoted 1 line
And if it is set there, the secondary process will also use it as its
preferred mbuf pool type.
If user decides to use a thread unsafe mempool, wouldn't it break in every PMD as well.
Yes, except if the application only uses one thread for packet processing. Then thread safety is not necessary.
Quoted 5 lines · On Tue, 7 Nov 2023 18:38:32 +0100 Morten Brørup <[email protected]>…
Quoted 2 lines
If user decides to use a thread unsafe mempool, wouldn't it break in every PMD as well.
Yes, except if the application only uses one thread for packet processing. Then thread safety is not necessary.
If application only used single thread then single consumer pool would work fine for dumpcap. Only the primary thread would ever allocate. But both secondary and primary could be freeing mbuf.
Morten Brørup
Acked-byQuoted 15 lines
From: Stephen Hemminger [mailto:[email protected]]
Sent: Wednesday, 8 November 2023 17.51Quoted 1 line · On Tue, 7 Nov 2023 18:38:32 +0100 Morten Brørup <[email protected]>…
Quoted 1 line
If user decides to use a thread unsafe mempool, wouldn't it break
in
Quoted 3 lines
Quoted 1 line
every PMD as well.
Yes, except if the application only uses one thread for packet
processing. Then thread safety is not necessary.
Quoted 1 line
Quoted 5 lines
If application only used single thread then single consumer pool would work fine
for dumpcap. Only the primary thread would ever allocate. But both secondary and
primary could be freeing mbuf.
I think it is an acceptable risk. Perhaps mention somewhere that dumpcap breaks the application if it uses non-thread safe mbuf ops.
Optionally, dumpcap could compare the return value of rte_mbuf_best_mempool_ops() [1] with the few known unsafe ops. However, this is probably going to be forgotten, if new non-thread safe mempool ops are ever added to DPDK.
[1]: https://elixir.bootlin.com/dpdk/latest/source/lib/mbuf/rte_mbuf_pool_ops.c#L88
Quoted 10 lines · On Tue, 7 Nov 2023 08:22:37 +0100 Morten Brørup <[email protected]>…
Quoted 7 lines
Quoted 1 line
Or are you saying that if they want to use dumpcap, they must choose
a thread safe pool type for their application (regardless if the application is single-threaded or not)?
There is no command line of EAL nature in dumpcap. This is intentional.
QED: overriding default pool type is not going to be a possibleThe preferred mbuf pool type can configured in the primary process by EAL params. If so configured, it is stored in a memzone named "
mbuf_user_pool_ops". And if it is set there, the secondary process will also use it as its preferred mbuf pool type.
I notice that no other app or example is using the create_by_ops except pdump/pcapng/dumpcap.
~/DPDK/main/examples $ git grep rte_pktmbuf_pool_create_by_ops
~/DPDK/main/app $ git grep rte_pktmbuf_pool_create_by_ops
dumpcap/main.c: mp = rte_pktmbuf_pool_create_by_ops(pool_name, num_mbufs,
pdump/main.c: mbuf_pool = rte_pktmbuf_pool_create_by_ops(mempool_name,
test/test_pcapng.c: mp = rte_pktmbuf_pool_create_by_ops("pcapng_test_pool", IOV_MAX + NUM_PACKETS,Quoted 4 lines · On Mon, 2 Oct 2023 09:33:50 +0200 David Marchand <[email protected]>…
Switching to
rte_pktmbuf_pool_create()still leaves the user with the possibility to shoot himself in the foot (I was thinking of setting some --mbuf-pool-ops-name EAL option).
Quoted 8 lines
This application has explicit requirements in terms of concurrent access (and I don't think the mempool library exposes per driver capabilities in that regard).
The application was enforcing the use of mempool/ring so far.I think it is safer to go with an explicit
rte_pktmbuf_pool_create_by_ops(... "ring_mp_mc").
WDYT?
The dumpcap command does not have EAL command line arguments that can be changed by user.
PATCH v2 1/1 dumpcap: fix mbuf pool ring type
In reply to Stephen HemmingerThe internal buffer pool used for copies of mbufs captured needs to be thread safe. If capturing on multiple interfaces or multiple queues, the same pool will be used (consumers). And if the capture ring gets full, the queues will need to put back the capture buffer which leads to multiple producers.
Since this is the same use case as normal drivers and the default pool time can not be overridden on dumpcap command line, it is OK to use the default pool type.
Bugzilla ID: 1271
---
v2 - reword commit message
app/dumpcap/main.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-)
app/dumpcap/main.c +3 -4
@@ -691,10 +691,9 @@ static struct rte_mempool *create_mempool(void) data_size = mbuf_size; } - mp =rte_pktmbuf_pool_create_by_ops (pool_name, num_mbufs, -MBUF_POOL_CACHE_SIZE ,0 , - data_size, -rte_socket_id (),"ring_mp_sc" ); + mp =rte_pktmbuf_pool_create (pool_name, num_mbufs, +MBUF_POOL_CACHE_SIZE ,0 , + data_size,rte_socket_id ());if (mp ==NULL ) rte_exit(EXIT_FAILURE, "Mempool (%s) creation failed: %s\n", pool_name,
Signature · 1 line
2.39.2
PATCH v3 1/1 dumpcap: fix mbuf pool ring type
In reply to Stephen HemmingerThe internal buffer pool used for copies of mbufs captured needs to be thread safe. If capturing on multiple interfaces or multiple queues, the same pool will be used (consumers). And if the capture ring gets full, the queues will need to put back the capture buffer which leads to multiple producers.
Bugzilla ID: 1271
---
v3 - just change ops, don't use default pool ops
app/dumpcap/main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
app/dumpcap/main.c +1 -1
@@ -694,7 +694,7 @@ static struct rte_mempool *create_mempool(void) mp =rte_pktmbuf_pool_create_by_ops (pool_name, num_mbufs,MBUF_POOL_CACHE_SIZE ,0 , data_size, -rte_socket_id (),"ring_mp_sc" ); +rte_socket_id (),"ring_mp_mc" );if (mp ==NULL ) rte_exit(EXIT_FAILURE, "Mempool (%s) creation failed: %s\n", pool_name,
Signature · 1 line
2.39.2
Quoted 9 lines
From: Stephen Hemminger [mailto:[email protected]]
Sent: Wednesday, 8 November 2023 18.48The internal buffer pool used for copies of mbufs captured needs to be thread safe. If capturing on multiple interfaces or multiple queues, the same pool will be used (consumers). And if the capture ring gets full, the queues will need to put back the capture buffer which leads to multiple producers.
Quoted 4 lines
Bugzilla ID: 1271
Fixes cbb44143be74 ("app/dumpcap: add new packet capture application")Signed-off-by Stephen Hemminger <[email protected]>---
Thomas Monjalon
Applied09/11/2023 08:21, Morten Brørup:
Quoted 9 lines
Quoted 9 lines
From: Stephen Hemminger [mailto:[email protected]]
Sent: Wednesday, 8 November 2023 18.48The internal buffer pool used for copies of mbufs captured needs to be thread safe. If capturing on multiple interfaces or multiple queues, the same pool will be used (consumers). And if the capture ring gets full, the queues will need to put back the capture buffer which leads to multiple producers.
Quoted 5 lines
Quoted 3 lines
Bugzilla ID: 1271
Fixes cbb44143be74 ("app/dumpcap: add new packet capture application")Signed-off-by Stephen Hemminger <[email protected]>Reviewed-by Morten Brørup <[email protected]>
Applied, thanks.