config: make queues per port a meson config option
Subsystems (2)
RFC PATCH 1/1 config: make queues per port a meson config option
The default number of ethernet queues per port is currently set to 1k which is more than enough for most applications, but still is lower than the total number of queues which may be available on modern NICs. Rather than increasing the max queues further, which will increase the memory footprint (since the value is used in array dimensioning), we can instead make the value a meson tunable option - and reduce the default value to 256 in the process. This means that:
* most apps which don't need hundreds of queues will see lower mem use.
* apps which do need to use thousands of queues can configure DPDK to allow this, without having to modify DPDK files (i.e. rte_config.h)
--- config/meson.build | 1 + config/rte_config.h | 1 - meson_options.txt | 2 ++ 3 files changed, 3 insertions(+), 1 deletion(-)
config/meson.build +1 -0
@@ -352,6 +352,7 @@ endforeach # set other values pulled from the build options dpdk_conf.set('RTE_MAX_ETHPORTS', get_option('max_ethports')) +dpdk_conf.set('RTE_MAX_QUEUES_PER_PORT', get_option('max_queues_per_ethport')) dpdk_conf.set('RTE_LIBEAL_USE_HPET', get_option('use_hpet')) dpdk_conf.set('RTE_ENABLE_STDATOMIC', get_option('enable_stdatomic')) dpdk_conf.set('RTE_ENABLE_TRACE_FP', get_option('enable_trace_fp'))
config/rte_config.h +0 -1
@@ -64,7 +64,6 @@#define RTE_MBUF_DEFAULT_MEMPOOL_OPS "ring_mp_mc"/* ether defines */ -#define RTE_MAX_QUEUES_PER_PORT 1024#define RTE_ETHDEV_QUEUE_STAT_CNTRS 16/* max 256 */ #define RTE_ETHDEV_RXTX_CALLBACKS 1#define RTE_MAX_MULTI_HOST_CTRLS 4
meson_options.txt +2 -0
@@ -44,6 +44,8 @@ option('max_lcores', type: 'string', value: 'default', description: 'Set maximum number of cores/threads supported by EAL; "default" is different per-arch, "detect" detects the number of cores on the build machine.') option('max_numa_nodes', type: 'string', value: 'default', description: 'Set the highest NUMA node supported by EAL; "default" is different per-arch, "detect" detects the highest NUMA node on the build machine.') +option('max_queues_per_ethport', type: 'integer', value: 256, description: + 'maximum number of queues on an Ethernet device') option('enable_iova_as_pa', type: 'boolean', value: true, description: 'Support the use of physical addresses for IO addresses, such as used by UIO or VFIO in no-IOMMU mode. When disabled, DPDK can only run with IOMMU support for address mappings, but will have more space available in the mbuf structure.') option('mbuf_refcnt_atomic', type: 'boolean', value: true, description:
Signature · 1 line
2.43.0
Quoted 9 lines
From: Bruce Richardson [mailto:[email protected]]
The default number of ethernet queues per port is currently set to 1k which is more than enough for most applications, but still is lower than the total number of queues which may be available on modern NICs. Rather than increasing the max queues further, which will increase the memory footprint (since the value is used in array dimensioning), we can instead make the value a meson tunable option - and reduce the default value to 256 in the process.
Overall, I agree that this tunable is not very exotic, and can be exposed as suggested. The reduction of the default value must be mentioned in the release notes.
Quoted 4 lines
# set other values pulled from the build options
dpdk_conf.set('RTE_MAX_ETHPORTS', get_option('max_ethports'))
+dpdk_conf.set('RTE_MAX_QUEUES_PER_PORT',
get_option('max_queues_per_ethport'))Please rename RTE_MAX_QUEUES_PER_PORT to _PER_ETHPORT, so it resembles MAX_ETHPORTS. For API backwards compatibility, you can add:
#define RTE_MAX_QUEUES_PER_PORT RTE_MAX_QUEUES_PER_ETHPORT
I wonder if it would be possible to have separate max sizes for RX and TX queues? If it can save a non-negligible amount of memory, it might be useful for some applications.
With suggested changes (splitting RX/TX maximums not required),
Quoted 10 lines · Morten Brørup
Quoted 9 lines
From: Bruce Richardson [mailto:[email protected]]
The default number of ethernet queues per port is currently set to 1k which is more than enough for most applications, but still is lower than the total number of queues which may be available on modern NICs. Rather than increasing the max queues further, which will increase the memory footprint (since the value is used in array dimensioning), we can instead make the value a meson tunable option - and reduce the default value to 256 in the process.
Quoted 3 lines
Overall, I agree that this tunable is not very exotic, and can be exposed as suggested. The reduction of the default value must be mentioned in the release notes.
Yes, good point. I'll add that in any next version.
Quoted 9 lines
Quoted 5 lines
# set other values pulled from the build options dpdk_conf.set('RTE_MAX_ETHPORTS', get_option('max_ethports')) +dpdk_conf.set('RTE_MAX_QUEUES_PER_PORT', get_option('max_queues_per_ethport'))Please rename
RTE_MAX_QUEUES_PER_PORTto_PER_ETHPORT, so it resemblesMAX_ETHPORTS. For API backwards compatibility, you can add:
#defineRTE_MAX_QUEUES_PER_PORTRTE_MAX_QUEUES_PER_ETHPORT
Agree that would more consistent. That would probably best be a separate patch, since we'd want to convert all internal use over. Will make this a two-patch set in next version.
Quoted 3 lines
I wonder if it would be possible to have separate max sizes for RX and TX queues? If it can save a non-negligible amount of memory, it might be useful for some applications.
That is an interesting idea. It would certainly allow saving memory for use-cases where you want a large number of rx queues only, or tx queues only. However, the defaults are still likely to remain the same. The main issue I would have with it, is that it would mean having two build time options rather than 1, and I'm a bit concerned at the number of options we seem to be accumulating in DPDK.
Overall, I'm tending towards suggesting that we not do that split, but I'm open to being convinced on it.
Quoted 4 lines
With suggested changes (splitting RX/TX maximums not required),
Acked-by Morten Brørup <[email protected]>
Quoted 13 lines
From: Bruce Richardson [mailto:[email protected]]
Quoted 10 lines · Morten Brørup
Quoted 9 lines
From: Bruce Richardson [mailto:[email protected]]
The default number of ethernet queues per port is currently set to 1k which is more than enough for most applications, but still is lower than the total number of queues which may be available on modern NICs. Rather than increasing the max queues further, which will increase the memory footprint (since the value is used in array dimensioning), we can instead make the value a meson tunable option - and reduce the default value to 256 in the process.
Quoted 6 lines
Quoted 1 line
Overall, I agree that this tunable is not very exotic, and can be exposed as
suggested.
Quoted 2 lines
The reduction of the default value must be mentioned in the release notes.
Yes, good point. I'll add that in any next version.
ACK.
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
# set other values pulled from the build options dpdk_conf.set('RTE_MAX_ETHPORTS', get_option('max_ethports')) +dpdk_conf.set('RTE_MAX_QUEUES_PER_PORT', get_option('max_queues_per_ethport'))
Quoted 8 lines
Quoted 1 line
Please rename
RTE_MAX_QUEUES_PER_PORTto_PER_ETHPORT, so it resembles
MAX_ETHPORTS. For API backwards compatibility, you can add:Quoted 2 lines
#define
RTE_MAX_QUEUES_PER_PORTRTE_MAX_QUEUES_PER_ETHPORTAgree that would more consistent. That would probably best be a separate patch, since we'd want to convert all internal use over. Will make this a two-patch set in next version.
ACK. And agree about two-patch series.
Quoted 14 lines
Quoted 3 lines
I wonder if it would be possible to have separate max sizes for RX and TX
queues? If it can save a non-negligible amount of memory, it might be useful for some applications.
Quoted 1 line
That is an interesting idea. It would certainly allow saving memory for use-cases where you want a large number of rx queues only, or tx queues only. However, the defaults are still likely to remain the same. The main issue I would have with it, is that it would mean having two build time options rather than 1, and I'm a bit concerned at the number of options we seem to be accumulating in DPDK.
Quoted 2 lines
Overall, I'm tending towards suggesting that we not do that split, but I'm open to being convinced on it.
I would guess that many applications have an asymmetrical split of number of RX/TX queues. So I would argue that:
The reason to make this configurable in meson is to conserve memory, so why only go half the way if there is more memory to be conserved?
The distros will use oversize maximums anyway, but custom built applications might benefit.
Here's a weird thought:
Perhaps RX and TX maximums can be controlled individually by changing rte_config.h, and they can be overridden via one meson configuration parameter to set both (to the same value).
Quoted 5 lines
Quoted 5 lines
With suggested changes (splitting RX/TX maximums not required),
Acked-by Morten Brørup <[email protected]>
My ACK remains; splitting RX/TX maximums is not Must Have, it is Nice To Have.
Quoted 67 lines · Morten Brørup
Quoted 13 lines
From: Bruce Richardson [mailto:[email protected]]
Quoted 10 lines · Morten Brørup
Quoted 9 lines
From: Bruce Richardson [mailto:[email protected]]
The default number of ethernet queues per port is currently set to 1k which is more than enough for most applications, but still is lower than the total number of queues which may be available on modern NICs. Rather than increasing the max queues further, which will increase the memory footprint (since the value is used in array dimensioning), we can instead make the value a meson tunable option - and reduce the default value to 256 in the process.
Quoted 6 lines
Quoted 1 line
Overall, I agree that this tunable is not very exotic, and can be exposed as
suggested.
Quoted 2 lines
The reduction of the default value must be mentioned in the release notes.
Yes, good point. I'll add that in any next version.
ACK.
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
# set other values pulled from the build options dpdk_conf.set('RTE_MAX_ETHPORTS', get_option('max_ethports')) +dpdk_conf.set('RTE_MAX_QUEUES_PER_PORT', get_option('max_queues_per_ethport'))Quoted 8 lines
Quoted 1 line
Please rename
RTE_MAX_QUEUES_PER_PORTto_PER_ETHPORT, so it resembles
MAX_ETHPORTS. For API backwards compatibility, you can add:Quoted 2 lines
#define
RTE_MAX_QUEUES_PER_PORTRTE_MAX_QUEUES_PER_ETHPORTAgree that would more consistent. That would probably best be a separate patch, since we'd want to convert all internal use over. Will make this a two-patch set in next version.
ACK. And agree about two-patch series.
Quoted 14 lines
Quoted 3 lines
I wonder if it would be possible to have separate max sizes for RX and TX
queues? If it can save a non-negligible amount of memory, it might be useful for some applications.
Quoted 1 line
That is an interesting idea. It would certainly allow saving memory for use-cases where you want a large number of rx queues only, or tx queues only. However, the defaults are still likely to remain the same. The main issue I would have with it, is that it would mean having two build time options rather than 1, and I'm a bit concerned at the number of options we seem to be accumulating in DPDK.
Quoted 2 lines
Overall, I'm tending towards suggesting that we not do that split, but I'm open to being convinced on it.
I would guess that many applications have an asymmetrical split of number of RX/TX queues. So I would argue that:
The reason to make this configurable in meson is to conserve memory, so why only go half the way if there is more memory to be conserved?The distros will use oversize maximums anyway, but custom built applications might benefit.
Here's a weird thought:
Perhaps RX and TX maximums can be controlled individually by changingrte_config.h, and they can be overridden via one meson configuration parameter to set both (to the same value).Quoted 2 lines
Quoted 2 lines
Quoted 6 lines
Quoted 3 lines
Quoted 3 lines
With suggested changes (splitting RX/TX maximums not required),
Acked-by Morten Brørup <[email protected]>My ACK remains; splitting RX/TX maximums is not Must Have, it is Nice To Have.
Let me see how much in involved in trying to split...
v2 cover add meson config options for queues per port
In reply to Bruce RichardsonThere are a number of issues with the current RTE_MAX_QUEUES_PER_PORT setting in DPDK that are addressed by this patchset:
* The name does not make it clear that this is intended as an ethdev-only setting
* A number of other libraries are using this define rather than having more relevant defines for the particular usecase.
* The define is hard-coded in DPDK source code and is not adjustable via a build-time/meson option
* Because of the lack of configurability, the max is therefore set to a conservatively-high value, wasting memory.
* There is an assumption that the number of Rx queues and Tx queues should have the same maximum value. Depending on application, it may be desirable to have fan-in with multiple Rx queues e.g. for classification/filtering, feed a single Tx queue, or the opposite where, e.g. for QoS Tx scheduling, a few Rx queues feeds a very large number of Tx queues.
This patchset therefore addresses these by:
* replacing the single define for max queues with independent defines for Rx and Tx queues.
* adjusts the name to ensure that it is clear the defines are for ethports only. [ethports being used in the RTE_MAX_ETHPORTS setting].
* replaces occurances of RTE_MAX_QUEUES_PER_PORT with appropriate defines for non-ethdev use cases
* replaces all other internal occurances of the define with the new per-Rx and per-Tx definitions.
* adds meson config options to allow build-time configuration of the max Rx and Tx queue values.
Naming Note:
* The new meson config options are called "max_ethport_rx_queues" and "max_ethport_tx_queues" so that in the meson options list they appear alphabetically beside the existing "max_ethports" option.
* For naming consistency, the new C defines are therefore RTE_MAX_ETHPORT_RX_QUEUES and RTE_MAX_ETHPORT_TX_QUEUES.
V2:
* What was a single patch with "3 insertions(+), 1 deletion(-)" has now become a 26-patch set! :-)
* Created separate Rx and Tx defines
* Ensured that the name makes it clear that the define is for ethdev
* When updating internal use, created one patch per component for easier maintainer review. In most cases it was obvious whether Rx or Tx define should be used, but a few cases were less clear.
* Added documentation updates for the changes (release notes and deprecation notice), spread across 3 of the patches.
Bruce Richardson (26):
cryptodev: remove use of ethdev max queues definition
eventdev: remove use of ethev queues define app/test-bbdev: remove use of ethdev queue count value
config: add separate defines for max Rx and Tx queues
ethdev: use separate Rx and Tx queue limits
bpf: use separate Rx and Tx queue limits
latencystats: use separate Rx and Tx queue limits
pdump: use separate Rx and Tx queue limits
power: use separate Rx and Tx queue limits net/af_xdp: use separate Rx and Tx queue limits net/cnxk: use separate Rx and Tx queue limits net/failsafe: use separate Rx and Tx queue limits net/hns3: use separate Rx and Tx queue limits net/mlx5: use separate Rx and Tx queue limits net/null: use separate Rx and Tx queue limits net/sfc: use separate Rx and Tx queue limits net/thunderx: use separate Rx and Tx queue limits net/vhost: use separate Rx and Tx queue limits app/dumpcap: use separate Rx and Tx queue limits app/test-pmd: use separate Rx and Tx queue limits examples/ipsec-secgw: use separate Rx and Tx queue limits examples/l3fwd-power: use separate Rx and Tx queue limits examples/l3fwd: use separate Rx and Tx queue limits examples/vhost: use separate Rx and Tx queue limits
config: make queues per port a meson config option
config: add computed max queues define for compatibility
app/dumpcap/main.c | 2 +- app/test-bbdev/test_bbdev.c | 4 ++-- app/test-pmd/testpmd.c | 7 ++++--- app/test-pmd/testpmd.h | 16 ++++++++-------- config/meson.build | 10 ++++++++++ config/rte_config.h | 2 +- doc/guides/rel_notes/deprecation.rst | 11 +++++++++++ doc/guides/rel_notes/release_24_11.rst | 21 +++++++++++++++++++++ drivers/net/af_xdp/rte_eth_af_xdp.c | 2 +- drivers/net/cnxk/cnxk_ethdev_ops.c | 4 ++-- drivers/net/failsafe/failsafe_ops.c | 4 ++-- drivers/net/hns3/hns3_tm.c | 2 +- drivers/net/mlx5/mlx5_flow.c | 2 +- drivers/net/mlx5/mlx5_flow_hw.c | 2 +- drivers/net/null/rte_eth_null.c | 4 ++-- drivers/net/sfc/sfc_sw_stats.c | 6 ++++-- drivers/net/thunderx/nicvf_ethdev.c | 2 +- drivers/net/vhost/rte_eth_vhost.c | 7 ++++--- examples/ipsec-secgw/ipsec-secgw.c | 2 +- examples/ipsec-secgw/ipsec.c | 2 +- examples/l3fwd-power/main.c | 2 +- examples/l3fwd-power/perf_core.c | 2 +- examples/l3fwd/main.c | 2 +- examples/vhost/main.c | 2 +- examples/vhost/main.h | 2 +- lib/bpf/bpf_pkt.c | 3 ++- lib/cryptodev/cryptodev_pmd.c | 4 ++-- lib/ethdev/ethdev_driver.h | 8 ++++---- lib/ethdev/ethdev_private.c | 24 ++++++++++++++---------- lib/ethdev/rte_ethdev.c | 16 +++++++--------- lib/ethdev/rte_ethdev.h | 18 +++++++++--------- lib/eventdev/eventdev_private.c | 2 +- lib/latencystats/rte_latencystats.c | 4 ++-- lib/pdump/rte_pdump.c | 18 +++++++++--------- lib/power/rte_power_pmd_mgmt.c | 4 ++-- meson_options.txt | 4 ++++ 36 files changed, 140 insertions(+), 87 deletions(-)
--
2.43.0
RE: [RFC PATCH v2 00/26] add meson config options for queues per port
Quoted 40 lines
From: Bruce Richardson [mailto:[email protected]]
There are a number of issues with the current
RTE_MAX_QUEUES_PER_PORTsetting in DPDK that are addressed by this patchset:* The name does not make it clear that this is intended as an ethdev-only setting
* A number of other libraries are using this define rather than having more relevant defines for the particular usecase.
* The define is hard-coded in DPDK source code and is not adjustable via a build-time/meson option
* Because of the lack of configurability, the max is therefore set to a conservatively-high value, wasting memory.
* There is an assumption that the number of Rx queues and Tx queues should have the same maximum value. Depending on application, it may be desirable to have fan-in with multiple Rx queues e.g. for classification/filtering, feed a single Tx queue, or the opposite where, e.g. for QoS Tx scheduling, a few Rx queues feeds a very large number of Tx queues.This patchset therefore addresses these by:
* replacing the single define for max queues with independent defines for Rx and Tx queues.
* adjusts the name to ensure that it is clear the defines are for ethports only. [ethports being used in theRTE_MAX_ETHPORTSsetting].
* replaces occurances ofRTE_MAX_QUEUES_PER_PORTwith appropriate defines for non-ethdev use cases
* replaces all other internal occurances of the define with the new per-Rx and per-Tx definitions.
* adds meson config options to allow build-time configuration of the max Rx and Tx queue values.Naming Note:
* The new meson config options are called "max_ethport_rx_queues" and "max_ethport_tx_queues" so that in the meson options list they appear alphabetically beside the existing "max_ethports" option.
* For naming consistency, the new C defines are thereforeRTE_MAX_ETHPORT_RX_QUEUESandRTE_MAX_ETHPORT_TX_QUEUES.
Quoted 10 lines
V2:
* What was a single patch with "3 insertions(+), 1 deletion(-)" has now become a 26-patch set! :-)
* Created separate Rx and Tx defines
* Ensured that the name makes it clear that the define is for ethdev
* When updating internal use, created one patch per component for easier maintainer review. In most cases it was obvious whether Rx or Tx define should be used, but a few cases were less clear.
* Added documentation updates for the changes (release notes and deprecation notice), spread across 3 of the patches.
Thanks.
For the series,
RFC PATCH v2 1/26 cryptodev: remove use of ethdev max queues definition
In reply to Bruce RichardsonThe number of queue pairs supported by cryptodev should not be dependent on the number of ethdev Rx or Tx queues, so add a new define for cryptodev specifically.
--- config/rte_config.h | 1 + lib/cryptodev/cryptodev_pmd.c | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-)
config/rte_config.h +1 -0
@@ -71,6 +71,7 @@/* cryptodev defines */ #define RTE_CRYPTO_MAX_DEVS 64 +#define RTE_CRYPTO_MAX_QPS_PER_DEV 256#define RTE_CRYPTODEV_NAME_LEN 64#define RTE_CRYPTO_CALLBACKS 1
lib/cryptodev/cryptodev_pmd.c +2 -2
@@ -212,8 +212,8 @@ dummy_crypto_dequeue_burst(__rte_unused void *qp,void cryptodev_fp_ops_reset (struct rte_crypto_fp_ops *fp_ops) { -static struct rte_cryptodev_cb_rcu dummy_cb[RTE_MAX_QUEUES_PER_PORT ]; -static void *dummy_data[RTE_MAX_QUEUES_PER_PORT ]; +static struct rte_cryptodev_cb_rcu dummy_cb[RTE_CRYPTO_MAX_QPS_PER_DEV ]; +static void *dummy_data[RTE_CRYPTO_MAX_QPS_PER_DEV ];static const struct rte_crypto_fp_ops dummy = { .enqueue_burst = dummy_crypto_enqueue_burst, .dequeue_burst = dummy_crypto_dequeue_burst,
Signature · 1 line
2.43.0
RFC PATCH v2 2/26 eventdev: remove use of ethev queues define
In reply to Bruce RichardsonThe max queues per port definition is intended for use for ethdevs, so it's use for a dummy array in eventdev is likely wrong. Since the array is intended to be for eventdev ports, use RTE_EVENT_MAX_PORTS_PER_DEV define instead.
--- lib/eventdev/eventdev_private.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
lib/eventdev/eventdev_private.c +1 -1
@@ -99,7 +99,7 @@ dummy_event_port_profile_switch(__rte_unused void *port, __rte_unused uint8_t prvoid event_dev_fp_ops_reset (struct rte_event_fp_ops *fp_op) { -static void *dummy_data[RTE_MAX_QUEUES_PER_PORT ]; +static void *dummy_data[RTE_EVENT_MAX_PORTS_PER_DEV ];static const struct rte_event_fp_ops dummy = { .enqueue = dummy_event_enqueue, .enqueue_burst = dummy_event_enqueue_burst,
Signature · 1 line
2.43.0
RFC PATCH v2 3/26 app/test-bbdev: remove use of ethdev queue count value
In reply to Bruce RichardsonReplace the use of RTE_MAX_QUEUES_PER_PORT, which is intended as an ethdev define, with the limit from the bbdev library.
--- app/test-bbdev/test_bbdev.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
app/test-bbdev/test_bbdev.c +2 -2
@@ -297,7 +297,7 @@ test_bbdev_configure_stop_queue(void)"Failed test for rte_bbdev_queue_stop " "invalid dev_id "); -TEST_ASSERT_FAIL (rte_bbdev_queue_stop (dev_id,RTE_MAX_QUEUES_PER_PORT ), +TEST_ASSERT_FAIL (rte_bbdev_queue_stop (dev_id,RTE_BBDEV_DEFAULT_MAX_NB_QUEUES ),"Failed test for rte_bbdev_queue_stop " "invalid queue_id " );
@@ -1176,7 +1176,7 @@ test_bbdev_invalid_driver(void) "invalid dev_id ");TEST_ASSERT_FAIL (rte_bbdev_queue_info_get (dev_id, -RTE_MAX_QUEUES_PER_PORT , &qinfo), +RTE_BBDEV_DEFAULT_MAX_NB_QUEUES , &qinfo),"Failed test for rte_bbdev_info_get: " "invalid queue_id " );
Signature · 1 line
2.43.0
RFC PATCH v2 4/26 config: add separate defines for max Rx and Tx queues
In reply to Bruce RichardsonRather than having a single define for maximum queues per ethernet port, we can set the max values for Rx queues and Tx queue independently. This allows future memory saving for apps which only need large numbers of Rx queues or only large numbers of Tx queues.
--- config/rte_config.h | 2 ++ doc/guides/rel_notes/release_24_11.rst | 6 ++++++ 2 files changed, 8 insertions(+)
config/rte_config.h +2 -0
@@ -65,6 +65,8 @@/* ether defines */ #define RTE_MAX_QUEUES_PER_PORT 1024 +#define RTE_MAX_ETHPORT_RX_QUEUES 1024 +#define RTE_MAX_ETHPORT_TX_QUEUES 1024#define RTE_ETHDEV_QUEUE_STAT_CNTRS 16/* max 256 */ #define RTE_ETHDEV_RXTX_CALLBACKS 1#define RTE_MAX_MULTI_HOST_CTRLS 4
doc/guides/rel_notes/release_24_11.rst +6 -0
@@ -55,6 +55,12 @@ New Features Also, make sure to start the actual text at the margin. ======================================================= +* **New configuration settings for Rx and Tx Ethernet Queues.** + + New defines have been added to DPDK to allow independent tracking of the maximum number of Rx and Tx queues. + These defines as ``RTE_MAX_ETHPORT_RX_QUEUES`` and ``RTE_MAX_ETHPORT_TX_QUEUES``, + and should be used in place of the older macro ``RTE_MAX_QUEUES_PER_PORT``. + Removed Items -------------
Signature · 1 line
2.43.0
RFC PATCH v2 5/26 ethdev: use separate Rx and Tx queue limits
In reply to Bruce RichardsonUpdate library to use the new defines RTE_MAX_ETHPORT_TX_QUEUES and RTE_MAX_ETHPORT_RX_QUEUES rather than the old define RTE_MAX_QUEUES_PER_PORT.
--- lib/ethdev/ethdev_driver.h | 8 ++++---- lib/ethdev/ethdev_private.c | 24 ++++++++++++++---------- lib/ethdev/rte_ethdev.c | 16 +++++++--------- lib/ethdev/rte_ethdev.h | 18 +++++++++--------- 4 files changed, 34 insertions(+), 32 deletions(-)
lib/ethdev/ethdev_driver.h +4 -4
@@ -84,12 +84,12 @@ struct __rte_cache_aligned rte_eth_dev {* User-supplied functions called from rx_burst to post-process * received packets before passing them to the user */ -RTE_ATOMIC (struct rte_eth_rxtx_callback *) post_rx_burst_cbs[RTE_MAX_QUEUES_PER_PORT ]; +RTE_ATOMIC (struct rte_eth_rxtx_callback *) post_rx_burst_cbs[RTE_MAX_ETHPORT_RX_QUEUES ];/** * User-supplied functions called from tx_burst to pre-process * received packets before passing them to the driver for transmission */ -RTE_ATOMIC (struct rte_eth_rxtx_callback *) pre_tx_burst_cbs[RTE_MAX_QUEUES_PER_PORT ]; +RTE_ATOMIC (struct rte_eth_rxtx_callback *) pre_tx_burst_cbs[RTE_MAX_ETHPORT_TX_QUEUES ];enum rte_eth_dev_state state;/**< Flag indicating the port state */ void *security_ctx;/**< Context for security ops */
@@ -165,9 +165,9 @@ struct __rte_cache_aligned rte_eth_dev_data { flow_configured :1 ;/** Queues state: HAIRPIN(2) / STARTED(1) / STOPPED(0) */ -uint8_t rx_queue_state[RTE_MAX_QUEUES_PER_PORT ]; +uint8_t rx_queue_state[RTE_MAX_ETHPORT_RX_QUEUES ];/** Queues state: HAIRPIN(2) / STARTED(1) / STOPPED(0) */ -uint8_t tx_queue_state[RTE_MAX_QUEUES_PER_PORT ]; +uint8_t tx_queue_state[RTE_MAX_ETHPORT_TX_QUEUES ];uint32_t dev_flags;/**< Capabilities */ int numa_node;/**< NUMA node connection */
lib/ethdev/ethdev_private.c +14 -10
@@ -190,7 +190,8 @@ struct dummy_queue {bool rx_warn_once;bool tx_warn_once; }; -static struct dummy_queue *dummy_queues_array[RTE_MAX_ETHPORTS ][RTE_MAX_QUEUES_PER_PORT ]; +static struct dummy_queue *dummy_rxq_array[RTE_MAX_ETHPORTS ][RTE_MAX_ETHPORT_RX_QUEUES ]; +static struct dummy_queue *dummy_txq_array[RTE_MAX_ETHPORTS ][RTE_MAX_ETHPORT_TX_QUEUES ];static struct dummy_queue per_port_queues[RTE_MAX_ETHPORTS ];RTE_INIT (dummy_queue_init) {
@@ -199,8 +200,10 @@ RTE_INIT(dummy_queue_init)for (port_id =0 ; port_id <RTE_DIM (per_port_queues); port_id++) {unsigned int q; -for (q =0 ; q <RTE_DIM (dummy_queues_array[port_id]); q++) - dummy_queues_array[port_id][q] = &per_port_queues[port_id]; +for (q =0 ; q <RTE_DIM (dummy_rxq_array[port_id]); q++) + dummy_rxq_array[port_id][q] = &per_port_queues[port_id]; +for (q =0 ; q <RTE_DIM (dummy_txq_array[port_id]); q++) + dummy_txq_array[port_id][q] = &per_port_queues[port_id]; } }
@@ -245,7 +248,8 @@ dummy_eth_tx_burst(void *txq,void eth_dev_fp_ops_reset (struct rte_eth_fp_ops *fpo) { -static RTE_ATOMIC (void *) dummy_data[RTE_MAX_QUEUES_PER_PORT ]; +static RTE_ATOMIC (void *) dummy_rx_data[RTE_MAX_ETHPORT_RX_QUEUES ]; +static RTE_ATOMIC (void *) dummy_tx_data[RTE_MAX_ETHPORT_TX_QUEUES ];uintptr_t port_id = fpo - rte_eth_fp_ops; per_port_queues[port_id].rx_warn_once = false;
@@ -254,12 +258,12 @@ eth_dev_fp_ops_reset(struct rte_eth_fp_ops *fpo) .rx_pkt_burst = dummy_eth_rx_burst, .tx_pkt_burst = dummy_eth_tx_burst, .rxq = { - .data = (void **)&dummy_queues_array[port_id], - .clbk = dummy_data, + .data = (void **)&dummy_rxq_array[port_id], + .clbk = dummy_rx_data, }, .txq = { - .data = (void **)&dummy_queues_array[port_id], - .clbk = dummy_data, + .data = (void **)&dummy_txq_array[port_id], + .clbk = dummy_tx_data, }, }; }
@@ -420,7 +424,7 @@ eth_dev_rx_queue_config(struct rte_eth_dev *dev, uint16_t nb_queues)if (dev->data->rx_queues ==NULL && nb_queues !=0 ) {/* first time configuration */ dev->data->rx_queues =rte_zmalloc ("ethdev->rx_queues" ,sizeof (dev->data->rx_queues[0 ]) * -RTE_MAX_QUEUES_PER_PORT , +RTE_MAX_ETHPORT_RX_QUEUES ,RTE_CACHE_LINE_SIZE );if (dev->data->rx_queues ==NULL ) { dev->data->nb_rx_queues =0 ;
@@ -450,7 +454,7 @@ eth_dev_tx_queue_config(struct rte_eth_dev *dev, uint16_t nb_queues)if (dev->data->tx_queues ==NULL && nb_queues !=0 ) {/* first time configuration */ dev->data->tx_queues =rte_zmalloc ("ethdev->tx_queues" ,sizeof (dev->data->tx_queues[0 ]) * -RTE_MAX_QUEUES_PER_PORT , +RTE_MAX_ETHPORT_TX_QUEUES ,RTE_CACHE_LINE_SIZE );if (dev->data->tx_queues ==NULL ) { dev->data->nb_tx_queues =0 ;
lib/ethdev/rte_ethdev.c +7 -9
@@ -1367,18 +1367,18 @@ rte_eth_dev_configure(uint16_t port_id, uint16_t nb_rx_q, uint16_t nb_tx_q, nb_tx_q =RTE_ETH_DEV_FALLBACK_TX_NBQUEUES ; } -if (nb_rx_q >RTE_MAX_QUEUES_PER_PORT ) { +if (nb_rx_q >RTE_MAX_ETHPORT_RX_QUEUES ) {RTE_ETHDEV_LOG_LINE (ERR ,"Number of Rx queues requested (%u) is greater than max supported(%d)" , - nb_rx_q,RTE_MAX_QUEUES_PER_PORT ); + nb_rx_q,RTE_MAX_ETHPORT_RX_QUEUES ); ret = -EINVAL ; goto rollback; } -if (nb_tx_q >RTE_MAX_QUEUES_PER_PORT ) { +if (nb_tx_q >RTE_MAX_ETHPORT_TX_QUEUES ) {RTE_ETHDEV_LOG_LINE (ERR ,"Number of Tx queues requested (%u) is greater than max supported(%d)" , - nb_tx_q,RTE_MAX_QUEUES_PER_PORT ); + nb_tx_q,RTE_MAX_ETHPORT_TX_QUEUES ); ret = -EINVAL ; goto rollback; }
@@ -3811,11 +3811,9 @@ rte_eth_dev_info_get(uint16_t port_id, struct rte_eth_dev_info *dev_info)return eth_err (port_id, diag); } -/* Maximum number of queues should be <= RTE_MAX_QUEUES_PER_PORT */ - dev_info->max_rx_queues =RTE_MIN (dev_info->max_rx_queues, -RTE_MAX_QUEUES_PER_PORT ); - dev_info->max_tx_queues =RTE_MIN (dev_info->max_tx_queues, -RTE_MAX_QUEUES_PER_PORT ); +/* Maximum number of queues should be <= RTE_MAX_ETHPORT_RX/TX_QUEUES */ + dev_info->max_rx_queues =RTE_MIN (dev_info->max_rx_queues,RTE_MAX_ETHPORT_RX_QUEUES ); + dev_info->max_tx_queues =RTE_MIN (dev_info->max_tx_queues,RTE_MAX_ETHPORT_TX_QUEUES ); dev_info->driver_name = dev->device->driver->name; dev_info->nb_rx_queues = dev->data->nb_rx_queues;
lib/ethdev/rte_ethdev.h +9 -9
@@ -6090,7 +6090,7 @@ rte_eth_rx_burst(uint16_t port_id, uint16_t queue_id,#ifdef RTE_ETHDEV_DEBUG_RX if (port_id >=RTE_MAX_ETHPORTS || - queue_id >=RTE_MAX_QUEUES_PER_PORT ) { + queue_id >=RTE_MAX_ETHPORT_RX_QUEUES ) {RTE_ETHDEV_LOG_LINE (ERR ,"Invalid port_id=%u or queue_id=%u" , port_id, queue_id);
@@ -6161,7 +6161,7 @@ rte_eth_rx_queue_count(uint16_t port_id, uint16_t queue_id)#ifdef RTE_ETHDEV_DEBUG_RX if (port_id >=RTE_MAX_ETHPORTS || - queue_id >=RTE_MAX_QUEUES_PER_PORT ) { + queue_id >=RTE_MAX_ETHPORT_RX_QUEUES ) {RTE_ETHDEV_LOG_LINE (ERR ,"Invalid port_id=%u or queue_id=%u" , port_id, queue_id);
@@ -6234,7 +6234,7 @@ rte_eth_rx_descriptor_status(uint16_t port_id, uint16_t queue_id,#ifdef RTE_ETHDEV_DEBUG_RX if (port_id >=RTE_MAX_ETHPORTS || - queue_id >=RTE_MAX_QUEUES_PER_PORT ) { + queue_id >=RTE_MAX_ETHPORT_RX_QUEUES ) {RTE_ETHDEV_LOG_LINE (ERR ,"Invalid port_id=%u or queue_id=%u" , port_id, queue_id);
@@ -6305,7 +6305,7 @@ static inline int rte_eth_tx_descriptor_status(uint16_t port_id,#ifdef RTE_ETHDEV_DEBUG_TX if (port_id >=RTE_MAX_ETHPORTS || - queue_id >=RTE_MAX_QUEUES_PER_PORT ) { + queue_id >=RTE_MAX_ETHPORT_TX_QUEUES ) {RTE_ETHDEV_LOG_LINE (ERR ,"Invalid port_id=%u or queue_id=%u" , port_id, queue_id);
@@ -6429,7 +6429,7 @@ rte_eth_tx_burst(uint16_t port_id, uint16_t queue_id,#ifdef RTE_ETHDEV_DEBUG_TX if (port_id >=RTE_MAX_ETHPORTS || - queue_id >=RTE_MAX_QUEUES_PER_PORT ) { + queue_id >=RTE_MAX_ETHPORT_TX_QUEUES ) {RTE_ETHDEV_LOG_LINE (ERR ,"Invalid port_id=%u or queue_id=%u" , port_id, queue_id);
@@ -6539,7 +6539,7 @@ rte_eth_tx_prepare(uint16_t port_id, uint16_t queue_id,#ifdef RTE_ETHDEV_DEBUG_TX if (port_id >=RTE_MAX_ETHPORTS || - queue_id >=RTE_MAX_QUEUES_PER_PORT ) { + queue_id >=RTE_MAX_ETHPORT_TX_QUEUES ) {RTE_ETHDEV_LOG_LINE (ERR ,"Invalid port_id=%u or queue_id=%u" , port_id, queue_id);
@@ -6744,7 +6744,7 @@ rte_eth_recycle_mbufs(uint16_t rx_port_id, uint16_t rx_queue_id,#ifdef RTE_ETHDEV_DEBUG_TX if (tx_port_id >=RTE_MAX_ETHPORTS || - tx_queue_id >=RTE_MAX_QUEUES_PER_PORT ) { + tx_queue_id >=RTE_MAX_ETHPORT_TX_QUEUES ) {RTE_ETHDEV_LOG_LINE (ERR ,"Invalid tx_port_id=%u or tx_queue_id=%u" , tx_port_id, tx_queue_id);
@@ -6770,7 +6770,7 @@ rte_eth_recycle_mbufs(uint16_t rx_port_id, uint16_t rx_queue_id,#ifdef RTE_ETHDEV_DEBUG_RX if (rx_port_id >=RTE_MAX_ETHPORTS || - rx_queue_id >=RTE_MAX_QUEUES_PER_PORT ) { + rx_queue_id >=RTE_MAX_ETHPORT_RX_QUEUES ) {RTE_ETHDEV_LOG_LINE (ERR ,"Invalid rx_port_id=%u or rx_queue_id=%u" , rx_port_id, rx_queue_id);return 0 ;
@@ -6890,7 +6890,7 @@ rte_eth_tx_queue_count(uint16_t port_id, uint16_t queue_id) goto out; } -if (queue_id >=RTE_MAX_QUEUES_PER_PORT ) { +if (queue_id >=RTE_MAX_ETHPORT_TX_QUEUES ) {RTE_ETHDEV_LOG_LINE (ERR ,"Invalid queue_id=%u for port_id=%u" , queue_id, port_id); rc = -EINVAL ;
Signature · 1 line
2.43.0
RFC PATCH v2 6/26 bpf: use separate Rx and Tx queue limits
In reply to Bruce RichardsonUpdate library to use the new defines RTE_MAX_ETHPORT_TX_QUEUES and RTE_MAX_ETHPORT_RX_QUEUES rather than the old define RTE_MAX_QUEUES_PER_PORT.
--- lib/bpf/bpf_pkt.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
lib/bpf/bpf_pkt.c +2 -1
@@ -503,7 +503,8 @@ bpf_eth_elf_load(struct bpf_eth_cbh *cbh, uint16_t port, uint16_t queue, ftx =NULL ;if (prm ==NULL ||rte_eth_dev_is_valid_port (port) ==0 || - queue >=RTE_MAX_QUEUES_PER_PORT ) + queue >= (cbh->type ==BPF_ETH_RX ? +RTE_MAX_ETHPORT_RX_QUEUES :RTE_MAX_ETHPORT_TX_QUEUES ))return -EINVAL ;if (cbh->type ==BPF_ETH_RX )
Signature · 1 line
2.43.0
RFC PATCH v2 7/26 latencystats: use separate Rx and Tx queue limits
In reply to Bruce RichardsonUpdate library to use the new defines RTE_MAX_ETHPORT_TX_QUEUES and RTE_MAX_ETHPORT_RX_QUEUES rather than the old define RTE_MAX_QUEUES_PER_PORT.
--- lib/latencystats/rte_latencystats.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
lib/latencystats/rte_latencystats.c +2 -2
@@ -60,8 +60,8 @@ struct rxtx_cbs {const struct rte_eth_rxtx_callback *cb; }; -static struct rxtx_cbs rx_cbs[RTE_MAX_ETHPORTS ][RTE_MAX_QUEUES_PER_PORT ]; -static struct rxtx_cbs tx_cbs[RTE_MAX_ETHPORTS ][RTE_MAX_QUEUES_PER_PORT ]; +static struct rxtx_cbs rx_cbs[RTE_MAX_ETHPORTS ][RTE_MAX_ETHPORT_RX_QUEUES ]; +static struct rxtx_cbs tx_cbs[RTE_MAX_ETHPORTS ][RTE_MAX_ETHPORT_TX_QUEUES ];struct latency_stats_nameoff {char name[RTE_ETH_XSTATS_NAME_SIZE ];
Signature · 1 line
2.43.0
RFC PATCH v2 8/26 pdump: use separate Rx and Tx queue limits
In reply to Bruce RichardsonUpdate library to use the new defines RTE_MAX_ETHPORT_TX_QUEUES and RTE_MAX_ETHPORT_RX_QUEUES rather than the old define RTE_MAX_QUEUES_PER_PORT.
--- lib/pdump/rte_pdump.c | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-)
lib/pdump/rte_pdump.c +9 -9
@@ -61,8 +61,8 @@ static struct pdump_rxtx_cbs {const struct rte_bpf *filter;enum pdump_version ver;uint32_t snaplen; -} rx_cbs[RTE_MAX_ETHPORTS ][RTE_MAX_QUEUES_PER_PORT ], -tx_cbs[RTE_MAX_ETHPORTS ][RTE_MAX_QUEUES_PER_PORT ]; +} rx_cbs[RTE_MAX_ETHPORTS ][RTE_MAX_ETHPORT_RX_QUEUES ], +tx_cbs[RTE_MAX_ETHPORTS ][RTE_MAX_ETHPORT_TX_QUEUES ];/*
@@ -72,8 +72,8 @@ tx_cbs[RTE_MAX_ETHPORTS][RTE_MAX_QUEUES_PER_PORT];*/ static const char MZ_RTE_PDUMP_STATS [] ="rte_pdump_stats" ;static struct { -struct rte_pdump_stats rx[RTE_MAX_ETHPORTS ][RTE_MAX_QUEUES_PER_PORT ]; -struct rte_pdump_stats tx[RTE_MAX_ETHPORTS ][RTE_MAX_QUEUES_PER_PORT ]; +struct rte_pdump_stats rx[RTE_MAX_ETHPORTS ][RTE_MAX_ETHPORT_RX_QUEUES ]; +struct rte_pdump_stats tx[RTE_MAX_ETHPORTS ][RTE_MAX_ETHPORT_TX_QUEUES ];const struct rte_memzone *mz; } *pdump_stats;
@@ -708,8 +708,8 @@ rte_pdump_disable_by_deviceid(char *device_id, uint16_t queue, }static void -pdump_sum_stats (uint16_t port,uint16_t nq, -struct rte_pdump_stats stats[RTE_MAX_ETHPORTS ][RTE_MAX_QUEUES_PER_PORT ], +pdump_sum_stats (uint16_t nq, +struct rte_pdump_stats *stats,struct rte_pdump_stats *total) {uint64_t *sum = (uint64_t *)total;
@@ -718,7 +718,7 @@ pdump_sum_stats(uint16_t port, uint16_t nq,uint16_t qid;for (qid =0 ; qid < nq; qid++) { - const RTE_ATOMIC(uint64_t) *perq = (const uint64_t __rte_atomic *)&stats[port][qid]; + const RTE_ATOMIC(uint64_t) *perq = (const uint64_t __rte_atomic *)&stats[qid];for (i =0 ; i <sizeof (*total) /sizeof (uint64_t ); i++) { val =rte_atomic_load_explicit (&perq[i], rte_memory_order_relaxed);
@@ -762,7 +762,7 @@ rte_pdump_stats(uint16_t port, struct rte_pdump_stats *stats) pdump_stats = mz->addr; } -pdump_sum_stats (port, dev_info.nb_rx_queues, pdump_stats->rx, stats); -pdump_sum_stats (port, dev_info.nb_tx_queues, pdump_stats->tx, stats); +pdump_sum_stats (dev_info.nb_rx_queues, pdump_stats->rx[port], stats); +pdump_sum_stats (dev_info.nb_tx_queues, pdump_stats->tx[port], stats);return 0 ; }
Signature · 1 line
2.43.0
RFC PATCH v2 9/26 power: use separate Rx and Tx queue limits
In reply to Bruce RichardsonUpdate library to use the new defines RTE_MAX_ETHPORT_TX_QUEUES and RTE_MAX_ETHPORT_RX_QUEUES rather than the old define RTE_MAX_QUEUES_PER_PORT.
--- lib/power/rte_power_pmd_mgmt.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
lib/power/rte_power_pmd_mgmt.c +2 -2
@@ -494,7 +494,7 @@ rte_power_ethdev_pmgmt_queue_enable(unsigned int lcore_id, uint16_t port_id,RTE_ETH_VALID_PORTID_OR_ERR_RET (port_id, -EINVAL ); -if (queue_id >=RTE_MAX_QUEUES_PER_PORT || lcore_id >=RTE_MAX_LCORE ) { +if (queue_id >=RTE_MAX_ETHPORT_RX_QUEUES || lcore_id >=RTE_MAX_LCORE ) { ret = -EINVAL ; goto end; }
@@ -608,7 +608,7 @@ rte_power_ethdev_pmgmt_queue_disable(unsigned int lcore_id,RTE_ETH_VALID_PORTID_OR_ERR_RET (port_id, -EINVAL ); -if (lcore_id >=RTE_MAX_LCORE || queue_id >=RTE_MAX_QUEUES_PER_PORT ) +if (lcore_id >=RTE_MAX_LCORE || queue_id >=RTE_MAX_ETHPORT_RX_QUEUES )return -EINVAL ;/* check if the queue is stopped */
Signature · 1 line
2.43.0
RFC PATCH v2 10/26 net/af_xdp: use separate Rx and Tx queue limits
In reply to Bruce RichardsonUpdate driver to use the new defines RTE_MAX_ETHPORT_TX_QUEUES and RTE_MAX_ETHPORT_RX_QUEUES rather than the old define RTE_MAX_QUEUES_PER_PORT.
--- drivers/net/af_xdp/rte_eth_af_xdp.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/af_xdp/rte_eth_af_xdp.c +1 -1
@@ -185,7 +185,7 @@ struct pmd_internals { };struct pmd_process_private { -int rxq_xsk_fds[RTE_MAX_QUEUES_PER_PORT ]; +int rxq_xsk_fds[RTE_MAX_ETHPORT_RX_QUEUES ]; };#define ETH_AF_XDP_IFACE_ARG "iface"
Signature · 1 line
2.43.0
RFC PATCH v2 11/26 net/cnxk: use separate Rx and Tx queue limits
In reply to Bruce RichardsonUpdate driver to use the new defines RTE_MAX_ETHPORT_TX_QUEUES and RTE_MAX_ETHPORT_RX_QUEUES rather than the old define RTE_MAX_QUEUES_PER_PORT.
--- drivers/net/cnxk/cnxk_ethdev_ops.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/net/cnxk/cnxk_ethdev_ops.c +2 -2
@@ -16,8 +16,8 @@ cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *devinfo) devinfo->min_rx_bufsize =NIX_MIN_HW_FRS +RTE_ETHER_CRC_LEN ; devinfo->max_rx_pktlen = max_rx_pktlen; - devinfo->max_rx_queues =RTE_MAX_QUEUES_PER_PORT ; - devinfo->max_tx_queues =RTE_MAX_QUEUES_PER_PORT ; + devinfo->max_rx_queues =RTE_MAX_ETHPORT_RX_QUEUES ; + devinfo->max_tx_queues =RTE_MAX_ETHPORT_TX_QUEUES ; devinfo->max_mac_addrs = dev->max_mac_entries; devinfo->max_vfs = pci_dev->max_vfs; devinfo->max_mtu = devinfo->max_rx_pktlen -CNXK_NIX_L2_OVERHEAD ;
Signature · 1 line
2.43.0
RFC PATCH v2 12/26 net/failsafe: use separate Rx and Tx queue limits
In reply to Bruce RichardsonUpdate driver to use the new defines RTE_MAX_ETHPORT_TX_QUEUES and RTE_MAX_ETHPORT_RX_QUEUES rather than the old define RTE_MAX_QUEUES_PER_PORT.
--- drivers/net/failsafe/failsafe_ops.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/net/failsafe/failsafe_ops.c +2 -2
@@ -1205,8 +1205,8 @@ fs_dev_infos_get(struct rte_eth_dev *dev, infos->min_mtu =RTE_ETHER_MIN_MTU ; infos->max_mtu =UINT16_MAX ; infos->max_rx_pktlen =UINT32_MAX ; - infos->max_rx_queues =RTE_MAX_QUEUES_PER_PORT ; - infos->max_tx_queues =RTE_MAX_QUEUES_PER_PORT ; + infos->max_rx_queues =RTE_MAX_ETHPORT_RX_QUEUES ; + infos->max_tx_queues =RTE_MAX_ETHPORT_TX_QUEUES ; infos->max_mac_addrs =FAILSAFE_MAX_ETHADDR ; infos->max_hash_mac_addrs =UINT32_MAX ; infos->max_vfs =UINT16_MAX ;
Signature · 1 line
2.43.0
RFC PATCH v2 13/26 net/hns3: use separate Rx and Tx queue limits
In reply to Bruce RichardsonUpdate driver to use the new defines RTE_MAX_ETHPORT_TX_QUEUES and RTE_MAX_ETHPORT_RX_QUEUES rather than the old define RTE_MAX_QUEUES_PER_PORT.
--- drivers/net/hns3/hns3_tm.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/hns3/hns3_tm.c +1 -1
@@ -21,7 +21,7 @@ hns3_tm_max_tx_queues_get(struct rte_eth_dev *dev)memset (&dev_info,0 ,sizeof (dev_info)); (void )hns3_dev_infos_get (dev, &dev_info); -return RTE_MIN (dev_info.max_tx_queues,RTE_MAX_QUEUES_PER_PORT ); +return RTE_MIN (dev_info.max_tx_queues,RTE_MAX_ETHPORT_TX_QUEUES ); }void
Signature · 1 line
2.43.0
RFC PATCH v2 14/26 net/mlx5: use separate Rx and Tx queue limits
In reply to Bruce RichardsonUpdate driver to use the new defines RTE_MAX_ETHPORT_TX_QUEUES and RTE_MAX_ETHPORT_RX_QUEUES rather than the old define RTE_MAX_QUEUES_PER_PORT.
--- drivers/net/mlx5/mlx5_flow.c | 2 +- drivers/net/mlx5/mlx5_flow_hw.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
drivers/net/mlx5/mlx5_flow.c +1 -1
@@ -7344,7 +7344,7 @@ flow_legacy_list_create(struct rte_eth_dev *dev, enum mlx5_flow_type type,struct rte_flow_item items[MLX5_MAX_SPLIT_ITEMS ];uint8_t buffer[2048 ]; } items_tx; -struct mlx5_rte_flow_item_sq sq_specs[RTE_MAX_QUEUES_PER_PORT ]; +struct mlx5_rte_flow_item_sq sq_specs[RTE_MAX_ETHPORT_RX_QUEUES ];struct mlx5_flow_expand_rss *buf = &expand_buffer.buf;struct mlx5_flow_rss_desc *rss_desc;const struct rte_flow_action *p_actions_rx;
drivers/net/mlx5/mlx5_flow_hw.c +1 -1
@@ -10978,7 +10978,7 @@ flow_hw_create_ctrl_rx_rss_template(struct rte_eth_dev *dev,struct rte_flow_actions_template_attr attr = { .ingress =1 , }; -uint16_t queue[RTE_MAX_QUEUES_PER_PORT ]; +uint16_t queue[RTE_MAX_ETHPORT_RX_QUEUES ];struct rte_flow_action_rss rss_conf = { .func =RTE_ETH_HASH_FUNCTION_DEFAULT , .level =0 ,
Signature · 1 line
2.43.0
RFC PATCH v2 15/26 net/null: use separate Rx and Tx queue limits
In reply to Bruce RichardsonUpdate driver to use the new defines RTE_MAX_ETHPORT_TX_QUEUES and RTE_MAX_ETHPORT_RX_QUEUES rather than the old define RTE_MAX_QUEUES_PER_PORT.
--- drivers/net/null/rte_eth_null.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/net/null/rte_eth_null.c +2 -2
@@ -53,8 +53,8 @@ struct pmd_internals {unsigned int no_rx;uint16_t port_id; -struct null_queue rx_null_queues[RTE_MAX_QUEUES_PER_PORT ]; -struct null_queue tx_null_queues[RTE_MAX_QUEUES_PER_PORT ]; +struct null_queue rx_null_queues[RTE_MAX_ETHPORT_RX_QUEUES ]; +struct null_queue tx_null_queues[RTE_MAX_ETHPORT_TX_QUEUES ];struct rte_ether_addr eth_addr;/** Bit mask of RSS offloads, the bit offset also means flow type */
Signature · 1 line
2.43.0
RFC PATCH v2 16/26 net/sfc: use separate Rx and Tx queue limits
In reply to Bruce RichardsonUpdate driver to use the new defines RTE_MAX_ETHPORT_TX_QUEUES and RTE_MAX_ETHPORT_RX_QUEUES rather than the old define RTE_MAX_QUEUES_PER_PORT.
--- drivers/net/sfc/sfc_sw_stats.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
drivers/net/sfc/sfc_sw_stats.c +4 -2
@@ -825,9 +825,11 @@ sfc_sw_xstats_alloc_queues_bitmap(struct sfc_adapter *sa)struct rte_bitmap **queues_bitmap = &sa->sw_stats.queues_bitmap;void **queues_bitmap_mem = &sa->sw_stats.queues_bitmap_mem;uint32_t bmp_size; +uint32_t max_queues =RTE_MAX (RTE_MAX_ETHPORT_RX_QUEUES , +RTE_MAX_ETHPORT_TX_QUEUES );int rc; - bmp_size =rte_bitmap_get_memory_footprint (RTE_MAX_QUEUES_PER_PORT ); + bmp_size =rte_bitmap_get_memory_footprint (max_queues); *queues_bitmap_mem =NULL ; *queues_bitmap =NULL ;
@@ -836,7 +838,7 @@ sfc_sw_xstats_alloc_queues_bitmap(struct sfc_adapter *sa)if (*queues_bitmap_mem ==NULL )return ENOMEM ; - *queues_bitmap =rte_bitmap_init (RTE_MAX_QUEUES_PER_PORT , + *queues_bitmap =rte_bitmap_init (max_queues, *queues_bitmap_mem, bmp_size);if (*queues_bitmap ==NULL ) { rc =EINVAL ;
Signature · 1 line
2.43.0
RFC PATCH v2 17/26 net/thunderx: use separate Rx and Tx queue limits
In reply to Bruce RichardsonUpdate driver to use the new defines RTE_MAX_ETHPORT_TX_QUEUES and RTE_MAX_ETHPORT_RX_QUEUES rather than the old define RTE_MAX_QUEUES_PER_PORT.
--- drivers/net/thunderx/nicvf_ethdev.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/thunderx/nicvf_ethdev.c +1 -1
@@ -907,7 +907,7 @@ nicvf_configure_rss_reta(struct rte_eth_dev *dev) {struct nicvf *nic =nicvf_pmd_priv (dev);unsigned int idx, qmap_size; -uint8_t qmap[RTE_MAX_QUEUES_PER_PORT ]; +uint8_t qmap[RTE_MAX_ETHPORT_RX_QUEUES ];uint8_t default_reta[NIC_MAX_RSS_IDR_TBL_SIZE ];if (nic->cpi_alg !=CPI_ALG_NONE )
Signature · 1 line
2.43.0
RFC PATCH v2 18/26 net/vhost: use separate Rx and Tx queue limits
In reply to Bruce RichardsonUpdate driver to use the new defines RTE_MAX_ETHPORT_TX_QUEUES and RTE_MAX_ETHPORT_RX_QUEUES rather than the old define RTE_MAX_QUEUES_PER_PORT.
--- drivers/net/vhost/rte_eth_vhost.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
drivers/net/vhost/rte_eth_vhost.c +4 -3
@@ -117,8 +117,8 @@ static struct rte_eth_link pmd_link = {struct rte_vhost_vring_state {rte_spinlock_t lock; -bool cur[RTE_MAX_QUEUES_PER_PORT *2 ]; -bool seen[RTE_MAX_QUEUES_PER_PORT *2 ]; +bool cur[RTE_MAX_ETHPORT_RX_QUEUES +RTE_MAX_ETHPORT_TX_QUEUES ]; +bool seen[RTE_MAX_ETHPORT_RX_QUEUES +RTE_MAX_ETHPORT_TX_QUEUES ];unsigned int index;unsigned int max_vring; };
@@ -1648,7 +1648,8 @@ rte_pmd_vhost_probe(struct rte_vdev_device *dev)if (rte_kvargs_count (kvlist,ETH_VHOST_QUEUES_ARG ) ==1 ) { ret =rte_kvargs_process (kvlist,ETH_VHOST_QUEUES_ARG , &open_int, &queues); -if (ret <0 || queues >RTE_MAX_QUEUES_PER_PORT ) +if (ret <0 || queues >RTE_MAX_ETHPORT_RX_QUEUES + || queues >RTE_MAX_ETHPORT_TX_QUEUES ) goto out_free; }else
Signature · 1 line
2.43.0
RFC PATCH v2 19/26 app/dumpcap: use separate Rx and Tx queue limits
In reply to Bruce RichardsonUpdate app to use the new defines RTE_MAX_ETHPORT_TX_QUEUES and RTE_MAX_ETHPORT_RX_QUEUES rather than the old define RTE_MAX_QUEUES_PER_PORT.
--- app/dumpcap/main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
app/dumpcap/main.c +1 -1
@@ -95,7 +95,7 @@ struct interface {struct rte_bpf_prm *bpf_prm;char name[RTE_ETH_NAME_MAX_LEN ]; -struct rte_rxtx_callback *rx_cb[RTE_MAX_QUEUES_PER_PORT ]; +struct rte_rxtx_callback *rx_cb[RTE_MAX_ETHPORT_RX_QUEUES ];const char *ifname;const char *ifdescr; };
Signature · 1 line
2.43.0
RFC PATCH v2 20/26 app/test-pmd: use separate Rx and Tx queue limits
In reply to Bruce RichardsonUpdate app to use the new defines RTE_MAX_ETHPORT_TX_QUEUES and RTE_MAX_ETHPORT_RX_QUEUES rather than the old define RTE_MAX_QUEUES_PER_PORT.
--- app/test-pmd/testpmd.c | 7 ++++--- app/test-pmd/testpmd.h | 16 ++++++++-------- 2 files changed, 12 insertions(+), 11 deletions(-)
app/test-pmd/testpmd.c +4 -3
@@ -1305,7 +1305,7 @@ check_socket_id(const unsigned int socket_id)queueid_t get_allowed_max_nb_rxq (portid_t *pid) { -queueid_t allowed_max_rxq =RTE_MAX_QUEUES_PER_PORT ; +queueid_t allowed_max_rxq =RTE_MAX_ETHPORT_RX_QUEUES ;bool max_rxq_valid = false;portid_t pi;struct rte_eth_dev_info dev_info;
@@ -1353,7 +1353,7 @@ check_nb_rxq(queueid_t rxq)queueid_t get_allowed_max_nb_txq (portid_t *pid) { -queueid_t allowed_max_txq =RTE_MAX_QUEUES_PER_PORT ; +queueid_t allowed_max_txq =RTE_MAX_ETHPORT_TX_QUEUES ;bool max_txq_valid = false;portid_t pi;struct rte_eth_dev_info dev_info;
@@ -1564,7 +1564,8 @@ check_nb_txd(queueid_t txd)queueid_t get_allowed_max_nb_hairpinq (portid_t *pid) { -queueid_t allowed_max_hairpinq =RTE_MAX_QUEUES_PER_PORT ; +queueid_t allowed_max_hairpinq =RTE_MIN (RTE_MAX_ETHPORT_RX_QUEUES , +RTE_MAX_ETHPORT_TX_QUEUES );portid_t pi;struct rte_eth_hairpin_cap cap;
app/test-pmd/testpmd.h +8 -8
@@ -332,10 +332,10 @@ struct rte_port {uint8_t need_reconfig_queues;/**< need reconfiguring queues or not */ uint8_t rss_flag;/**< enable rss or not */ uint8_t dcb_flag;/**< enable dcb */ -uint16_t nb_rx_desc[RTE_MAX_QUEUES_PER_PORT +1 ];/**< per queue rx desc number */ -uint16_t nb_tx_desc[RTE_MAX_QUEUES_PER_PORT +1 ];/**< per queue tx desc number */ -struct port_rxqueue rxq[RTE_MAX_QUEUES_PER_PORT +1 ];/**< per queue Rx config and state */ -struct port_txqueue txq[RTE_MAX_QUEUES_PER_PORT +1 ];/**< per queue Tx config and state */ +uint16_t nb_rx_desc[RTE_MAX_ETHPORT_RX_QUEUES +1 ];/**< per queue rx desc number */ +uint16_t nb_tx_desc[RTE_MAX_ETHPORT_TX_QUEUES +1 ];/**< per queue tx desc number */ +struct port_rxqueue rxq[RTE_MAX_ETHPORT_RX_QUEUES +1 ];/**< per queue Rx config and state */ +struct port_txqueue txq[RTE_MAX_ETHPORT_TX_QUEUES +1 ];/**< per queue Tx config and state */ struct rte_ether_addr *mc_addr_pool;/**< pool of multicast addrs */ uint32_t mc_addr_nb;/**< nb. of addr. in mc_addr_pool */ queueid_t queue_nb;/**< nb. of queues for flow rules */
@@ -351,14 +351,14 @@ struct rte_port {struct port_indirect_action *actions_list;/**< Associated indirect actions. */ LIST_HEAD(, port_flow_tunnel) flow_tunnel_list; -const struct rte_eth_rxtx_callback *rx_dump_cb[RTE_MAX_QUEUES_PER_PORT +1 ]; -const struct rte_eth_rxtx_callback *tx_dump_cb[RTE_MAX_QUEUES_PER_PORT +1 ]; +const struct rte_eth_rxtx_callback *rx_dump_cb[RTE_MAX_ETHPORT_RX_QUEUES +1 ]; +const struct rte_eth_rxtx_callback *tx_dump_cb[RTE_MAX_ETHPORT_TX_QUEUES +1 ];/**< metadata value to insert in Tx packets. */ uint32_t tx_metadata; -const struct rte_eth_rxtx_callback *tx_set_md_cb[RTE_MAX_QUEUES_PER_PORT +1 ]; +const struct rte_eth_rxtx_callback *tx_set_md_cb[RTE_MAX_ETHPORT_TX_QUEUES +1 ];/**< dynamic flags. */ uint64_t mbuf_dynf; -const struct rte_eth_rxtx_callback *tx_set_dynf_cb[RTE_MAX_QUEUES_PER_PORT +1 ]; +const struct rte_eth_rxtx_callback *tx_set_dynf_cb[RTE_MAX_ETHPORT_TX_QUEUES +1 ];struct xstat_display_info xstats_info; };
Signature · 1 line
2.43.0
RFC PATCH v2 21/26 examples/ipsec-secgw: use separate Rx and Tx queue limits
In reply to Bruce RichardsonUpdate example app to use the new defines RTE_MAX_ETHPORT_TX_QUEUES and RTE_MAX_ETHPORT_RX_QUEUES rather than the old define RTE_MAX_QUEUES_PER_PORT.
--- examples/ipsec-secgw/ipsec-secgw.c | 2 +- examples/ipsec-secgw/ipsec.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
examples/ipsec-secgw/ipsec-secgw.c +1 -1
@@ -1050,7 +1050,7 @@ parse_config(const char *q_arg)uint32_t size;uint32_t max_fld[_NUM_FLD] = {RTE_MAX_ETHPORTS , -RTE_MAX_QUEUES_PER_PORT , +RTE_MAX_ETHPORT_RX_QUEUES ,RTE_MAX_LCORE };
examples/ipsec-secgw/ipsec.c +1 -1
@@ -584,7 +584,7 @@ create_inline_session(struct socket_ctx *skt_ctx, struct ipsec_sa *sa, .rss_key_len = sizeof(rss_key), };struct rte_eth_dev_info dev_info; -uint16_t queue[RTE_MAX_QUEUES_PER_PORT ]; +uint16_t queue[RTE_MAX_ETHPORT_RX_QUEUES ];struct rte_flow_action_rss action_rss;unsigned int i;unsigned int j;
Signature · 1 line
2.43.0
RFC PATCH v2 22/26 examples/l3fwd-power: use separate Rx and Tx queue limits
In reply to Bruce RichardsonUpdate example app to use the new defines RTE_MAX_ETHPORT_TX_QUEUES and RTE_MAX_ETHPORT_RX_QUEUES rather than the old define RTE_MAX_QUEUES_PER_PORT.
--- examples/l3fwd-power/main.c | 2 +- examples/l3fwd-power/perf_core.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
examples/l3fwd-power/main.c +1 -1
@@ -1660,7 +1660,7 @@ parse_config(const char *q_arg)unsigned size;unsigned int max_fld[_NUM_FLD] = {RTE_MAX_ETHPORTS , -RTE_MAX_QUEUES_PER_PORT , +RTE_MAX_ETHPORT_RX_QUEUES ,RTE_MAX_LCORE };
examples/l3fwd-power/perf_core.c +1 -1
@@ -134,7 +134,7 @@ parse_perf_config(const char *q_arg)unsigned int size;unsigned int max_fld[_NUM_FLD] = {RTE_MAX_ETHPORTS , -RTE_MAX_QUEUES_PER_PORT , +RTE_MAX_ETHPORT_RX_QUEUES ,255 ,RTE_MAX_LCORE };
Signature · 1 line
2.43.0
RFC PATCH v2 23/26 examples/l3fwd: use separate Rx and Tx queue limits
In reply to Bruce RichardsonUpdate example app to use the new defines RTE_MAX_ETHPORT_TX_QUEUES and RTE_MAX_ETHPORT_RX_QUEUES rather than the old define RTE_MAX_QUEUES_PER_PORT.
--- examples/l3fwd/main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
examples/l3fwd/main.c +1 -1
@@ -502,7 +502,7 @@ parse_config(const char *q_arg)unsigned size;uint16_t max_fld[_NUM_FLD] = {RTE_MAX_ETHPORTS , -RTE_MAX_QUEUES_PER_PORT , +RTE_MAX_ETHPORT_RX_QUEUES ,RTE_MAX_LCORE };
Signature · 1 line
2.43.0
RFC PATCH v2 24/26 examples/vhost: use separate Rx and Tx queue limits
In reply to Bruce RichardsonUpdate example app to use the new defines RTE_MAX_ETHPORT_TX_QUEUES and RTE_MAX_ETHPORT_RX_QUEUES rather than the old define RTE_MAX_QUEUES_PER_PORT.
--- examples/vhost/main.c | 2 +- examples/vhost/main.h | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
examples/vhost/main.c +1 -1
@@ -1891,7 +1891,7 @@ reset_dma(void)for (i =0 ; i <RTE_MAX_VHOST_DEVICE ; i++) {int j; -for (j =0 ; j <RTE_MAX_QUEUES_PER_PORT *2 ; j++) { +for (j =0 ; j <RTE_MAX_ETHPORT_RX_QUEUES +RTE_MAX_ETHPORT_TX_QUEUES ; j++) { dma_bind[i].dmas[j].dev_id =INVALID_DMA_ID ; dma_bind[i].dmas[j].async_enabled = false; }
examples/vhost/main.h +1 -1
@@ -99,7 +99,7 @@ struct dma_info { };struct dma_for_vhost { -struct dma_info dmas[RTE_MAX_QUEUES_PER_PORT *2 ]; +struct dma_info dmas[RTE_MAX_ETHPORT_RX_QUEUES +RTE_MAX_ETHPORT_TX_QUEUES ];uint32_t async_flag; };
Signature · 1 line
2.43.0
RFC PATCH v2 25/26 config: make queues per port a meson config option
In reply to Bruce RichardsonFrom: Bruce Richardson <[email protected]>
The default number of ethernet queues per port is currently set to 1k which is more than enough for most applications, but still is lower than the total number of queues which may be available on modern NICs. Rather than increasing the max queues further, which will increase the memory footprint (since the value is used in array dimensioning), we can instead make the value a meson tunable option - and reduce the default value to 256 in the process. This means that:
* most apps which don't need hundreds of queues will see lower mem use.
* apps which do need to use thousands of queues can configure DPDK to allow this, without having to modify DPDK files (i.e. rte_config.h)
--- config/meson.build | 2 ++ config/rte_config.h | 3 --- doc/guides/rel_notes/release_24_11.rst | 13 +++++++++++-- meson_options.txt | 4 ++++ 4 files changed, 17 insertions(+), 5 deletions(-)
config/meson.build +2 -0
@@ -352,6 +352,8 @@ endforeach # set other values pulled from the build options dpdk_conf.set('RTE_MAX_ETHPORTS', get_option('max_ethports')) +dpdk_conf.set('RTE_MAX_ETHPORT_RX_QUEUES', get_option('max_ethport_rx_queues')) +dpdk_conf.set('RTE_MAX_ETHPORT_TX_QUEUES', get_option('max_ethport_tx_queues')) dpdk_conf.set('RTE_LIBEAL_USE_HPET', get_option('use_hpet')) dpdk_conf.set('RTE_ENABLE_STDATOMIC', get_option('enable_stdatomic')) dpdk_conf.set('RTE_ENABLE_TRACE_FP', get_option('enable_trace_fp'))
config/rte_config.h +0 -3
@@ -64,9 +64,6 @@#define RTE_MBUF_DEFAULT_MEMPOOL_OPS "ring_mp_mc"/* ether defines */ -#define RTE_MAX_QUEUES_PER_PORT 1024 -#define RTE_MAX_ETHPORT_RX_QUEUES 1024 -#define RTE_MAX_ETHPORT_TX_QUEUES 1024#define RTE_ETHDEV_QUEUE_STAT_CNTRS 16/* max 256 */ #define RTE_ETHDEV_RXTX_CALLBACKS 1#define RTE_MAX_MULTI_HOST_CTRLS 4
doc/guides/rel_notes/release_24_11.rst +11 -2
@@ -58,8 +58,17 @@ New Features * **New configuration settings for Rx and Tx Ethernet Queues.** New defines have been added to DPDK to allow independent tracking of the maximum number of Rx and Tx queues. - These defines as ``RTE_MAX_ETHPORT_RX_QUEUES`` and ``RTE_MAX_ETHPORT_TX_QUEUES``, - and should be used in place of the older macro ``RTE_MAX_QUEUES_PER_PORT``. + These defines are ``RTE_MAX_ETHPORT_RX_QUEUES`` and ``RTE_MAX_ETHPORT_TX_QUEUES``, + and these new defines should be used in place of the older define ``RTE_MAX_QUEUES_PER_PORT``. + Unlike the older define, these new defines are build-time configurable via meson options: + ``max_ethport_rx_queues`` and ``max_ethport_tx_queues``. + +.. note:: + + The default max values for Rx and Tx queue limits are reduced from 1024, in previous releases, + to 256 in this release. + For application that require large numbers of queues, + these defaults can be changed via the meson configuration options described above. Removed Items
meson_options.txt +4 -0
@@ -40,6 +40,10 @@ option('machine', type: 'string', value: 'auto', description: 'Alias of cpu_instruction_set.') option('max_ethports', type: 'integer', value: 32, description: 'maximum number of Ethernet devices') +option('max_ethport_rx_queues', type: 'integer', value: 256, description: + 'maximum number of Rx queues on an Ethernet device') +option('max_ethport_tx_queues', type: 'integer', value: 256, description: + 'maximum number of Tx queues on an Ethernet device') option('max_lcores', type: 'string', value: 'default', description: 'Set maximum number of cores/threads supported by EAL; "default" is different per-arch, "detect" detects the number of cores on the build machine.') option('max_numa_nodes', type: 'string', value: 'default', description:
Signature · 1 line
2.43.0
RFC PATCH v2 26/26 config: add computed max queues define for compatibility
In reply to Bruce RichardsonEnd applications may use the RTE_MAX_QUEUES_PER_PORT define in their structure definitions, so keep a define present in DPDK for backward compatibility. Rather than having a hard-coded value, we can use the maximum of the Rx and Tx values as the overall max value. Rather than using a macro which does the MAX() calculation inside it, we can compute the actual value at configuration time and write it using meson.
--- config/meson.build | 8 ++++++++ doc/guides/rel_notes/deprecation.rst | 11 +++++++++++ doc/guides/rel_notes/release_24_11.rst | 8 +++++++- 3 files changed, 26 insertions(+), 1 deletion(-)
config/meson.build +8 -0
@@ -372,6 +372,14 @@ if get_option('mbuf_refcnt_atomic') endif dpdk_conf.set10('RTE_IOVA_IN_MBUF', get_option('enable_iova_as_pa')) +# set old MAX_QUEUES_PER_PORT option for compatibility. Compute +# value as max of Rx and Tx counts +if get_option('max_ethport_rx_queues') > get_option('max_ethport_tx_queues') + dpdk_conf.set('RTE_MAX_QUEUES_PER_PORT', get_option('max_ethport_rx_queues')) +else + dpdk_conf.set('RTE_MAX_QUEUES_PER_PORT', get_option('max_ethport_tx_queues')) +endif + compile_time_cpuflags = [] subdir(arch_subdir) dpdk_conf.set('RTE_COMPILE_TIME_CPUFLAGS', ','.join(compile_time_cpuflags))
doc/guides/rel_notes/deprecation.rst +11 -0
@@ -17,6 +17,17 @@ Other API and ABI deprecation notices are to be posted below. Deprecation Notices ------------------- +* config: The define ``RTE_MAX_QUEUES_PER_PORT`` should be considered deprecated + and may be removed in a future release. + Its use in apps should be replaced by ``RTE_MAX_ETHPORT_RX_QUEUES`` or ``RTE_MAX_ETHPORT_TX_QUEUES``, + as appropriate. + +* config: The ``RTE_MAX_QUEUES_PER_PORT`` value is no longer hard-coded to 1024. + Its value is now computed at configuration time to be the maximum of the configured max Rx and Tx queue values, + given by the meson options ``max_ethport_rx_queues`` and ``max_ethport_tx_queues``. + If these are unmodified from the defaults, + the value of ``RTE_MAX_QUEUES_PER_PORT`` will be 256. + * build: The ``enable_kmods`` option is deprecated and will be removed in a future release. Setting/clearing the option has no impact on the build. Instead, kernel modules will be always built for OS's where out-of-tree kernel modules
doc/guides/rel_notes/release_24_11.rst +7 -1
@@ -67,9 +67,15 @@ New Features The default max values for Rx and Tx queue limits are reduced from 1024, in previous releases, to 256 in this release. - For application that require large numbers of queues, + For applications that require large numbers of queues, these defaults can be changed via the meson configuration options described above. +.. note:: + + The define ``RTE_MAX_QUEUES_PER_PORT`` is kept for backward compatibility. + Its value is no longer hard-coded, + but is set, at configuration time, to the maximum of the configured max Rx and Tx queue values. + Removed Items -------------
Signature · 1 line
2.43.0
Re: [RFC PATCH v2 26/26] config: add computed max queues define for compatibility
doc/guides/rel_notes/release_24_11.rst New Features · line 77
Quoted 5 lines · On Tue, 13 Aug 2024 17:00:03 +0100 Bruce Richardson <[email protected]>…
+.. note::
doc/guides/rel_notes/release_24_11.rst +4
-0+ + The define ``RTE_MAX_QUEUES_PER_PORT`` is kept for backward compatibility. + Its value is no longer hard-coded, + but is set, at configuration time, to the maximum of the configured max Rx and Tx queue values.
Would rather it just be marked __rte_deprecated because a runtime calculation is going to break code that uses this to size arrays at compile time.
Re: [RFC PATCH v2 26/26] config: add computed max queues define for compatibility
Quoted 11 lines · Stephen Hemminger
doc/guides/rel_notes/release_24_11.rst New Features · line 77
Quoted 5 lines · On Tue, 13 Aug 2024 17:00:03 +0100 Bruce Richardson <[email protected]>…
+.. note::
doc/guides/rel_notes/release_24_11.rst +4
-0+ + The define ``RTE_MAX_QUEUES_PER_PORT`` is kept for backward compatibility. + Its value is no longer hard-coded, + but is set, at configuration time, to the maximum of the configured max Rx and Tx queue values.Would rather it just be marked
__rte_deprecatedbecause a runtime calculation is going to break code that uses this to size arrays at compile time.
"runtime calculation is going to break" -> yep, that's why it's computed at configuration time using meson, so all C code sees is a hard-coded value. See commit log message, which calls this out too.
In terms of deprecating the old value, I'd really rather not do so, as I suspect that this value is likely used in many end-user applications. While I know according to our policy we could remove this, the cost of keeping around a backward-compatible version of this seems pretty small and may avoid breaking lots of end-applications. Then again, we may want to deprecate it for internal use in DPDK.
/Bruce
RE: [RFC PATCH v2 00/26] add meson config options for queues per port
In reply to Bruce RichardsonQuoted 40 lines
From: Bruce Richardson [mailto:[email protected]]
There are a number of issues with the current
RTE_MAX_QUEUES_PER_PORTsetting in DPDK that are addressed by this patchset:* The name does not make it clear that this is intended as an ethdev-only setting
* A number of other libraries are using this define rather than having more relevant defines for the particular usecase.
* The define is hard-coded in DPDK source code and is not adjustable via a build-time/meson option
* Because of the lack of configurability, the max is therefore set to a conservatively-high value, wasting memory.
* There is an assumption that the number of Rx queues and Tx queues should have the same maximum value. Depending on application, it may be desirable to have fan-in with multiple Rx queues e.g. for classification/filtering, feed a single Tx queue, or the opposite where, e.g. for QoS Tx scheduling, a few Rx queues feeds a very large number of Tx queues.This patchset therefore addresses these by:
* replacing the single define for max queues with independent defines for Rx and Tx queues.
* adjusts the name to ensure that it is clear the defines are for ethports only. [ethports being used in theRTE_MAX_ETHPORTSsetting].
* replaces occurances ofRTE_MAX_QUEUES_PER_PORTwith appropriate defines for non-ethdev use cases
* replaces all other internal occurances of the define with the new per-Rx and per-Tx definitions.
* adds meson config options to allow build-time configuration of the max Rx and Tx queue values.Naming Note:
* The new meson config options are called "max_ethport_rx_queues" and "max_ethport_tx_queues" so that in the meson options list they appear alphabetically beside the existing "max_ethports" option.
* For naming consistency, the new C defines are thereforeRTE_MAX_ETHPORT_RX_QUEUESandRTE_MAX_ETHPORT_TX_QUEUES.
Quoted 10 lines
V2:
* What was a single patch with "3 insertions(+), 1 deletion(-)" has now become a 26-patch set! :-)
* Created separate Rx and Tx defines
* Ensured that the name makes it clear that the define is for ethdev
* When updating internal use, created one patch per component for easier maintainer review. In most cases it was obvious whether Rx or Tx define should be used, but a few cases were less clear.
* Added documentation updates for the changes (release notes and deprecation notice), spread across 3 of the patches.
Thanks.
For the series,
@Bruce: There's something wrong with your "From" email address; bruce.richards@ bounces. So I resent this reply to your bruce.richardson@ address.
Re: [RFC PATCH v2 00/26] add meson config options for queues per port
Quoted 51 lines · Morten Brørup
Quoted 40 lines
From: Bruce Richardson [mailto:[email protected]]
There are a number of issues with the current
RTE_MAX_QUEUES_PER_PORTsetting in DPDK that are addressed by this patchset:* The name does not make it clear that this is intended as an ethdev-only setting
* A number of other libraries are using this define rather than having more relevant defines for the particular usecase.
* The define is hard-coded in DPDK source code and is not adjustable via a build-time/meson option
* Because of the lack of configurability, the max is therefore set to a conservatively-high value, wasting memory.
* There is an assumption that the number of Rx queues and Tx queues should have the same maximum value. Depending on application, it may be desirable to have fan-in with multiple Rx queues e.g. for classification/filtering, feed a single Tx queue, or the opposite where, e.g. for QoS Tx scheduling, a few Rx queues feeds a very large number of Tx queues.This patchset therefore addresses these by:
* replacing the single define for max queues with independent defines for Rx and Tx queues.
* adjusts the name to ensure that it is clear the defines are for ethports only. [ethports being used in theRTE_MAX_ETHPORTSsetting].
* replaces occurances ofRTE_MAX_QUEUES_PER_PORTwith appropriate defines for non-ethdev use cases
* replaces all other internal occurances of the define with the new per-Rx and per-Tx definitions.
* adds meson config options to allow build-time configuration of the max Rx and Tx queue values.Naming Note:
* The new meson config options are called "max_ethport_rx_queues" and "max_ethport_tx_queues" so that in the meson options list they appear alphabetically beside the existing "max_ethports" option.
* For naming consistency, the new C defines are thereforeRTE_MAX_ETHPORT_RX_QUEUESandRTE_MAX_ETHPORT_TX_QUEUES.Quoted 10 lines
V2:
* What was a single patch with "3 insertions(+), 1 deletion(-)" has now become a 26-patch set! :-)
* Created separate Rx and Tx defines
* Ensured that the name makes it clear that the define is for ethdev
* When updating internal use, created one patch per component for easier maintainer review. In most cases it was obvious whether Rx or Tx define should be used, but a few cases were less clear.
* Added documentation updates for the changes (release notes and deprecation notice), spread across 3 of the patches.
Quoted 8 lines
Thanks.
For the series,
Acked-by Morten Brørup <[email protected]>@Bruce: There's something wrong with your "From" email address; bruce.richards@ bounces. So I resent this reply to your bruce.richardson@ address.
Yes, indeed. Something has indeed got messed up - probably in my git configuration here. I'll resend a v3 to try and correct it, so that others don't get any bounces.
/Bruce
v3 cover add meson config options for queues per port
In reply to Bruce RichardsonThere are a number of issues with the current RTE_MAX_QUEUES_PER_PORT setting in DPDK that are addressed by this patchset:
* The name does not make it clear that this is intended as an ethdev-only setting
* A number of other libraries are using this define rather than having more relevant defines for the particular usecase.
* The define is hard-coded in DPDK source code and is not adjustable via a build-time/meson option
* Because of the lack of configurability, the max is therefore set to a conservatively-high value, wasting memory.
* There is an assumption that the number of Rx queues and Tx queues should have the same maximum value. Depending on application, it may be desirable to have fan-in with multiple Rx queues e.g. for classification/filtering, feed a single Tx queue, or the opposite where, e.g. for QoS Tx scheduling, a few Rx queues feeds a very large number of Tx queues.
This patchset therefore addresses these by:
* replacing the single define for max queues with independent defines for Rx and Tx queues.
* adjusts the name to ensure that it is clear the defines are for ethports only. [ethports being used in the RTE_MAX_ETHPORTS setting].
* replaces occurances of RTE_MAX_QUEUES_PER_PORT with appropriate defines for non-ethdev use cases
* replaces all other internal occurances of the define with the new per-Rx and per-Tx definitions.
* adds meson config options to allow build-time configuration of the max Rx and Tx queue values.
Naming Note:
* The new meson config options are called "max_ethport_rx_queues" and "max_ethport_tx_queues" so that in the meson options list they appear alphabetically beside the existing "max_ethports" option.
* For naming consistency, the new C defines are therefore RTE_MAX_ETHPORT_RX_QUEUES and RTE_MAX_ETHPORT_TX_QUEUES.
V3:
* Resend of v2 with correct author email, to avoid reply bounces
* drop "rfc" prefix from patches
V2:
* What was a single patch with "3 insertions(+), 1 deletion(-)" has now become a 26-patch set! :-)
* Created separate Rx and Tx defines
* Ensured that the name makes it clear that the define is for ethdev
* When updating internal use, created one patch per component for easier maintainer review. In most cases it was obvious whether Rx or Tx define should be used, but a few cases were less clear.
* Added documentation updates for the changes (release notes and deprecation notice), spread across 3 of the patches.
Bruce Richardson (26):
cryptodev: remove use of ethdev max queues definition
eventdev: remove use of ethev queues define app/test-bbdev: remove use of ethdev queue count value
config: add separate defines for max Rx and Tx queues
ethdev: use separate Rx and Tx queue limits
bpf: use separate Rx and Tx queue limits
latencystats: use separate Rx and Tx queue limits
pdump: use separate Rx and Tx queue limits
power: use separate Rx and Tx queue limits net/af_xdp: use separate Rx and Tx queue limits net/cnxk: use separate Rx and Tx queue limits net/failsafe: use separate Rx and Tx queue limits net/hns3: use separate Rx and Tx queue limits net/mlx5: use separate Rx and Tx queue limits net/null: use separate Rx and Tx queue limits net/sfc: use separate Rx and Tx queue limits net/thunderx: use separate Rx and Tx queue limits net/vhost: use separate Rx and Tx queue limits app/dumpcap: use separate Rx and Tx queue limits app/test-pmd: use separate Rx and Tx queue limits examples/ipsec-secgw: use separate Rx and Tx queue limits examples/l3fwd-power: use separate Rx and Tx queue limits examples/l3fwd: use separate Rx and Tx queue limits examples/vhost: use separate Rx and Tx queue limits
config: make queues per port a meson config option
config: add computed max queues define for compatibility
app/dumpcap/main.c | 2 +- app/test-bbdev/test_bbdev.c | 4 ++-- app/test-pmd/testpmd.c | 7 ++++--- app/test-pmd/testpmd.h | 16 ++++++++-------- config/meson.build | 10 ++++++++++ config/rte_config.h | 2 +- doc/guides/rel_notes/deprecation.rst | 11 +++++++++++ doc/guides/rel_notes/release_24_11.rst | 21 +++++++++++++++++++++ drivers/net/af_xdp/rte_eth_af_xdp.c | 2 +- drivers/net/cnxk/cnxk_ethdev_ops.c | 4 ++-- drivers/net/failsafe/failsafe_ops.c | 4 ++-- drivers/net/hns3/hns3_tm.c | 2 +- drivers/net/mlx5/mlx5_flow.c | 2 +- drivers/net/mlx5/mlx5_flow_hw.c | 2 +- drivers/net/null/rte_eth_null.c | 4 ++-- drivers/net/sfc/sfc_sw_stats.c | 6 ++++-- drivers/net/thunderx/nicvf_ethdev.c | 2 +- drivers/net/vhost/rte_eth_vhost.c | 7 ++++--- examples/ipsec-secgw/ipsec-secgw.c | 2 +- examples/ipsec-secgw/ipsec.c | 2 +- examples/l3fwd-power/main.c | 2 +- examples/l3fwd-power/perf_core.c | 2 +- examples/l3fwd/main.c | 2 +- examples/vhost/main.c | 2 +- examples/vhost/main.h | 2 +- lib/bpf/bpf_pkt.c | 3 ++- lib/cryptodev/cryptodev_pmd.c | 4 ++-- lib/ethdev/ethdev_driver.h | 8 ++++---- lib/ethdev/ethdev_private.c | 24 ++++++++++++++---------- lib/ethdev/rte_ethdev.c | 16 +++++++--------- lib/ethdev/rte_ethdev.h | 18 +++++++++--------- lib/eventdev/eventdev_private.c | 2 +- lib/latencystats/rte_latencystats.c | 4 ++-- lib/pdump/rte_pdump.c | 18 +++++++++--------- lib/power/rte_power_pmd_mgmt.c | 4 ++-- meson_options.txt | 4 ++++ 36 files changed, 140 insertions(+), 87 deletions(-)
--
2.43.0
PATCH v3 1/26 cryptodev: remove use of ethdev max queues definition
The number of queue pairs supported by cryptodev should not be dependent on the number of ethdev Rx or Tx queues, so add a new define for cryptodev specifically.
--- config/rte_config.h | 1 + lib/cryptodev/cryptodev_pmd.c | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-)
config/rte_config.h +1 -0
@@ -71,6 +71,7 @@/* cryptodev defines */ #define RTE_CRYPTO_MAX_DEVS 64 +#define RTE_CRYPTO_MAX_QPS_PER_DEV 256#define RTE_CRYPTODEV_NAME_LEN 64#define RTE_CRYPTO_CALLBACKS 1
lib/cryptodev/cryptodev_pmd.c +2 -2
@@ -212,8 +212,8 @@ dummy_crypto_dequeue_burst(__rte_unused void *qp,void cryptodev_fp_ops_reset (struct rte_crypto_fp_ops *fp_ops) { -static struct rte_cryptodev_cb_rcu dummy_cb[RTE_MAX_QUEUES_PER_PORT ]; -static void *dummy_data[RTE_MAX_QUEUES_PER_PORT ]; +static struct rte_cryptodev_cb_rcu dummy_cb[RTE_CRYPTO_MAX_QPS_PER_DEV ]; +static void *dummy_data[RTE_CRYPTO_MAX_QPS_PER_DEV ];static const struct rte_crypto_fp_ops dummy = { .enqueue_burst = dummy_crypto_enqueue_burst, .dequeue_burst = dummy_crypto_dequeue_burst,
Signature · 1 line
2.43.0
Re: [PATCH v3 01/26] cryptodev: remove use of ethdev max queues definition
Quoted 17 lines · Bruce Richardson
The number of queue pairs supported by cryptodev should not be dependent on the number of ethdev Rx or Tx queues, so add a new define for cryptodev specifically.
Signed-off-by Bruce Richardson <[email protected]>Acked-by Morten Brørup <[email protected]>--- config/rte_config.h | 1 + lib/cryptodev/cryptodev_pmd.c | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-)config/rte_config.h +0
-0
config/rte_config.h line 74
Quoted 4 lines
config/rte_config.h +1
-0@@ -71,6 +71,7 @@/* cryptodev defines */ #define RTE_CRYPTO_MAX_DEVS 64 +#define RTE_CRYPTO_MAX_QPS_PER_DEV 256
Before this patch, the dummy_cb array could hold 1024 entries, so I wonder if this is enough / what the reason is to go to 256.
Additionnally, should we check rte_cryptodev_pmd_init_params->max_nb_queue_pairs in rte_cryptodev_pmd_create() ?
Quoted 21 lines
config/rte_config.h +0
-0#define RTE_CRYPTODEV_NAME_LEN 64#define RTE_CRYPTO_CALLBACKS 1lib/cryptodev/cryptodev_pmd.c +2
-3@@ -212,8 +212,8 @@ dummy_crypto_dequeue_burst(__rte_unused void *qp,void cryptodev_fp_ops_reset (struct rte_crypto_fp_ops *fp_ops) { - static struct rte_cryptodev_cb_rcu dummy_cb[RTE_MAX_QUEUES_PER_PORT]; - static void *dummy_data[RTE_MAX_QUEUES_PER_PORT]; +static struct rte_cryptodev_cb_rcu dummy_cb[RTE_CRYPTO_MAX_QPS_PER_DEV ]; +static void *dummy_data[RTE_CRYPTO_MAX_QPS_PER_DEV ];static const struct rte_crypto_fp_ops dummy = { .enqueue_burst = dummy_crypto_enqueue_burst, .dequeue_burst = dummy_crypto_dequeue_burst, --2.43.0
Signature · 1 line
David Marchand
PATCH v3 2/26 eventdev: remove use of ethev queues define
In reply to Bruce RichardsonThe max queues per port definition is intended for use for ethdevs, so it's use for a dummy array in eventdev is likely wrong. Since the array is intended to be for eventdev ports, use RTE_EVENT_MAX_PORTS_PER_DEV define instead.
--- lib/eventdev/eventdev_private.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
lib/eventdev/eventdev_private.c +1 -1
@@ -99,7 +99,7 @@ dummy_event_port_profile_switch(__rte_unused void *port, __rte_unused uint8_t prvoid event_dev_fp_ops_reset (struct rte_event_fp_ops *fp_op) { -static void *dummy_data[RTE_MAX_QUEUES_PER_PORT ]; +static void *dummy_data[RTE_EVENT_MAX_PORTS_PER_DEV ];static const struct rte_event_fp_ops dummy = { .enqueue = dummy_event_enqueue, .enqueue_burst = dummy_event_enqueue_burst,
Signature · 1 line
2.43.0
PATCH v3 3/26 app/test-bbdev: remove use of ethdev queue count value
In reply to Bruce RichardsonReplace the use of RTE_MAX_QUEUES_PER_PORT, which is intended as an ethdev define, with the limit from the bbdev library.
--- app/test-bbdev/test_bbdev.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
app/test-bbdev/test_bbdev.c +2 -2
@@ -297,7 +297,7 @@ test_bbdev_configure_stop_queue(void)"Failed test for rte_bbdev_queue_stop " "invalid dev_id "); -TEST_ASSERT_FAIL (rte_bbdev_queue_stop (dev_id,RTE_MAX_QUEUES_PER_PORT ), +TEST_ASSERT_FAIL (rte_bbdev_queue_stop (dev_id,RTE_BBDEV_DEFAULT_MAX_NB_QUEUES ),"Failed test for rte_bbdev_queue_stop " "invalid queue_id " );
@@ -1176,7 +1176,7 @@ test_bbdev_invalid_driver(void) "invalid dev_id ");TEST_ASSERT_FAIL (rte_bbdev_queue_info_get (dev_id, -RTE_MAX_QUEUES_PER_PORT , &qinfo), +RTE_BBDEV_DEFAULT_MAX_NB_QUEUES , &qinfo),"Failed test for rte_bbdev_info_get: " "invalid queue_id " );
Signature · 1 line
2.43.0
Re: [PATCH v3 03/26] app/test-bbdev: remove use of ethdev queue count value
Quoted 6 lines · Bruce Richardson
Replace the use of
RTE_MAX_QUEUES_PER_PORT, which is intended as an ethdev define, with the limit from the bbdev library.Signed-off-by Bruce Richardson <[email protected]>Acked-by Morten Brørup <[email protected]>
Quoted 9 lines
--- app/test-bbdev/test_bbdev.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)app/test-bbdev/test_bbdev.c +0
-0
app/test-bbdev/test_bbdev.c test_bbdev_configure_stop_queue(void) · line 300
Quoted 5 lines
app/test-bbdev/test_bbdev.c +1
-1@@ -297,7 +297,7 @@ test_bbdev_configure_stop_queue(void)"Failed test for rte_bbdev_queue_stop " "invalid dev_id "); - TEST_ASSERT_FAIL(rte_bbdev_queue_stop(dev_id, RTE_MAX_QUEUES_PER_PORT), + TEST_ASSERT_FAIL(rte_bbdev_queue_stop(dev_id, RTE_BBDEV_DEFAULT_MAX_NB_QUEUES),
According to lib/bbdev/rte_bbdev_pmd.h:
/** Suggested value for SW based devices */ #define RTE_BBDEV_DEFAULT_MAX_NB_QUEUES RTE_MAX_LCORE
So it is not clear to me that this is a safe maximum value.
I would instead pass a queue id which is >= devinfo.num_queues.
app/test-bbdev/test_bbdev.c test_bbdev_invalid_driver(void) · line 1179
Quoted 9 lines
app/test-bbdev/test_bbdev.c +1
-1"Failed test for rte_bbdev_queue_stop " "invalid queue_id ");@@ -1176,7 +1176,7 @@ test_bbdev_invalid_driver(void) "invalid dev_id "); TEST_ASSERT_FAIL(rte_bbdev_queue_info_get(dev_id, - RTE_MAX_QUEUES_PER_PORT, &qinfo), + RTE_BBDEV_DEFAULT_MAX_NB_QUEUES, &qinfo),
Idem.
Quoted 3 lines
app/test-bbdev/test_bbdev.c +0
-0"Failed test for rte_bbdev_info_get: " "invalid queue_id ");
Signature · 1 line
David Marchand
PATCH v3 4/26 config: add separate defines for max Rx and Tx queues
In reply to Bruce RichardsonRather than having a single define for maximum queues per ethernet port, we can set the max values for Rx queues and Tx queue independently. This allows future memory saving for apps which only need large numbers of Rx queues or only large numbers of Tx queues.
--- config/rte_config.h | 2 ++ doc/guides/rel_notes/release_24_11.rst | 6 ++++++ 2 files changed, 8 insertions(+)
config/rte_config.h +2 -0
@@ -65,6 +65,8 @@/* ether defines */ #define RTE_MAX_QUEUES_PER_PORT 1024 +#define RTE_MAX_ETHPORT_RX_QUEUES 1024 +#define RTE_MAX_ETHPORT_TX_QUEUES 1024#define RTE_ETHDEV_QUEUE_STAT_CNTRS 16/* max 256 */ #define RTE_ETHDEV_RXTX_CALLBACKS 1#define RTE_MAX_MULTI_HOST_CTRLS 4
doc/guides/rel_notes/release_24_11.rst +6 -0
@@ -55,6 +55,12 @@ New Features Also, make sure to start the actual text at the margin. ======================================================= +* **New configuration settings for Rx and Tx Ethernet Queues.** + + New defines have been added to DPDK to allow independent tracking of the maximum number of Rx and Tx queues. + These defines as ``RTE_MAX_ETHPORT_RX_QUEUES`` and ``RTE_MAX_ETHPORT_TX_QUEUES``, + and should be used in place of the older macro ``RTE_MAX_QUEUES_PER_PORT``. + Removed Items -------------
Signature · 1 line
2.43.0
Re: [PATCH v3 04/26] config: add separate defines for max Rx and Tx queues
Quoted 17 lines · Bruce Richardson
Rather than having a single define for maximum queues per ethernet port, we can set the max values for Rx queues and Tx queue independently. This allows future memory saving for apps which only need large numbers of Rx queues or only large numbers of Tx queues.
Signed-off-by Bruce Richardson <[email protected]>Acked-by Morten Brørup <[email protected]>--- config/rte_config.h | 2 ++ doc/guides/rel_notes/release_24_11.rst | 6 ++++++ 2 files changed, 8 insertions(+)config/rte_config.h +0
-0
config/rte_config.h line 69
Quoted 5 lines
config/rte_config.h +2
-0@@ -65,6 +65,8 @@/* ether defines */ #define RTE_MAX_QUEUES_PER_PORT 1024 +#define RTE_MAX_ETHPORT_RX_QUEUES 1024 +#define RTE_MAX_ETHPORT_TX_QUEUES 1024
The Rx Queues != Tx Queues is not a mainstream scenario (at least from most of DPDK user as I know), rename it (not separate Rx/Tx) with eth meaning and make it as a compile option is enough.
Quoted 21 lines
config/rte_config.h +0
-0#define RTE_ETHDEV_QUEUE_STAT_CNTRS 16/* max 256 */ #define RTE_ETHDEV_RXTX_CALLBACKS 1#define RTE_MAX_MULTI_HOST_CTRLS 4doc/guides/rel_notes/release_24_11.rst +6
-0@@ -55,6 +55,12 @@ New Features Also, make sure to start the actual text at the margin. ======================================================= +* **New configuration settings for Rx and Tx Ethernet Queues.** + + New defines have been added to DPDK to allow independent tracking of the maximum number of Rx and Tx queues. + These defines as ``RTE_MAX_ETHPORT_RX_QUEUES`` and ``RTE_MAX_ETHPORT_TX_QUEUES``, + and should be used in place of the older macro ``RTE_MAX_QUEUES_PER_PORT``. + Removed Items -------------
Re: [PATCH v3 04/26] config: add separate defines for max Rx and Tx queues
Quoted 19 lines · On Tue, 10 Sep 2024 10:54:37 +0800 fengchengwen <[email protected]> wrote:
Quoted 18 lines · Bruce Richardson
Rather than having a single define for maximum queues per ethernet port, we can set the max values for Rx queues and Tx queue independently. This allows future memory saving for apps which only need large numbers of Rx queues or only large numbers of Tx queues.
Signed-off-by Bruce Richardson <[email protected]>Acked-by Morten Brørup <[email protected]>--- config/rte_config.h | 2 ++ doc/guides/rel_notes/release_24_11.rst | 6 ++++++ 2 files changed, 8 insertions(+)config/rte_config.h +0
-0@@ -65,6 +65,8 @@
Quoted 7 lines
config/rte_config.h line 69
Quoted 4 lines
config/rte_config.h +2
-0/* ether defines */ #define RTE_MAX_QUEUES_PER_PORT 1024 +#define RTE_MAX_ETHPORT_RX_QUEUES 1024 +#define RTE_MAX_ETHPORT_TX_QUEUES 1024The Rx Queues != Tx Queues is not a mainstream scenario (at least from most of DPDK user as I know), rename it (not separate Rx/Tx) with eth meaning and make it as a compile option is enough.
Agree, allowing max Tx != Rx creates more test cases and other things. Lets not open up that can of worms.
Re: [PATCH v3 04/26] config: add separate defines for max Rx and Tx queues
Quoted 31 lines · Stephen Hemminger
Quoted 19 lines · On Tue, 10 Sep 2024 10:54:37 +0800 fengchengwen <[email protected]> wrote:
Quoted 18 lines · Bruce Richardson
Rather than having a single define for maximum queues per ethernet port, we can set the max values for Rx queues and Tx queue independently. This allows future memory saving for apps which only need large numbers of Rx queues or only large numbers of Tx queues.
Signed-off-by Bruce Richardson <[email protected]>Acked-by Morten Brørup <[email protected]>--- config/rte_config.h | 2 ++ doc/guides/rel_notes/release_24_11.rst | 6 ++++++ 2 files changed, 8 insertions(+)config/rte_config.h +0
-0@@ -65,6 +65,8 @@Quoted 7 lines
config/rte_config.h line 69
Quoted 4 lines
config/rte_config.h +2
-0/* ether defines */ #define RTE_MAX_QUEUES_PER_PORT 1024 +#define RTE_MAX_ETHPORT_RX_QUEUES 1024 +#define RTE_MAX_ETHPORT_TX_QUEUES 1024The Rx Queues != Tx Queues is not a mainstream scenario (at least from most of DPDK user as I know), rename it (not separate Rx/Tx) with eth meaning and make it as a compile option is enough.
Quoted 2 lines
Agree, allowing max Tx != Rx creates more test cases and other things. Lets not open up that can of worms.
I fail to see why it would be that problematic requiring additional test cases. I also think it's reasonable to give that level of control - thinking particularly of cases where one side may require thousands of queues e.g. the rte_tm cases with thousands of TX queues - you'll pay a large penalty for the other side's (RX) data-structures unnecessarily.
/Bruce
PATCH v3 5/26 ethdev: use separate Rx and Tx queue limits
In reply to Bruce RichardsonUpdate library to use the new defines RTE_MAX_ETHPORT_TX_QUEUES and RTE_MAX_ETHPORT_RX_QUEUES rather than the old define RTE_MAX_QUEUES_PER_PORT.
--- lib/ethdev/ethdev_driver.h | 8 ++++---- lib/ethdev/ethdev_private.c | 24 ++++++++++++++---------- lib/ethdev/rte_ethdev.c | 16 +++++++--------- lib/ethdev/rte_ethdev.h | 18 +++++++++--------- 4 files changed, 34 insertions(+), 32 deletions(-)
lib/ethdev/ethdev_driver.h +4 -4
@@ -84,12 +84,12 @@ struct __rte_cache_aligned rte_eth_dev {* User-supplied functions called from rx_burst to post-process * received packets before passing them to the user */ -RTE_ATOMIC (struct rte_eth_rxtx_callback *) post_rx_burst_cbs[RTE_MAX_QUEUES_PER_PORT ]; +RTE_ATOMIC (struct rte_eth_rxtx_callback *) post_rx_burst_cbs[RTE_MAX_ETHPORT_RX_QUEUES ];/** * User-supplied functions called from tx_burst to pre-process * received packets before passing them to the driver for transmission */ -RTE_ATOMIC (struct rte_eth_rxtx_callback *) pre_tx_burst_cbs[RTE_MAX_QUEUES_PER_PORT ]; +RTE_ATOMIC (struct rte_eth_rxtx_callback *) pre_tx_burst_cbs[RTE_MAX_ETHPORT_TX_QUEUES ];enum rte_eth_dev_state state;/**< Flag indicating the port state */ void *security_ctx;/**< Context for security ops */
@@ -165,9 +165,9 @@ struct __rte_cache_aligned rte_eth_dev_data { flow_configured :1 ;/** Queues state: HAIRPIN(2) / STARTED(1) / STOPPED(0) */ -uint8_t rx_queue_state[RTE_MAX_QUEUES_PER_PORT ]; +uint8_t rx_queue_state[RTE_MAX_ETHPORT_RX_QUEUES ];/** Queues state: HAIRPIN(2) / STARTED(1) / STOPPED(0) */ -uint8_t tx_queue_state[RTE_MAX_QUEUES_PER_PORT ]; +uint8_t tx_queue_state[RTE_MAX_ETHPORT_TX_QUEUES ];uint32_t dev_flags;/**< Capabilities */ int numa_node;/**< NUMA node connection */
lib/ethdev/ethdev_private.c +14 -10
@@ -190,7 +190,8 @@ struct dummy_queue {bool rx_warn_once;bool tx_warn_once; }; -static struct dummy_queue *dummy_queues_array[RTE_MAX_ETHPORTS ][RTE_MAX_QUEUES_PER_PORT ]; +static struct dummy_queue *dummy_rxq_array[RTE_MAX_ETHPORTS ][RTE_MAX_ETHPORT_RX_QUEUES ]; +static struct dummy_queue *dummy_txq_array[RTE_MAX_ETHPORTS ][RTE_MAX_ETHPORT_TX_QUEUES ];static struct dummy_queue per_port_queues[RTE_MAX_ETHPORTS ];RTE_INIT (dummy_queue_init) {
@@ -199,8 +200,10 @@ RTE_INIT(dummy_queue_init)for (port_id =0 ; port_id <RTE_DIM (per_port_queues); port_id++) {unsigned int q; -for (q =0 ; q <RTE_DIM (dummy_queues_array[port_id]); q++) - dummy_queues_array[port_id][q] = &per_port_queues[port_id]; +for (q =0 ; q <RTE_DIM (dummy_rxq_array[port_id]); q++) + dummy_rxq_array[port_id][q] = &per_port_queues[port_id]; +for (q =0 ; q <RTE_DIM (dummy_txq_array[port_id]); q++) + dummy_txq_array[port_id][q] = &per_port_queues[port_id]; } }
@@ -245,7 +248,8 @@ dummy_eth_tx_burst(void *txq,void eth_dev_fp_ops_reset (struct rte_eth_fp_ops *fpo) { -static RTE_ATOMIC (void *) dummy_data[RTE_MAX_QUEUES_PER_PORT ]; +static RTE_ATOMIC (void *) dummy_rx_data[RTE_MAX_ETHPORT_RX_QUEUES ]; +static RTE_ATOMIC (void *) dummy_tx_data[RTE_MAX_ETHPORT_TX_QUEUES ];uintptr_t port_id = fpo - rte_eth_fp_ops; per_port_queues[port_id].rx_warn_once = false;
@@ -254,12 +258,12 @@ eth_dev_fp_ops_reset(struct rte_eth_fp_ops *fpo) .rx_pkt_burst = dummy_eth_rx_burst, .tx_pkt_burst = dummy_eth_tx_burst, .rxq = { - .data = (void **)&dummy_queues_array[port_id], - .clbk = dummy_data, + .data = (void **)&dummy_rxq_array[port_id], + .clbk = dummy_rx_data, }, .txq = { - .data = (void **)&dummy_queues_array[port_id], - .clbk = dummy_data, + .data = (void **)&dummy_txq_array[port_id], + .clbk = dummy_tx_data, }, }; }
@@ -420,7 +424,7 @@ eth_dev_rx_queue_config(struct rte_eth_dev *dev, uint16_t nb_queues)if (dev->data->rx_queues ==NULL && nb_queues !=0 ) {/* first time configuration */ dev->data->rx_queues =rte_zmalloc ("ethdev->rx_queues" ,sizeof (dev->data->rx_queues[0 ]) * -RTE_MAX_QUEUES_PER_PORT , +RTE_MAX_ETHPORT_RX_QUEUES ,RTE_CACHE_LINE_SIZE );if (dev->data->rx_queues ==NULL ) { dev->data->nb_rx_queues =0 ;
@@ -450,7 +454,7 @@ eth_dev_tx_queue_config(struct rte_eth_dev *dev, uint16_t nb_queues)if (dev->data->tx_queues ==NULL && nb_queues !=0 ) {/* first time configuration */ dev->data->tx_queues =rte_zmalloc ("ethdev->tx_queues" ,sizeof (dev->data->tx_queues[0 ]) * -RTE_MAX_QUEUES_PER_PORT , +RTE_MAX_ETHPORT_TX_QUEUES ,RTE_CACHE_LINE_SIZE );if (dev->data->tx_queues ==NULL ) { dev->data->nb_tx_queues =0 ;
lib/ethdev/rte_ethdev.c +7 -9
@@ -1367,18 +1367,18 @@ rte_eth_dev_configure(uint16_t port_id, uint16_t nb_rx_q, uint16_t nb_tx_q, nb_tx_q =RTE_ETH_DEV_FALLBACK_TX_NBQUEUES ; } -if (nb_rx_q >RTE_MAX_QUEUES_PER_PORT ) { +if (nb_rx_q >RTE_MAX_ETHPORT_RX_QUEUES ) {RTE_ETHDEV_LOG_LINE (ERR ,"Number of Rx queues requested (%u) is greater than max supported(%d)" , - nb_rx_q,RTE_MAX_QUEUES_PER_PORT ); + nb_rx_q,RTE_MAX_ETHPORT_RX_QUEUES ); ret = -EINVAL ; goto rollback; } -if (nb_tx_q >RTE_MAX_QUEUES_PER_PORT ) { +if (nb_tx_q >RTE_MAX_ETHPORT_TX_QUEUES ) {RTE_ETHDEV_LOG_LINE (ERR ,"Number of Tx queues requested (%u) is greater than max supported(%d)" , - nb_tx_q,RTE_MAX_QUEUES_PER_PORT ); + nb_tx_q,RTE_MAX_ETHPORT_TX_QUEUES ); ret = -EINVAL ; goto rollback; }
@@ -3811,11 +3811,9 @@ rte_eth_dev_info_get(uint16_t port_id, struct rte_eth_dev_info *dev_info)return eth_err (port_id, diag); } -/* Maximum number of queues should be <= RTE_MAX_QUEUES_PER_PORT */ - dev_info->max_rx_queues =RTE_MIN (dev_info->max_rx_queues, -RTE_MAX_QUEUES_PER_PORT ); - dev_info->max_tx_queues =RTE_MIN (dev_info->max_tx_queues, -RTE_MAX_QUEUES_PER_PORT ); +/* Maximum number of queues should be <= RTE_MAX_ETHPORT_RX/TX_QUEUES */ + dev_info->max_rx_queues =RTE_MIN (dev_info->max_rx_queues,RTE_MAX_ETHPORT_RX_QUEUES ); + dev_info->max_tx_queues =RTE_MIN (dev_info->max_tx_queues,RTE_MAX_ETHPORT_TX_QUEUES ); dev_info->driver_name = dev->device->driver->name; dev_info->nb_rx_queues = dev->data->nb_rx_queues;
lib/ethdev/rte_ethdev.h +9 -9
@@ -6090,7 +6090,7 @@ rte_eth_rx_burst(uint16_t port_id, uint16_t queue_id,#ifdef RTE_ETHDEV_DEBUG_RX if (port_id >=RTE_MAX_ETHPORTS || - queue_id >=RTE_MAX_QUEUES_PER_PORT ) { + queue_id >=RTE_MAX_ETHPORT_RX_QUEUES ) {RTE_ETHDEV_LOG_LINE (ERR ,"Invalid port_id=%u or queue_id=%u" , port_id, queue_id);
@@ -6161,7 +6161,7 @@ rte_eth_rx_queue_count(uint16_t port_id, uint16_t queue_id)#ifdef RTE_ETHDEV_DEBUG_RX if (port_id >=RTE_MAX_ETHPORTS || - queue_id >=RTE_MAX_QUEUES_PER_PORT ) { + queue_id >=RTE_MAX_ETHPORT_RX_QUEUES ) {RTE_ETHDEV_LOG_LINE (ERR ,"Invalid port_id=%u or queue_id=%u" , port_id, queue_id);
@@ -6234,7 +6234,7 @@ rte_eth_rx_descriptor_status(uint16_t port_id, uint16_t queue_id,#ifdef RTE_ETHDEV_DEBUG_RX if (port_id >=RTE_MAX_ETHPORTS || - queue_id >=RTE_MAX_QUEUES_PER_PORT ) { + queue_id >=RTE_MAX_ETHPORT_RX_QUEUES ) {RTE_ETHDEV_LOG_LINE (ERR ,"Invalid port_id=%u or queue_id=%u" , port_id, queue_id);
@@ -6305,7 +6305,7 @@ static inline int rte_eth_tx_descriptor_status(uint16_t port_id,#ifdef RTE_ETHDEV_DEBUG_TX if (port_id >=RTE_MAX_ETHPORTS || - queue_id >=RTE_MAX_QUEUES_PER_PORT ) { + queue_id >=RTE_MAX_ETHPORT_TX_QUEUES ) {RTE_ETHDEV_LOG_LINE (ERR ,"Invalid port_id=%u or queue_id=%u" , port_id, queue_id);
@@ -6429,7 +6429,7 @@ rte_eth_tx_burst(uint16_t port_id, uint16_t queue_id,#ifdef RTE_ETHDEV_DEBUG_TX if (port_id >=RTE_MAX_ETHPORTS || - queue_id >=RTE_MAX_QUEUES_PER_PORT ) { + queue_id >=RTE_MAX_ETHPORT_TX_QUEUES ) {RTE_ETHDEV_LOG_LINE (ERR ,"Invalid port_id=%u or queue_id=%u" , port_id, queue_id);
@@ -6539,7 +6539,7 @@ rte_eth_tx_prepare(uint16_t port_id, uint16_t queue_id,#ifdef RTE_ETHDEV_DEBUG_TX if (port_id >=RTE_MAX_ETHPORTS || - queue_id >=RTE_MAX_QUEUES_PER_PORT ) { + queue_id >=RTE_MAX_ETHPORT_TX_QUEUES ) {RTE_ETHDEV_LOG_LINE (ERR ,"Invalid port_id=%u or queue_id=%u" , port_id, queue_id);
@@ -6744,7 +6744,7 @@ rte_eth_recycle_mbufs(uint16_t rx_port_id, uint16_t rx_queue_id,#ifdef RTE_ETHDEV_DEBUG_TX if (tx_port_id >=RTE_MAX_ETHPORTS || - tx_queue_id >=RTE_MAX_QUEUES_PER_PORT ) { + tx_queue_id >=RTE_MAX_ETHPORT_TX_QUEUES ) {RTE_ETHDEV_LOG_LINE (ERR ,"Invalid tx_port_id=%u or tx_queue_id=%u" , tx_port_id, tx_queue_id);
@@ -6770,7 +6770,7 @@ rte_eth_recycle_mbufs(uint16_t rx_port_id, uint16_t rx_queue_id,#ifdef RTE_ETHDEV_DEBUG_RX if (rx_port_id >=RTE_MAX_ETHPORTS || - rx_queue_id >=RTE_MAX_QUEUES_PER_PORT ) { + rx_queue_id >=RTE_MAX_ETHPORT_RX_QUEUES ) {RTE_ETHDEV_LOG_LINE (ERR ,"Invalid rx_port_id=%u or rx_queue_id=%u" , rx_port_id, rx_queue_id);return 0 ;
@@ -6890,7 +6890,7 @@ rte_eth_tx_queue_count(uint16_t port_id, uint16_t queue_id) goto out; } -if (queue_id >=RTE_MAX_QUEUES_PER_PORT ) { +if (queue_id >=RTE_MAX_ETHPORT_TX_QUEUES ) {RTE_ETHDEV_LOG_LINE (ERR ,"Invalid queue_id=%u for port_id=%u" , queue_id, port_id); rc = -EINVAL ;
Signature · 1 line
2.43.0
PATCH v3 6/26 bpf: use separate Rx and Tx queue limits
In reply to Bruce RichardsonUpdate library to use the new defines RTE_MAX_ETHPORT_TX_QUEUES and RTE_MAX_ETHPORT_RX_QUEUES rather than the old define RTE_MAX_QUEUES_PER_PORT.
--- lib/bpf/bpf_pkt.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
lib/bpf/bpf_pkt.c +2 -1
@@ -503,7 +503,8 @@ bpf_eth_elf_load(struct bpf_eth_cbh *cbh, uint16_t port, uint16_t queue, ftx =NULL ;if (prm ==NULL ||rte_eth_dev_is_valid_port (port) ==0 || - queue >=RTE_MAX_QUEUES_PER_PORT ) + queue >= (cbh->type ==BPF_ETH_RX ? +RTE_MAX_ETHPORT_RX_QUEUES :RTE_MAX_ETHPORT_TX_QUEUES ))return -EINVAL ;if (cbh->type ==BPF_ETH_RX )
Signature · 1 line
2.43.0
PATCH v3 7/26 latencystats: use separate Rx and Tx queue limits
In reply to Bruce RichardsonUpdate library to use the new defines RTE_MAX_ETHPORT_TX_QUEUES and RTE_MAX_ETHPORT_RX_QUEUES rather than the old define RTE_MAX_QUEUES_PER_PORT.
--- lib/latencystats/rte_latencystats.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
lib/latencystats/rte_latencystats.c +2 -2
@@ -60,8 +60,8 @@ struct rxtx_cbs {const struct rte_eth_rxtx_callback *cb; }; -static struct rxtx_cbs rx_cbs[RTE_MAX_ETHPORTS ][RTE_MAX_QUEUES_PER_PORT ]; -static struct rxtx_cbs tx_cbs[RTE_MAX_ETHPORTS ][RTE_MAX_QUEUES_PER_PORT ]; +static struct rxtx_cbs rx_cbs[RTE_MAX_ETHPORTS ][RTE_MAX_ETHPORT_RX_QUEUES ]; +static struct rxtx_cbs tx_cbs[RTE_MAX_ETHPORTS ][RTE_MAX_ETHPORT_TX_QUEUES ];struct latency_stats_nameoff {char name[RTE_ETH_XSTATS_NAME_SIZE ];
Signature · 1 line
2.43.0
PATCH v3 8/26 pdump: use separate Rx and Tx queue limits
In reply to Bruce RichardsonUpdate library to use the new defines RTE_MAX_ETHPORT_TX_QUEUES and RTE_MAX_ETHPORT_RX_QUEUES rather than the old define RTE_MAX_QUEUES_PER_PORT.
--- lib/pdump/rte_pdump.c | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-)
lib/pdump/rte_pdump.c +9 -9
@@ -61,8 +61,8 @@ static struct pdump_rxtx_cbs {const struct rte_bpf *filter;enum pdump_version ver;uint32_t snaplen; -} rx_cbs[RTE_MAX_ETHPORTS ][RTE_MAX_QUEUES_PER_PORT ], -tx_cbs[RTE_MAX_ETHPORTS ][RTE_MAX_QUEUES_PER_PORT ]; +} rx_cbs[RTE_MAX_ETHPORTS ][RTE_MAX_ETHPORT_RX_QUEUES ], +tx_cbs[RTE_MAX_ETHPORTS ][RTE_MAX_ETHPORT_TX_QUEUES ];/*
@@ -72,8 +72,8 @@ tx_cbs[RTE_MAX_ETHPORTS][RTE_MAX_QUEUES_PER_PORT];*/ static const char MZ_RTE_PDUMP_STATS [] ="rte_pdump_stats" ;static struct { -struct rte_pdump_stats rx[RTE_MAX_ETHPORTS ][RTE_MAX_QUEUES_PER_PORT ]; -struct rte_pdump_stats tx[RTE_MAX_ETHPORTS ][RTE_MAX_QUEUES_PER_PORT ]; +struct rte_pdump_stats rx[RTE_MAX_ETHPORTS ][RTE_MAX_ETHPORT_RX_QUEUES ]; +struct rte_pdump_stats tx[RTE_MAX_ETHPORTS ][RTE_MAX_ETHPORT_TX_QUEUES ];const struct rte_memzone *mz; } *pdump_stats;
@@ -708,8 +708,8 @@ rte_pdump_disable_by_deviceid(char *device_id, uint16_t queue, }static void -pdump_sum_stats (uint16_t port,uint16_t nq, -struct rte_pdump_stats stats[RTE_MAX_ETHPORTS ][RTE_MAX_QUEUES_PER_PORT ], +pdump_sum_stats (uint16_t nq, +struct rte_pdump_stats *stats,struct rte_pdump_stats *total) {uint64_t *sum = (uint64_t *)total;
@@ -718,7 +718,7 @@ pdump_sum_stats(uint16_t port, uint16_t nq,uint16_t qid;for (qid =0 ; qid < nq; qid++) { - const RTE_ATOMIC(uint64_t) *perq = (const uint64_t __rte_atomic *)&stats[port][qid]; + const RTE_ATOMIC(uint64_t) *perq = (const uint64_t __rte_atomic *)&stats[qid];for (i =0 ; i <sizeof (*total) /sizeof (uint64_t ); i++) { val =rte_atomic_load_explicit (&perq[i], rte_memory_order_relaxed);
@@ -762,7 +762,7 @@ rte_pdump_stats(uint16_t port, struct rte_pdump_stats *stats) pdump_stats = mz->addr; } -pdump_sum_stats (port, dev_info.nb_rx_queues, pdump_stats->rx, stats); -pdump_sum_stats (port, dev_info.nb_tx_queues, pdump_stats->tx, stats); +pdump_sum_stats (dev_info.nb_rx_queues, pdump_stats->rx[port], stats); +pdump_sum_stats (dev_info.nb_tx_queues, pdump_stats->tx[port], stats);return 0 ; }
Signature · 1 line
2.43.0
PATCH v3 9/26 power: use separate Rx and Tx queue limits
In reply to Bruce RichardsonUpdate library to use the new defines RTE_MAX_ETHPORT_TX_QUEUES and RTE_MAX_ETHPORT_RX_QUEUES rather than the old define RTE_MAX_QUEUES_PER_PORT.
--- lib/power/rte_power_pmd_mgmt.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
lib/power/rte_power_pmd_mgmt.c +2 -2
@@ -494,7 +494,7 @@ rte_power_ethdev_pmgmt_queue_enable(unsigned int lcore_id, uint16_t port_id,RTE_ETH_VALID_PORTID_OR_ERR_RET (port_id, -EINVAL ); -if (queue_id >=RTE_MAX_QUEUES_PER_PORT || lcore_id >=RTE_MAX_LCORE ) { +if (queue_id >=RTE_MAX_ETHPORT_RX_QUEUES || lcore_id >=RTE_MAX_LCORE ) { ret = -EINVAL ; goto end; }
@@ -608,7 +608,7 @@ rte_power_ethdev_pmgmt_queue_disable(unsigned int lcore_id,RTE_ETH_VALID_PORTID_OR_ERR_RET (port_id, -EINVAL ); -if (lcore_id >=RTE_MAX_LCORE || queue_id >=RTE_MAX_QUEUES_PER_PORT ) +if (lcore_id >=RTE_MAX_LCORE || queue_id >=RTE_MAX_ETHPORT_RX_QUEUES )return -EINVAL ;/* check if the queue is stopped */
Signature · 1 line
2.43.0
PATCH v3 10/26 net/af_xdp: use separate Rx and Tx queue limits
In reply to Bruce RichardsonUpdate driver to use the new defines RTE_MAX_ETHPORT_TX_QUEUES and RTE_MAX_ETHPORT_RX_QUEUES rather than the old define RTE_MAX_QUEUES_PER_PORT.
--- drivers/net/af_xdp/rte_eth_af_xdp.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/af_xdp/rte_eth_af_xdp.c +1 -1
@@ -185,7 +185,7 @@ struct pmd_internals { };struct pmd_process_private { -int rxq_xsk_fds[RTE_MAX_QUEUES_PER_PORT ]; +int rxq_xsk_fds[RTE_MAX_ETHPORT_RX_QUEUES ]; };#define ETH_AF_XDP_IFACE_ARG "iface"
Signature · 1 line
2.43.0
PATCH v3 11/26 net/cnxk: use separate Rx and Tx queue limits
In reply to Bruce RichardsonUpdate driver to use the new defines RTE_MAX_ETHPORT_TX_QUEUES and RTE_MAX_ETHPORT_RX_QUEUES rather than the old define RTE_MAX_QUEUES_PER_PORT.
--- drivers/net/cnxk/cnxk_ethdev_ops.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/net/cnxk/cnxk_ethdev_ops.c +2 -2
@@ -16,8 +16,8 @@ cnxk_nix_info_get(struct rte_eth_dev *eth_dev, struct rte_eth_dev_info *devinfo) devinfo->min_rx_bufsize =NIX_MIN_HW_FRS +RTE_ETHER_CRC_LEN ; devinfo->max_rx_pktlen = max_rx_pktlen; - devinfo->max_rx_queues =RTE_MAX_QUEUES_PER_PORT ; - devinfo->max_tx_queues =RTE_MAX_QUEUES_PER_PORT ; + devinfo->max_rx_queues =RTE_MAX_ETHPORT_RX_QUEUES ; + devinfo->max_tx_queues =RTE_MAX_ETHPORT_TX_QUEUES ; devinfo->max_mac_addrs = dev->max_mac_entries; devinfo->max_vfs = pci_dev->max_vfs; devinfo->max_mtu = devinfo->max_rx_pktlen -CNXK_NIX_L2_OVERHEAD ;
Signature · 1 line
2.43.0
PATCH v3 12/26 net/failsafe: use separate Rx and Tx queue limits
In reply to Bruce RichardsonUpdate driver to use the new defines RTE_MAX_ETHPORT_TX_QUEUES and RTE_MAX_ETHPORT_RX_QUEUES rather than the old define RTE_MAX_QUEUES_PER_PORT.
--- drivers/net/failsafe/failsafe_ops.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/net/failsafe/failsafe_ops.c +2 -2
@@ -1205,8 +1205,8 @@ fs_dev_infos_get(struct rte_eth_dev *dev, infos->min_mtu =RTE_ETHER_MIN_MTU ; infos->max_mtu =UINT16_MAX ; infos->max_rx_pktlen =UINT32_MAX ; - infos->max_rx_queues =RTE_MAX_QUEUES_PER_PORT ; - infos->max_tx_queues =RTE_MAX_QUEUES_PER_PORT ; + infos->max_rx_queues =RTE_MAX_ETHPORT_RX_QUEUES ; + infos->max_tx_queues =RTE_MAX_ETHPORT_TX_QUEUES ; infos->max_mac_addrs =FAILSAFE_MAX_ETHADDR ; infos->max_hash_mac_addrs =UINT32_MAX ; infos->max_vfs =UINT16_MAX ;
Signature · 1 line
2.43.0
PATCH v3 13/26 net/hns3: use separate Rx and Tx queue limits
In reply to Bruce RichardsonUpdate driver to use the new defines RTE_MAX_ETHPORT_TX_QUEUES and RTE_MAX_ETHPORT_RX_QUEUES rather than the old define RTE_MAX_QUEUES_PER_PORT.
--- drivers/net/hns3/hns3_tm.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/hns3/hns3_tm.c +1 -1
@@ -21,7 +21,7 @@ hns3_tm_max_tx_queues_get(struct rte_eth_dev *dev)memset (&dev_info,0 ,sizeof (dev_info)); (void )hns3_dev_infos_get (dev, &dev_info); -return RTE_MIN (dev_info.max_tx_queues,RTE_MAX_QUEUES_PER_PORT ); +return RTE_MIN (dev_info.max_tx_queues,RTE_MAX_ETHPORT_TX_QUEUES ); }void
Signature · 1 line
2.43.0
PATCH v3 14/26 net/mlx5: use separate Rx and Tx queue limits
In reply to Bruce RichardsonUpdate driver to use the new defines RTE_MAX_ETHPORT_TX_QUEUES and RTE_MAX_ETHPORT_RX_QUEUES rather than the old define RTE_MAX_QUEUES_PER_PORT.
--- drivers/net/mlx5/mlx5_flow.c | 2 +- drivers/net/mlx5/mlx5_flow_hw.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
drivers/net/mlx5/mlx5_flow.c +1 -1
@@ -7344,7 +7344,7 @@ flow_legacy_list_create(struct rte_eth_dev *dev, enum mlx5_flow_type type,struct rte_flow_item items[MLX5_MAX_SPLIT_ITEMS ];uint8_t buffer[2048 ]; } items_tx; -struct mlx5_rte_flow_item_sq sq_specs[RTE_MAX_QUEUES_PER_PORT ]; +struct mlx5_rte_flow_item_sq sq_specs[RTE_MAX_ETHPORT_RX_QUEUES ];struct mlx5_flow_expand_rss *buf = &expand_buffer.buf;struct mlx5_flow_rss_desc *rss_desc;const struct rte_flow_action *p_actions_rx;
drivers/net/mlx5/mlx5_flow_hw.c +1 -1
@@ -10978,7 +10978,7 @@ flow_hw_create_ctrl_rx_rss_template(struct rte_eth_dev *dev,struct rte_flow_actions_template_attr attr = { .ingress =1 , }; -uint16_t queue[RTE_MAX_QUEUES_PER_PORT ]; +uint16_t queue[RTE_MAX_ETHPORT_RX_QUEUES ];struct rte_flow_action_rss rss_conf = { .func =RTE_ETH_HASH_FUNCTION_DEFAULT , .level =0 ,
Signature · 1 line
2.43.0
PATCH v3 15/26 net/null: use separate Rx and Tx queue limits
In reply to Bruce RichardsonUpdate driver to use the new defines RTE_MAX_ETHPORT_TX_QUEUES and RTE_MAX_ETHPORT_RX_QUEUES rather than the old define RTE_MAX_QUEUES_PER_PORT.
--- drivers/net/null/rte_eth_null.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/net/null/rte_eth_null.c +2 -2
@@ -53,8 +53,8 @@ struct pmd_internals {unsigned int no_rx;uint16_t port_id; -struct null_queue rx_null_queues[RTE_MAX_QUEUES_PER_PORT ]; -struct null_queue tx_null_queues[RTE_MAX_QUEUES_PER_PORT ]; +struct null_queue rx_null_queues[RTE_MAX_ETHPORT_RX_QUEUES ]; +struct null_queue tx_null_queues[RTE_MAX_ETHPORT_TX_QUEUES ];struct rte_ether_addr eth_addr;/** Bit mask of RSS offloads, the bit offset also means flow type */
Signature · 1 line
2.43.0
PATCH v3 16/26 net/sfc: use separate Rx and Tx queue limits
In reply to Bruce RichardsonUpdate driver to use the new defines RTE_MAX_ETHPORT_TX_QUEUES and RTE_MAX_ETHPORT_RX_QUEUES rather than the old define RTE_MAX_QUEUES_PER_PORT.
--- drivers/net/sfc/sfc_sw_stats.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
drivers/net/sfc/sfc_sw_stats.c +4 -2
@@ -825,9 +825,11 @@ sfc_sw_xstats_alloc_queues_bitmap(struct sfc_adapter *sa)struct rte_bitmap **queues_bitmap = &sa->sw_stats.queues_bitmap;void **queues_bitmap_mem = &sa->sw_stats.queues_bitmap_mem;uint32_t bmp_size; +uint32_t max_queues =RTE_MAX (RTE_MAX_ETHPORT_RX_QUEUES , +RTE_MAX_ETHPORT_TX_QUEUES );int rc; - bmp_size =rte_bitmap_get_memory_footprint (RTE_MAX_QUEUES_PER_PORT ); + bmp_size =rte_bitmap_get_memory_footprint (max_queues); *queues_bitmap_mem =NULL ; *queues_bitmap =NULL ;
@@ -836,7 +838,7 @@ sfc_sw_xstats_alloc_queues_bitmap(struct sfc_adapter *sa)if (*queues_bitmap_mem ==NULL )return ENOMEM ; - *queues_bitmap =rte_bitmap_init (RTE_MAX_QUEUES_PER_PORT , + *queues_bitmap =rte_bitmap_init (max_queues, *queues_bitmap_mem, bmp_size);if (*queues_bitmap ==NULL ) { rc =EINVAL ;
Signature · 1 line
2.43.0
PATCH v3 17/26 net/thunderx: use separate Rx and Tx queue limits
In reply to Bruce RichardsonUpdate driver to use the new defines RTE_MAX_ETHPORT_TX_QUEUES and RTE_MAX_ETHPORT_RX_QUEUES rather than the old define RTE_MAX_QUEUES_PER_PORT.
--- drivers/net/thunderx/nicvf_ethdev.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/thunderx/nicvf_ethdev.c +1 -1
@@ -907,7 +907,7 @@ nicvf_configure_rss_reta(struct rte_eth_dev *dev) {struct nicvf *nic =nicvf_pmd_priv (dev);unsigned int idx, qmap_size; -uint8_t qmap[RTE_MAX_QUEUES_PER_PORT ]; +uint8_t qmap[RTE_MAX_ETHPORT_RX_QUEUES ];uint8_t default_reta[NIC_MAX_RSS_IDR_TBL_SIZE ];if (nic->cpi_alg !=CPI_ALG_NONE )
Signature · 1 line
2.43.0
PATCH v3 18/26 net/vhost: use separate Rx and Tx queue limits
In reply to Bruce RichardsonUpdate driver to use the new defines RTE_MAX_ETHPORT_TX_QUEUES and RTE_MAX_ETHPORT_RX_QUEUES rather than the old define RTE_MAX_QUEUES_PER_PORT.
--- drivers/net/vhost/rte_eth_vhost.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
drivers/net/vhost/rte_eth_vhost.c +4 -3
@@ -117,8 +117,8 @@ static struct rte_eth_link pmd_link = {struct rte_vhost_vring_state {rte_spinlock_t lock; -bool cur[RTE_MAX_QUEUES_PER_PORT *2 ]; -bool seen[RTE_MAX_QUEUES_PER_PORT *2 ]; +bool cur[RTE_MAX_ETHPORT_RX_QUEUES +RTE_MAX_ETHPORT_TX_QUEUES ]; +bool seen[RTE_MAX_ETHPORT_RX_QUEUES +RTE_MAX_ETHPORT_TX_QUEUES ];unsigned int index;unsigned int max_vring; };
@@ -1648,7 +1648,8 @@ rte_pmd_vhost_probe(struct rte_vdev_device *dev)if (rte_kvargs_count (kvlist,ETH_VHOST_QUEUES_ARG ) ==1 ) { ret =rte_kvargs_process (kvlist,ETH_VHOST_QUEUES_ARG , &open_int, &queues); -if (ret <0 || queues >RTE_MAX_QUEUES_PER_PORT ) +if (ret <0 || queues >RTE_MAX_ETHPORT_RX_QUEUES + || queues >RTE_MAX_ETHPORT_TX_QUEUES ) goto out_free; }else
Signature · 1 line
2.43.0
PATCH v3 19/26 app/dumpcap: use separate Rx and Tx queue limits
In reply to Bruce RichardsonUpdate app to use the new defines RTE_MAX_ETHPORT_TX_QUEUES and RTE_MAX_ETHPORT_RX_QUEUES rather than the old define RTE_MAX_QUEUES_PER_PORT.
--- app/dumpcap/main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
app/dumpcap/main.c +1 -1
@@ -95,7 +95,7 @@ struct interface {struct rte_bpf_prm *bpf_prm;char name[RTE_ETH_NAME_MAX_LEN ]; -struct rte_rxtx_callback *rx_cb[RTE_MAX_QUEUES_PER_PORT ]; +struct rte_rxtx_callback *rx_cb[RTE_MAX_ETHPORT_RX_QUEUES ];const char *ifname;const char *ifdescr; };
Signature · 1 line
2.43.0
Re: [PATCH v3 19/26] app/dumpcap: use separate Rx and Tx queue limits
Quoted 15 lines · On Wed, 14 Aug 2024 11:49:25 +0100 Bruce Richardson…
Update app to use the new defines
RTE_MAX_ETHPORT_TX_QUEUESandRTE_MAX_ETHPORT_RX_QUEUESrather than the old defineRTE_MAX_QUEUES_PER_PORT.Signed-off-by Bruce Richardson <[email protected]>Acked-by Morten Brørup <[email protected]>--- app/dumpcap/main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)app/dumpcap/main.c +0
-0
app/dumpcap/main.c struct interface { · line 101
Quoted 8 lines
app/dumpcap/main.c +1
-1@@ -95,7 +95,7 @@ struct interface {struct rte_bpf_prm *bpf_prm;char name[RTE_ETH_NAME_MAX_LEN ]; -struct rte_rxtx_callback *rx_cb[RTE_MAX_QUEUES_PER_PORT ]; +struct rte_rxtx_callback *rx_cb[RTE_MAX_ETHPORT_RX_QUEUES ];const char *ifname;const char *ifdescr; };
Looking closer, this field is defined and never used, it can go.
That structure is used to gather the command line arguments before applying them.
PATCH v3 20/26 app/test-pmd: use separate Rx and Tx queue limits
In reply to Bruce RichardsonUpdate app to use the new defines RTE_MAX_ETHPORT_TX_QUEUES and RTE_MAX_ETHPORT_RX_QUEUES rather than the old define RTE_MAX_QUEUES_PER_PORT.
--- app/test-pmd/testpmd.c | 7 ++++--- app/test-pmd/testpmd.h | 16 ++++++++-------- 2 files changed, 12 insertions(+), 11 deletions(-)
app/test-pmd/testpmd.c +4 -3
@@ -1305,7 +1305,7 @@ check_socket_id(const unsigned int socket_id)queueid_t get_allowed_max_nb_rxq (portid_t *pid) { -queueid_t allowed_max_rxq =RTE_MAX_QUEUES_PER_PORT ; +queueid_t allowed_max_rxq =RTE_MAX_ETHPORT_RX_QUEUES ;bool max_rxq_valid = false;portid_t pi;struct rte_eth_dev_info dev_info;
@@ -1353,7 +1353,7 @@ check_nb_rxq(queueid_t rxq)queueid_t get_allowed_max_nb_txq (portid_t *pid) { -queueid_t allowed_max_txq =RTE_MAX_QUEUES_PER_PORT ; +queueid_t allowed_max_txq =RTE_MAX_ETHPORT_TX_QUEUES ;bool max_txq_valid = false;portid_t pi;struct rte_eth_dev_info dev_info;
@@ -1564,7 +1564,8 @@ check_nb_txd(queueid_t txd)queueid_t get_allowed_max_nb_hairpinq (portid_t *pid) { -queueid_t allowed_max_hairpinq =RTE_MAX_QUEUES_PER_PORT ; +queueid_t allowed_max_hairpinq =RTE_MIN (RTE_MAX_ETHPORT_RX_QUEUES , +RTE_MAX_ETHPORT_TX_QUEUES );portid_t pi;struct rte_eth_hairpin_cap cap;
app/test-pmd/testpmd.h +8 -8
@@ -332,10 +332,10 @@ struct rte_port {uint8_t need_reconfig_queues;/**< need reconfiguring queues or not */ uint8_t rss_flag;/**< enable rss or not */ uint8_t dcb_flag;/**< enable dcb */ -uint16_t nb_rx_desc[RTE_MAX_QUEUES_PER_PORT +1 ];/**< per queue rx desc number */ -uint16_t nb_tx_desc[RTE_MAX_QUEUES_PER_PORT +1 ];/**< per queue tx desc number */ -struct port_rxqueue rxq[RTE_MAX_QUEUES_PER_PORT +1 ];/**< per queue Rx config and state */ -struct port_txqueue txq[RTE_MAX_QUEUES_PER_PORT +1 ];/**< per queue Tx config and state */ +uint16_t nb_rx_desc[RTE_MAX_ETHPORT_RX_QUEUES +1 ];/**< per queue rx desc number */ +uint16_t nb_tx_desc[RTE_MAX_ETHPORT_TX_QUEUES +1 ];/**< per queue tx desc number */ +struct port_rxqueue rxq[RTE_MAX_ETHPORT_RX_QUEUES +1 ];/**< per queue Rx config and state */ +struct port_txqueue txq[RTE_MAX_ETHPORT_TX_QUEUES +1 ];/**< per queue Tx config and state */ struct rte_ether_addr *mc_addr_pool;/**< pool of multicast addrs */ uint32_t mc_addr_nb;/**< nb. of addr. in mc_addr_pool */ queueid_t queue_nb;/**< nb. of queues for flow rules */
@@ -351,14 +351,14 @@ struct rte_port {struct port_indirect_action *actions_list;/**< Associated indirect actions. */ LIST_HEAD(, port_flow_tunnel) flow_tunnel_list; -const struct rte_eth_rxtx_callback *rx_dump_cb[RTE_MAX_QUEUES_PER_PORT +1 ]; -const struct rte_eth_rxtx_callback *tx_dump_cb[RTE_MAX_QUEUES_PER_PORT +1 ]; +const struct rte_eth_rxtx_callback *rx_dump_cb[RTE_MAX_ETHPORT_RX_QUEUES +1 ]; +const struct rte_eth_rxtx_callback *tx_dump_cb[RTE_MAX_ETHPORT_TX_QUEUES +1 ];/**< metadata value to insert in Tx packets. */ uint32_t tx_metadata; -const struct rte_eth_rxtx_callback *tx_set_md_cb[RTE_MAX_QUEUES_PER_PORT +1 ]; +const struct rte_eth_rxtx_callback *tx_set_md_cb[RTE_MAX_ETHPORT_TX_QUEUES +1 ];/**< dynamic flags. */ uint64_t mbuf_dynf; -const struct rte_eth_rxtx_callback *tx_set_dynf_cb[RTE_MAX_QUEUES_PER_PORT +1 ]; +const struct rte_eth_rxtx_callback *tx_set_dynf_cb[RTE_MAX_ETHPORT_TX_QUEUES +1 ];struct xstat_display_info xstats_info; };
Signature · 1 line
2.43.0
PATCH v3 21/26 examples/ipsec-secgw: use separate Rx and Tx queue limits
In reply to Bruce RichardsonUpdate example app to use the new defines RTE_MAX_ETHPORT_TX_QUEUES and RTE_MAX_ETHPORT_RX_QUEUES rather than the old define RTE_MAX_QUEUES_PER_PORT.
--- examples/ipsec-secgw/ipsec-secgw.c | 2 +- examples/ipsec-secgw/ipsec.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
examples/ipsec-secgw/ipsec-secgw.c +1 -1
@@ -1050,7 +1050,7 @@ parse_config(const char *q_arg)uint32_t size;uint32_t max_fld[_NUM_FLD] = {RTE_MAX_ETHPORTS , -RTE_MAX_QUEUES_PER_PORT , +RTE_MAX_ETHPORT_RX_QUEUES ,RTE_MAX_LCORE };
examples/ipsec-secgw/ipsec.c +1 -1
@@ -584,7 +584,7 @@ create_inline_session(struct socket_ctx *skt_ctx, struct ipsec_sa *sa, .rss_key_len = sizeof(rss_key), };struct rte_eth_dev_info dev_info; -uint16_t queue[RTE_MAX_QUEUES_PER_PORT ]; +uint16_t queue[RTE_MAX_ETHPORT_RX_QUEUES ];struct rte_flow_action_rss action_rss;unsigned int i;unsigned int j;
Signature · 1 line
2.43.0
PATCH v3 22/26 examples/l3fwd-power: use separate Rx and Tx queue limits
In reply to Bruce RichardsonUpdate example app to use the new defines RTE_MAX_ETHPORT_TX_QUEUES and RTE_MAX_ETHPORT_RX_QUEUES rather than the old define RTE_MAX_QUEUES_PER_PORT.
--- examples/l3fwd-power/main.c | 2 +- examples/l3fwd-power/perf_core.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
examples/l3fwd-power/main.c +1 -1
@@ -1660,7 +1660,7 @@ parse_config(const char *q_arg)unsigned size;unsigned int max_fld[_NUM_FLD] = {RTE_MAX_ETHPORTS , -RTE_MAX_QUEUES_PER_PORT , +RTE_MAX_ETHPORT_RX_QUEUES ,RTE_MAX_LCORE };
examples/l3fwd-power/perf_core.c +1 -1
@@ -134,7 +134,7 @@ parse_perf_config(const char *q_arg)unsigned int size;unsigned int max_fld[_NUM_FLD] = {RTE_MAX_ETHPORTS , -RTE_MAX_QUEUES_PER_PORT , +RTE_MAX_ETHPORT_RX_QUEUES ,255 ,RTE_MAX_LCORE };
Signature · 1 line
2.43.0
PATCH v3 23/26 examples/l3fwd: use separate Rx and Tx queue limits
In reply to Bruce RichardsonUpdate example app to use the new defines RTE_MAX_ETHPORT_TX_QUEUES and RTE_MAX_ETHPORT_RX_QUEUES rather than the old define RTE_MAX_QUEUES_PER_PORT.
--- examples/l3fwd/main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
examples/l3fwd/main.c +1 -1
@@ -502,7 +502,7 @@ parse_config(const char *q_arg)unsigned size;uint16_t max_fld[_NUM_FLD] = {RTE_MAX_ETHPORTS , -RTE_MAX_QUEUES_PER_PORT , +RTE_MAX_ETHPORT_RX_QUEUES ,RTE_MAX_LCORE };
Signature · 1 line
2.43.0
PATCH v3 24/26 examples/vhost: use separate Rx and Tx queue limits
In reply to Bruce RichardsonUpdate example app to use the new defines RTE_MAX_ETHPORT_TX_QUEUES and RTE_MAX_ETHPORT_RX_QUEUES rather than the old define RTE_MAX_QUEUES_PER_PORT.
--- examples/vhost/main.c | 2 +- examples/vhost/main.h | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
examples/vhost/main.c +1 -1
@@ -1891,7 +1891,7 @@ reset_dma(void)for (i =0 ; i <RTE_MAX_VHOST_DEVICE ; i++) {int j; -for (j =0 ; j <RTE_MAX_QUEUES_PER_PORT *2 ; j++) { +for (j =0 ; j <RTE_MAX_ETHPORT_RX_QUEUES +RTE_MAX_ETHPORT_TX_QUEUES ; j++) { dma_bind[i].dmas[j].dev_id =INVALID_DMA_ID ; dma_bind[i].dmas[j].async_enabled = false; }
examples/vhost/main.h +1 -1
@@ -99,7 +99,7 @@ struct dma_info { };struct dma_for_vhost { -struct dma_info dmas[RTE_MAX_QUEUES_PER_PORT *2 ]; +struct dma_info dmas[RTE_MAX_ETHPORT_RX_QUEUES +RTE_MAX_ETHPORT_TX_QUEUES ];uint32_t async_flag; };
Signature · 1 line
2.43.0
PATCH v3 25/26 config: make queues per port a meson config option
In reply to Bruce RichardsonThe default number of ethernet queues per port is currently set to 1k which is more than enough for most applications, but still is lower than the total number of queues which may be available on modern NICs. Rather than increasing the max queues further, which will increase the memory footprint (since the value is used in array dimensioning), we can instead make the value a meson tunable option - and reduce the default value to 256 in the process. This means that:
* most apps which don't need hundreds of queues will see lower mem use.
* apps which do need to use thousands of queues can configure DPDK to allow this, without having to modify DPDK files (i.e. rte_config.h)
--- config/meson.build | 2 ++ config/rte_config.h | 3 --- doc/guides/rel_notes/release_24_11.rst | 13 +++++++++++-- meson_options.txt | 4 ++++ 4 files changed, 17 insertions(+), 5 deletions(-)
config/meson.build +2 -0
@@ -352,6 +352,8 @@ endforeach # set other values pulled from the build options dpdk_conf.set('RTE_MAX_ETHPORTS', get_option('max_ethports')) +dpdk_conf.set('RTE_MAX_ETHPORT_RX_QUEUES', get_option('max_ethport_rx_queues')) +dpdk_conf.set('RTE_MAX_ETHPORT_TX_QUEUES', get_option('max_ethport_tx_queues')) dpdk_conf.set('RTE_LIBEAL_USE_HPET', get_option('use_hpet')) dpdk_conf.set('RTE_ENABLE_STDATOMIC', get_option('enable_stdatomic')) dpdk_conf.set('RTE_ENABLE_TRACE_FP', get_option('enable_trace_fp'))
config/rte_config.h +0 -3
@@ -64,9 +64,6 @@#define RTE_MBUF_DEFAULT_MEMPOOL_OPS "ring_mp_mc"/* ether defines */ -#define RTE_MAX_QUEUES_PER_PORT 1024 -#define RTE_MAX_ETHPORT_RX_QUEUES 1024 -#define RTE_MAX_ETHPORT_TX_QUEUES 1024#define RTE_ETHDEV_QUEUE_STAT_CNTRS 16/* max 256 */ #define RTE_ETHDEV_RXTX_CALLBACKS 1#define RTE_MAX_MULTI_HOST_CTRLS 4
doc/guides/rel_notes/release_24_11.rst +11 -2
@@ -58,8 +58,17 @@ New Features * **New configuration settings for Rx and Tx Ethernet Queues.** New defines have been added to DPDK to allow independent tracking of the maximum number of Rx and Tx queues. - These defines as ``RTE_MAX_ETHPORT_RX_QUEUES`` and ``RTE_MAX_ETHPORT_TX_QUEUES``, - and should be used in place of the older macro ``RTE_MAX_QUEUES_PER_PORT``. + These defines are ``RTE_MAX_ETHPORT_RX_QUEUES`` and ``RTE_MAX_ETHPORT_TX_QUEUES``, + and these new defines should be used in place of the older define ``RTE_MAX_QUEUES_PER_PORT``. + Unlike the older define, these new defines are build-time configurable via meson options: + ``max_ethport_rx_queues`` and ``max_ethport_tx_queues``. + +.. note:: + + The default max values for Rx and Tx queue limits are reduced from 1024, in previous releases, + to 256 in this release. + For application that require large numbers of queues, + these defaults can be changed via the meson configuration options described above. Removed Items
meson_options.txt +4 -0
@@ -40,6 +40,10 @@ option('machine', type: 'string', value: 'auto', description: 'Alias of cpu_instruction_set.') option('max_ethports', type: 'integer', value: 32, description: 'maximum number of Ethernet devices') +option('max_ethport_rx_queues', type: 'integer', value: 256, description: + 'maximum number of Rx queues on an Ethernet device') +option('max_ethport_tx_queues', type: 'integer', value: 256, description: + 'maximum number of Tx queues on an Ethernet device') option('max_lcores', type: 'string', value: 'default', description: 'Set maximum number of cores/threads supported by EAL; "default" is different per-arch, "detect" detects the number of cores on the build machine.') option('max_numa_nodes', type: 'string', value: 'default', description:
Signature · 1 line
2.43.0
PATCH v3 26/26 config: add computed max queues define for compatibility
In reply to Bruce RichardsonEnd applications may use the RTE_MAX_QUEUES_PER_PORT define in their structure definitions, so keep a define present in DPDK for backward compatibility. Rather than having a hard-coded value, we can use the maximum of the Rx and Tx values as the overall max value. Rather than using a macro which does the MAX() calculation inside it, we can compute the actual value at configuration time and write it using meson.
--- config/meson.build | 8 ++++++++ doc/guides/rel_notes/deprecation.rst | 11 +++++++++++ doc/guides/rel_notes/release_24_11.rst | 8 +++++++- 3 files changed, 26 insertions(+), 1 deletion(-)
config/meson.build +8 -0
@@ -372,6 +372,14 @@ if get_option('mbuf_refcnt_atomic') endif dpdk_conf.set10('RTE_IOVA_IN_MBUF', get_option('enable_iova_as_pa')) +# set old MAX_QUEUES_PER_PORT option for compatibility. Compute +# value as max of Rx and Tx counts +if get_option('max_ethport_rx_queues') > get_option('max_ethport_tx_queues') + dpdk_conf.set('RTE_MAX_QUEUES_PER_PORT', get_option('max_ethport_rx_queues')) +else + dpdk_conf.set('RTE_MAX_QUEUES_PER_PORT', get_option('max_ethport_tx_queues')) +endif + compile_time_cpuflags = [] subdir(arch_subdir) dpdk_conf.set('RTE_COMPILE_TIME_CPUFLAGS', ','.join(compile_time_cpuflags))
doc/guides/rel_notes/deprecation.rst +11 -0
@@ -17,6 +17,17 @@ Other API and ABI deprecation notices are to be posted below. Deprecation Notices ------------------- +* config: The define ``RTE_MAX_QUEUES_PER_PORT`` should be considered deprecated + and may be removed in a future release. + Its use in apps should be replaced by ``RTE_MAX_ETHPORT_RX_QUEUES`` or ``RTE_MAX_ETHPORT_TX_QUEUES``, + as appropriate. + +* config: The ``RTE_MAX_QUEUES_PER_PORT`` value is no longer hard-coded to 1024. + Its value is now computed at configuration time to be the maximum of the configured max Rx and Tx queue values, + given by the meson options ``max_ethport_rx_queues`` and ``max_ethport_tx_queues``. + If these are unmodified from the defaults, + the value of ``RTE_MAX_QUEUES_PER_PORT`` will be 256. + * build: The ``enable_kmods`` option is deprecated and will be removed in a future release. Setting/clearing the option has no impact on the build. Instead, kernel modules will be always built for OS's where out-of-tree kernel modules
doc/guides/rel_notes/release_24_11.rst +7 -1
@@ -67,9 +67,15 @@ New Features The default max values for Rx and Tx queue limits are reduced from 1024, in previous releases, to 256 in this release. - For application that require large numbers of queues, + For applications that require large numbers of queues, these defaults can be changed via the meson configuration options described above. +.. note:: + + The define ``RTE_MAX_QUEUES_PER_PORT`` is kept for backward compatibility. + Its value is no longer hard-coded, + but is set, at configuration time, to the maximum of the configured max Rx and Tx queue values. + Removed Items -------------
Signature · 1 line
2.43.0
Re: [PATCH v3 00/26] add meson config options for queues per port
In reply to Bruce RichardsonQuoted 111 lines · Bruce Richardson
There are a number of issues with the current
RTE_MAX_QUEUES_PER_PORTsetting in DPDK that are addressed by this patchset:* The name does not make it clear that this is intended as an ethdev-only setting
* A number of other libraries are using this define rather than having more relevant defines for the particular usecase.
* The define is hard-coded in DPDK source code and is not adjustable via a build-time/meson option
* Because of the lack of configurability, the max is therefore set to a conservatively-high value, wasting memory.
* There is an assumption that the number of Rx queues and Tx queues should have the same maximum value. Depending on application, it may be desirable to have fan-in with multiple Rx queues e.g. for classification/filtering, feed a single Tx queue, or the opposite where, e.g. for QoS Tx scheduling, a few Rx queues feeds a very large number of Tx queues.This patchset therefore addresses these by:
* replacing the single define for max queues with independent defines for Rx and Tx queues.
* adjusts the name to ensure that it is clear the defines are for ethports only. [ethports being used in theRTE_MAX_ETHPORTSsetting].
* replaces occurances ofRTE_MAX_QUEUES_PER_PORTwith appropriate defines for non-ethdev use cases
* replaces all other internal occurances of the define with the new per-Rx and per-Tx definitions.
* adds meson config options to allow build-time configuration of the max Rx and Tx queue values.Naming Note:
* The new meson config options are called "max_ethport_rx_queues" and "max_ethport_tx_queues" so that in the meson options list they appear alphabetically beside the existing "max_ethports" option.
* For naming consistency, the new C defines are thereforeRTE_MAX_ETHPORT_RX_QUEUESandRTE_MAX_ETHPORT_TX_QUEUES.V3:
* Resend of v2 with correct author email, to avoid reply bounces
* drop "rfc" prefix from patchesV2:
* What was a single patch with "3 insertions(+), 1 deletion(-)" has now become a 26-patch set! :-)
* Created separate Rx and Tx defines
* Ensured that the name makes it clear that the define is for ethdev
* When updating internal use, created one patch per component for easier maintainer review. In most cases it was obvious whether Rx or Tx define should be used, but a few cases were less clear.
* Added documentation updates for the changes (release notes and deprecation notice), spread across 3 of the patches.Bruce Richardson (26):
cryptodev: remove use of ethdev max queues definition
eventdev: remove use of ethev queues define app/test-bbdev: remove use of ethdev queue count value
config: add separate defines for max Rx and Tx queues
ethdev: use separate Rx and Tx queue limits
bpf: use separate Rx and Tx queue limits
latencystats: use separate Rx and Tx queue limits
pdump: use separate Rx and Tx queue limits
power: use separate Rx and Tx queue limitsnet/af_xdp: use separate Rx and Tx queue limits net/cnxk: use separate Rx and Tx queue limits net/failsafe: use separate Rx and Tx queue limits net/hns3: use separate Rx and Tx queue limits net/mlx5: use separate Rx and Tx queue limits net/null: use separate Rx and Tx queue limits net/sfc: use separate Rx and Tx queue limits net/thunderx: use separate Rx and Tx queue limits net/vhost: use separate Rx and Tx queue limits app/dumpcap: use separate Rx and Tx queue limits app/test-pmd: use separate Rx and Tx queue limits examples/ipsec-secgw: use separate Rx and Tx queue limits examples/l3fwd-power: use separate Rx and Tx queue limits examples/l3fwd: use separate Rx and Tx queue limits examples/vhost: use separate Rx and Tx queue limits
config: make queues per port a meson config option
config: add computed max queues define for compatibilityapp/dumpcap/main.c | 2 +- app/test-bbdev/test_bbdev.c | 4 ++-- app/test-pmd/testpmd.c | 7 ++++--- app/test-pmd/testpmd.h | 16 ++++++++-------- config/meson.build | 10 ++++++++++ config/rte_config.h | 2 +- doc/guides/rel_notes/deprecation.rst | 11 +++++++++++ doc/guides/rel_notes/release_24_11.rst | 21 +++++++++++++++++++++ drivers/net/af_xdp/rte_eth_af_xdp.c | 2 +- drivers/net/cnxk/cnxk_ethdev_ops.c | 4 ++-- drivers/net/failsafe/failsafe_ops.c | 4 ++-- drivers/net/hns3/hns3_tm.c | 2 +- drivers/net/mlx5/mlx5_flow.c | 2 +- drivers/net/mlx5/mlx5_flow_hw.c | 2 +- drivers/net/null/rte_eth_null.c | 4 ++-- drivers/net/sfc/sfc_sw_stats.c | 6 ++++-- drivers/net/thunderx/nicvf_ethdev.c | 2 +- drivers/net/vhost/rte_eth_vhost.c | 7 ++++--- examples/ipsec-secgw/ipsec-secgw.c | 2 +- examples/ipsec-secgw/ipsec.c | 2 +- examples/l3fwd-power/main.c | 2 +- examples/l3fwd-power/perf_core.c | 2 +- examples/l3fwd/main.c | 2 +- examples/vhost/main.c | 2 +- examples/vhost/main.h | 2 +- lib/bpf/bpf_pkt.c | 3 ++- lib/cryptodev/cryptodev_pmd.c | 4 ++-- lib/ethdev/ethdev_driver.h | 8 ++++---- lib/ethdev/ethdev_private.c | 24 ++++++++++++++----------
Quoted 9 lines
lib/ethdev/rte_ethdev.c | 16 +++++++--------- lib/ethdev/rte_ethdev.h | 18 +++++++++--------- lib/eventdev/eventdev_private.c | 2 +- lib/latencystats/rte_latencystats.c | 4 ++-- lib/pdump/rte_pdump.c | 18 +++++++++--------- lib/power/rte_power_pmd_mgmt.c | 4 ++-- meson_options.txt | 4 ++++ 36 files changed, 140 insertions(+), 87 deletions(-)
I sent some comments.
Patch 2 has a typo "ethev" in its title.
I would squash the drivers changes into a single patch, as those are mechanical.
The last two patches may have to be squashed as I suspect compilation is broken for applications relying on RTE_MAX_QUEUES_PER_PORT if we stop between the two changes.
Otherwise, lgtm.
Signature · 1 line
David Marchand
Re: [PATCH v3 00/26] add meson config options for queues per port
Quoted 123 lines · David Marchand
Quoted 111 lines · Bruce Richardson
There are a number of issues with the current
RTE_MAX_QUEUES_PER_PORTsetting in DPDK that are addressed by this patchset:* The name does not make it clear that this is intended as an ethdev-only setting
* A number of other libraries are using this define rather than having more relevant defines for the particular usecase.
* The define is hard-coded in DPDK source code and is not adjustable via a build-time/meson option
* Because of the lack of configurability, the max is therefore set to a conservatively-high value, wasting memory.
* There is an assumption that the number of Rx queues and Tx queues should have the same maximum value. Depending on application, it may be desirable to have fan-in with multiple Rx queues e.g. for classification/filtering, feed a single Tx queue, or the opposite where, e.g. for QoS Tx scheduling, a few Rx queues feeds a very large number of Tx queues.This patchset therefore addresses these by:
* replacing the single define for max queues with independent defines for Rx and Tx queues.
* adjusts the name to ensure that it is clear the defines are for ethports only. [ethports being used in theRTE_MAX_ETHPORTSsetting].
* replaces occurances ofRTE_MAX_QUEUES_PER_PORTwith appropriate defines for non-ethdev use cases
* replaces all other internal occurances of the define with the new per-Rx and per-Tx definitions.
* adds meson config options to allow build-time configuration of the max Rx and Tx queue values.Naming Note:
* The new meson config options are called "max_ethport_rx_queues" and "max_ethport_tx_queues" so that in the meson options list they appear alphabetically beside the existing "max_ethports" option.
* For naming consistency, the new C defines are thereforeRTE_MAX_ETHPORT_RX_QUEUESandRTE_MAX_ETHPORT_TX_QUEUES.V3:
* Resend of v2 with correct author email, to avoid reply bounces
* drop "rfc" prefix from patchesV2:
* What was a single patch with "3 insertions(+), 1 deletion(-)" has now become a 26-patch set! :-)
* Created separate Rx and Tx defines
* Ensured that the name makes it clear that the define is for ethdev
* When updating internal use, created one patch per component for easier maintainer review. In most cases it was obvious whether Rx or Tx define should be used, but a few cases were less clear.
* Added documentation updates for the changes (release notes and deprecation notice), spread across 3 of the patches.Bruce Richardson (26):
cryptodev: remove use of ethdev max queues definition
eventdev: remove use of ethev queues define app/test-bbdev: remove use of ethdev queue count value
config: add separate defines for max Rx and Tx queues
ethdev: use separate Rx and Tx queue limits
bpf: use separate Rx and Tx queue limits
latencystats: use separate Rx and Tx queue limits
pdump: use separate Rx and Tx queue limits
power: use separate Rx and Tx queue limitsnet/af_xdp: use separate Rx and Tx queue limits net/cnxk: use separate Rx and Tx queue limits net/failsafe: use separate Rx and Tx queue limits net/hns3: use separate Rx and Tx queue limits net/mlx5: use separate Rx and Tx queue limits net/null: use separate Rx and Tx queue limits net/sfc: use separate Rx and Tx queue limits net/thunderx: use separate Rx and Tx queue limits net/vhost: use separate Rx and Tx queue limits app/dumpcap: use separate Rx and Tx queue limits app/test-pmd: use separate Rx and Tx queue limits examples/ipsec-secgw: use separate Rx and Tx queue limits examples/l3fwd-power: use separate Rx and Tx queue limits examples/l3fwd: use separate Rx and Tx queue limits examples/vhost: use separate Rx and Tx queue limits
config: make queues per port a meson config option
config: add computed max queues define for compatibilityapp/dumpcap/main.c | 2 +- app/test-bbdev/test_bbdev.c | 4 ++-- app/test-pmd/testpmd.c | 7 ++++--- app/test-pmd/testpmd.h | 16 ++++++++-------- config/meson.build | 10 ++++++++++ config/rte_config.h | 2 +- doc/guides/rel_notes/deprecation.rst | 11 +++++++++++ doc/guides/rel_notes/release_24_11.rst | 21 +++++++++++++++++++++ drivers/net/af_xdp/rte_eth_af_xdp.c | 2 +- drivers/net/cnxk/cnxk_ethdev_ops.c | 4 ++-- drivers/net/failsafe/failsafe_ops.c | 4 ++-- drivers/net/hns3/hns3_tm.c | 2 +- drivers/net/mlx5/mlx5_flow.c | 2 +- drivers/net/mlx5/mlx5_flow_hw.c | 2 +- drivers/net/null/rte_eth_null.c | 4 ++-- drivers/net/sfc/sfc_sw_stats.c | 6 ++++-- drivers/net/thunderx/nicvf_ethdev.c | 2 +- drivers/net/vhost/rte_eth_vhost.c | 7 ++++--- examples/ipsec-secgw/ipsec-secgw.c | 2 +- examples/ipsec-secgw/ipsec.c | 2 +- examples/l3fwd-power/main.c | 2 +- examples/l3fwd-power/perf_core.c | 2 +- examples/l3fwd/main.c | 2 +- examples/vhost/main.c | 2 +- examples/vhost/main.h | 2 +- lib/bpf/bpf_pkt.c | 3 ++- lib/cryptodev/cryptodev_pmd.c | 4 ++-- lib/ethdev/ethdev_driver.h | 8 ++++---- lib/ethdev/ethdev_private.c | 24 ++++++++++++++----------Quoted 9 lines
lib/ethdev/rte_ethdev.c | 16 +++++++--------- lib/ethdev/rte_ethdev.h | 18 +++++++++--------- lib/eventdev/eventdev_private.c | 2 +- lib/latencystats/rte_latencystats.c | 4 ++-- lib/pdump/rte_pdump.c | 18 +++++++++--------- lib/power/rte_power_pmd_mgmt.c | 4 ++-- meson_options.txt | 4 ++++ 36 files changed, 140 insertions(+), 87 deletions(-)
Quoted 3 lines
I sent some comments.
Patch 2 has a typo "ethev" in its title.
Sure. Will do a new revision in future if others are similarly ok with it.
Quoted 3 lines
I would squash the drivers changes into a single patch, as those are mechanical.
Yep, makes sense. I split them out for easier review. Are you or Thomas ok to squash those on apply as I'd rather keep them separate for maintainers while the changes are still in patchwork?
Quoted 3 lines
The last two patches may have to be squashed as I suspect compilation is broken for applications relying on
RTE_MAX_QUEUES_PER_PORTif we stop between the two changes.
Yes, good point that it might be. Will squash in later revisions.
Quoted 3 lines
Otherwise, lgtm.
One open question is whether we are doing the right thing to have separate defines for Tx queues and Rx queues?
I think it's useful to have the separate defines, but there has been a comment suggesting that we are better keeping the single define. My own thinking is that the single-define is appropriate for offload devices since the queues tend to come in pairs - since everything sent down to the device comes back up again - but for the NICs, we can have wild assymmetry depending on use case, for example, for QoS on Tx.
I take it you are ok with the two-define solution then?
/Bruce