Skip to contentSkip to conversation

TAP device can not support 17 queues

DPDK/ethdev Bug 1381 [email protected] 2 participants 2 messages Feb 20, 2024

lore
Share & tools
lore

[email protected]

author Reviewed-by Ferruh Yigit
#0 lore raw

https://bugs.dpdk.org/show_bug.cgi?id=1381

            Bug ID: 1381
           Summary: TAP device can not support 17 queues
           Product: DPDK
           Version: 23.11
          Hardware: All
                OS: All
            Status: UNCONFIRMED
          Severity: normal
          Priority: Normal
         Component: ethdev
          Assignee: [email protected]
          Reporter: [email protected]

Target Milestone: ---

If you try:

# dpdk-testpmd --log-level=pmd.net.tap:debug -l 1-2 --vdev=net_tap0 -- -i
--rxq=8 --txq=8

It will fail because:
EAL: Detected CPU lcores: 8
EAL: Detected NUMA nodes: 1
EAL: Detected static linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'VA'
rte_pmd_tap_probe(): Initializing pmd_tap for net_tap0 eth_dev_tap_create(): TAP device on numa 0
tun_alloc(): /dev/net/tun Features 00007173
tun_alloc(): Multi-queue support for 16 queues
tun_alloc(): Device name is 'dtap0'
tun_alloc(): Using rt-signal 35
eth_dev_tap_create(): allocated dtap0
Interactive-mode selected
testpmd: create a new mbuf pool <mb_pool_0>: n=155456, size=2176, socket=0
testpmd: preferred mempool ops selected: ring_mp_mc

Warning! port-topology=paired and odd forward ports number, the last port will pair with itself.

Configuring Port 0 (socket 0)

tap_dev_configure(): net_tap0: dtap0: TX configured queues number: 8
tap_dev_configure(): net_tap0: dtap0: RX configured queues number: 8
tun_alloc(): /dev/net/tun Features 00007173
tun_alloc():   Multi-queue support for 16 queues
tun_alloc(): Device name is 'dtap0'
tap_setup_queue(): dtap0: add tx queue for qid 0 fd 26
tap_tx_queue_setup():   TX TUNTAP device name dtap0, qid 0 on fd 26 csum off
tun_alloc(): /dev/net/tun Features 00007173
tun_alloc():   Multi-queue support for 16 queues
tun_alloc(): Device name is 'dtap0'
tap_setup_queue(): dtap0: add tx queue for qid 1 fd 212
tap_tx_queue_setup():   TX TUNTAP device name dtap0, qid 1 on fd 212 csum off
tun_alloc(): /dev/net/tun Features 00007173
tun_alloc():   Multi-queue support for 16 queues
tun_alloc(): Device name is 'dtap0'
tap_setup_queue(): dtap0: add tx queue for qid 2 fd 213
tap_tx_queue_setup():   TX TUNTAP device name dtap0, qid 2 on fd 213 csum off
tun_alloc(): /dev/net/tun Features 00007173
tun_alloc():   Multi-queue support for 16 queues
tun_alloc(): Device name is 'dtap0'
tap_setup_queue(): dtap0: add tx queue for qid 3 fd 214
tap_tx_queue_setup():   TX TUNTAP device name dtap0, qid 3 on fd 214 csum off
tun_alloc(): /dev/net/tun Features 00007173
tun_alloc():   Multi-queue support for 16 queues
tun_alloc(): Device name is 'dtap0'
tap_setup_queue(): dtap0: add tx queue for qid 4 fd 215
tap_tx_queue_setup():   TX TUNTAP device name dtap0, qid 4 on fd 215 csum off
tun_alloc(): /dev/net/tun Features 00007173
tun_alloc():   Multi-queue support for 16 queues
tun_alloc(): Device name is 'dtap0'
tap_setup_queue(): dtap0: add tx queue for qid 5 fd 216
tap_tx_queue_setup():   TX TUNTAP device name dtap0, qid 5 on fd 216 csum off
tun_alloc(): /dev/net/tun Features 00007173
tun_alloc():   Multi-queue support for 16 queues
tun_alloc(): Device name is 'dtap0'
tap_setup_queue(): dtap0: add tx queue for qid 6 fd 217
tap_tx_queue_setup():   TX TUNTAP device name dtap0, qid 6 on fd 217 csum off
tun_alloc(): /dev/net/tun Features 00007173
tun_alloc():   Multi-queue support for 16 queues
tun_alloc(): Device name is 'dtap0'
tap_setup_queue(): dtap0: add tx queue for qid 7 fd 218
tap_tx_queue_setup():   TX TUNTAP device name dtap0, qid 7 on fd 218 csum off
tap_setup_queue(): dtap0: dup fd 26 for rx queue qid 0 (219)
tap_rx_queue_setup():   RX TUNTAP device name dtap0, qid 0 on fd 219
tap_setup_queue(): dtap0: dup fd 212 for rx queue qid 1 (220)
tap_rx_queue_setup():   RX TUNTAP device name dtap0, qid 1 on fd 220
tap_setup_queue(): dtap0: dup fd 213 for rx queue qid 2 (221)
tap_rx_queue_setup():   RX TUNTAP device name dtap0, qid 2 on fd 221
tap_setup_queue(): dtap0: dup fd 214 for rx queue qid 3 (222)
tap_rx_queue_setup():   RX TUNTAP device name dtap0, qid 3 on fd 222
tap_setup_queue(): dtap0: dup fd 215 for rx queue qid 4 (223)
tap_rx_queue_setup():   RX TUNTAP device name dtap0, qid 4 on fd 223
tap_setup_queue(): dtap0: dup fd 216 for rx queue qid 5 (224)
tap_rx_queue_setup():   RX TUNTAP device name dtap0, qid 5 on fd 224
tap_setup_queue(): dtap0: dup fd 217 for rx queue qid 6 (225)
tap_rx_queue_setup():   RX TUNTAP device name dtap0, qid 6 on fd 225
tap_setup_queue(): dtap0: dup fd 218 for rx queue qid 7 (226)
tap_rx_queue_setup():   RX TUNTAP device name dtap0, qid 7 on fd 226
EAL: Cannot send more than 8 FDs
tap_mp_req_on_rxtx(): Failed to send start req to secondary 7

This is a regression caused by:
commit c36ce7099c2187926cd62cff7ebd479823554929

Author: Kumara Parameshwaran <[email protected]>
Date:   Mon Jan 31 20:02:34 2022 +0530

net/tap: fix to populate FDs in secondary process

When a tap device is hotplugged to primary process which in turn adds the device to all secondary process, the secondary process does a tap_mp_attach_queues, but the fds are not populated in the primary during the probe they are populated during the queue_setup, added a fix to sync the queues during rte_eth_dev_start

Fixes 4852aa8f6e21 ("drivers/net: enable hotplug on secondary process")
Signature · 2 lines
You are receiving this mail because:
You are the assignee for the bug.
#1 lore raw
Quoted 115 lines · On Tue, 20 Feb 2024 03:29:04 +0000 [email protected] wrote:

https://bugs.dpdk.org/show_bug.cgi?id=1381

            Bug ID: 1381
           Summary: TAP device can not support 17 queues
           Product: DPDK
           Version: 23.11
          Hardware: All
                OS: All
            Status: UNCONFIRMED
          Severity: normal
          Priority: Normal
         Component: ethdev
          Assignee: [email protected]
          Reporter: [email protected]

Target Milestone: ---

If you try:

# dpdk-testpmd --log-level=pmd.net.tap:debug -l 1-2 --vdev=net_tap0 -- -i
--rxq=8 --txq=8

It will fail because:
EAL: Detected CPU lcores: 8
EAL: Detected NUMA nodes: 1
EAL: Detected static linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'VA'
rte_pmd_tap_probe(): Initializing pmd_tap for net_tap0 eth_dev_tap_create(): TAP device on numa 0
tun_alloc(): /dev/net/tun Features 00007173
tun_alloc(): Multi-queue support for 16 queues
tun_alloc(): Device name is 'dtap0'
tun_alloc(): Using rt-signal 35
eth_dev_tap_create(): allocated dtap0
Interactive-mode selected
testpmd: create a new mbuf pool <mb_pool_0>: n=155456, size=2176, socket=0
testpmd: preferred mempool ops selected: ring_mp_mc

Warning! port-topology=paired and odd forward ports number, the last port will pair with itself.

Configuring Port 0 (socket 0)

tap_dev_configure(): net_tap0: dtap0: TX configured queues number: 8
tap_dev_configure(): net_tap0: dtap0: RX configured queues number: 8
tun_alloc(): /dev/net/tun Features 00007173
tun_alloc():   Multi-queue support for 16 queues
tun_alloc(): Device name is 'dtap0'
tap_setup_queue(): dtap0: add tx queue for qid 0 fd 26
tap_tx_queue_setup():   TX TUNTAP device name dtap0, qid 0 on fd 26 csum off
tun_alloc(): /dev/net/tun Features 00007173
tun_alloc():   Multi-queue support for 16 queues
tun_alloc(): Device name is 'dtap0'
tap_setup_queue(): dtap0: add tx queue for qid 1 fd 212
tap_tx_queue_setup():   TX TUNTAP device name dtap0, qid 1 on fd 212 csum off
tun_alloc(): /dev/net/tun Features 00007173
tun_alloc():   Multi-queue support for 16 queues
tun_alloc(): Device name is 'dtap0'
tap_setup_queue(): dtap0: add tx queue for qid 2 fd 213
tap_tx_queue_setup():   TX TUNTAP device name dtap0, qid 2 on fd 213 csum off
tun_alloc(): /dev/net/tun Features 00007173
tun_alloc():   Multi-queue support for 16 queues
tun_alloc(): Device name is 'dtap0'
tap_setup_queue(): dtap0: add tx queue for qid 3 fd 214
tap_tx_queue_setup():   TX TUNTAP device name dtap0, qid 3 on fd 214 csum off
tun_alloc(): /dev/net/tun Features 00007173
tun_alloc():   Multi-queue support for 16 queues
tun_alloc(): Device name is 'dtap0'
tap_setup_queue(): dtap0: add tx queue for qid 4 fd 215
tap_tx_queue_setup():   TX TUNTAP device name dtap0, qid 4 on fd 215 csum off
tun_alloc(): /dev/net/tun Features 00007173
tun_alloc():   Multi-queue support for 16 queues
tun_alloc(): Device name is 'dtap0'
tap_setup_queue(): dtap0: add tx queue for qid 5 fd 216
tap_tx_queue_setup():   TX TUNTAP device name dtap0, qid 5 on fd 216 csum off
tun_alloc(): /dev/net/tun Features 00007173
tun_alloc():   Multi-queue support for 16 queues
tun_alloc(): Device name is 'dtap0'
tap_setup_queue(): dtap0: add tx queue for qid 6 fd 217
tap_tx_queue_setup():   TX TUNTAP device name dtap0, qid 6 on fd 217 csum off
tun_alloc(): /dev/net/tun Features 00007173
tun_alloc():   Multi-queue support for 16 queues
tun_alloc(): Device name is 'dtap0'
tap_setup_queue(): dtap0: add tx queue for qid 7 fd 218
tap_tx_queue_setup():   TX TUNTAP device name dtap0, qid 7 on fd 218 csum off
tap_setup_queue(): dtap0: dup fd 26 for rx queue qid 0 (219)
tap_rx_queue_setup():   RX TUNTAP device name dtap0, qid 0 on fd 219
tap_setup_queue(): dtap0: dup fd 212 for rx queue qid 1 (220)
tap_rx_queue_setup():   RX TUNTAP device name dtap0, qid 1 on fd 220
tap_setup_queue(): dtap0: dup fd 213 for rx queue qid 2 (221)
tap_rx_queue_setup():   RX TUNTAP device name dtap0, qid 2 on fd 221
tap_setup_queue(): dtap0: dup fd 214 for rx queue qid 3 (222)
tap_rx_queue_setup():   RX TUNTAP device name dtap0, qid 3 on fd 222
tap_setup_queue(): dtap0: dup fd 215 for rx queue qid 4 (223)
tap_rx_queue_setup():   RX TUNTAP device name dtap0, qid 4 on fd 223
tap_setup_queue(): dtap0: dup fd 216 for rx queue qid 5 (224)
tap_rx_queue_setup():   RX TUNTAP device name dtap0, qid 5 on fd 224
tap_setup_queue(): dtap0: dup fd 217 for rx queue qid 6 (225)
tap_rx_queue_setup():   RX TUNTAP device name dtap0, qid 6 on fd 225
tap_setup_queue(): dtap0: dup fd 218 for rx queue qid 7 (226)
tap_rx_queue_setup():   RX TUNTAP device name dtap0, qid 7 on fd 226
EAL: Cannot send more than 8 FDs
tap_mp_req_on_rxtx(): Failed to send start req to secondary 7

This is a regression caused by:
commit c36ce7099c2187926cd62cff7ebd479823554929

Author: Kumara Parameshwaran <[email protected]>
Date:   Mon Jan 31 20:02:34 2022 +0530

net/tap: fix to populate FDs in secondary process

When a tap device is hotplugged to primary process which in turn adds the device to all secondary process, the secondary process does a tap_mp_attach_queues, but the fds are not populated in the primary during the probe they are populated during the queue_setup, added a fix to sync the queues during rte_eth_dev_start

Quoted 6 lines
Fixes 4852aa8f6e21 ("drivers/net: enable hotplug on secondary process")

The number of file descriptors allowed in rte_mp_msg should be much larger. The Linux kernel has an upper limit SCM_MAX_FD which is 253 (see net/scm.h)

But fixing this will break ABI because rte_mp_msg structure was exposed in rte_eal.h. It should have been internal!

Alternatively, since fds[] is the last field in rte_mp_msg, and num_fds is also there, fds[] could have been a flexible array, rather than hard coded.