bus/pci: probe PCI devices in whitelisted order
PATCH 1/1 bus/pci: probe PCI devices in whitelisted order
From: Vamsi Attunuru <[email protected]>
Current pci bus driver scans pci devices in the order that it read from sysfs. Accordingly all or whitelisted devices are getting probed.
Patch modifies the probing order of whitelisted pci devices in a sequence the devices are whitelisted(using EAL flags).
It ensures the eth devices that application uses are probed in device whitelisted sequence, in turn it facilitates the packet forwarding applications to work without any packet loss or performance drop when the underneath network ports have different bandwidths. By altering the whitelist order applications like testpmd, l2fwd can forward the ingress traffic to egress port that has of equivalent bandwidth.
--- drivers/bus/pci/pci_common.c | 67 ++++++++++++++++++++++++++++++-------------- 1 file changed, 46 insertions(+), 21 deletions(-)
drivers/bus/pci/pci_common.c +46 -21
@@ -293,32 +293,57 @@ rte_pci_probe(void)struct rte_pci_device *dev =NULL ;size_t probed =0 , failed =0 ;struct rte_devargs *devargs; -int probe_all =0 ; +struct rte_pci_addr addr;int ret =0 ; -if (rte_pci_bus.bus.conf.scan_mode !=RTE_BUS_SCAN_WHITELIST ) - probe_all =1 ; - -FOREACH_DEVICE_ON_PCIBUS (dev) { - probed++; +if (rte_pci_bus.bus.conf.scan_mode !=RTE_BUS_SCAN_WHITELIST ) { +/* Probe all devices */ +FOREACH_DEVICE_ON_PCIBUS (dev) { + probed++; - devargs = dev->device.devargs; -/* probe all or only whitelisted devices */ -if (probe_all) ret =pci_probe_all_drivers (dev); -else if (devargs !=NULL && - devargs->policy ==RTE_DEV_WHITELISTED ) - ret =pci_probe_all_drivers (dev); -if (ret <0 ) { -if (ret != -EEXIST ) { -RTE_LOG (ERR ,EAL ,"Requested device " -PCI_PRI_FMT " cannot be used\n" , - dev->addr.domain, dev->addr.bus, - dev->addr.devid, dev->addr.function); - rte_errno = errno; - failed++; +if (ret <0 ) { +if (ret != -EEXIST ) { +RTE_LOG (ERR ,EAL ,"Requested device " +PCI_PRI_FMT " cannot be used\n" , + dev->addr.domain, dev->addr.bus, + dev->addr.devid, + dev->addr.function); + rte_errno = errno; + failed++; + } + ret =0 ; + } + } + }else { +/* Probe only whitelisted devices */ + RTE_EAL_DEVARGS_FOREACH("pci", devargs) { +if (devargs->policy !=RTE_DEV_WHITELISTED ) +continue ; + + devargs->bus->parse (devargs->name, &addr); + +FOREACH_DEVICE_ON_PCIBUS (dev) { +if (rte_pci_addr_cmp (&dev->addr, &addr)) +continue ; + probed++; + ret =pci_probe_all_drivers (dev); +if (ret <0 ) { +if (ret != -EEXIST ) { +RTE_LOG (ERR ,EAL , +"Requested device " +PCI_PRI_FMT +"cannot be used\n" , + dev->addr.domain, + dev->addr.bus, + dev->addr.devid, + dev->addr.function); + rte_errno = errno; + failed++; + } + ret =0 ; + } } - ret =0 ; } }
Signature · 1 line
2.8.4
Quoted 10 lines
-----Original Message-----
From: dev <[email protected]> On Behalf Of [email protected]
Sent: Monday, September 23, 2019 14:57
To: [email protected]
Cc: [email protected]; [email protected]; [email protected]; Thomas Monjalon <[email protected]>; [email protected]; Vamsi Attunuru <[email protected]>
Subject: [dpdk-dev] [PATCH v1 1/1] bus/pci: probe PCI devices in whitelisted order
Quoted 7 lines
From: Vamsi Attunuru <[email protected]>
Current pci bus driver scans pci devices in the order that it read from sysfs. Accordingly all or whitelisted devices are getting probed.
Patch modifies the probing order of whitelisted pci devices in a sequence the devices are whitelisted(using EAL flags).
Thanks, it would be nice to have opportunity to control probing order, it might be useful for bonded devices and representors either.
Quoted 103 lines
It ensures the eth devices that application uses are probed in device whitelisted sequence, in turn it facilitates the packet forwarding applications to work without any packet loss or performance drop when the underneath network ports have different bandwidths. By altering the whitelist order applications like testpmd, l2fwd can forward the ingress traffic to egress port that has of equivalent bandwidth.
Signed-off-by Vamsi Attunuru <[email protected]>--- drivers/bus/pci/pci_common.c | 67 ++++++++++++++++++++++++++++++---- ---------- 1 file changed, 46 insertions(+), 21 deletions(-)drivers/bus/pci/pci_common.c +8
-20@@ -293,32 +293,57 @@ rte_pci_probe(void)struct rte_pci_device *dev =NULL ;size_t probed =0 , failed =0 ;struct rte_devargs *devargs; -int probe_all =0 ; +struct rte_pci_addr addr;int ret =0 ; -if (rte_pci_bus.bus.conf.scan_mode !=RTE_BUS_SCAN_WHITELIST ) - probe_all =1 ; - -FOREACH_DEVICE_ON_PCIBUS (dev) { - probed++; +if (rte_pci_bus.bus.conf.scan_mode !=RTE_BUS_SCAN_WHITELIST ) { +/* Probe all devices */ +FOREACH_DEVICE_ON_PCIBUS (dev) { + probed++; - devargs = dev->device.devargs; -/* probe all or only whitelisted devices */ -if (probe_all) ret =pci_probe_all_drivers (dev); -else if (devargs !=NULL && - devargs->policy ==RTE_DEV_WHITELISTED ) - ret =pci_probe_all_drivers (dev); -if (ret <0 ) { -if (ret != -EEXIST ) { -RTE_LOG (ERR ,EAL ,"Requested device " -PCI_PRI_FMT " cannot be used\n" , - dev->addr.domain, dev->addr.bus, - dev->addr.devid, dev- >addr.function); - rte_errno = errno; - failed++; +if (ret <0 ) { +if (ret != -EEXIST ) { + RTE_LOG(ERR, EAL, "Requesteddevice "
drivers/bus/pci/pci_common.c +1
-0+ PCI_PRI_FMT " cannot beused\n",
drivers/bus/pci/pci_common.c +1
-0+ dev->addr.domain, dev-Quoted 1 line
addr.bus,
drivers/bus/pci/pci_common.c +36
-2+ dev->addr.devid, + dev->addr.function); + rte_errno = errno; + failed++; + } + ret =0 ; + } + } + }else { +/* Probe only whitelisted devices */ + RTE_EAL_DEVARGS_FOREACH("pci", devargs) { +if (devargs->policy !=RTE_DEV_WHITELISTED ) +continue ; + + devargs->bus->parse (devargs->name, &addr); + +FOREACH_DEVICE_ON_PCIBUS (dev) { +if (rte_pci_addr_cmp (&dev->addr, &addr)) +continue ; + probed++; + ret =pci_probe_all_drivers (dev); +if (ret <0 ) { +if (ret != -EEXIST ) { +RTE_LOG (ERR ,EAL , +"Requested device " +PCI_PRI_FMT +"cannot be used\n" , + dev->addr.domain, + dev->addr.bus, + dev->addr.devid, + dev->addr.function); + rte_errno = errno; + failed++; + } + ret =0 ; + } } - ret =0 ; } } --2.8.4
Quoted 15 lines · Slava Ovsiienko
Quoted 15 lines
-----Original Message-----
From: dev <[email protected]> On Behalf Of [email protected]
Sent: Monday, September 23, 2019 14:57
To: [email protected]
Cc: [email protected]; [email protected]; [email protected]; Thomas Monjalon <[email protected]>; [email protected]; Vamsi Attunuru <[email protected]>
Subject: [dpdk-dev] [PATCH v1 1/1] bus/pci: probe PCI devices in whitelisted orderFrom: Vamsi Attunuru <[email protected]>
Current pci bus driver scans pci devices in the order that it read from sysfs. Accordingly all or whitelisted devices are getting probed.
Quoted 17 lines
Quoted 2 lines
Patch modifies the probing order of whitelisted pci devices in a sequence the devices are whitelisted(using EAL flags).
Thanks, it would be nice to have opportunity to control probing order, it might be useful for bonded devices and representors either.
Quoted 9 lines
It ensures the eth devices that application uses are probed in device whitelisted sequence, in turn it facilitates the packet forwarding applications to work without any packet loss or performance drop when the underneath network ports have different bandwidths. By altering the whitelist order applications like testpmd, l2fwd can forward the ingress traffic to egress port that has of equivalent bandwidth.
Signed-off-by Vamsi Attunuru <[email protected]>
Hello Vamsi, Viacheslav,
This is a nice patch. I agree that port dependency could be better handled. The port-mapping part however should be managed at the app level.
Vamsi, you gave the example of l2fwd and testpmd, being able to properly configure forwarding directions implicitly. I think the better approach here is to add these configurations items within the apps directly. Configuring the mapping at the port level is not precise enough. The proper control is about cores, port and queues, not only ports. This patch only solves a limited part of this issue with testpmd.
I wrote a command to do this, that collided with some stream rework from Intel at the time (3, 4 years back?), so I did not take the time to force it through. If there is a need we could discuss about adding this back. I had needed it to write a PMD, that could be useful to others.
As you say Viacheslav, there are use-cases that will rely on fine-grained probe order. However, this patch solves this issue only regarding PCI devices, depending on other PCI devices. We have in EAL an improper hack about it, forcing the vdev probe last, because usually ports depending on others are virtual ones. As this patch shows, the hack is not sufficient, and as the hack shows, this patch does not cover everything.
A solution, would be an EAL parameter (I propose --no-dev), that disable probing for all buses. Applications and devices requiring a fine-grained probe order, are then free to start in this mode (and maybe force it through EAL conf), then hotplug ports as they see fit.
This will keep the existing behavior stable for current apps, while allowing flexibility for the more advanced ones.
Signature · 2 lines
Gaëtan Rivet 6WIND
-----Original Message-----
From: dev <[email protected]> On Behalf Of Gaëtan Rivet
Sent: Wednesday, September 25, 2019 2:37 PM
To: Slava Ovsiienko <[email protected]>
Cc: Vamsi Krishna Attunuru <[email protected]>; [email protected]; [email protected]; [email protected]; Thomas Monjalon <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>
Subject: Re: [dpdk-dev] [PATCH v1 1/1] bus/pci: probe PCI devices in whitelisted order
Quoted 15 lines · Slava Ovsiienko
Quoted 15 lines
-----Original Message-----
From: dev <[email protected]> On Behalf Of [email protected]
Sent: Monday, September 23, 2019 14:57
To: [email protected]
Cc: [email protected]; [email protected]; [email protected]; Thomas Monjalon <[email protected]>; [email protected]; Vamsi Attunuru <[email protected]>
Subject: [dpdk-dev] [PATCH v1 1/1] bus/pci: probe PCI devices in whitelisted orderFrom: Vamsi Attunuru <[email protected]>
Current pci bus driver scans pci devices in the order that it read from sysfs. Accordingly all or whitelisted devices are getting probed.
Quoted 17 lines
Quoted 2 lines
Patch modifies the probing order of whitelisted pci devices in a sequence the devices are whitelisted(using EAL flags).
Thanks, it would be nice to have opportunity to control probing order, it might be useful for bonded devices and representors either.
Quoted 9 lines
It ensures the eth devices that application uses are probed in
device whitelisted sequence, in turn it facilitates the packet
forwarding applications to work without any packet loss or
performance drop when the underneath network ports have different
bandwidths. By altering the whitelist order applications like
testpmd, l2fwd can forward the ingress traffic to egress port that has of equivalent bandwidth.Signed-off-by Vamsi Attunuru <[email protected]>
Hello Vamsi, Viacheslav,
This is a nice patch. I agree that port dependency could be better handled. The port-mapping part however should be managed at the app level.
Vamsi, you gave the example of l2fwd and testpmd, being able to properly configure forwarding directions implicitly. I think the better approach here is to add these configurations items within the apps directly. Configuring the mapping at the port level is not precise enough. The proper control is about cores, port and queues, not only ports. This patch only solves a limited part of this issue with testpmd.
I wrote a command to do this, that collided with some stream rework from Intel at the time (3, 4 years back?), so I did not take the time to force it through. If there is a need we could discuss about adding this back. I had needed it to write a PMD, that could be useful to others.
As you say Viacheslav, there are use-cases that will rely on fine-grained probe order. However, this patch solves this issue only regarding PCI devices, depending on other PCI devices. We have in EAL an improper hack about it, forcing the vdev probe last, because usually ports depending on others are virtual ones. As this patch shows, the hack is not sufficient, and as the hack shows, this patch does not cover everything.
A solution, would be an EAL parameter (I propose --no-dev), that disable probing for all buses. Applications and devices requiring a fine-grained probe order, are then free to start in this mode (and maybe force it through EAL conf), then hotplug ports as they see fit.
This will keep the existing behavior stable for current apps, while allowing flexibility for the more advanced ones.
Hi Gaetan,
Thanks, vdev part was not taken care in this patch. Rather than imposing hotplug for every application which requires port mapping, If vdev probing order is also handled same as pdevs(in whitelist order), existing whitelisting feature will serve the port mapping requirement, right. Also the existing applications get benefited instead of overloading them with more configuration options. If these probing order is not needed by default, it can be triggered using an EAL parameter(not added yet).
Regards,
A Vamsi
--
Gaëtan Rivet
6WIND
Quoted 59 lines · Vamsi Krishna Attunuru
-----Original Message-----
From: dev <[email protected]> On Behalf Of Gaëtan Rivet
Sent: Wednesday, September 25, 2019 2:37 PM
To: Slava Ovsiienko <[email protected]>
Cc: Vamsi Krishna Attunuru <[email protected]>; [email protected]; [email protected]; [email protected]; Thomas Monjalon <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>
Subject: Re: [dpdk-dev] [PATCH v1 1/1] bus/pci: probe PCI devices in whitelisted orderQuoted 15 lines · Slava Ovsiienko
Quoted 15 lines
-----Original Message-----
From: dev <[email protected]> On Behalf Of [email protected]
Sent: Monday, September 23, 2019 14:57
To: [email protected]
Cc: [email protected]; [email protected]; [email protected]; Thomas Monjalon <[email protected]>; [email protected]; Vamsi Attunuru <[email protected]>
Subject: [dpdk-dev] [PATCH v1 1/1] bus/pci: probe PCI devices in whitelisted orderFrom: Vamsi Attunuru <[email protected]>
Current pci bus driver scans pci devices in the order that it read from sysfs. Accordingly all or whitelisted devices are getting probed.
Quoted 17 lines
Quoted 2 lines
Patch modifies the probing order of whitelisted pci devices in a sequence the devices are whitelisted(using EAL flags).
Thanks, it would be nice to have opportunity to control probing order, it might be useful for bonded devices and representors either.
Quoted 9 lines
It ensures the eth devices that application uses are probed in
device whitelisted sequence, in turn it facilitates the packet
forwarding applications to work without any packet loss or
performance drop when the underneath network ports have different
bandwidths. By altering the whitelist order applications like
testpmd, l2fwd can forward the ingress traffic to egress port that has of equivalent bandwidth.Signed-off-by Vamsi Attunuru <[email protected]>Hello Vamsi, Viacheslav,
This is a nice patch. I agree that port dependency could be better handled. The port-mapping part however should be managed at the app level.
Vamsi, you gave the example of l2fwd and testpmd, being able to properly configure forwarding directions implicitly. I think the better approach here is to add these configurations items within the apps directly. Configuring the mapping at the port level is not precise enough. The proper control is about cores, port and queues, not only ports. This patch only solves a limited part of this issue with testpmd.
I wrote a command to do this, that collided with some stream rework from Intel at the time (3, 4 years back?), so I did not take the time to force it through. If there is a need we could discuss about adding this back. I had needed it to write a PMD, that could be useful to others.
As you say Viacheslav, there are use-cases that will rely on fine-grained probe order. However, this patch solves this issue only regarding PCI devices, depending on other PCI devices. We have in EAL an improper hack about it, forcing the vdev probe last, because usually ports depending on others are virtual ones. As this patch shows, the hack is not sufficient, and as the hack shows, this patch does not cover everything.
A solution, would be an EAL parameter (I propose --no-dev), that disable probing for all buses. Applications and devices requiring a fine-grained probe order, are then free to start in this mode (and maybe force it through EAL conf), then hotplug ports as they see fit.
This will keep the existing behavior stable for current apps, while allowing flexibility for the more advanced ones.
Quoted 6 lines
Hi Gaetan,
Thanks, vdev part was not taken care in this patch. Rather than imposing hotplug for every application which requires port mapping, If vdev probing order is also handled same as pdevs(in whitelist order), existing whitelisting feature will serve the port mapping requirement, right. Also the existing applications get benefited instead of overloading them with more configuration options. If these probing order is not needed by default, it can be triggered using an EAL parameter(not added yet).
Regards,
A Vamsi
Hi,
The way buses are written right now, they will each do a whole scan, then they each probe all their devices.
You cannot intersperse probes across several buses, i.e. probe a PCI device, then a vdev, then another PCI device.
Changing this structure could be difficult. A possible way to do what you want without breaking everything would be to do what the app would have done in my solution above, but from within the EAL: block all probes, then go over a mixed list of (-w) and (--vdev) parameters and hotplug them in order. This would require the --no-dev (or --no-probe, or --no-auto-probe) flag anyway (or as a conf item, or something at least telling the EAL to behave this way).
Would this way of doing it work for you?
In any case, controlling the probe order should be fixed properly for all buses and the general use-case if possible, instead of limiting the patch to the PCI bus.
Kind regards,
Signature · 2 lines
Gaëtan Rivet 6WIND
Quoted 118 lines
-----Original Message-----
From: Gaëtan Rivet <[email protected]>
Sent: Thursday, September 26, 2019 1:34 PM
To: Vamsi Krishna Attunuru <[email protected]>
Cc: Slava Ovsiienko <[email protected]>; [email protected]; [email protected]; [email protected]; Thomas Monjalon <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>
Subject: [EXT] Re: [dpdk-dev] [PATCH v1 1/1] bus/pci: probe PCI devices in whitelisted orderExternal Email
----------------------------------------------------------------------
Quoted 12 lines · Vamsi Krishna Attunuru
-----Original Message-----
From: dev <[email protected]> On Behalf Of Gaëtan Rivet
Sent: Wednesday, September 25, 2019 2:37 PM
To: Slava Ovsiienko <[email protected]>
Cc: Vamsi Krishna Attunuru <[email protected]>; [email protected]; [email protected]; [email protected]; Thomas Monjalon <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>
Subject: Re: [dpdk-dev] [PATCH v1 1/1] bus/pci: probe PCI devices in whitelisted orderQuoted 32 lines
Quoted 31 lines · Slava Ovsiienko
Quoted 11 lines
-----Original Message-----
From: dev <[email protected]> On Behalf Of [email protected]
Sent: Monday, September 23, 2019 14:57
To: [email protected]
Cc: [email protected]; [email protected]; [email protected]; Thomas Monjalon <[email protected]>; [email protected]; Vamsi Attunuru <[email protected]>
Subject: [dpdk-dev] [PATCH v1 1/1] bus/pci: probe PCI devices in whitelisted orderQuoted 7 lines
From: Vamsi Attunuru <[email protected]>
Current pci bus driver scans pci devices in the order that it read from sysfs. Accordingly all or whitelisted devices are getting probed.
Patch modifies the probing order of whitelisted pci devices in a sequence the devices are whitelisted(using EAL flags).
Thanks, it would be nice to have opportunity to control probing order, it might be useful for bonded devices and representors either.
Quoted 7 lines
It ensures the eth devices that application uses are probed in device whitelisted sequence, in turn it facilitates the packet forwarding applications to work without any packet loss or performance drop when the underneath network ports have different bandwidths. By altering the whitelist order applications like testpmd, l2fwd can forward the ingress traffic to egress port that has of
equivalent bandwidth.
Quoted 6 lines
Quoted 2 lines
Quoted 2 lines
Signed-off-by Vamsi Attunuru <[email protected]>Hello Vamsi, Viacheslav,
This is a nice patch. I agree that port dependency could be better handled. The
port-mapping part however should be managed at the app level.
Quoted 2 lines
Vamsi, you gave the example of l2fwd and testpmd, being able to properly
configure forwarding directions implicitly. I think the better approach here is to add these configurations items within the apps directly. Configuring the mapping at the port level is not precise enough. The proper control is about cores, port and queues, not only ports.
Quoted 3 lines
This patch only solves a limited part of this issue with testpmd.
I wrote a command to do this, that collided with some stream rework from
Intel at the time (3, 4 years back?), so I did not take the time to force it through. If there is a need we could discuss about adding this back. I had needed it to write a PMD, that could be useful to others.
Quoted 2 lines
As you say Viacheslav, there are use-cases that will rely on fine-grained probe
order. However, this patch solves this issue only regarding PCI devices, depending on other PCI devices. We have in EAL an improper hack about it, forcing the vdev probe last, because usually ports depending on others are virtual ones. As this patch shows, the hack is not sufficient, and as the hack shows, this patch does not cover everything.
Quoted 2 lines
A solution, would be an EAL parameter (I propose --no-dev), that disable
probing for all buses. Applications and devices requiring a fine-grained probe order, are then free to start in this mode (and maybe force it through EAL conf), then hotplug ports as they see fit.
Quoted 2 lines
This will keep the existing behavior stable for current apps, while allowing
flexibility for the more advanced ones.
Quoted 5 lines
Hi Gaetan,
Thanks, vdev part was not taken care in this patch. Rather than imposing
hotplug for every application which requires port mapping, If vdev probing order is also handled same as pdevs(in whitelist order), existing whitelisting feature will serve the port mapping requirement, right. Also the existing applications get benefited instead of overloading them with more configuration options. If these probing order is not needed by default, it can be triggered using an EAL parameter(not added yet).
Quoted 3 lines
Regards,
A VamsiHi,
The way buses are written right now, they will each do a whole scan, then they each probe all their devices.
You cannot intersperse probes across several buses, i.e. probe a PCI device, then a vdev, then another PCI device.
Quoted 8 lines
Changing this structure could be difficult. A possible way to do what you want without breaking everything would be to do what the app would have done in my solution above, but from within the EAL: block all probes, then go over a mixed list of (-w) and (--vdev) parameters and hotplug them in order. This would require the --no-dev (or --no-probe, or --no-auto-probe) flag anyway (or as a conf item, or something at least telling the EAL to behave this way).
Would this way of doing it work for you?
Yes, above approach sounds fine to me and it works without breaking everything.
Quoted 3 lines
In any case, controlling the probe order should be fixed properly for all buses and the general use-case if possible, instead of limiting the patch to the PCI bus.
Ack
Quoted 5 lines
Kind regards,
--
Gaëtan Rivet
6WIND
PATCH 1/1 eal: add manual probing option
Add a new EAL option enabling manual probing in the EAL. This command line option will configure the EAL so that buses will not trigger their probe step on their own.
Applications are then expected to hotplug devices as they see fit.
Devices declared on the command line by the user (using -w and --vdev), will be probed using the hotplug API, in the order they are declared.
This has the effect of offering a way for users to control probe order of their devices, for drivers requiring it.
---
I haven't heard many opinions on the matter, please shout if you see an issue with this approach.
@Slava: I have tested rather quickly that it does not break anything,
and that it works as intended for basic cases.
Can you test it further for your use-case and tell me if it works fine?
Beyond the obvious difference between both probe mode, something to keep in mind:
while using -w on invalid devices would not block (PCI) bus probing, it will stop manual probing in its track. All devices need to exist and be valid device IDs.
doc/guides/rel_notes/release_19_11.rst | 9 +++++++ lib/librte_eal/common/eal_common_bus.c | 6 +++++ lib/librte_eal/common/eal_common_dev.c | 41 ++++++++++++++++++++++++++++++ lib/librte_eal/common/eal_common_options.c | 8 ++++++ lib/librte_eal/common/eal_internal_cfg.h | 1 + lib/librte_eal/common/eal_options.h | 2 ++ lib/librte_eal/common/eal_private.h | 9 +++++++ lib/librte_eal/common/include/rte_eal.h | 17 +++++++++++++ lib/librte_eal/freebsd/eal/eal.c | 5 ++++ lib/librte_eal/linux/eal/eal.c | 5 ++++ lib/librte_eal/rte_eal_version.map | 7 +++++ 11 files changed, 110 insertions(+)
doc/guides/rel_notes/release_19_11.rst +9 -0
@@ -56,6 +56,15 @@ New Features Also, make sure to start the actual text at the margin. ========================================================= +* **EAL wil now propose to enforce manual probing of devices.** + + Previously, a user could not force an order when probing declared devices. + This could cause issues for drivers depending on another device being present. + A new option ``--manual-probe`` is now available to do just that. + This new option relies on the device bus supporting hotplug. It can + also be used to disable automatic probing from the ``PCI`` bus without + having to disable the whole bus. + Removed Items -------------
lib/librte_eal/common/eal_common_bus.c +6 -0
@@ -6,6 +6,7 @@#include <string.h> #include <sys/queue.h> +#include <rte_eal.h> #include <rte_bus.h> #include <rte_debug.h> #include <rte_string_fns.h>
@@ -63,6 +64,11 @@ rte_bus_probe(void)int ret;struct rte_bus *bus, *vbus =NULL ; +if (rte_eal_manual_probe ()) { +RTE_LOG (DEBUG ,EAL ,"Manual probing enabled.\n" ); +return rte_dev_probe_devargs_list (); + } + TAILQ_FOREACH(bus, &rte_bus_list, next) {if (!strcmp (bus->name,"vdev" )) { vbus = bus;
lib/librte_eal/common/eal_common_dev.c +41 -0
47 lines not on this page · raw message
lib/librte_eal/common/eal_common_options.c +8 -0
@@ -81,6 +81,7 @@ eal_long_options[] = { {OPT_LEGACY_MEM, 0, NULL, OPT_LEGACY_MEM_NUM }, {OPT_SINGLE_FILE_SEGMENTS, 0, NULL, OPT_SINGLE_FILE_SEGMENTS_NUM}, {OPT_MATCH_ALLOCATIONS, 0, NULL, OPT_MATCH_ALLOCATIONS_NUM}, + {OPT_MANUAL_PROBE, 0, NULL, OPT_MANUAL_PROBE_NUM }, {0, 0, NULL, 0 } };
@@ -1408,6 +1409,9 @@ eal_parse_common_option(int opt, const char *optarg,return -1 ; }break ; +case OPT_MANUAL_PROBE_NUM : + conf->manual_probe =1 ; +break ;/* don't know what to do, leave this to caller */ default :
@@ -1634,6 +1638,10 @@ eal_common_usage(void)" --" OPT_VDEV " Add a virtual device.\n" " The argument format is <driver><id>[,key=val,...]\n" " (ex: --vdev=net_pcap0,iface=eth2).\n" +" --" OPT_MANUAL_PROBE " Enable manual probing.\n" +" Disable probe step for all buses.\n" +" Devices will need to be probed using the hotplug API.\n" +" PCI and vdev declarations will be treated in order as hotplug commands.\n" " --" OPT_IOVA_MODE " Set IOVA mode. 'pa' for IOVA_PA\n" " 'va' for IOVA_VA\n" " -d LIB.so|DIR Add a driver or driver directory\n"
lib/librte_eal/common/eal_internal_cfg.h +1 -0
@@ -44,6 +44,7 @@ struct internal_config {unsigned hugepage_unlink;/**< true to unlink backing files */ volatile unsigned no_pci;/**< true to disable PCI */ volatile unsigned no_hpet;/**< true to disable HPET */ +volatile unsigned manual_probe;/**< true to enable manual device probing. */ volatile unsigned vmware_tsc_map;/**< true to use VMware TSC mapping * instead of native TSC */ volatile unsigned no_shconf;/**< true if there is no shared config */
lib/librte_eal/common/eal_options.h +2 -0
@@ -69,6 +69,8 @@ enum {OPT_IOVA_MODE_NUM , #define OPT_MATCH_ALLOCATIONS "match-allocations"OPT_MATCH_ALLOCATIONS_NUM , +#define OPT_MANUAL_PROBE "manual-probe" +OPT_MANUAL_PROBE_NUM , OPT_LONG_MAX_NUM };
lib/librte_eal/common/eal_private.h +9 -0
@@ -381,4 +381,13 @@ rte_option_init(void);void rte_option_usage (void ); +/** +* Go through the devargs list and probe everything in order. +* +* @return +* 0 on success, negative on error. +*/ +int +rte_dev_probe_devargs_list (void ); +#endif /* _EAL_PRIVATE_H_ */
lib/librte_eal/common/include/rte_eal.h +17 -0
@@ -465,6 +465,23 @@ int rte_eal_has_hugepages(void);int rte_eal_has_pci (void );/** +* Whether EAL probe is manual. +* Enabled by the --manual-probe option. +* +* When manual probing is enabled, batched bus probe of +* their devices is disabled. All devices need to be probed +* using the proper rte_dev API. +* +* In this mode, devices declared on the command line will +* be probed using the bus hotplug API. It is used to enforce +* a specific probe order. +* +* @return +* Nonzero if manual device probing is enabled. +*/ +int rte_eal_manual_probe (void ); + +/** * Whether the EAL was asked to create UIO device. * * @return
lib/librte_eal/freebsd/eal/eal.c +5 -0
@@ -961,6 +961,11 @@ int rte_eal_has_pci(void)return !internal_config.no_pci; } +int rte_eal_manual_probe (void ) +{ +return internal_config.manual_probe; +} +int rte_eal_create_uio_dev (void ) {return internal_config.create_uio_dev;
lib/librte_eal/linux/eal/eal.c +5 -0
@@ -1376,6 +1376,11 @@ int rte_eal_create_uio_dev(void)return internal_config.create_uio_dev; } +int rte_eal_manual_probe (void ) +{ +return internal_config.manual_probe; +} +enum rte_intr_mode rte_eal_vfio_intr_mode (void ) {
lib/librte_eal/rte_eal_version.map +7 -0
@@ -312,6 +312,13 @@ DPDK_19.08 { } DPDK_19.05; +DPDK_19.05 { + global: + + rte_eal_manual_probe + +} DPDK_19.11; + EXPERIMENTAL { global:
Signature · 1 line
2.11.0
Re: [dpdk-dev] [PATCH v1] eal: add manual probing option
Quoted 276 lines · Gaetan Rivet
Add a new EAL option enabling manual probing in the EAL. This command line option will configure the EAL so that buses will not trigger their probe step on their own.
Applications are then expected to hotplug devices as they see fit.
Devices declared on the command line by the user (using -w and --vdev), will be probed using the hotplug API, in the order they are declared.
This has the effect of offering a way for users to control probe order of their devices, for drivers requiring it.
Signed-off-by Gaetan Rivet <[email protected]>---
I haven't heard many opinions on the matter, please shout if you see an issue with this approach.
@Slava: I have tested rather quickly that it does not break anything,
and that it works as intended for basic cases.
Can you test it further for your use-case and tell me if it works fine?Beyond the obvious difference between both probe mode, something to keep in mind:
while using -w on invalid devices would not block (PCI) bus probing, it will stop manual probing in its track. All devices need to exist and be valid device IDs.doc/guides/rel_notes/release_19_11.rst | 9 +++++++ lib/librte_eal/common/eal_common_bus.c | 6 +++++ lib/librte_eal/common/eal_common_dev.c | 41 ++++++++++++++++++++++++++++++ lib/librte_eal/common/eal_common_options.c | 8 ++++++ lib/librte_eal/common/eal_internal_cfg.h | 1 + lib/librte_eal/common/eal_options.h | 2 ++ lib/librte_eal/common/eal_private.h | 9 +++++++ lib/librte_eal/common/include/rte_eal.h | 17 +++++++++++++ lib/librte_eal/freebsd/eal/eal.c | 5 ++++ lib/librte_eal/linux/eal/eal.c | 5 ++++ lib/librte_eal/rte_eal_version.map | 7 +++++ 11 files changed, 110 insertions(+)doc/guides/rel_notes/release_19_11.rst +9
-0@@ -56,6 +56,15 @@ New Features Also, make sure to start the actual text at the margin. ========================================================= +* **EAL wil now propose to enforce manual probing of devices.** + + Previously, a user could not force an order when probing declared devices. + This could cause issues for drivers depending on another device being present. + A new option ``--manual-probe`` is now available to do just that. + This new option relies on the device bus supporting hotplug. It can + also be used to disable automatic probing from the ``PCI`` bus without + having to disable the whole bus. + Removed Items -------------lib/librte_eal/common/eal_common_bus.c +6
-0@@ -6,6 +6,7 @@#include <string.h> #include <sys/queue.h> +#include <rte_eal.h> #include <rte_bus.h> #include <rte_debug.h> #include <rte_string_fns.h> @@ -63,6 +64,11 @@ rte_bus_probe(void)int ret;struct rte_bus *bus, *vbus =NULL ; +if (rte_eal_manual_probe ()) { +RTE_LOG (DEBUG ,EAL ,"Manual probing enabled.\n" ); +return rte_dev_probe_devargs_list (); + } + TAILQ_FOREACH(bus, &rte_bus_list, next) {if (!strcmp (bus->name,"vdev" )) { vbus = bus;lib/librte_eal/common/eal_common_dev.c +41
-0@@ -109,6 +109,47 @@ build_devargs(const char *busname, const char *devname, }int +rte_dev_probe_devargs_list (void ) +{ +struct rte_device *dev; +struct rte_devargs *da; +int ret; + + RTE_EAL_DEVARGS_FOREACH(NULL, da) { + dev = da->bus->find_device (NULL , cmp_dev_name, da->name); +if (dev ==NULL ) { +RTE_LOG (ERR ,EAL ,"Unable to find device %s on bus %s\n" , + da->name, da->bus->name); +continue ; + } + +if (rte_dev_is_probed (dev)) +continue ; + +if (dev->bus->plug ==NULL ) { +RTE_LOG (ERR ,EAL ,"Manual probing (hotplug) not supported by bus %s, " +"required by device %s\n" , + dev->bus->name, dev->name); +continue ; + } + + ret = dev->bus->plug (dev); +/* Ignore positive return values, they are possibly +* triggered by blacklisted devices on the PCI bus. Probing +* should then continue. +*/ +if (ret <0 || !rte_dev_is_probed (dev)) { +RTE_LOG (ERR ,EAL ,"Driver cannot attach device %s\n" , + dev->name); +/* Fail on first real probe error. */ +return ret; + } + } + +return 0 ; +} + +int rte_eal_hotplug_add (const char *busname,const char *devname,const char *drvargs) {lib/librte_eal/common/eal_common_options.c +8
-0@@ -81,6 +81,7 @@ eal_long_options[] = { {OPT_LEGACY_MEM, 0, NULL, OPT_LEGACY_MEM_NUM }, {OPT_SINGLE_FILE_SEGMENTS, 0, NULL, OPT_SINGLE_FILE_SEGMENTS_NUM}, {OPT_MATCH_ALLOCATIONS, 0, NULL, OPT_MATCH_ALLOCATIONS_NUM}, + {OPT_MANUAL_PROBE, 0, NULL, OPT_MANUAL_PROBE_NUM }, {0, 0, NULL, 0 } };@@ -1408,6 +1409,9 @@ eal_parse_common_option(int opt, const char *optarg,return -1 ; }break ; +case OPT_MANUAL_PROBE_NUM : + conf->manual_probe =1 ; +break ;/* don't know what to do, leave this to caller */ default :@@ -1634,6 +1638,10 @@ eal_common_usage(void)" --" OPT_VDEV " Add a virtual device.\n" " The argument format is <driver><id>[,key=val,...]\n" " (ex: --vdev=net_pcap0,iface=eth2).\n" +" --" OPT_MANUAL_PROBE " Enable manual probing.\n" +" Disable probe step for all buses.\n" +" Devices will need to be probed using the hotplug API.\n" +" PCI and vdev declarations will be treated in order as hotplug commands.\n" " --" OPT_IOVA_MODE " Set IOVA mode. 'pa' for IOVA_PA\n" " 'va' for IOVA_VA\n" " -d LIB.so|DIR Add a driver or driver directory\n" lib/librte_eal/common/eal_internal_cfg.h +1
-0@@ -44,6 +44,7 @@ struct internal_config {unsigned hugepage_unlink;/**< true to unlink backing files */ volatile unsigned no_pci;/**< true to disable PCI */ volatile unsigned no_hpet;/**< true to disable HPET */ +volatile unsigned manual_probe;/**< true to enable manual device probing. */ volatile unsigned vmware_tsc_map;/**< true to use VMware TSC mapping * instead of native TSC */ volatile unsigned no_shconf;/**< true if there is no shared config */ lib/librte_eal/common/eal_options.h +2
-0@@ -69,6 +69,8 @@ enum {OPT_IOVA_MODE_NUM , #define OPT_MATCH_ALLOCATIONS "match-allocations"OPT_MATCH_ALLOCATIONS_NUM , +#define OPT_MANUAL_PROBE "manual-probe" +OPT_MANUAL_PROBE_NUM , OPT_LONG_MAX_NUM };lib/librte_eal/common/eal_private.h +9
-0@@ -381,4 +381,13 @@ rte_option_init(void);void rte_option_usage (void ); +/** +* Go through the devargs list and probe everything in order. +* +* @return +* 0 on success, negative on error. +*/ +int +rte_dev_probe_devargs_list (void ); +#endif /* _EAL_PRIVATE_H_ */ lib/librte_eal/common/include/rte_eal.h +17
-0@@ -465,6 +465,23 @@ int rte_eal_has_hugepages(void);int rte_eal_has_pci (void );/** +* Whether EAL probe is manual. +* Enabled by the --manual-probe option. +* +* When manual probing is enabled, batched bus probe of +* their devices is disabled. All devices need to be probed +* using the proper rte_dev API. +* +* In this mode, devices declared on the command line will +* be probed using the bus hotplug API. It is used to enforce +* a specific probe order. +* +* @return +* Nonzero if manual device probing is enabled. +*/ +int rte_eal_manual_probe (void ); + +/** * Whether the EAL was asked to create UIO device. * * @return lib/librte_eal/freebsd/eal/eal.c +5
-0@@ -961,6 +961,11 @@ int rte_eal_has_pci(void)return !internal_config.no_pci; } +int rte_eal_manual_probe (void ) +{ +return internal_config.manual_probe; +} +int rte_eal_create_uio_dev (void ) {return internal_config.create_uio_dev;lib/librte_eal/linux/eal/eal.c +5
-0@@ -1376,6 +1376,11 @@ int rte_eal_create_uio_dev(void)return internal_config.create_uio_dev; } +int rte_eal_manual_probe (void ) +{ +return internal_config.manual_probe; +} +enum rte_intr_mode rte_eal_vfio_intr_mode (void ) {lib/librte_eal/rte_eal_version.map +0
-0
lib/librte_eal/rte_eal_version.map DPDK_19.08 { · line 318
Quoted 7 lines
lib/librte_eal/rte_eal_version.map +4
-0@@ -312,6 +312,13 @@ DPDK_19.08 { } DPDK_19.05; +DPDK_19.05 { + global: + + rte_eal_manual_probe
Syntax error - needs a ';' at the end of this symbol.
Quoted 5 lines
lib/librte_eal/rte_eal_version.map +3
-0+ +} DPDK_19.11; + EXPERIMENTAL { global:
Re: [dpdk-dev] [PATCH v1] eal: add manual probing option
Quoted 279 lines · Aaron Conole
Quoted 277 lines · Gaetan Rivet
Add a new EAL option enabling manual probing in the EAL. This command line option will configure the EAL so that buses will not trigger their probe step on their own.
Applications are then expected to hotplug devices as they see fit.
Devices declared on the command line by the user (using -w and --vdev), will be probed using the hotplug API, in the order they are declared.
This has the effect of offering a way for users to control probe order of their devices, for drivers requiring it.
Signed-off-by Gaetan Rivet <[email protected]>---
I haven't heard many opinions on the matter, please shout if you see an issue with this approach.
@Slava: I have tested rather quickly that it does not break anything,
and that it works as intended for basic cases.
Can you test it further for your use-case and tell me if it works fine?Beyond the obvious difference between both probe mode, something to keep in mind:
while using -w on invalid devices would not block (PCI) bus probing, it will stop manual probing in its track. All devices need to exist and be valid device IDs.doc/guides/rel_notes/release_19_11.rst | 9 +++++++ lib/librte_eal/common/eal_common_bus.c | 6 +++++ lib/librte_eal/common/eal_common_dev.c | 41 ++++++++++++++++++++++++++++++ lib/librte_eal/common/eal_common_options.c | 8 ++++++ lib/librte_eal/common/eal_internal_cfg.h | 1 + lib/librte_eal/common/eal_options.h | 2 ++ lib/librte_eal/common/eal_private.h | 9 +++++++ lib/librte_eal/common/include/rte_eal.h | 17 +++++++++++++ lib/librte_eal/freebsd/eal/eal.c | 5 ++++ lib/librte_eal/linux/eal/eal.c | 5 ++++ lib/librte_eal/rte_eal_version.map | 7 +++++ 11 files changed, 110 insertions(+)doc/guides/rel_notes/release_19_11.rst +9
-0@@ -56,6 +56,15 @@ New Features Also, make sure to start the actual text at the margin. ========================================================= +* **EAL wil now propose to enforce manual probing of devices.** + + Previously, a user could not force an order when probing declared devices. + This could cause issues for drivers depending on another device being present. + A new option ``--manual-probe`` is now available to do just that. + This new option relies on the device bus supporting hotplug. It can + also be used to disable automatic probing from the ``PCI`` bus without + having to disable the whole bus. + Removed Items -------------lib/librte_eal/common/eal_common_bus.c +6
-0@@ -6,6 +6,7 @@#include <string.h> #include <sys/queue.h> +#include <rte_eal.h> #include <rte_bus.h> #include <rte_debug.h> #include <rte_string_fns.h> @@ -63,6 +64,11 @@ rte_bus_probe(void)int ret;struct rte_bus *bus, *vbus =NULL ; +if (rte_eal_manual_probe ()) { +RTE_LOG (DEBUG ,EAL ,"Manual probing enabled.\n" ); +return rte_dev_probe_devargs_list (); + } + TAILQ_FOREACH(bus, &rte_bus_list, next) {if (!strcmp (bus->name,"vdev" )) { vbus = bus;lib/librte_eal/common/eal_common_dev.c +41
-0@@ -109,6 +109,47 @@ build_devargs(const char *busname, const char *devname, }int +rte_dev_probe_devargs_list (void ) +{ +struct rte_device *dev; +struct rte_devargs *da; +int ret; + + RTE_EAL_DEVARGS_FOREACH(NULL, da) { + dev = da->bus->find_device (NULL , cmp_dev_name, da->name); +if (dev ==NULL ) { +RTE_LOG (ERR ,EAL ,"Unable to find device %s on bus %s\n" , + da->name, da->bus->name); +continue ; + } + +if (rte_dev_is_probed (dev)) +continue ; + +if (dev->bus->plug ==NULL ) { +RTE_LOG (ERR ,EAL ,"Manual probing (hotplug) not supported by bus %s, " +"required by device %s\n" , + dev->bus->name, dev->name); +continue ; + } + + ret = dev->bus->plug (dev); +/* Ignore positive return values, they are possibly +* triggered by blacklisted devices on the PCI bus. Probing +* should then continue. +*/ +if (ret <0 || !rte_dev_is_probed (dev)) { +RTE_LOG (ERR ,EAL ,"Driver cannot attach device %s\n" , + dev->name); +/* Fail on first real probe error. */ +return ret; + } + } + +return 0 ; +} + +int rte_eal_hotplug_add (const char *busname,const char *devname,const char *drvargs) {lib/librte_eal/common/eal_common_options.c +8
-0@@ -81,6 +81,7 @@ eal_long_options[] = { {OPT_LEGACY_MEM, 0, NULL, OPT_LEGACY_MEM_NUM }, {OPT_SINGLE_FILE_SEGMENTS, 0, NULL, OPT_SINGLE_FILE_SEGMENTS_NUM}, {OPT_MATCH_ALLOCATIONS, 0, NULL, OPT_MATCH_ALLOCATIONS_NUM}, + {OPT_MANUAL_PROBE, 0, NULL, OPT_MANUAL_PROBE_NUM }, {0, 0, NULL, 0 } };@@ -1408,6 +1409,9 @@ eal_parse_common_option(int opt, const char *optarg,return -1 ; }break ; +case OPT_MANUAL_PROBE_NUM : + conf->manual_probe =1 ; +break ;/* don't know what to do, leave this to caller */ default :@@ -1634,6 +1638,10 @@ eal_common_usage(void)" --" OPT_VDEV " Add a virtual device.\n" " The argument format is <driver><id>[,key=val,...]\n" " (ex: --vdev=net_pcap0,iface=eth2).\n" +" --" OPT_MANUAL_PROBE " Enable manual probing.\n" +" Disable probe step for all buses.\n" +" Devices will need to be probed using the hotplug API.\n" +" PCI and vdev declarations will be treated in order as hotplug commands.\n" " --" OPT_IOVA_MODE " Set IOVA mode. 'pa' for IOVA_PA\n" " 'va' for IOVA_VA\n" " -d LIB.so|DIR Add a driver or driver directory\n" lib/librte_eal/common/eal_internal_cfg.h +1
-0@@ -44,6 +44,7 @@ struct internal_config {unsigned hugepage_unlink;/**< true to unlink backing files */ volatile unsigned no_pci;/**< true to disable PCI */ volatile unsigned no_hpet;/**< true to disable HPET */ +volatile unsigned manual_probe;/**< true to enable manual device probing. */ volatile unsigned vmware_tsc_map;/**< true to use VMware TSC mapping * instead of native TSC */ volatile unsigned no_shconf;/**< true if there is no shared config */ lib/librte_eal/common/eal_options.h +2
-0@@ -69,6 +69,8 @@ enum {OPT_IOVA_MODE_NUM , #define OPT_MATCH_ALLOCATIONS "match-allocations"OPT_MATCH_ALLOCATIONS_NUM , +#define OPT_MANUAL_PROBE "manual-probe" +OPT_MANUAL_PROBE_NUM , OPT_LONG_MAX_NUM };lib/librte_eal/common/eal_private.h +9
-0@@ -381,4 +381,13 @@ rte_option_init(void);void rte_option_usage (void ); +/** +* Go through the devargs list and probe everything in order. +* +* @return +* 0 on success, negative on error. +*/ +int +rte_dev_probe_devargs_list (void ); +#endif /* _EAL_PRIVATE_H_ */ lib/librte_eal/common/include/rte_eal.h +17
-0@@ -465,6 +465,23 @@ int rte_eal_has_hugepages(void);int rte_eal_has_pci (void );/** +* Whether EAL probe is manual. +* Enabled by the --manual-probe option. +* +* When manual probing is enabled, batched bus probe of +* their devices is disabled. All devices need to be probed +* using the proper rte_dev API. +* +* In this mode, devices declared on the command line will +* be probed using the bus hotplug API. It is used to enforce +* a specific probe order. +* +* @return +* Nonzero if manual device probing is enabled. +*/ +int rte_eal_manual_probe (void ); + +/** * Whether the EAL was asked to create UIO device. * * @return lib/librte_eal/freebsd/eal/eal.c +5
-0@@ -961,6 +961,11 @@ int rte_eal_has_pci(void)return !internal_config.no_pci; } +int rte_eal_manual_probe (void ) +{ +return internal_config.manual_probe; +} +int rte_eal_create_uio_dev (void ) {return internal_config.create_uio_dev;lib/librte_eal/linux/eal/eal.c +5
-0@@ -1376,6 +1376,11 @@ int rte_eal_create_uio_dev(void)return internal_config.create_uio_dev; } +int rte_eal_manual_probe (void ) +{ +return internal_config.manual_probe; +} +enum rte_intr_mode rte_eal_vfio_intr_mode (void ) {lib/librte_eal/rte_eal_version.map +0
-0@@ -312,6 +312,13 @@ DPDK_19.08 {
Quoted 9 lines
lib/librte_eal/rte_eal_version.map DPDK_19.08 { · line 318
Quoted 6 lines
lib/librte_eal/rte_eal_version.map +4
-0} DPDK_19.05; +DPDK_19.05 { + global: + + rte_eal_manual_probeSyntax error - needs a ';' at the end of this symbol.
Ah yes, sorry.
I'm thinking there is a script or a validation tool using this, I should try it.
Quoted 5 lines
Quoted 5 lines
lib/librte_eal/rte_eal_version.map +3
-0+ +} DPDK_19.11; + EXPERIMENTAL { global:
Signature · 2 lines
Gaëtan Rivet 6WIND
Re: [dpdk-dev] [PATCH v1] eal: add manual probing option
Quoted 290 lines · Gaëtan Rivet
Quoted 279 lines · Aaron Conole
Quoted 277 lines · Gaetan Rivet
Add a new EAL option enabling manual probing in the EAL. This command line option will configure the EAL so that buses will not trigger their probe step on their own.
Applications are then expected to hotplug devices as they see fit.
Devices declared on the command line by the user (using -w and --vdev), will be probed using the hotplug API, in the order they are declared.
This has the effect of offering a way for users to control probe order of their devices, for drivers requiring it.
Signed-off-by Gaetan Rivet <[email protected]>---
I haven't heard many opinions on the matter, please shout if you see an issue with this approach.
@Slava: I have tested rather quickly that it does not break anything,
and that it works as intended for basic cases.
Can you test it further for your use-case and tell me if it works fine?Beyond the obvious difference between both probe mode, something to keep in mind:
while using -w on invalid devices would not block (PCI) bus probing, it will stop manual probing in its track. All devices need to exist and be valid device IDs.doc/guides/rel_notes/release_19_11.rst | 9 +++++++ lib/librte_eal/common/eal_common_bus.c | 6 +++++ lib/librte_eal/common/eal_common_dev.c | 41 ++++++++++++++++++++++++++++++ lib/librte_eal/common/eal_common_options.c | 8 ++++++ lib/librte_eal/common/eal_internal_cfg.h | 1 + lib/librte_eal/common/eal_options.h | 2 ++ lib/librte_eal/common/eal_private.h | 9 +++++++ lib/librte_eal/common/include/rte_eal.h | 17 +++++++++++++ lib/librte_eal/freebsd/eal/eal.c | 5 ++++ lib/librte_eal/linux/eal/eal.c | 5 ++++ lib/librte_eal/rte_eal_version.map | 7 +++++ 11 files changed, 110 insertions(+)doc/guides/rel_notes/release_19_11.rst +9
-0@@ -56,6 +56,15 @@ New Features Also, make sure to start the actual text at the margin. ========================================================= +* **EAL wil now propose to enforce manual probing of devices.** + + Previously, a user could not force an order when probing declared devices. + This could cause issues for drivers depending on another device being present. + A new option ``--manual-probe`` is now available to do just that. + This new option relies on the device bus supporting hotplug. It can + also be used to disable automatic probing from the ``PCI`` bus without + having to disable the whole bus. + Removed Items -------------lib/librte_eal/common/eal_common_bus.c +6
-0@@ -6,6 +6,7 @@#include <string.h> #include <sys/queue.h> +#include <rte_eal.h> #include <rte_bus.h> #include <rte_debug.h> #include <rte_string_fns.h> @@ -63,6 +64,11 @@ rte_bus_probe(void)int ret;struct rte_bus *bus, *vbus =NULL ; +if (rte_eal_manual_probe ()) { +RTE_LOG (DEBUG ,EAL ,"Manual probing enabled.\n" ); +return rte_dev_probe_devargs_list (); + } + TAILQ_FOREACH(bus, &rte_bus_list, next) {if (!strcmp (bus->name,"vdev" )) { vbus = bus;lib/librte_eal/common/eal_common_dev.c +41
-0@@ -109,6 +109,47 @@ build_devargs(const char *busname, const char *devname, }int +rte_dev_probe_devargs_list (void ) +{ +struct rte_device *dev; +struct rte_devargs *da; +int ret; + + RTE_EAL_DEVARGS_FOREACH(NULL, da) { + dev = da->bus->find_device (NULL , cmp_dev_name, da->name); +if (dev ==NULL ) { +RTE_LOG (ERR ,EAL ,"Unable to find device %s on bus %s\n" , + da->name, da->bus->name); +continue ; + } + +if (rte_dev_is_probed (dev)) +continue ; + +if (dev->bus->plug ==NULL ) { +RTE_LOG (ERR ,EAL ,"Manual probing (hotplug) not supported by bus %s, " +"required by device %s\n" , + dev->bus->name, dev->name); +continue ; + } + + ret = dev->bus->plug (dev); +/* Ignore positive return values, they are possibly +* triggered by blacklisted devices on the PCI bus. Probing +* should then continue. +*/ +if (ret <0 || !rte_dev_is_probed (dev)) { +RTE_LOG (ERR ,EAL ,"Driver cannot attach device %s\n" , + dev->name); +/* Fail on first real probe error. */ +return ret; + } + } + +return 0 ; +} + +int rte_eal_hotplug_add (const char *busname,const char *devname,const char *drvargs) {lib/librte_eal/common/eal_common_options.c +8
-0@@ -81,6 +81,7 @@ eal_long_options[] = { {OPT_LEGACY_MEM, 0, NULL, OPT_LEGACY_MEM_NUM }, {OPT_SINGLE_FILE_SEGMENTS, 0, NULL, OPT_SINGLE_FILE_SEGMENTS_NUM}, {OPT_MATCH_ALLOCATIONS, 0, NULL, OPT_MATCH_ALLOCATIONS_NUM}, + {OPT_MANUAL_PROBE, 0, NULL, OPT_MANUAL_PROBE_NUM }, {0, 0, NULL, 0 } };@@ -1408,6 +1409,9 @@ eal_parse_common_option(int opt, const char *optarg,return -1 ; }break ; +case OPT_MANUAL_PROBE_NUM : + conf->manual_probe =1 ; +break ;/* don't know what to do, leave this to caller */ default :@@ -1634,6 +1638,10 @@ eal_common_usage(void)" --" OPT_VDEV " Add a virtual device.\n" " The argument format is <driver><id>[,key=val,...]\n" " (ex: --vdev=net_pcap0,iface=eth2).\n" +" --" OPT_MANUAL_PROBE " Enable manual probing.\n" +" Disable probe step for all buses.\n" +" Devices will need to be probed using the hotplug API.\n" +" PCI and vdev declarations will be treated in order as hotplug commands.\n" " --" OPT_IOVA_MODE " Set IOVA mode. 'pa' for IOVA_PA\n" " 'va' for IOVA_VA\n" " -d LIB.so|DIR Add a driver or driver directory\n" lib/librte_eal/common/eal_internal_cfg.h +1
-0@@ -44,6 +44,7 @@ struct internal_config {unsigned hugepage_unlink;/**< true to unlink backing files */ volatile unsigned no_pci;/**< true to disable PCI */ volatile unsigned no_hpet;/**< true to disable HPET */ +volatile unsigned manual_probe;/**< true to enable manual device probing. */ volatile unsigned vmware_tsc_map;/**< true to use VMware TSC mapping * instead of native TSC */ volatile unsigned no_shconf;/**< true if there is no shared config */ lib/librte_eal/common/eal_options.h +2
-0@@ -69,6 +69,8 @@ enum {OPT_IOVA_MODE_NUM , #define OPT_MATCH_ALLOCATIONS "match-allocations"OPT_MATCH_ALLOCATIONS_NUM , +#define OPT_MANUAL_PROBE "manual-probe" +OPT_MANUAL_PROBE_NUM , OPT_LONG_MAX_NUM };lib/librte_eal/common/eal_private.h +9
-0@@ -381,4 +381,13 @@ rte_option_init(void);void rte_option_usage (void ); +/** +* Go through the devargs list and probe everything in order. +* +* @return +* 0 on success, negative on error. +*/ +int +rte_dev_probe_devargs_list (void ); +#endif /* _EAL_PRIVATE_H_ */ lib/librte_eal/common/include/rte_eal.h +17
-0@@ -465,6 +465,23 @@ int rte_eal_has_hugepages(void);int rte_eal_has_pci (void );/** +* Whether EAL probe is manual. +* Enabled by the --manual-probe option. +* +* When manual probing is enabled, batched bus probe of +* their devices is disabled. All devices need to be probed +* using the proper rte_dev API. +* +* In this mode, devices declared on the command line will +* be probed using the bus hotplug API. It is used to enforce +* a specific probe order. +* +* @return +* Nonzero if manual device probing is enabled. +*/ +int rte_eal_manual_probe (void ); + +/** * Whether the EAL was asked to create UIO device. * * @return lib/librte_eal/freebsd/eal/eal.c +5
-0@@ -961,6 +961,11 @@ int rte_eal_has_pci(void)return !internal_config.no_pci; } +int rte_eal_manual_probe (void ) +{ +return internal_config.manual_probe; +} +int rte_eal_create_uio_dev (void ) {return internal_config.create_uio_dev;lib/librte_eal/linux/eal/eal.c +5
-0@@ -1376,6 +1376,11 @@ int rte_eal_create_uio_dev(void)return internal_config.create_uio_dev; } +int rte_eal_manual_probe (void ) +{ +return internal_config.manual_probe; +} +enum rte_intr_mode rte_eal_vfio_intr_mode (void ) {lib/librte_eal/rte_eal_version.map +0
-0@@ -312,6 +312,13 @@ DPDK_19.08 {Quoted 9 lines
lib/librte_eal/rte_eal_version.map DPDK_19.08 { · line 318
Quoted 6 lines
lib/librte_eal/rte_eal_version.map +4
-0} DPDK_19.05; +DPDK_19.05 { + global: + + rte_eal_manual_probeSyntax error - needs a ';' at the end of this symbol.
Quoted 2 lines
Ah yes, sorry.
I'm thinking there is a script or a validation tool using this, I should try it.
One method I use is to push to my personal github account, linked to Travis-CI - then I get emails when it fails ;)
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
lib/librte_eal/rte_eal_version.map +3
-0+ +} DPDK_19.11; + EXPERIMENTAL { global:
Re: [dpdk-dev] [PATCH v1] eal: add manual probing option
In reply to Gaetan RivetQuoted 6 lines · On Mon, 30 Sep 2019 14:51:03 +0200 Gaetan Rivet <[email protected]> wrote:
Add a new EAL option enabling manual probing in the EAL. This command line option will configure the EAL so that buses will not trigger their probe step on their own.
Applications are then expected to hotplug devices as they see fit.
Quoted 7 lines
Devices declared on the command line by the user (using -w and --vdev), will be probed using the hotplug API, in the order they are declared.
This has the effect of offering a way for users to control probe order of their devices, for drivers requiring it.
Signed-off-by Gaetan Rivet <[email protected]>
I have no problems with the patch, but it would help if there was better way to handle device naming policy in DPDK. Applications that depend on particular port number are prone to get broken by changes in surrounding OS or hardware environment. Just like Linux applications that are built to depend on "eth0"; which is unfortunately all too common.
Re: [dpdk-dev] [PATCH v1] eal: add manual probing option
Quoted 15 lines · Stephen Hemminger
Quoted 12 lines · On Mon, 30 Sep 2019 14:51:03 +0200 Gaetan Rivet <[email protected]> wrote:
Add a new EAL option enabling manual probing in the EAL. This command line option will configure the EAL so that buses will not trigger their probe step on their own.
Applications are then expected to hotplug devices as they see fit.
Devices declared on the command line by the user (using -w and --vdev), will be probed using the hotplug API, in the order they are declared.
This has the effect of offering a way for users to control probe order of their devices, for drivers requiring it.
Quoted 7 lines
Quoted 1 line
Signed-off-by Gaetan Rivet <[email protected]>I have no problems with the patch, but it would help if there was better way to handle device naming policy in DPDK. Applications that depend on particular port number are prone to get broken by changes in surrounding OS or hardware environment. Just like Linux applications that are built to depend on "eth0"; which is unfortunately all too common.
Hello Stephen,
This patch is a way to avoid having the PCI bus defining the probe order with the current hardware environment. It seems to be a step in the right direction for the issue you identify.
There is a tight coupling between device names and driver matches for the vdev bus, but that seems difficult to avoid.
Do you see other EAL APIs fostering an over reliance of downstream systems on device names?
I pushed a few months back a way to iterate / match devices by their properties. If you identify other pain points, this could certainly be improved as well.
Signature · 2 lines
Gaëtan Rivet 6WIND
Re: [dpdk-dev] [PATCH v1] eal: add manual probing option
Quoted 34 lines · Gaëtan Rivet
Quoted 15 lines · Stephen Hemminger
Quoted 12 lines · On Mon, 30 Sep 2019 14:51:03 +0200 Gaetan Rivet <[email protected]> wrote:
Add a new EAL option enabling manual probing in the EAL. This command line option will configure the EAL so that buses will not trigger their probe step on their own.
Applications are then expected to hotplug devices as they see fit.
Devices declared on the command line by the user (using -w and --vdev), will be probed using the hotplug API, in the order they are declared.
This has the effect of offering a way for users to control probe order of their devices, for drivers requiring it.
Quoted 7 lines
Quoted 1 line
Signed-off-by Gaetan Rivet <[email protected]>I have no problems with the patch, but it would help if there was better way to handle device naming policy in DPDK. Applications that depend on particular port number are prone to get broken by changes in surrounding OS or hardware environment. Just like Linux applications that are built to depend on "eth0"; which is unfortunately all too common.
Hello Stephen,
This patch is a way to avoid having the PCI bus defining the probe order with the current hardware environment. It seems to be a step in the right direction for the issue you identify.
There is a tight coupling between device names and driver matches for the vdev bus, but that seems difficult to avoid.
Quoted 6 lines
Do you see other EAL APIs fostering an over reliance of downstream systems on device names?
I pushed a few months back a way to iterate / match devices by their properties. If you identify other pain points, this could certainly be improved as well.
And this mode will be kicked in only when "--manual-probe" selected on eal arguments.
So it won't change the behavior of the existing applications.
Quoted 4 lines
--
Gaėtan Rivet
6WIND
Re: [dpdk-dev] [PATCH v1] eal: add manual probing option
Quoted 38 lines · Jerin Jacob
Quoted 37 lines · Gaëtan Rivet
Quoted 15 lines · Stephen Hemminger
Quoted 12 lines · On Mon, 30 Sep 2019 14:51:03 +0200 Gaetan Rivet <[email protected]> wrote:
Add a new EAL option enabling manual probing in the EAL. This command line option will configure the EAL so that buses will not trigger their probe step on their own.
Applications are then expected to hotplug devices as they see fit.
Devices declared on the command line by the user (using -w and --vdev), will be probed using the hotplug API, in the order they are declared.
This has the effect of offering a way for users to control probe order of their devices, for drivers requiring it.
Quoted 7 lines
Quoted 1 line
Signed-off-by Gaetan Rivet <[email protected]>I have no problems with the patch, but it would help if there was better way to handle device naming policy in DPDK. Applications that depend on particular port number are prone to get broken by changes in surrounding OS or hardware environment. Just like Linux applications that are built to depend on "eth0"; which is unfortunately all too common.
Hello Stephen,
This patch is a way to avoid having the PCI bus defining the probe order with the current hardware environment. It seems to be a step in the right direction for the issue you identify.
There is a tight coupling between device names and driver matches for the vdev bus, but that seems difficult to avoid.
Do you see other EAL APIs fostering an over reliance of downstream systems on device names?
Quoted 8 lines
Quoted 3 lines
I pushed a few months back a way to iterate / match devices by their properties. If you identify other pain points, this could certainly be improved as well.
And this mode will be kicked in only when "--manual-probe" selected on eal arguments.
So it won't change the behavior of the existing applications.
If I read you correctly, if hardware independence is the proper way to function, we should switch entirely to it.
I agree, but that means rewriting entirely the probe step of rte_bus. This patch is a incremental step, I preferred to keep risks low.
It is not clear from your remark whether you are considering this limitation a good or a bad thing however :)
Can you be a bit more explicit?
Signature · 2 lines
Gaëtan Rivet 6WIND
Re: [dpdk-dev] [PATCH v1] eal: add manual probing option
Quoted 49 lines · Gaëtan Rivet
Quoted 38 lines · Jerin Jacob
Quoted 37 lines · Gaėtan Rivet
Quoted 15 lines · Stephen Hemminger
Quoted 12 lines · On Mon, 30 Sep 2019 14:51:03 +0200 Gaetan Rivet <[email protected]> wrote:
Add a new EAL option enabling manual probing in the EAL. This command line option will configure the EAL so that buses will not trigger their probe step on their own.
Applications are then expected to hotplug devices as they see fit.
Devices declared on the command line by the user (using -w and --vdev), will be probed using the hotplug API, in the order they are declared.
This has the effect of offering a way for users to control probe order of their devices, for drivers requiring it.
Quoted 7 lines
Quoted 1 line
Signed-off-by Gaetan Rivet <[email protected]>I have no problems with the patch, but it would help if there was better way to handle device naming policy in DPDK. Applications that depend on particular port number are prone to get broken by changes in surrounding OS or hardware environment. Just like Linux applications that are built to depend on "eth0"; which is unfortunately all too common.
Hello Stephen,
This patch is a way to avoid having the PCI bus defining the probe order with the current hardware environment. It seems to be a step in the right direction for the issue you identify.
There is a tight coupling between device names and driver matches for the vdev bus, but that seems difficult to avoid.
Do you see other EAL APIs fostering an over reliance of downstream systems on device names?
Quoted 8 lines
Quoted 3 lines
I pushed a few months back a way to iterate / match devices by their properties. If you identify other pain points, this could certainly be improved as well.
And this mode will be kicked in only when "--manual-probe" selected on eal arguments.
So it won't change the behavior of the existing applications.
Quoted 5 lines
If I read you correctly, if hardware independence is the proper way to function, we should switch entirely to it.
I agree, but that means rewriting entirely the probe step of
rte_bus. This patch is a incremental step, I preferred to keep risks low.
+1
Quoted 3 lines
It is not clear from your remark whether you are considering this limitation a good or a bad thing however :)
I am considering it as a good step.
Quoted 5 lines
Can you be a bit more explicit?
--
Gaėtan Rivet
6WIND
PATCH v2 1/1 eal: add manual probing option
In reply to Gaetan RivetAdd a new EAL option enabling manual probing in the EAL. This command line option will configure the EAL so that buses will not trigger their probe step on their own.
Applications are then expected to hotplug devices as they see fit.
Devices declared on the command line by the user (using -w and --vdev), will be probed using the hotplug API, in the order they are declared.
This has the effect of offering a way for users to control probe order of their devices, for drivers requiring it.
---
I haven't heard many opinions on the matter, please shout if you see an issue with this approach.
@Slava: I have tested rather quickly that it does not break anything,
and that it works as intended for basic cases.
Can you test it further for your use-case and tell me if it works fine?
Beyond the obvious difference between both probe mode, something to keep in mind:
while using -w on invalid devices would not block (PCI) bus probing, it will stop manual probing in its track. All devices need to exist and be valid device IDs.
v2: fixed a few typos, map file (and used Travis to validate).
Slava, are you able to test this patch?
doc/guides/rel_notes/release_19_11.rst | 9 +++++++ lib/librte_eal/common/eal_common_bus.c | 6 +++++ lib/librte_eal/common/eal_common_dev.c | 41 ++++++++++++++++++++++++++++++ lib/librte_eal/common/eal_common_options.c | 8 ++++++ lib/librte_eal/common/eal_internal_cfg.h | 1 + lib/librte_eal/common/eal_options.h | 2 ++ lib/librte_eal/common/eal_private.h | 9 +++++++ lib/librte_eal/common/include/rte_eal.h | 17 +++++++++++++ lib/librte_eal/freebsd/eal/eal.c | 5 ++++ lib/librte_eal/linux/eal/eal.c | 5 ++++ lib/librte_eal/rte_eal_version.map | 7 +++++ 11 files changed, 110 insertions(+)
doc/guides/rel_notes/release_19_11.rst +9 -0
@@ -56,6 +56,15 @@ New Features Also, make sure to start the actual text at the margin. ========================================================= +* **EAL will now propose to enforce manual probing of devices.** + + Previously, a user could not force an order when probing declared devices. + This could cause issues for drivers depending on another device being present. + A new option ``--manual-probe`` is now available to do just that. + This new option relies on the device bus supporting hotplug. It can + also be used to disable automatic probing from the ``PCI`` bus without + having to disable the whole bus. + Removed Items -------------
lib/librte_eal/common/eal_common_bus.c +6 -0
@@ -6,6 +6,7 @@#include <string.h> #include <sys/queue.h> +#include <rte_eal.h> #include <rte_bus.h> #include <rte_debug.h> #include <rte_string_fns.h>
@@ -63,6 +64,11 @@ rte_bus_probe(void)int ret;struct rte_bus *bus, *vbus =NULL ; +if (rte_eal_manual_probe ()) { +RTE_LOG (DEBUG ,EAL ,"Manual probing enabled.\n" ); +return rte_dev_probe_devargs_list (); + } + TAILQ_FOREACH(bus, &rte_bus_list, next) {if (!strcmp (bus->name,"vdev" )) { vbus = bus;
lib/librte_eal/common/eal_common_dev.c +41 -0
47 lines not on this page · raw message
lib/librte_eal/common/eal_common_options.c +8 -0
@@ -81,6 +81,7 @@ eal_long_options[] = { {OPT_LEGACY_MEM, 0, NULL, OPT_LEGACY_MEM_NUM }, {OPT_SINGLE_FILE_SEGMENTS, 0, NULL, OPT_SINGLE_FILE_SEGMENTS_NUM}, {OPT_MATCH_ALLOCATIONS, 0, NULL, OPT_MATCH_ALLOCATIONS_NUM}, + {OPT_MANUAL_PROBE, 0, NULL, OPT_MANUAL_PROBE_NUM }, {0, 0, NULL, 0 } };
@@ -1408,6 +1409,9 @@ eal_parse_common_option(int opt, const char *optarg,return -1 ; }break ; +case OPT_MANUAL_PROBE_NUM : + conf->manual_probe =1 ; +break ;/* don't know what to do, leave this to caller */ default :
@@ -1634,6 +1638,10 @@ eal_common_usage(void)" --" OPT_VDEV " Add a virtual device.\n" " The argument format is <driver><id>[,key=val,...]\n" " (ex: --vdev=net_pcap0,iface=eth2).\n" +" --" OPT_MANUAL_PROBE " Enable manual probing.\n" +" Disable probe step for all buses.\n" +" Devices will need to be probed using the hotplug API.\n" +" PCI and vdev declarations will be treated in order as hotplug commands.\n" " --" OPT_IOVA_MODE " Set IOVA mode. 'pa' for IOVA_PA\n" " 'va' for IOVA_VA\n" " -d LIB.so|DIR Add a driver or driver directory\n"
lib/librte_eal/common/eal_internal_cfg.h +1 -0
@@ -44,6 +44,7 @@ struct internal_config {unsigned hugepage_unlink;/**< true to unlink backing files */ volatile unsigned no_pci;/**< true to disable PCI */ volatile unsigned no_hpet;/**< true to disable HPET */ +volatile unsigned manual_probe;/**< true to enable manual device probing. */ volatile unsigned vmware_tsc_map;/**< true to use VMware TSC mapping * instead of native TSC */ volatile unsigned no_shconf;/**< true if there is no shared config */
lib/librte_eal/common/eal_options.h +2 -0
@@ -69,6 +69,8 @@ enum {OPT_IOVA_MODE_NUM , #define OPT_MATCH_ALLOCATIONS "match-allocations"OPT_MATCH_ALLOCATIONS_NUM , +#define OPT_MANUAL_PROBE "manual-probe" +OPT_MANUAL_PROBE_NUM , OPT_LONG_MAX_NUM };
lib/librte_eal/common/eal_private.h +9 -0
@@ -381,4 +381,13 @@ rte_option_init(void);void rte_option_usage (void ); +/** +* Go through the devargs list and probe everything in order. +* +* @return +* 0 on success, negative on error. +*/ +int +rte_dev_probe_devargs_list (void ); +#endif /* _EAL_PRIVATE_H_ */
lib/librte_eal/common/include/rte_eal.h +17 -0
@@ -465,6 +465,23 @@ int rte_eal_has_hugepages(void);int rte_eal_has_pci (void );/** +* Whether EAL probe is manual. +* Enabled by the --manual-probe option. +* +* When manual probing is enabled, batched bus probe of +* their devices is disabled. All devices need to be probed +* using the proper rte_dev API. +* +* In this mode, devices declared on the command line will +* be probed using the bus hotplug API. It is used to enforce +* a specific probe order. +* +* @return +* Nonzero if manual device probing is enabled. +*/ +int rte_eal_manual_probe (void ); + +/** * Whether the EAL was asked to create UIO device. * * @return
lib/librte_eal/freebsd/eal/eal.c +5 -0
@@ -961,6 +961,11 @@ int rte_eal_has_pci(void)return !internal_config.no_pci; } +int rte_eal_manual_probe (void ) +{ +return internal_config.manual_probe; +} +int rte_eal_create_uio_dev (void ) {return internal_config.create_uio_dev;
lib/librte_eal/linux/eal/eal.c +5 -0
@@ -1376,6 +1376,11 @@ int rte_eal_create_uio_dev(void)return internal_config.create_uio_dev; } +int rte_eal_manual_probe (void ) +{ +return internal_config.manual_probe; +} +enum rte_intr_mode rte_eal_vfio_intr_mode (void ) {
lib/librte_eal/rte_eal_version.map +7 -0
@@ -312,6 +312,13 @@ DPDK_19.08 { } DPDK_19.05; +DPDK_19.11 { + global: + + rte_eal_manual_probe; + +} DPDK_19.05; + EXPERIMENTAL { global:
Signature · 1 line
2.11.0
PATCH v3 1/1 eal: add manual probing option
Add a new EAL option enabling manual probing in the EAL. This command line option will configure the EAL so that buses will not trigger their probe step on their own.
Applications are then expected to hotplug devices as they see fit.
Devices declared on the command line by the user (using -w and --vdev), will be probed using the hotplug API, in the order they are declared.
This has the effect of offering a way for users to control probe order of their devices, for drivers requiring it.
---
I haven't heard many opinions on the matter, please shout if you see an issue with this approach.
@Slava: I have tested rather quickly that it does not break anything,
and that it works as intended for basic cases.
Can you test it further for your use-case and tell me if it works fine?
Beyond the obvious difference between both probe mode, something to keep in mind:
while using -w on invalid devices would not block (PCI) bus probing, it will stop manual probing in its track. All devices need to exist and be valid device IDs.
v2: fixed a few typos, map file (and used Travis to validate).
Slava, are you able to test this patch?
v3: properly fixed the map file (herited 19.08 instead of 19.05).
Added a function to set the probe manual from the application, without having the user do it from the command line.
Stopped spamming Slava about it, Vamsi was actually the one interested in it!
Standing issue worth chiming in:
Currently manual-probe will cut off probing from all buses. It could be interesting to be able to only cut buses supporting hotplug, given that they are the one able to probe devices afterward.
No real use-case for this right now, so leaving as-is. Might be worth considering in the future.
doc/guides/rel_notes/release_19_11.rst | 9 +++++++ lib/librte_eal/common/eal_common_bus.c | 6 +++++ lib/librte_eal/common/eal_common_dev.c | 41 ++++++++++++++++++++++++++++++ lib/librte_eal/common/eal_common_options.c | 8 ++++++ lib/librte_eal/common/eal_internal_cfg.h | 1 + lib/librte_eal/common/eal_options.h | 2 ++ lib/librte_eal/common/eal_private.h | 9 +++++++ lib/librte_eal/common/include/rte_eal.h | 34 +++++++++++++++++++++++++ lib/librte_eal/freebsd/eal/eal.c | 10 ++++++++ lib/librte_eal/linux/eal/eal.c | 10 ++++++++ lib/librte_eal/rte_eal_version.map | 8 ++++++ 11 files changed, 138 insertions(+)
doc/guides/rel_notes/release_19_11.rst +9 -0
@@ -56,6 +56,15 @@ New Features Also, make sure to start the actual text at the margin. ========================================================= +* **EAL will now allow manual probing devices.** + + Previously, a user could not force an order when probing declared devices. + This could cause issues for drivers depending on another device being present. + A new option ``--manual-probe`` is now available to do just that. + This new option relies on the device bus supporting hotplug. It can + also be used to disable automatic probing from the ``PCI`` bus without + having to disable the whole bus. + Removed Items -------------
lib/librte_eal/common/eal_common_bus.c +6 -0
@@ -6,6 +6,7 @@#include <string.h> #include <sys/queue.h> +#include <rte_eal.h> #include <rte_bus.h> #include <rte_debug.h> #include <rte_string_fns.h>
@@ -63,6 +64,11 @@ rte_bus_probe(void)int ret;struct rte_bus *bus, *vbus =NULL ; +if (rte_eal_manual_probe ()) { +RTE_LOG (DEBUG ,EAL ,"Manual probing enabled.\n" ); +return rte_dev_probe_devargs_list (); + } + TAILQ_FOREACH(bus, &rte_bus_list, next) {if (!strcmp (bus->name,"vdev" )) { vbus = bus;
lib/librte_eal/common/eal_common_dev.c +41 -0
47 lines not on this page · raw message
lib/librte_eal/common/eal_common_options.c +8 -0
@@ -81,6 +81,7 @@ eal_long_options[] = { {OPT_LEGACY_MEM, 0, NULL, OPT_LEGACY_MEM_NUM }, {OPT_SINGLE_FILE_SEGMENTS, 0, NULL, OPT_SINGLE_FILE_SEGMENTS_NUM}, {OPT_MATCH_ALLOCATIONS, 0, NULL, OPT_MATCH_ALLOCATIONS_NUM}, + {OPT_MANUAL_PROBE, 0, NULL, OPT_MANUAL_PROBE_NUM }, {0, 0, NULL, 0 } };
@@ -1408,6 +1409,9 @@ eal_parse_common_option(int opt, const char *optarg,return -1 ; }break ; +case OPT_MANUAL_PROBE_NUM : + conf->manual_probe =1 ; +break ;/* don't know what to do, leave this to caller */ default :
@@ -1634,6 +1638,10 @@ eal_common_usage(void)" --" OPT_VDEV " Add a virtual device.\n" " The argument format is <driver><id>[,key=val,...]\n" " (ex: --vdev=net_pcap0,iface=eth2).\n" +" --" OPT_MANUAL_PROBE " Enable manual probing.\n" +" Disable probe step for all buses.\n" +" Devices will need to be probed using the hotplug API.\n" +" PCI and vdev declarations will be treated in order as hotplug commands.\n" " --" OPT_IOVA_MODE " Set IOVA mode. 'pa' for IOVA_PA\n" " 'va' for IOVA_VA\n" " -d LIB.so|DIR Add a driver or driver directory\n"
lib/librte_eal/common/eal_internal_cfg.h +1 -0
@@ -44,6 +44,7 @@ struct internal_config {unsigned hugepage_unlink;/**< true to unlink backing files */ volatile unsigned no_pci;/**< true to disable PCI */ volatile unsigned no_hpet;/**< true to disable HPET */ +volatile unsigned manual_probe;/**< true to enable manual device probing. */ volatile unsigned vmware_tsc_map;/**< true to use VMware TSC mapping * instead of native TSC */ volatile unsigned no_shconf;/**< true if there is no shared config */
lib/librte_eal/common/eal_options.h +2 -0
@@ -69,6 +69,8 @@ enum {OPT_IOVA_MODE_NUM , #define OPT_MATCH_ALLOCATIONS "match-allocations"OPT_MATCH_ALLOCATIONS_NUM , +#define OPT_MANUAL_PROBE "manual-probe" +OPT_MANUAL_PROBE_NUM , OPT_LONG_MAX_NUM };
lib/librte_eal/common/eal_private.h +9 -0
@@ -381,4 +381,13 @@ rte_option_init(void);void rte_option_usage (void ); +/** +* Go through the devargs list and probe everything in order. +* +* @return +* 0 on success, negative on error. +*/ +int +rte_dev_probe_devargs_list (void ); +#endif /* _EAL_PRIVATE_H_ */
lib/librte_eal/common/include/rte_eal.h +34 -0
@@ -465,6 +465,40 @@ int rte_eal_has_hugepages(void);int rte_eal_has_pci (void );/** +* Whether EAL probe is manual. +* Enabled by the --manual-probe option or by +* using rte_eal_manual_probe_set(). +* +* When manual probing is enabled, batched bus probe of +* their devices is disabled. All devices need to be probed +* using the proper rte_dev API. +* +* In this mode, devices declared on the command line will +* be probed using the bus hotplug API. It is used to enforce +* a specific probe order. +* +* @return +* Nonzero if manual device probing is enabled. +* +* @see rte_eal_manual_probe_set +*/ +int rte_eal_manual_probe (void ); + +/** +* Configure EAL probe mode -- manual or automatic. +* +* Enable or disable manual probe mode in EAL. +* This function can be called at any time, but must be used +* before calling rte_eal_init() to have any effect. +* +* @param enabled +* zero to disable manual probe, non-zero to enable it. +* +* @see rte_eal_manual_probe +*/ +void rte_eal_manual_probe_set (int enabled); + +/** * Whether the EAL was asked to create UIO device. * * @return
lib/librte_eal/freebsd/eal/eal.c +10 -0
@@ -972,6 +972,16 @@ rte_eal_vfio_intr_mode(void)return RTE_INTR_MODE_NONE ; } +int rte_eal_manual_probe (void ) +{ +return internal_config.manual_probe; +} + +void rte_eal_manual_probe_set (int enabled) +{ + internal_config.manual_probe = !!enabled; +} + int rte_vfio_setup_device(__rte_unused const char *sysfs_base, __rte_unused const char *dev_addr, __rte_unused int *vfio_dev_fd,
lib/librte_eal/linux/eal/eal.c +10 -0
@@ -1382,6 +1382,16 @@ rte_eal_vfio_intr_mode(void)return internal_config.vfio_intr_mode; } +int rte_eal_manual_probe (void ) +{ +return internal_config.manual_probe; +} + +void rte_eal_manual_probe_set (int enabled) +{ + internal_config.manual_probe = !!enabled; +} +int rte_eal_check_module (const char *module_name) {
lib/librte_eal/rte_eal_version.map +8 -0
@@ -312,6 +312,14 @@ DPDK_19.08 { } DPDK_19.05; +DPDK_19.11 { + global: + + rte_eal_manual_probe; + rte_eal_manual_probe_set; + +} DPDK_19.08; + EXPERIMENTAL { global:
Signature · 1 line
2.11.0
Re: [dpdk-dev] [PATCH v3] eal: add manual probing option
Quoted 44 lines
-----Original Message-----
From: dev <[email protected]> On Behalf Of Gaetan Rivet
Sent: Friday, October 4, 2019 6:25 PM
To: [email protected]; Vamsi Krishna Attunuru <[email protected]>
Cc: Gaetan Rivet <[email protected]>; Ferruh Yigit <[email protected]>; Anatoly Burakov <[email protected]>; Thomas Monjalon <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>; David Marchand <[email protected]>
Subject: [dpdk-dev] [PATCH v3] eal: add manual probing optionAdd a new EAL option enabling manual probing in the EAL.
This command line option will configure the EAL so that buses will not trigger their probe step on their own.Applications are then expected to hotplug devices as they see fit.
Devices declared on the command line by the user (using -w and --vdev), will be probed using the hotplug API, in the order they are declared.
This has the effect of offering a way for users to control probe order of their devices, for drivers requiring it.
Signed-off-by Gaetan Rivet <[email protected]>---
I haven't heard many opinions on the matter, please shout if you see an issue with this approach.
@Slava: I have tested rather quickly that it does not break anything,
and that it works as intended for basic cases.
Can you test it further for your use-case and tell me if it works fine?Beyond the obvious difference between both probe mode, something to keep in mind:
while using -w on invalid devices would not block (PCI) bus probing, it will stop manual probing in its track. All devices need to exist and be valid device IDs.v2: fixed a few typos, map file (and used Travis to validate).
Slava, are you able to test this patch?
Quoted 7 lines
v3: properly fixed the map file (herited 19.08 instead of 19.05).
Added a function to set the probe manual from the application, without having the user do it from the command line.
Stopped spamming Slava about it, Vamsi was actually the one interested in
it!
@Gaetan: Thanks a lot for providing this patch. I have tested it in our use cases and it works as expected, also verified multiple combinations by passing whitelisted pdevs, vdevs and mix of both. Probing sequence happens as required.
Quoted 319 lines
Standing issue worth chiming in:
Currently manual-probe will cut off probing from all buses. It could be interesting to be able to only cut buses supporting hotplug, given that they are the one able to probe devices afterward.
No real use-case for this right now, so leaving as-is. Might be worth considering in the future.
doc/guides/rel_notes/release_19_11.rst | 9 +++++++ lib/librte_eal/common/eal_common_bus.c | 6 +++++ lib/librte_eal/common/eal_common_dev.c | 41 ++++++++++++++++++++++++++++++ lib/librte_eal/common/eal_common_options.c | 8 ++++++ lib/librte_eal/common/eal_internal_cfg.h | 1 + lib/librte_eal/common/eal_options.h | 2 ++ lib/librte_eal/common/eal_private.h | 9 +++++++ lib/librte_eal/common/include/rte_eal.h | 34 +++++++++++++++++++++++++ lib/librte_eal/freebsd/eal/eal.c | 10 ++++++++ lib/librte_eal/linux/eal/eal.c | 10 ++++++++ lib/librte_eal/rte_eal_version.map | 8 ++++++ 11 files changed, 138 insertions(+)doc/guides/rel_notes/release_19_11.rst +4
-0@@ -56,6 +56,15 @@ New Features Also, make sure to start the actual text at the margin. ========================================================= +* **EAL will now allow manual probing devices.** + + Previously, a user could not force an order when probing declared devices. + This could cause issues for drivers depending on another device beingpresent.
doc/guides/rel_notes/release_19_11.rst +5
-0+ A new option ``--manual-probe`` is now available to do just that. + This new option relies on the device bus supporting hotplug. It can + also be used to disable automatic probing from the ``PCI`` bus without + having to disable the whole bus. + Removed Items -------------lib/librte_eal/common/eal_common_bus.c +6
-0@@ -6,6 +6,7 @@#include <string.h> #include <sys/queue.h> +#include <rte_eal.h> #include <rte_bus.h> #include <rte_debug.h> #include <rte_string_fns.h> @@ -63,6 +64,11 @@ rte_bus_probe(void)int ret;struct rte_bus *bus, *vbus =NULL ; +if (rte_eal_manual_probe ()) { +RTE_LOG (DEBUG ,EAL ,"Manual probing enabled.\n" ); +return rte_dev_probe_devargs_list (); + } + TAILQ_FOREACH(bus, &rte_bus_list, next) {if (!strcmp (bus->name,"vdev" )) { vbus = bus;lib/librte_eal/common/eal_common_dev.c +19
-0@@ -109,6 +109,47 @@ build_devargs(const char *busname, const char *devname, }int +rte_dev_probe_devargs_list (void ) +{ +struct rte_device *dev; +struct rte_devargs *da; +int ret; + + RTE_EAL_DEVARGS_FOREACH(NULL, da) { + dev = da->bus->find_device (NULL , cmp_dev_name, da- >name); +if (dev ==NULL ) { + RTE_LOG(ERR, EAL, "Unable to find device %s on bus %s\n", + da->name, da->bus->name); +continue ; + } + +if (rte_dev_is_probed (dev)) +continue ; + +if (dev->bus->plug ==NULL ) { + RTE_LOG(ERR, EAL, "Manual probing (hotplug) notsupported by bus %s, "
lib/librte_eal/common/eal_common_dev.c +12
-0+"required by device %s\n" , + dev->bus->name, dev->name); +continue ; + } + + ret = dev->bus->plug (dev); +/* Ignore positive return values, they are possibly +* triggered by blacklisted devices on the PCI bus. Probing +* should then continue. +*/ +if (ret <0 ) { + RTE_LOG(ERR, EAL, "Driver cannot attach device%s\n",
lib/librte_eal/common/eal_common_dev.c +10
-0+ dev->name); +/* Fail on first real probe error. */ +return ret; + } + } + +return 0 ; +} + +int rte_eal_hotplug_add (const char *busname,const char *devname,const char *drvargs) {lib/librte_eal/common/eal_common_options.c +1
-0@@ -81,6 +81,7 @@ eal_long_options[] = { {OPT_LEGACY_MEM, 0, NULL, OPT_LEGACY_MEM_NUM }, {OPT_SINGLE_FILE_SEGMENTS, 0, NULL, OPT_SINGLE_FILE_SEGMENTS_NUM}, {OPT_MATCH_ALLOCATIONS, 0, NULL, OPT_MATCH_ALLOCATIONS_NUM}, + {OPT_MANUAL_PROBE, 0, NULL, OPT_MANUAL_PROBE_NUM }, {0, 0, NULL, 0 }};
lib/librte_eal/common/eal_common_options.c +3
-0@@ -1408,6 +1409,9 @@ eal_parse_common_option(int opt, const char *optarg, return -1 ; }break ; +case OPT_MANUAL_PROBE_NUM : + conf->manual_probe =1 ; +break ;/* don't know what to do, leave this to caller */ lib/librte_eal/common/eal_common_options.c +3
-0default :@@ -1634,6 +1638,10 @@ eal_common_usage(void)" --" OPT_VDEV " Add a virtual device.\n" " The argument format is <driver><id>[,key=val,...]\n" " (ex: --vdev=net_pcap0,iface=eth2).\n" +" --" OPT_MANUAL_PROBE " Enable manual probing.\n" +" Disable probe step for all buses.\n" + " Devices will need to be probed using the hotplugAPI.\n"
lib/librte_eal/common/eal_common_options.c +1
-0+ " PCI and vdev declarations will be treated in order ashotplug commands.\n"
lib/librte_eal/common/eal_common_options.c +0
-0" --" OPT_IOVA_MODE " Set IOVA mode. 'pa' for IOVA_PA\n" " 'va' for IOVA_VA\n" " -d LIB.so|DIR Add a driver or driver directory\n" lib/librte_eal/common/eal_internal_cfg.h +1
-0@@ -44,6 +44,7 @@ struct internal_config {unsigned hugepage_unlink;/**< true to unlink backing files */ volatile unsigned no_pci;/**< true to disable PCI */ volatile unsigned no_hpet;/**< true to disable HPET */ +volatile unsigned manual_probe;/**< true to enable manual device probing. */
volatile unsignedvmware_tsc_map; /**< true to use VMware TSC
mapping* instead of native TSC */
volatile unsignedno_shconf; /**< true if there is no shared config
*/lib/librte_eal/common/eal_options.h +2
-0@@ -69,6 +69,8 @@ enum {OPT_IOVA_MODE_NUM , #define OPT_MATCH_ALLOCATIONS "match-allocations"OPT_MATCH_ALLOCATIONS_NUM , +#define OPT_MANUAL_PROBE "manual-probe" +OPT_MANUAL_PROBE_NUM , OPT_LONG_MAX_NUM };lib/librte_eal/common/eal_private.h +9
-0@@ -381,4 +381,13 @@ rte_option_init(void); void rte_option_usage(void); +/** +* Go through the devargs list and probe everything in order. +* +* @return +* 0 on success, negative on error. +*/ +int +rte_dev_probe_devargs_list (void ); +#endif /* _EAL_PRIVATE_H_ */ lib/librte_eal/common/include/rte_eal.h +34
-0@@ -465,6 +465,40 @@ int rte_eal_has_hugepages(void); intrte_eal_has_pci (void);/** +* Whether EAL probe is manual. +* Enabled by the --manual-probe option or by +* using rte_eal_manual_probe_set(). +* +* When manual probing is enabled, batched bus probe of +* their devices is disabled. All devices need to be probed +* using the proper rte_dev API. +* +* In this mode, devices declared on the command line will +* be probed using the bus hotplug API. It is used to enforce +* a specific probe order. +* +* @return +* Nonzero if manual device probing is enabled. +* +* @see rte_eal_manual_probe_set +*/ +int rte_eal_manual_probe (void ); + +/** +* Configure EAL probe mode -- manual or automatic. +* +* Enable or disable manual probe mode in EAL. +* This function can be called at any time, but must be used +* before calling rte_eal_init() to have any effect. +* +* @param enabled +* zero to disable manual probe, non-zero to enable it. +* +* @see rte_eal_manual_probe +*/ +void rte_eal_manual_probe_set (int enabled); + +/** * Whether the EAL was asked to create UIO device. * * @return lib/librte_eal/freebsd/eal/eal.c +8
-0@@ -972,6 +972,16 @@ rte_eal_vfio_intr_mode(void)return RTE_INTR_MODE_NONE ; } +int rte_eal_manual_probe (void ) +{ +return internal_config.manual_probe; +} + +void rte_eal_manual_probe_set (int enabled) { + internal_config.manual_probe = !!enabled; } + int rte_vfio_setup_device(__rte_unused const char *sysfs_base, __rte_unused const char *dev_addr, __rte_unused int *vfio_dev_fd,lib/librte_eal/linux/eal/eal.c index renamed from lib/librte_eal/linux/eal/eal.c +0
-0
946222ccd..da00eb14d100644lib/librte_eal/linux/eal/eal.c +8
-0@@ -1382,6 +1382,16 @@ rte_eal_vfio_intr_mode(void)return internal_config.vfio_intr_mode; } +int rte_eal_manual_probe (void ) +{ +return internal_config.manual_probe; +} + +void rte_eal_manual_probe_set (int enabled) { + internal_config.manual_probe = !!enabled; } +int rte_eal_check_module(const char *module_name) { diff --gita/lib/librte_eal/rte_eal_version.map b/lib/librte_eal/rte_eal_version.map index
7cbf82d37..ccc4ffb21100644lib/librte_eal/rte_eal_version.map +8
-1@@ -312,6 +312,14 @@ DPDK_19.08 { } DPDK_19.05; +DPDK_19.11 { + global: + + rte_eal_manual_probe; + rte_eal_manual_probe_set; + +} DPDK_19.08; + EXPERIMENTAL { global: --2.11.0
Re: [dpdk-dev] [PATCH v3] eal: add manual probing option
Hi,
Quoted 52 lines · Vamsi Krishna Attunuru
Quoted 44 lines
-----Original Message-----
From: dev <[email protected]> On Behalf Of Gaetan Rivet
Sent: Friday, October 4, 2019 6:25 PM
To: [email protected]; Vamsi Krishna Attunuru <[email protected]>
Cc: Gaetan Rivet <[email protected]>; Ferruh Yigit <[email protected]>; Anatoly Burakov <[email protected]>; Thomas Monjalon <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>; David Marchand <[email protected]>
Subject: [dpdk-dev] [PATCH v3] eal: add manual probing optionAdd a new EAL option enabling manual probing in the EAL.
This command line option will configure the EAL so that buses will not trigger their probe step on their own.Applications are then expected to hotplug devices as they see fit.
Devices declared on the command line by the user (using -w and --vdev), will be probed using the hotplug API, in the order they are declared.
This has the effect of offering a way for users to control probe order of their devices, for drivers requiring it.
Signed-off-by Gaetan Rivet <[email protected]>---
I haven't heard many opinions on the matter, please shout if you see an issue with this approach.
@Slava: I have tested rather quickly that it does not break anything,
and that it works as intended for basic cases.
Can you test it further for your use-case and tell me if it works fine?Beyond the obvious difference between both probe mode, something to keep in mind:
while using -w on invalid devices would not block (PCI) bus probing, it will stop manual probing in its track. All devices need to exist and be valid device IDs.v2: fixed a few typos, map file (and used Travis to validate).
Slava, are you able to test this patch?
Quoted 7 lines
v3: properly fixed the map file (herited 19.08 instead of 19.05).
Added a function to set the probe manual from the application, without having the user do it from the command line.
Stopped spamming Slava about it, Vamsi was actually the one interested in
it!
Quoted 17 lines
@Gaetan: Thanks a lot for providing this patch. I have tested it in our use cases and it works as expected, also verified multiple combinations by passing whitelisted pdevs, vdevs and mix of both. Probing sequence happens as required.
Acked-by Vamsi Attunuru < [email protected] >Tested-by Vamsi Attunuru < [email protected] >Quoted 10 lines
Standing issue worth chiming in:
Currently manual-probe will cut off probing from all buses. It could be interesting to be able to only cut buses supporting hotplug, given that they are the one able to probe devices afterward.
No real use-case for this right now, so leaving as-is. Might be worth considering in the future.
Any further comment for this patch? Can someone review it?
Signature · 2 lines
Gaëtan Rivet 6WIND
Re: [dpdk-dev] [PATCH v3] eal: add manual probing option
In reply to Gaetan RivetQuoted 93 lines · Gaetan Rivet
Add a new EAL option enabling manual probing in the EAL. This command line option will configure the EAL so that buses will not trigger their probe step on their own.
Applications are then expected to hotplug devices as they see fit.
Devices declared on the command line by the user (using -w and --vdev), will be probed using the hotplug API, in the order they are declared.
This has the effect of offering a way for users to control probe order of their devices, for drivers requiring it.
Signed-off-by Gaetan Rivet <[email protected]>---
I haven't heard many opinions on the matter, please shout if you see an issue with this approach.
@Slava: I have tested rather quickly that it does not break anything,
and that it works as intended for basic cases.
Can you test it further for your use-case and tell me if it works fine?Beyond the obvious difference between both probe mode, something to keep in mind:
while using -w on invalid devices would not block (PCI) bus probing, it will stop manual probing in its track. All devices need to exist and be valid device IDs.v2: fixed a few typos, map file (and used Travis to validate).
Slava, are you able to test this patch?
v3: properly fixed the map file (herited 19.08 instead of 19.05).
Added a function to set the probe manual from the application, without having the user do it from the command line.
Stopped spamming Slava about it, Vamsi was actually the one interested in it!
Standing issue worth chiming in:
Currently manual-probe will cut off probing from all buses. It could be interesting to be able to only cut buses supporting hotplug, given that they are the one able to probe devices afterward.
No real use-case for this right now, so leaving as-is. Might be worth considering in the future.
doc/guides/rel_notes/release_19_11.rst | 9 +++++++ lib/librte_eal/common/eal_common_bus.c | 6 +++++ lib/librte_eal/common/eal_common_dev.c | 41 ++++++++++++++++++++++++++++++ lib/librte_eal/common/eal_common_options.c | 8 ++++++ lib/librte_eal/common/eal_internal_cfg.h | 1 + lib/librte_eal/common/eal_options.h | 2 ++ lib/librte_eal/common/eal_private.h | 9 +++++++ lib/librte_eal/common/include/rte_eal.h | 34 +++++++++++++++++++++++++ lib/librte_eal/freebsd/eal/eal.c | 10 ++++++++ lib/librte_eal/linux/eal/eal.c | 10 ++++++++ lib/librte_eal/rte_eal_version.map | 8 ++++++ 11 files changed, 138 insertions(+)doc/guides/rel_notes/release_19_11.rst +9
-0@@ -56,6 +56,15 @@ New Features Also, make sure to start the actual text at the margin. ========================================================= +* **EAL will now allow manual probing devices.** + + Previously, a user could not force an order when probing declared devices. + This could cause issues for drivers depending on another device being present. + A new option ``--manual-probe`` is now available to do just that. + This new option relies on the device bus supporting hotplug. It can + also be used to disable automatic probing from the ``PCI`` bus without + having to disable the whole bus. + Removed Items -------------lib/librte_eal/common/eal_common_bus.c +1
-0@@ -6,6 +6,7 @@#include <string.h> #include <sys/queue.h> +#include <rte_eal.h> #include <rte_bus.h> #include <rte_debug.h> #include <rte_string_fns.h>
lib/librte_eal/common/eal_common_bus.c rte_bus_probe(void) · line 67
Quoted 4 lines
lib/librte_eal/common/eal_common_bus.c +1
-0@@ -63,6 +64,11 @@ rte_bus_probe(void)int ret;struct rte_bus *bus, *vbus =NULL ; +if (rte_eal_manual_probe ()) {
See below,
Quoted 10 lines
lib/librte_eal/common/eal_common_bus.c +0
-0int rte_vfio_setup_device(__rte_unused const char *sysfs_base, __rte_unused const char *dev_addr, __rte_unused int *vfio_dev_fd,lib/librte_eal/linux/eal/eal.c +0
-0@@ -1382,6 +1382,16 @@ rte_eal_vfio_intr_mode(void)return internal_config.vfio_intr_mode; }
lib/librte_eal/linux/eal/eal.c rte_eal_vfio_intr_mode(void) · line 1393
Quoted 10 lines
lib/librte_eal/linux/eal/eal.c +9
-0+int rte_eal_manual_probe (void ) +{ +return internal_config.manual_probe; +} + +void rte_eal_manual_probe_set (int enabled) +{ + internal_config.manual_probe = !!enabled; +}
I don't think FreeBSD and Linux specific implementation is NOT required as internal_config accessible in eal/common. example diff.
[master][dpdk.org] $ git diff
(quoted diff) +0 -0
b/lib/librte_eal/common/eal_common_bus.c
index 145a96812..db4257cf1 100644
lib/librte_eal/common/eal_common_bus.c +2 -1
@@ -13,6 +13,7 @@#include <rte_errno.h> #include "eal_private.h" +#include "eal_internal_cfg.h" static struct rte_bus_list rte_bus_list =TAILQ_HEAD_INITIALIZER (rte_bus_list);
@@ -64,7 +65,7 @@ rte_bus_probe(void)int ret;struct rte_bus *bus, *vbus =NULL ; -if (rte_eal_manual_probe ()) { +if (internal_config.manual_probe) {RTE_LOG (DEBUG ,EAL ,"Manual probing enabled.\n" );return rte_dev_probe_devargs_list (); }
Quoted 8 lines
lib/librte_eal/linux/eal/eal.c +0
-0int rte_eal_check_module (const char *module_name) {lib/librte_eal/rte_eal_version.map +0
-0
lib/librte_eal/rte_eal_version.map DPDK_19.08 { · line 319
Quoted 8 lines
lib/librte_eal/rte_eal_version.map +5
-0@@ -312,6 +312,14 @@ DPDK_19.08 { } DPDK_19.05; +DPDK_19.11 { + global: + + rte_eal_manual_probe; + rte_eal_manual_probe_set;
Do we need public API for this? it is only used by eal lib.
checkpatch complains the following too.
ERROR: symbol rte_eal_manual_probe_set is added in the DPDK_19.11 section, but is expected to be added in the EXPERIMENTAL section of the version map
Re: [dpdk-dev] [PATCH v3] eal: add manual probing option
Quoted 162 lines · Jerin Jacob
Quoted 93 lines · Gaetan Rivet
Add a new EAL option enabling manual probing in the EAL. This command line option will configure the EAL so that buses will not trigger their probe step on their own.
Applications are then expected to hotplug devices as they see fit.
Devices declared on the command line by the user (using -w and --vdev), will be probed using the hotplug API, in the order they are declared.
This has the effect of offering a way for users to control probe order of their devices, for drivers requiring it.
Signed-off-by Gaetan Rivet <[email protected]>---
I haven't heard many opinions on the matter, please shout if you see an issue with this approach.
@Slava: I have tested rather quickly that it does not break anything,
and that it works as intended for basic cases.
Can you test it further for your use-case and tell me if it works fine?Beyond the obvious difference between both probe mode, something to keep in mind:
while using -w on invalid devices would not block (PCI) bus probing, it will stop manual probing in its track. All devices need to exist and be valid device IDs.v2: fixed a few typos, map file (and used Travis to validate).
Slava, are you able to test this patch?
v3: properly fixed the map file (herited 19.08 instead of 19.05).
Added a function to set the probe manual from the application, without having the user do it from the command line.
Stopped spamming Slava about it, Vamsi was actually the one interested in it!
Standing issue worth chiming in:
Currently manual-probe will cut off probing from all buses. It could be interesting to be able to only cut buses supporting hotplug, given that they are the one able to probe devices afterward.
No real use-case for this right now, so leaving as-is. Might be worth considering in the future.
doc/guides/rel_notes/release_19_11.rst | 9 +++++++ lib/librte_eal/common/eal_common_bus.c | 6 +++++ lib/librte_eal/common/eal_common_dev.c | 41 ++++++++++++++++++++++++++++++ lib/librte_eal/common/eal_common_options.c | 8 ++++++ lib/librte_eal/common/eal_internal_cfg.h | 1 + lib/librte_eal/common/eal_options.h | 2 ++ lib/librte_eal/common/eal_private.h | 9 +++++++ lib/librte_eal/common/include/rte_eal.h | 34 +++++++++++++++++++++++++ lib/librte_eal/freebsd/eal/eal.c | 10 ++++++++ lib/librte_eal/linux/eal/eal.c | 10 ++++++++ lib/librte_eal/rte_eal_version.map | 8 ++++++ 11 files changed, 138 insertions(+)doc/guides/rel_notes/release_19_11.rst +9
-0@@ -56,6 +56,15 @@ New Features Also, make sure to start the actual text at the margin. ========================================================= +* **EAL will now allow manual probing devices.** + + Previously, a user could not force an order when probing declared devices. + This could cause issues for drivers depending on another device being present. + A new option ``--manual-probe`` is now available to do just that. + This new option relies on the device bus supporting hotplug. It can + also be used to disable automatic probing from the ``PCI`` bus without + having to disable the whole bus. + Removed Items -------------lib/librte_eal/common/eal_common_bus.c +1
-0@@ -6,6 +6,7 @@#include <string.h> #include <sys/queue.h> +#include <rte_eal.h> #include <rte_bus.h> #include <rte_debug.h> #include <rte_string_fns.h> lib/librte_eal/common/eal_common_bus.c rte_bus_probe(void) · line 67
Quoted 4 lines
lib/librte_eal/common/eal_common_bus.c +1
-0@@ -63,6 +64,11 @@ rte_bus_probe(void)int ret;struct rte_bus *bus, *vbus =NULL ; +if (rte_eal_manual_probe ()) {See below,
Quoted 10 lines
lib/librte_eal/common/eal_common_bus.c +0
-0int rte_vfio_setup_device(__rte_unused const char *sysfs_base, __rte_unused const char *dev_addr, __rte_unused int *vfio_dev_fd,lib/librte_eal/linux/eal/eal.c +0
-0@@ -1382,6 +1382,16 @@ rte_eal_vfio_intr_mode(void)return internal_config.vfio_intr_mode; }lib/librte_eal/linux/eal/eal.c rte_eal_vfio_intr_mode(void) · line 1393
Quoted 10 lines
lib/librte_eal/linux/eal/eal.c +9
-0+int rte_eal_manual_probe (void ) +{ +return internal_config.manual_probe; +} + +void rte_eal_manual_probe_set (int enabled) +{ + internal_config.manual_probe = !!enabled; +}I don't think FreeBSD and Linux specific implementation is NOT required as
internal_configaccessible in eal/common. example diff.[master][dpdk.org] $ git diff
lib/librte_eal/common/eal_common_bus.c +2
-1@@ -13,6 +13,7 @@#include <rte_errno.h> #include "eal_private.h" +#include "eal_internal_cfg.h" static struct rte_bus_list rte_bus_list =TAILQ_HEAD_INITIALIZER (rte_bus_list);@@ -64,7 +65,7 @@ rte_bus_probe(void)int ret;struct rte_bus *bus, *vbus =NULL ; -if (rte_eal_manual_probe ()) { +if (internal_config.manual_probe) {RTE_LOG (DEBUG ,EAL ,"Manual probing enabled.\n" );return rte_dev_probe_devargs_list (); }Quoted 12 lines
lib/librte_eal/linux/eal/eal.c +0
-0int rte_eal_check_module (const char *module_name) {lib/librte_eal/rte_eal_version.map +0
-0@@ -312,6 +312,14 @@ DPDK_19.08 { } DPDK_19.05;
Quoted 8 lines
lib/librte_eal/rte_eal_version.map DPDK_19.08 { · line 319
Quoted 5 lines
lib/librte_eal/rte_eal_version.map +5
-0+DPDK_19.11 { + global: + + rte_eal_manual_probe; + rte_eal_manual_probe_set;Do we need public API for this? it is only used by eal lib.
I think some PMDs and some applications would need to issue warnings to the user if manual probing is not enabled (i.e. a port representor parameter was used in a PMD, but probing is automatic --> on some platform it would work and others it would fail with the same config).
Some applications would also prefer forcing it enabled without having to use the option on the command line.
This motivates the public API. Given that there is a public API, I think it's better to use it even from within the EAL.
However I agree that it does not need two separate implementations. I could move it to some eal_common_* part probably, maybe bus or dev?
Quoted 4 lines
checkpatch complains the following too.
ERROR: symbolrte_eal_manual_probe_setis added in the DPDK_19.11 section, but is expected to be added in the EXPERIMENTAL section of the version map
The API is simple enough IMO to make it immediately stable: it will allow applications forbidding the use of experimental API to make use of it.
This is only for the sake of simplicity for some users. I'm not opposed to making it experimental first if it's expected for all new APIs.
Signature · 2 lines
Gaëtan Rivet 6WIND
Re: [dpdk-dev] [PATCH v3] eal: add manual probing option
Quoted 24 lines
Quoted 10 lines
Quoted 10 lines
rte_eal_check_module(const char *module_name)
{lib/librte_eal/rte_eal_version.map +0
-0@@ -312,6 +312,14 @@ DPDK_19.08 { } DPDK_19.05;Quoted 8 lines
lib/librte_eal/rte_eal_version.map DPDK_19.08 { · line 319
Quoted 5 lines
lib/librte_eal/rte_eal_version.map +5
-0+DPDK_19.11 { + global: + + rte_eal_manual_probe; + rte_eal_manual_probe_set;Do we need public API for this? it is only used by eal lib.
I think some PMDs and some applications would need to issue warnings to the user if manual probing is not enabled (i.e. a port representor parameter was used in a PMD, but probing is automatic --> on some platform it would work and others it would fail with the same config).
Quoted 4 lines
Some applications would also prefer forcing it enabled without having to use the option on the command line.
This motivates the public API. Given that there is a public API, I think
it's better to use it even from within the EAL.
No issue from my side to expose through public API as well. Please rte_eal_manual_probe_set() when setting in flags in eal as well. Currently rte_eal_manual_probe_set() not used anywhere.
Quoted 3 lines
However I agree that it does not need two separate implementations. I could move it to some eal_common_* part probably, maybe bus or dev?
eal should be fine. No strong opinion on the place.
Quoted 6 lines
Quoted 5 lines
checkpatch complains the following too.
ERROR: symbolrte_eal_manual_probe_setis added in the DPDK_19.11 section, but is expected to be added in the EXPERIMENTAL section of the version map
Quoted 7 lines
The API is simple enough IMO to make it immediately stable: it will allow applications forbidding the use of experimental API to make use of it.
This is only for the sake of simplicity for some users. I'm not opposed to making it experimental first if it's expected for all new APIs.
All new APIs are treated as experimental. I don't think, we need to make an exception here.
Other these nits, This patch looks good.
Quoted 4 lines
--
Gaėtan Rivet
6WIND
PATCH v4 1/1 eal: add manual probing option
In reply to Gaetan RivetAdd a new EAL option enabling manual probing in the EAL. This command line option will configure the EAL so that buses will not trigger their probe step on their own.
Applications are then expected to hotplug devices as they see fit.
Devices declared on the command line by the user (using -w and --vdev), will be probed using the hotplug API, in the order they are declared.
This has the effect of offering a way for users to control probe order of their devices, for drivers requiring it.
---
I haven't heard many opinions on the matter, please shout if you see an issue with this approach.
@Slava: I have tested rather quickly that it does not break anything,
and that it works as intended for basic cases.
Can you test it further for your use-case and tell me if it works fine?
Beyond the obvious difference between both probe mode, something to keep in mind:
while using -w on invalid devices would not block (PCI) bus probing, it will stop manual probing in its track. All devices need to exist and be valid device IDs.
v2: fixed a few typos, map file (and used Travis to validate).
Slava, are you able to test this patch?
v3: properly fixed the map file (inherited 19.08 instead of 19.05).
Added a function to set the probe manual from the application, without having the user do it from the command line.
Stopped spamming Slava about it, Vamsi was actually the one interested in it!
Standing issue worth chiming in:
Currently manual-probe will cut off probing from all buses. It could be interesting to be able to only cut buses supporting hotplug, given that they are the one able to probe devices afterward.
No real use-case for this right now, so leaving as-is. Might be worth considering in the future.
v4: Rebased on master,
Moved implementation in common EAL,
Used public API within the EAL to set the option, Made the API experimental
doc/guides/rel_notes/release_19_11.rst | 9 +++++ lib/librte_eal/common/eal_common_bus.c | 6 ++++ lib/librte_eal/common/eal_common_dev.c | 54 ++++++++++++++++++++++++++++++ lib/librte_eal/common/eal_common_options.c | 8 +++++ lib/librte_eal/common/eal_internal_cfg.h | 1 + lib/librte_eal/common/eal_options.h | 2 ++ lib/librte_eal/common/eal_private.h | 9 +++++ lib/librte_eal/common/include/rte_eal.h | 36 ++++++++++++++++++++ lib/librte_eal/rte_eal_version.map | 4 +++ 9 files changed, 129 insertions(+)
doc/guides/rel_notes/release_19_11.rst +9 -0
@@ -56,6 +56,15 @@ New Features Also, make sure to start the actual text at the margin. ========================================================= +* **EAL will now allow probing devices manually.** + + Previously, a user could not force an order when probing declared devices. + This could cause issues for drivers depending on another device being present. + A new option ``--manual-probe`` is now available to do just that. + This new option relies on the device bus supporting hotplug. It can + also be used to disable automatic probing from the ``PCI`` bus without + having to disable the whole bus. + * **FreeBSD now supports `--base-virtaddr` EAL option.** FreeBSD version now also supports setting base virtual address for mapping
lib/librte_eal/common/eal_common_bus.c +6 -0
@@ -6,6 +6,7 @@#include <string.h> #include <sys/queue.h> +#include <rte_eal.h> #include <rte_bus.h> #include <rte_debug.h> #include <rte_string_fns.h>
@@ -63,6 +64,11 @@ rte_bus_probe(void)int ret;struct rte_bus *bus, *vbus =NULL ; +if (rte_eal_manual_probe ()) { +RTE_LOG (DEBUG ,EAL ,"Manual probing enabled.\n" ); +return rte_dev_probe_devargs_list (); + } + TAILQ_FOREACH(bus, &rte_bus_list, next) {if (!strcmp (bus->name,"vdev" )) { vbus = bus;
lib/librte_eal/common/eal_common_dev.c +54 -0
66 lines not on this page · raw message
lib/librte_eal/common/eal_common_options.c +8 -0
@@ -82,6 +82,7 @@ eal_long_options[] = { {OPT_LEGACY_MEM, 0, NULL, OPT_LEGACY_MEM_NUM }, {OPT_SINGLE_FILE_SEGMENTS, 0, NULL, OPT_SINGLE_FILE_SEGMENTS_NUM}, {OPT_MATCH_ALLOCATIONS, 0, NULL, OPT_MATCH_ALLOCATIONS_NUM}, + {OPT_MANUAL_PROBE, 0, NULL, OPT_MANUAL_PROBE_NUM }, {0, 0, NULL, 0 } };
@@ -1446,6 +1447,9 @@ eal_parse_common_option(int opt, const char *optarg,return -1 ; }break ; +case OPT_MANUAL_PROBE_NUM : +rte_eal_manual_probe_set (1 ); +break ;/* don't know what to do, leave this to caller */ default :
@@ -1672,6 +1676,10 @@ eal_common_usage(void)" --" OPT_VDEV " Add a virtual device.\n" " The argument format is <driver><id>[,key=val,...]\n" " (ex: --vdev=net_pcap0,iface=eth2).\n" +" --" OPT_MANUAL_PROBE " Enable manual probing.\n" +" Disable probe step for all buses.\n" +" Devices will need to be probed using the hotplug API.\n" +" PCI and vdev declarations will be treated in order as hotplug commands.\n" " --" OPT_IOVA_MODE " Set IOVA mode. 'pa' for IOVA_PA\n" " 'va' for IOVA_VA\n" " -d LIB.so|DIR Add a driver or driver directory\n"
lib/librte_eal/common/eal_internal_cfg.h +1 -0
@@ -44,6 +44,7 @@ struct internal_config {unsigned hugepage_unlink;/**< true to unlink backing files */ volatile unsigned no_pci;/**< true to disable PCI */ volatile unsigned no_hpet;/**< true to disable HPET */ +volatile unsigned manual_probe;/**< true to enable manual device probing. */ volatile unsigned vmware_tsc_map;/**< true to use VMware TSC mapping * instead of native TSC */ volatile unsigned no_shconf;/**< true if there is no shared config */
lib/librte_eal/common/eal_options.h +2 -0
@@ -69,6 +69,8 @@ enum {OPT_IOVA_MODE_NUM , #define OPT_MATCH_ALLOCATIONS "match-allocations"OPT_MATCH_ALLOCATIONS_NUM , +#define OPT_MANUAL_PROBE "manual-probe" +OPT_MANUAL_PROBE_NUM , OPT_LONG_MAX_NUM };
lib/librte_eal/common/eal_private.h +9 -0
@@ -381,4 +381,13 @@ rte_option_init(void);void rte_option_usage (void ); +/** +* Go through the devargs list and probe everything in order. +* +* @return +* 0 on success, negative on error. +*/ +int +rte_dev_probe_devargs_list (void ); +#endif /* _EAL_PRIVATE_H_ */
lib/librte_eal/common/include/rte_eal.h +36 -0
@@ -465,6 +465,42 @@ int rte_eal_has_hugepages(void);int rte_eal_has_pci (void );/** +* Whether EAL probe is manual. +* Enabled by the --manual-probe option or by +* using rte_eal_manual_probe_set(). +* +* When manual probing is enabled, batched bus probe of +* their devices is disabled. All devices need to be probed +* using the proper rte_dev API. +* +* In this mode, devices declared on the command line will +* be probed using the bus hotplug API. It is used to enforce +* a specific probe order. +* +* @return +* Nonzero if manual device probing is enabled. +* +* @see rte_eal_manual_probe_set +*/ +__rte_experimental +int rte_eal_manual_probe(void); + +/** +* Configure EAL probe mode -- manual or automatic. +* +* Enable or disable manual probe mode in EAL. +* This function can be called at any time, but must be used +* before calling rte_eal_init() to have any effect. +* +* @param enabled +* zero to disable manual probe, non-zero to enable it. +* +* @see rte_eal_manual_probe +*/ +__rte_experimental +void rte_eal_manual_probe_set(int enabled); + +/** * Whether the EAL was asked to create UIO device. * * @return
lib/librte_eal/rte_eal_version.map +4 -0
@@ -419,4 +419,8 @@ EXPERIMENTAL { rte_mcfg_timer_lock; rte_mcfg_timer_unlock; rte_rand_max; + + # added in 19.11 + rte_eal_manual_probe; + rte_eal_manual_probe_set; };
Signature · 1 line
2.11.0
Re: [dpdk-dev] [PATCH v4] eal: add manual probing option
Quoted 16 lines · Gaetan Rivet
Add a new EAL option enabling manual probing in the EAL. This command line option will configure the EAL so that buses will not trigger their probe step on their own.
Applications are then expected to hotplug devices as they see fit.
Devices declared on the command line by the user (using -w and --vdev), will be probed using the hotplug API, in the order they are declared.
This has the effect of offering a way for users to control probe order of their devices, for drivers requiring it.
Signed-off-by Gaetan Rivet <[email protected]>---
Quoted 9 lines
doc/guides/rel_notes/release_19_11.rst | 9 +++++ lib/librte_eal/common/eal_common_bus.c | 6 ++++ lib/librte_eal/common/eal_common_dev.c | 54 ++++++++++++++++++++++++++++++ lib/librte_eal/common/eal_common_options.c | 8 +++++ lib/librte_eal/common/eal_internal_cfg.h | 1 + lib/librte_eal/common/eal_options.h | 2 ++ lib/librte_eal/common/eal_private.h | 9 +++++ lib/librte_eal/common/include/rte_eal.h | 36 ++++++++++++++++++++ lib/librte_eal/rte_eal_version.map | 4 +++
Missed to update new eal argument indoc/guides/linux_gsg/eal_args.include.rst file.
With the above change:
PATCH v5 1/1 eal: add manual probing option
In reply to Gaetan RivetAdd a new EAL option enabling manual probing in the EAL. This command line option will configure the EAL so that buses will not trigger their probe step on their own.
Applications are then expected to hotplug devices as they see fit.
Devices declared on the command line by the user (using -w and --vdev), will be probed using the hotplug API, in the order they are declared.
This has the effect of offering a way for users to control probe order of their devices, for drivers requiring it.
---
I haven't heard many opinions on the matter, please shout if you see an issue with this approach.
@Slava: I have tested rather quickly that it does not break anything,
and that it works as intended for basic cases.
Can you test it further for your use-case and tell me if it works fine?
Beyond the obvious difference between both probe mode, something to keep in mind:
while using -w on invalid devices would not block (PCI) bus probing, it will stop manual probing in its track. All devices need to exist and be valid device IDs.
v2: fixed a few typos, map file (and used Travis to validate).
Slava, are you able to test this patch?
v3: properly fixed the map file (inherited 19.08 instead of 19.05).
Added a function to set the probe manual from the application, without having the user do it from the command line.
Stopped spamming Slava about it, Vamsi was actually the one interested in it!
Standing issue worth chiming in:
Currently manual-probe will cut off probing from all buses. It could be interesting to be able to only cut buses supporting hotplug, given that they are the one able to probe devices afterward.
No real use-case for this right now, so leaving as-is. Might be worth considering in the future.
v4: Rebased on master,
Moved implementation in common EAL,
Used public API within the EAL to set the option, Made the API experimental
v5: added note in the Getting Started Guide.
doc/guides/linux_gsg/eal_args.include.rst | 13 +++++++ doc/guides/rel_notes/release_19_11.rst | 9 +++++ lib/librte_eal/common/eal_common_bus.c | 6 ++++ lib/librte_eal/common/eal_common_dev.c | 54 ++++++++++++++++++++++++++++++ lib/librte_eal/common/eal_common_options.c | 8 +++++ lib/librte_eal/common/eal_internal_cfg.h | 1 + lib/librte_eal/common/eal_options.h | 2 ++ lib/librte_eal/common/eal_private.h | 9 +++++ lib/librte_eal/common/include/rte_eal.h | 36 ++++++++++++++++++++ lib/librte_eal/rte_eal_version.map | 4 +++ 10 files changed, 142 insertions(+)
doc/guides/linux_gsg/eal_args.include.rst +13 -0
@@ -69,6 +69,19 @@ Device-related options --vdev 'net_pcap0,rx_pcap=input.pcap,tx_pcap=output.pcap' +* ``--manual-probe`` + + Switch the ``EAL`` probe mode to manual. The main bus probe step + is disabled and applications are expected to manually probe + devices using ``rte_dev_probe()``. + + Devices declared on the command-line using ``-w`` and ``-vdev`` + are interpreted as hotplug commands. They are thus probed in the + order they are declared. + + This makes this option useful to enforce a specific device probe + order, instead of relying on each bus scan implementation details. + * ``-d <path to shared object or directory>`` Load external drivers. An argument can be a single shared object file, or a
doc/guides/rel_notes/release_19_11.rst +9 -0
@@ -56,6 +56,15 @@ New Features Also, make sure to start the actual text at the margin. ========================================================= +* **EAL will now allow probing devices manually.** + + Previously, a user could not force an order when probing declared devices. + This could cause issues for drivers depending on another device being present. + A new option ``--manual-probe`` is now available to do just that. + This new option relies on the device bus supporting hotplug. It can + also be used to disable automatic probing from the ``PCI`` bus without + having to disable the whole bus. + * **FreeBSD now supports `--base-virtaddr` EAL option.** FreeBSD version now also supports setting base virtual address for mapping
lib/librte_eal/common/eal_common_bus.c +6 -0
@@ -6,6 +6,7 @@#include <string.h> #include <sys/queue.h> +#include <rte_eal.h> #include <rte_bus.h> #include <rte_debug.h> #include <rte_string_fns.h>
@@ -63,6 +64,11 @@ rte_bus_probe(void)int ret;struct rte_bus *bus, *vbus =NULL ; +if (rte_eal_manual_probe ()) { +RTE_LOG (DEBUG ,EAL ,"Manual probing enabled.\n" ); +return rte_dev_probe_devargs_list (); + } + TAILQ_FOREACH(bus, &rte_bus_list, next) {if (!strcmp (bus->name,"vdev" )) { vbus = bus;
lib/librte_eal/common/eal_common_dev.c +54 -0
66 lines not on this page · raw message
lib/librte_eal/common/eal_common_options.c +8 -0
@@ -82,6 +82,7 @@ eal_long_options[] = { {OPT_LEGACY_MEM, 0, NULL, OPT_LEGACY_MEM_NUM }, {OPT_SINGLE_FILE_SEGMENTS, 0, NULL, OPT_SINGLE_FILE_SEGMENTS_NUM}, {OPT_MATCH_ALLOCATIONS, 0, NULL, OPT_MATCH_ALLOCATIONS_NUM}, + {OPT_MANUAL_PROBE, 0, NULL, OPT_MANUAL_PROBE_NUM }, {0, 0, NULL, 0 } };
@@ -1446,6 +1447,9 @@ eal_parse_common_option(int opt, const char *optarg,return -1 ; }break ; +case OPT_MANUAL_PROBE_NUM : +rte_eal_manual_probe_set (1 ); +break ;/* don't know what to do, leave this to caller */ default :
@@ -1672,6 +1676,10 @@ eal_common_usage(void)" --" OPT_VDEV " Add a virtual device.\n" " The argument format is <driver><id>[,key=val,...]\n" " (ex: --vdev=net_pcap0,iface=eth2).\n" +" --" OPT_MANUAL_PROBE " Enable manual probing.\n" +" Disable probe step for all buses.\n" +" Devices will need to be probed using the hotplug API.\n" +" PCI and vdev declarations will be treated in order as hotplug commands.\n" " --" OPT_IOVA_MODE " Set IOVA mode. 'pa' for IOVA_PA\n" " 'va' for IOVA_VA\n" " -d LIB.so|DIR Add a driver or driver directory\n"
lib/librte_eal/common/eal_internal_cfg.h +1 -0
@@ -44,6 +44,7 @@ struct internal_config {unsigned hugepage_unlink;/**< true to unlink backing files */ volatile unsigned no_pci;/**< true to disable PCI */ volatile unsigned no_hpet;/**< true to disable HPET */ +volatile unsigned manual_probe;/**< true to enable manual device probing. */ volatile unsigned vmware_tsc_map;/**< true to use VMware TSC mapping * instead of native TSC */ volatile unsigned no_shconf;/**< true if there is no shared config */
lib/librte_eal/common/eal_options.h +2 -0
@@ -69,6 +69,8 @@ enum {OPT_IOVA_MODE_NUM , #define OPT_MATCH_ALLOCATIONS "match-allocations"OPT_MATCH_ALLOCATIONS_NUM , +#define OPT_MANUAL_PROBE "manual-probe" +OPT_MANUAL_PROBE_NUM , OPT_LONG_MAX_NUM };
lib/librte_eal/common/eal_private.h +9 -0
@@ -381,4 +381,13 @@ rte_option_init(void);void rte_option_usage (void ); +/** +* Go through the devargs list and probe everything in order. +* +* @return +* 0 on success, negative on error. +*/ +int +rte_dev_probe_devargs_list (void ); +#endif /* _EAL_PRIVATE_H_ */
lib/librte_eal/common/include/rte_eal.h +36 -0
@@ -465,6 +465,42 @@ int rte_eal_has_hugepages(void);int rte_eal_has_pci (void );/** +* Whether EAL probe is manual. +* Enabled by the --manual-probe option or by +* using rte_eal_manual_probe_set(). +* +* When manual probing is enabled, batched bus probe of +* their devices is disabled. All devices need to be probed +* using the proper rte_dev API. +* +* In this mode, devices declared on the command line will +* be probed using the bus hotplug API. It is used to enforce +* a specific probe order. +* +* @return +* Nonzero if manual device probing is enabled. +* +* @see rte_eal_manual_probe_set +*/ +__rte_experimental +int rte_eal_manual_probe(void); + +/** +* Configure EAL probe mode -- manual or automatic. +* +* Enable or disable manual probe mode in EAL. +* This function can be called at any time, but must be used +* before calling rte_eal_init() to have any effect. +* +* @param enabled +* zero to disable manual probe, non-zero to enable it. +* +* @see rte_eal_manual_probe +*/ +__rte_experimental +void rte_eal_manual_probe_set(int enabled); + +/** * Whether the EAL was asked to create UIO device. * * @return
lib/librte_eal/rte_eal_version.map +4 -0
@@ -419,4 +419,8 @@ EXPERIMENTAL { rte_mcfg_timer_lock; rte_mcfg_timer_unlock; rte_rand_max; + + # added in 19.11 + rte_eal_manual_probe; + rte_eal_manual_probe_set; };
Signature · 1 line
2.11.0
Re: [dpdk-dev] [PATCH v5] eal: add manual probing option
Quoted 7 lines · Gaetan Rivet
Add a new EAL option enabling manual probing in the EAL. This command line option will configure the EAL so that buses will not trigger their probe step on their own.
Applications are then expected to hotplug devices as they see fit.
Quoted 7 lines
Devices declared on the command line by the user (using -w and --vdev), will be probed using the hotplug API, in the order they are declared.
This has the effect of offering a way for users to control probe order of their devices, for drivers requiring it.
Signed-off-by Gaetan Rivet <[email protected]>
Re: [dpdk-dev] [PATCH v5] eal: add manual probing option
In reply to Gaetan RivetPing
@[email protected]
Since, you weren't reachable on regular email I rebased the patch on ToT and sent a v6
http://patches.dpdk.org/patch/64235/
Quoted 395 lines
-----Original Message-----
From: dev <[email protected]> On Behalf Of Gaetan Rivet
Sent: Friday, October 25, 2019 9:17 PM
To: [email protected]
Cc: Gaetan Rivet <[email protected]>; Vamsi Krishna Attunuru <[email protected]>; David Marchand <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>
Subject: [dpdk-dev] [PATCH v5] eal: add manual probing optionAdd a new EAL option enabling manual probing in the EAL. This command line option will configure the EAL so that buses will not trigger their probe step on their own.
Applications are then expected to hotplug devices as they see fit.
Devices declared on the command line by the user (using -w and -- vdev),
will be probed using the hotplug API, in the order they are declared.This has the effect of offering a way for users to control probe order of their devices, for drivers requiring it.
Signed-off-by Gaetan Rivet <[email protected]>---
I haven't heard many opinions on the matter, please shout if you see an issue
with this approach.@Slava: I have tested rather quickly that it does not break anything,
and that it works as intended for basic cases.
Can you test it further for your use-case and tell me if it works fine?Beyond the obvious difference between both probe mode, something to keep in mind:
while using -w on invalid devices would not block (PCI) bus probing, it will stop manual
probing in its track. All devices need to exist and be valid device IDs.v2: fixed a few typos, map file (and used Travis to validate).
Slava, are you able to test this patch?
v3: properly fixed the map file (inherited 19.08 instead of 19.05).
Added a function to set the probe manual from the application, without having the user do it from the command line.
Stopped spamming Slava about it, Vamsi was actually the one
interested in it!Standing issue worth chiming in:
Currently manual-probe will cut off probing from all buses. It could be interesting to be able to only cut buses supporting hotplug, given that they are the one able to probe devices afterward.
No real use-case for this right now, so leaving as-is. Might be worth considering in the future.
v4: Rebased on master,
Moved implementation in common EAL,
Used public API within the EAL to set the option, Made the API experimentalv5: added note in the Getting Started Guide.
doc/guides/linux_gsg/eal_args.include.rst | 13 +++++++ doc/guides/rel_notes/release_19_11.rst | 9 +++++ lib/librte_eal/common/eal_common_bus.c | 6 ++++ lib/librte_eal/common/eal_common_dev.c | 54 ++++++++++++++++++++++++++++++ lib/librte_eal/common/eal_common_options.c | 8 +++++ lib/librte_eal/common/eal_internal_cfg.h | 1 + lib/librte_eal/common/eal_options.h | 2 ++ lib/librte_eal/common/eal_private.h | 9 +++++ lib/librte_eal/common/include/rte_eal.h | 36 ++++++++++++++++++++ lib/librte_eal/rte_eal_version.map | 4 +++ 10 files changed, 142 insertions(+)doc/guides/linux_gsg/eal_args.include.rst +13
-0@@ -69,6 +69,19 @@ Device-related options --vdev 'net_pcap0,rx_pcap=input.pcap,tx_pcap=output.pcap' +* ``--manual-probe`` + + Switch the ``EAL`` probe mode to manual. The main bus probe step + is disabled and applications are expected to manually probe + devices using ``rte_dev_probe()``. + + Devices declared on the command-line using ``-w`` and ``-vdev`` + are interpreted as hotplug commands. They are thus probed in the + order they are declared. + + This makes this option useful to enforce a specific device probe + order, instead of relying on each bus scan implementation details. + * ``-d <path to shared object or directory>`` Load external drivers. An argument can be a single shared object file,or a
doc/guides/rel_notes/release_19_11.rst +0
-0@@ -56,6 +56,15 @@ New Features Also, make sure to start the actual text at the margin.======================================================
===doc/guides/rel_notes/release_19_11.rst +3
-0+* **EAL will now allow probing devices manually.** + + Previously, a user could not force an order when probing declareddevices.
+ This could cause issues for drivers depending on another device being present.doc/guides/rel_notes/release_19_11.rst +5
-0+ A new option ``--manual-probe`` is now available to do just that. + This new option relies on the device bus supporting hotplug. It can + also be used to disable automatic probing from the ``PCI`` bus without + having to disable the whole bus. +* **FreeBSD now supports
--base-virtaddrEAL option.**FreeBSD version now also supports setting base virtual address for
mappinglib/librte_eal/common/eal_common_bus.c +6
-0@@ -6,6 +6,7 @@#include <string.h> #include <sys/queue.h> +#include <rte_eal.h> #include <rte_bus.h> #include <rte_debug.h> #include <rte_string_fns.h> @@ -63,6 +64,11 @@ rte_bus_probe(void)int ret;struct rte_bus *bus, *vbus =NULL ; +if (rte_eal_manual_probe ()) { +RTE_LOG (DEBUG ,EAL ,"Manual probing enabled.\n" ); +return rte_dev_probe_devargs_list (); + } + TAILQ_FOREACH(bus, &rte_bus_list, next) {if (!strcmp (bus->name,"vdev" )) { vbus = bus;lib/librte_eal/common/eal_common_dev.c +33
-0@@ -21,6 +21,7 @@#include <rte_malloc.h> #include <rte_string_fns.h> +#include "eal_internal_cfg.h" #include "eal_private.h" #include "hotplug_mp.h" @@ -83,6 +84,59 @@ rte_dev_is_probed(const struct rte_device *dev)return dev->driver !=NULL ; } +int +rte_eal_manual_probe (void ) +{ +return internal_config.manual_probe; +} + +void +rte_eal_manual_probe_set (int enabled) +{ + internal_config.manual_probe = !!enabled; +} + +int +rte_dev_probe_devargs_list (void ) +{ +struct rte_device *dev; +struct rte_devargs *da; +int ret; + + RTE_EAL_DEVARGS_FOREACH(NULL, da) { + dev = da->bus->find_device (NULL , cmp_dev_name, da->name); +if (dev ==NULL ) { + RTE_LOG(ERR, EAL, "Unable to find device %s on bus %s\n", + da->name, da->bus->name); +continue ; + } + +if (rte_dev_is_probed (dev)) +continue ; + +if (dev->bus->plug ==NULL ) { + RTE_LOG(ERR, EAL, "Manual probing (hotplug)not supported by bus %s, "
lib/librte_eal/common/eal_common_dev.c +8
-0+"required by device %s\n" , + dev->bus->name, dev->name); +continue ; + } + + ret = dev->bus->plug (dev); +/* Ignore positive return values, they are possibly +* triggered by blacklisted devices on the PCI bus. Probing
lib/librte_eal/common/eal_common_dev.c +4
-0+* should then continue. +*/ +if (ret <0 ) { + RTE_LOG(ERR, EAL, "Driver cannot attach device%s\n",
lib/librte_eal/common/eal_common_dev.c +9
-0+ dev->name); +/* Fail on first real probe error. */ +return ret; + } + } + +return 0 ; +} +/* helper function to build devargs, caller should free the memory */ static int
build_devargs(const char *busname, const char *devname,lib/librte_eal/common/eal_common_options.c +1
-0@@ -82,6 +82,7 @@ eal_long_options[] = { {OPT_LEGACY_MEM ,0 ,NULL ,OPT_LEGACY_MEM_NUM }, {OPT_SINGLE_FILE_SEGMENTS, 0, NULL, OPT_SINGLE_FILE_SEGMENTS_NUM}, {OPT_MATCH_ALLOCATIONS, 0, NULL, OPT_MATCH_ALLOCATIONS_NUM}, + {OPT_MANUAL_PROBE, 0, NULL,
OPT_MANUAL_PROBE_NUM},{0, 0, NULL, 0 } };lib/librte_eal/common/eal_common_options.c +3
-0@@ -1446,6 +1447,9 @@ eal_parse_common_option(int opt, constchar *optarg, return -1; }break ; +case OPT_MANUAL_PROBE_NUM : +rte_eal_manual_probe_set (1 ); +break ;/* don't know what to do, leave this to caller */ default:
lib/librte_eal/common/eal_common_options.c +0
-0@@ -1672,6 +1676,10 @@ eal_common_usage(void)" --" OPT_VDEV " Add a virtual device.\n" " The argument format is<driver><id>[,key=val,...]\n"
" (ex: --vdev=net_pcap0,iface=eth2).\n"lib/librte_eal/common/eal_common_options.c +3
-0+" --" OPT_MANUAL_PROBE " Enable manual probing.\n" +" Disable probe step for all buses.\n" + " Devices will need to be probed using the hotplugAPI.\n"
lib/librte_eal/common/eal_common_options.c +1
-0+ " PCI and vdev declarations will be treated inorder as hotplug commands.\n"
" --"OPT_IOVA_MODE" Set IOVA mode. 'pa' for
IOVA_PA\n"
" 'va' for IOVA_VA\n"
" -d LIB.so|DIR Add a driver or driver directory\n"lib/librte_eal/common/eal_internal_cfg.h +1
-0@@ -44,6 +44,7 @@ struct internal_config {unsigned hugepage_unlink;/**< true to unlink backing files */ volatile unsigned no_pci;/**< true to disable PCI */ volatile unsigned no_hpet;/**< true to disable HPET */ +volatile unsigned manual_probe;/**< true to enable manual device probing. */
volatile unsignedvmware_tsc_map; /**< true to use VMware
TSC mapping* instead of native TSC */
volatile unsignedno_shconf; /**< true if there is no shared
config */lib/librte_eal/common/eal_options.h +2
-0@@ -69,6 +69,8 @@ enum {OPT_IOVA_MODE_NUM , #define OPT_MATCH_ALLOCATIONS "match-allocations"OPT_MATCH_ALLOCATIONS_NUM , +#define OPT_MANUAL_PROBE "manual-probe" +OPT_MANUAL_PROBE_NUM , OPT_LONG_MAX_NUM };lib/librte_eal/common/eal_private.h +0
-0@@ -381,4 +381,13 @@ rte_option_init(void);void
rte_option_usage(void);lib/librte_eal/common/eal_private.h +9
-0+/** +* Go through the devargs list and probe everything in order. +* +* @return +* 0 on success, negative on error. +*/ +int +rte_dev_probe_devargs_list (void ); +#endif /* _EAL_PRIVATE_H_ */
lib/librte_eal/common/include/rte_eal.h +36
-0@@ -465,6 +465,42 @@ int rte_eal_has_hugepages(void);int rte_eal_has_pci (void );/** +* Whether EAL probe is manual. +* Enabled by the --manual-probe option or by +* using rte_eal_manual_probe_set(). +* +* When manual probing is enabled, batched bus probe of +* their devices is disabled. All devices need to be probed +* using the proper rte_dev API. +* +* In this mode, devices declared on the command line will +* be probed using the bus hotplug API. It is used to enforce +* a specific probe order. +* +* @return +* Nonzero if manual device probing is enabled. +* +* @see rte_eal_manual_probe_set +*/ +__rte_experimental +int rte_eal_manual_probe(void); + +/** +* Configure EAL probe mode -- manual or automatic. +* +* Enable or disable manual probe mode in EAL. +* This function can be called at any time, but must be used +* before calling rte_eal_init() to have any effect. +* +* @param enabled +* zero to disable manual probe, non-zero to enable it. +* +* @see rte_eal_manual_probe +*/ +__rte_experimental +void rte_eal_manual_probe_set(int enabled); + +/** * Whether the EAL was asked to create UIO device. * * @return lib/librte_eal/rte_eal_version.map +4
-1@@ -419,4 +419,8 @@ EXPERIMENTAL { rte_mcfg_timer_lock; rte_mcfg_timer_unlock; rte_rand_max; + + # added in 19.11 + rte_eal_manual_probe; + rte_eal_manual_probe_set; }; --2.11.0
Re: [dpdk-dev] [PATCH v5] eal: add manual probing option
Quoted 6 lines · Pavan Nikhilesh Bhagavatula
Ping
@[email protected]Since, you weren't reachable on regular email I rebased the patch on ToT and sent a v6
http://patches.dpdk.org/patch/64235/
Hello Pavan, indeed sorry about this.
I have not heard remarks about the patchset anyway in the meantime, not sure whether there is anything to change.
Thanks for the rebase, I will follow-up if there are remarks blocking the integration.
Quoted 396 lines
Quoted 395 lines
-----Original Message-----
From: dev <[email protected]> On Behalf Of Gaetan Rivet
Sent: Friday, October 25, 2019 9:17 PM
To: [email protected]
Cc: Gaetan Rivet <[email protected]>; Vamsi Krishna Attunuru <[email protected]>; David Marchand <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>
Subject: [dpdk-dev] [PATCH v5] eal: add manual probing optionAdd a new EAL option enabling manual probing in the EAL. This command line option will configure the EAL so that buses will not trigger their probe step on their own.
Applications are then expected to hotplug devices as they see fit.
Devices declared on the command line by the user (using -w and -- vdev),
will be probed using the hotplug API, in the order they are declared.This has the effect of offering a way for users to control probe order of their devices, for drivers requiring it.
Signed-off-by Gaetan Rivet <[email protected]>---
I haven't heard many opinions on the matter, please shout if you see an issue
with this approach.@Slava: I have tested rather quickly that it does not break anything,
and that it works as intended for basic cases.
Can you test it further for your use-case and tell me if it works fine?Beyond the obvious difference between both probe mode, something to keep in mind:
while using -w on invalid devices would not block (PCI) bus probing, it will stop manual
probing in its track. All devices need to exist and be valid device IDs.v2: fixed a few typos, map file (and used Travis to validate).
Slava, are you able to test this patch?
v3: properly fixed the map file (inherited 19.08 instead of 19.05).
Added a function to set the probe manual from the application, without having the user do it from the command line.
Stopped spamming Slava about it, Vamsi was actually the one
interested in it!Standing issue worth chiming in:
Currently manual-probe will cut off probing from all buses. It could be interesting to be able to only cut buses supporting hotplug, given that they are the one able to probe devices afterward.
No real use-case for this right now, so leaving as-is. Might be worth considering in the future.
v4: Rebased on master,
Moved implementation in common EAL,
Used public API within the EAL to set the option, Made the API experimentalv5: added note in the Getting Started Guide.
doc/guides/linux_gsg/eal_args.include.rst | 13 +++++++ doc/guides/rel_notes/release_19_11.rst | 9 +++++ lib/librte_eal/common/eal_common_bus.c | 6 ++++ lib/librte_eal/common/eal_common_dev.c | 54 ++++++++++++++++++++++++++++++ lib/librte_eal/common/eal_common_options.c | 8 +++++ lib/librte_eal/common/eal_internal_cfg.h | 1 + lib/librte_eal/common/eal_options.h | 2 ++ lib/librte_eal/common/eal_private.h | 9 +++++ lib/librte_eal/common/include/rte_eal.h | 36 ++++++++++++++++++++ lib/librte_eal/rte_eal_version.map | 4 +++ 10 files changed, 142 insertions(+)doc/guides/linux_gsg/eal_args.include.rst +13
-0@@ -69,6 +69,19 @@ Device-related options --vdev 'net_pcap0,rx_pcap=input.pcap,tx_pcap=output.pcap' +* ``--manual-probe`` + + Switch the ``EAL`` probe mode to manual. The main bus probe step + is disabled and applications are expected to manually probe + devices using ``rte_dev_probe()``. + + Devices declared on the command-line using ``-w`` and ``-vdev`` + are interpreted as hotplug commands. They are thus probed in the + order they are declared. + + This makes this option useful to enforce a specific device probe + order, instead of relying on each bus scan implementation details. + * ``-d <path to shared object or directory>`` Load external drivers. An argument can be a single shared object file,or a
doc/guides/rel_notes/release_19_11.rst +0
-0@@ -56,6 +56,15 @@ New Features Also, make sure to start the actual text at the margin.======================================================
===doc/guides/rel_notes/release_19_11.rst +3
-0+* **EAL will now allow probing devices manually.** + + Previously, a user could not force an order when probing declareddevices.
+ This could cause issues for drivers depending on another device being present.doc/guides/rel_notes/release_19_11.rst +5
-0+ A new option ``--manual-probe`` is now available to do just that. + This new option relies on the device bus supporting hotplug. It can + also be used to disable automatic probing from the ``PCI`` bus without + having to disable the whole bus. +* **FreeBSD now supports
--base-virtaddrEAL option.**FreeBSD version now also supports setting base virtual address for
mappinglib/librte_eal/common/eal_common_bus.c +6
-0@@ -6,6 +6,7 @@#include <string.h> #include <sys/queue.h> +#include <rte_eal.h> #include <rte_bus.h> #include <rte_debug.h> #include <rte_string_fns.h> @@ -63,6 +64,11 @@ rte_bus_probe(void)int ret;struct rte_bus *bus, *vbus =NULL ; +if (rte_eal_manual_probe ()) { +RTE_LOG (DEBUG ,EAL ,"Manual probing enabled.\n" ); +return rte_dev_probe_devargs_list (); + } + TAILQ_FOREACH(bus, &rte_bus_list, next) {if (!strcmp (bus->name,"vdev" )) { vbus = bus;lib/librte_eal/common/eal_common_dev.c +33
-0@@ -21,6 +21,7 @@#include <rte_malloc.h> #include <rte_string_fns.h> +#include "eal_internal_cfg.h" #include "eal_private.h" #include "hotplug_mp.h" @@ -83,6 +84,59 @@ rte_dev_is_probed(const struct rte_device *dev)return dev->driver !=NULL ; } +int +rte_eal_manual_probe (void ) +{ +return internal_config.manual_probe; +} + +void +rte_eal_manual_probe_set (int enabled) +{ + internal_config.manual_probe = !!enabled; +} + +int +rte_dev_probe_devargs_list (void ) +{ +struct rte_device *dev; +struct rte_devargs *da; +int ret; + + RTE_EAL_DEVARGS_FOREACH(NULL, da) { + dev = da->bus->find_device (NULL , cmp_dev_name, da->name); +if (dev ==NULL ) { + RTE_LOG(ERR, EAL, "Unable to find device %s on bus %s\n", + da->name, da->bus->name); +continue ; + } + +if (rte_dev_is_probed (dev)) +continue ; + +if (dev->bus->plug ==NULL ) { + RTE_LOG(ERR, EAL, "Manual probing (hotplug)not supported by bus %s, "
lib/librte_eal/common/eal_common_dev.c +8
-0+"required by device %s\n" , + dev->bus->name, dev->name); +continue ; + } + + ret = dev->bus->plug (dev); +/* Ignore positive return values, they are possibly +* triggered by blacklisted devices on the PCI bus. Probing
lib/librte_eal/common/eal_common_dev.c +4
-0+* should then continue. +*/ +if (ret <0 ) { + RTE_LOG(ERR, EAL, "Driver cannot attach device%s\n",
lib/librte_eal/common/eal_common_dev.c +9
-0+ dev->name); +/* Fail on first real probe error. */ +return ret; + } + } + +return 0 ; +} +/* helper function to build devargs, caller should free the memory */ static int
build_devargs(const char *busname, const char *devname,lib/librte_eal/common/eal_common_options.c +1
-0@@ -82,6 +82,7 @@ eal_long_options[] = { {OPT_LEGACY_MEM ,0 ,NULL ,OPT_LEGACY_MEM_NUM }, {OPT_SINGLE_FILE_SEGMENTS, 0, NULL, OPT_SINGLE_FILE_SEGMENTS_NUM}, {OPT_MATCH_ALLOCATIONS, 0, NULL, OPT_MATCH_ALLOCATIONS_NUM}, + {OPT_MANUAL_PROBE, 0, NULL,
OPT_MANUAL_PROBE_NUM},{0, 0, NULL, 0 } };lib/librte_eal/common/eal_common_options.c +3
-0@@ -1446,6 +1447,9 @@ eal_parse_common_option(int opt, constchar *optarg, return -1; }break ; +case OPT_MANUAL_PROBE_NUM : +rte_eal_manual_probe_set (1 ); +break ;/* don't know what to do, leave this to caller */ default:
lib/librte_eal/common/eal_common_options.c +0
-0@@ -1672,6 +1676,10 @@ eal_common_usage(void)" --" OPT_VDEV " Add a virtual device.\n" " The argument format is<driver><id>[,key=val,...]\n"
" (ex: --vdev=net_pcap0,iface=eth2).\n"lib/librte_eal/common/eal_common_options.c +3
-0+" --" OPT_MANUAL_PROBE " Enable manual probing.\n" +" Disable probe step for all buses.\n" + " Devices will need to be probed using the hotplugAPI.\n"
lib/librte_eal/common/eal_common_options.c +1
-0+ " PCI and vdev declarations will be treated inorder as hotplug commands.\n"
" --"OPT_IOVA_MODE" Set IOVA mode. 'pa' for
IOVA_PA\n"
" 'va' for IOVA_VA\n"
" -d LIB.so|DIR Add a driver or driver directory\n"lib/librte_eal/common/eal_internal_cfg.h +1
-0@@ -44,6 +44,7 @@ struct internal_config {unsigned hugepage_unlink;/**< true to unlink backing files */ volatile unsigned no_pci;/**< true to disable PCI */ volatile unsigned no_hpet;/**< true to disable HPET */ +volatile unsigned manual_probe;/**< true to enable manual device probing. */
volatile unsignedvmware_tsc_map; /**< true to use VMware
TSC mapping* instead of native TSC */
volatile unsignedno_shconf; /**< true if there is no shared
config */lib/librte_eal/common/eal_options.h +2
-0@@ -69,6 +69,8 @@ enum {OPT_IOVA_MODE_NUM , #define OPT_MATCH_ALLOCATIONS "match-allocations"OPT_MATCH_ALLOCATIONS_NUM , +#define OPT_MANUAL_PROBE "manual-probe" +OPT_MANUAL_PROBE_NUM , OPT_LONG_MAX_NUM };lib/librte_eal/common/eal_private.h +0
-0@@ -381,4 +381,13 @@ rte_option_init(void);void
rte_option_usage(void);lib/librte_eal/common/eal_private.h +9
-0+/** +* Go through the devargs list and probe everything in order. +* +* @return +* 0 on success, negative on error. +*/ +int +rte_dev_probe_devargs_list (void ); +#endif /* _EAL_PRIVATE_H_ */
lib/librte_eal/common/include/rte_eal.h +36
-0@@ -465,6 +465,42 @@ int rte_eal_has_hugepages(void);int rte_eal_has_pci (void );/** +* Whether EAL probe is manual. +* Enabled by the --manual-probe option or by +* using rte_eal_manual_probe_set(). +* +* When manual probing is enabled, batched bus probe of +* their devices is disabled. All devices need to be probed +* using the proper rte_dev API. +* +* In this mode, devices declared on the command line will +* be probed using the bus hotplug API. It is used to enforce +* a specific probe order. +* +* @return +* Nonzero if manual device probing is enabled. +* +* @see rte_eal_manual_probe_set +*/ +__rte_experimental +int rte_eal_manual_probe(void); + +/** +* Configure EAL probe mode -- manual or automatic. +* +* Enable or disable manual probe mode in EAL. +* This function can be called at any time, but must be used +* before calling rte_eal_init() to have any effect. +* +* @param enabled +* zero to disable manual probe, non-zero to enable it. +* +* @see rte_eal_manual_probe +*/ +__rte_experimental +void rte_eal_manual_probe_set(int enabled); + +/** * Whether the EAL was asked to create UIO device. * * @return lib/librte_eal/rte_eal_version.map +4
-1@@ -419,4 +419,8 @@ EXPERIMENTAL { rte_mcfg_timer_lock; rte_mcfg_timer_unlock; rte_rand_max; + + # added in 19.11 + rte_eal_manual_probe; + rte_eal_manual_probe_set; }; --2.11.0
PATCH v7 1/1 eal: add manual probing option
In reply to Gaetan RivetAdd a new EAL option enabling manual probing in the EAL. This command line option will configure the EAL so that buses will not trigger their probe step on their own.
Applications are then expected to hotplug devices as they see fit.
Devices declared on the command line by the user (using -w and --vdev), will be probed using the hotplug API, in the order they are declared.
This has the effect of offering a way for users to control probe order of their devices, for drivers requiring it.
---
haven't heard many opinions on the matter, please shout if you see an issue
with this approach.
@Slava: I have tested rather quickly that it does not break anything,
and that it works as intended for basic cases.
Can you test it further for your use-case and tell me if it works fine?
Beyond the obvious difference between both probe mode, something to keep in mind:
while using -w on invalid devices would not block (PCI) bus probing, it will stop manual probing in its track. All devices need to exist and be valid device IDs.
v2: fixed a few typos, map file (and used Travis to validate).
Slava, are you able to test this patch?
v3: properly fixed the map file (inherited 19.08 instead of 19.05).
Added a function to set the probe manual from the application, without having the user do it from the command line.
Stopped spamming Slava about it, Vamsi was actually the one interested in it!
Standing issue worth chiming in:
Currently manual-probe will cut off probing from all buses. It could be interesting to be able to only cut buses supporting hotplug, given that they are the one able to probe devices afterward.
No real use-case for this right now, so leaving as-is. Might be worth considering in the future.
v4: Rebased on master,
Moved implementation in common EAL,
Used public API within the EAL to set the option, Made the API experimental
v5: added note in the Getting Started Guide.
v6: Rebased on master
see http://mails.dpdk.org/archives/dev/2020-January/154178.html
for reference to this version, linking v7 to v5 thread.
v7: Updated author and SoB.
doc/guides/linux_gsg/eal_args.include.rst | 13 ++++++ doc/guides/rel_notes/release_20_02.rst | 9 ++++ lib/librte_eal/common/eal_common_bus.c | 6 +++ lib/librte_eal/common/eal_common_dev.c | 54 ++++++++++++++++++++++ lib/librte_eal/common/eal_common_options.c | 8 ++++ lib/librte_eal/common/eal_internal_cfg.h | 1 + lib/librte_eal/common/eal_options.h | 2 + lib/librte_eal/common/eal_private.h | 9 ++++ lib/librte_eal/common/include/rte_eal.h | 36 +++++++++++++++ lib/librte_eal/rte_eal_version.map | 4 ++ 10 files changed, 142 insertions(+)
doc/guides/linux_gsg/eal_args.include.rst +13 -0
@@ -69,6 +69,19 @@ Device-related options --vdev 'net_pcap0,rx_pcap=input.pcap,tx_pcap=output.pcap' +* ``--manual-probe`` + + Switch the ``EAL`` probe mode to manual. The main bus probe step + is disabled and applications are expected to manually probe + devices using ``rte_dev_probe()``. + + Devices declared on the command-line using ``-w`` and ``-vdev`` + are interpreted as hotplug commands. They are thus probed in the + order they are declared. + + This makes this option useful to enforce a specific device probe + order, instead of relying on each bus scan implementation details. + * ``-d <path to shared object or directory>`` Load external drivers. An argument can be a single shared object file, or a
doc/guides/rel_notes/release_20_02.rst +9 -0
@@ -56,6 +56,15 @@ New Features Also, make sure to start the actual text at the margin. ========================================================= +* **EAL will now allow probing devices manually.** + + Previously, a user could not force an order when probing declared devices. + This could cause issues for drivers depending on another device being present. + A new option ``--manual-probe`` is now available to do just that. + This new option relies on the device bus supporting hotplug. It can + also be used to disable automatic probing from the ``PCI`` bus without + having to disable the whole bus. + * **Added Wait Until Equal API.** A new API has been added to wait for a memory location to be updated with a
lib/librte_eal/common/eal_common_bus.c +6 -0
@@ -6,6 +6,7 @@#include <string.h> #include <sys/queue.h> +#include <rte_eal.h> #include <rte_bus.h> #include <rte_debug.h> #include <rte_string_fns.h>
@@ -63,6 +64,11 @@ rte_bus_probe(void)int ret;struct rte_bus *bus, *vbus =NULL ; +if (rte_eal_manual_probe ()) { +RTE_LOG (DEBUG ,EAL ,"Manual probing enabled.\n" ); +return rte_dev_probe_devargs_list (); + } + TAILQ_FOREACH(bus, &rte_bus_list, next) {if (!strcmp (bus->name,"vdev" )) { vbus = bus;
lib/librte_eal/common/eal_common_dev.c +54 -0
66 lines not on this page · raw message
lib/librte_eal/common/eal_common_options.c +8 -0
@@ -82,6 +82,7 @@ eal_long_options[] = { {OPT_LEGACY_MEM, 0, NULL, OPT_LEGACY_MEM_NUM }, {OPT_SINGLE_FILE_SEGMENTS, 0, NULL, OPT_SINGLE_FILE_SEGMENTS_NUM}, {OPT_MATCH_ALLOCATIONS, 0, NULL, OPT_MATCH_ALLOCATIONS_NUM}, + {OPT_MANUAL_PROBE, 0, NULL, OPT_MANUAL_PROBE_NUM }, {0, 0, NULL, 0 } };
@@ -1443,6 +1444,9 @@ eal_parse_common_option(int opt, const char *optarg,return -1 ; }break ; +case OPT_MANUAL_PROBE_NUM : +rte_eal_manual_probe_set (1 ); +break ;/* don't know what to do, leave this to caller */ default :
@@ -1669,6 +1673,10 @@ eal_common_usage(void)" --" OPT_VDEV " Add a virtual device.\n" " The argument format is <driver><id>[,key=val,...]\n" " (ex: --vdev=net_pcap0,iface=eth2).\n" +" --" OPT_MANUAL_PROBE " Enable manual probing.\n" +" Disable probe step for all buses.\n" +" Devices will need to be probed using the hotplug API.\n" +" PCI and vdev declarations will be treated in order as hotplug commands.\n" " --" OPT_IOVA_MODE " Set IOVA mode. 'pa' for IOVA_PA\n" " 'va' for IOVA_VA\n" " -d LIB.so|DIR Add a driver or driver directory\n"
lib/librte_eal/common/eal_internal_cfg.h +1 -0
@@ -44,6 +44,7 @@ struct internal_config {unsigned hugepage_unlink;/**< true to unlink backing files */ volatile unsigned no_pci;/**< true to disable PCI */ volatile unsigned no_hpet;/**< true to disable HPET */ +volatile unsigned manual_probe;/**< true to enable manual device probing. */ volatile unsigned vmware_tsc_map;/**< true to use VMware TSC mapping * instead of native TSC */ volatile unsigned no_shconf;/**< true if there is no shared config */
lib/librte_eal/common/eal_options.h +2 -0
@@ -69,6 +69,8 @@ enum {OPT_IOVA_MODE_NUM , #define OPT_MATCH_ALLOCATIONS "match-allocations"OPT_MATCH_ALLOCATIONS_NUM , +#define OPT_MANUAL_PROBE "manual-probe" +OPT_MANUAL_PROBE_NUM , OPT_LONG_MAX_NUM };
lib/librte_eal/common/eal_private.h +9 -0
@@ -443,4 +443,13 @@ rte_option_usage(void);uint64_t eal_get_baseaddr (void ); +/** +* Go through the devargs list and probe everything in order. +* +* @return +* 0 on success, negative on error. +*/ +int +rte_dev_probe_devargs_list (void ); +#endif /* _EAL_PRIVATE_H_ */
lib/librte_eal/common/include/rte_eal.h +36 -0
@@ -421,6 +421,42 @@ int rte_eal_has_hugepages(void);*/ int rte_eal_has_pci (void ); +/** +* Whether EAL probe is manual. +* Enabled by the --manual-probe option or by +* using rte_eal_manual_probe_set(). +* +* When manual probing is enabled, batched bus probe of +* their devices is disabled. All devices need to be probed +* using the proper rte_dev API. +* +* In this mode, devices declared on the command line will +* be probed using the bus hotplug API. It is used to enforce +* a specific probe order. +* +* @return +* Nonzero if manual device probing is enabled. +* +* @see rte_eal_manual_probe_set +*/ +__rte_experimental +int rte_eal_manual_probe(void); + +/** +* Configure EAL probe mode -- manual or automatic. +* +* Enable or disable manual probe mode in EAL. +* This function can be called at any time, but must be used +* before calling rte_eal_init() to have any effect. +* +* @param enabled +* zero to disable manual probe, non-zero to enable it. +* +* @see rte_eal_manual_probe +*/ +__rte_experimental +void rte_eal_manual_probe_set(int enabled); +/** * Whether the EAL was asked to create UIO device. *
lib/librte_eal/rte_eal_version.map +4 -0
@@ -332,4 +332,8 @@ EXPERIMENTAL { # added in 19.11 rte_log_get_stream; rte_mcfg_get_single_file_segments; + + # added in 20.02 + rte_eal_manual_probe; + rte_eal_manual_probe_set; };
Signature · 1 line
2.25.0
Re: [dpdk-dev] [PATCH v7] eal: add manual probing option
@David Marchand @[email protected]
Ping?
Are there any more changes required for this patch? It's been in queue since last October.
Quoted 407 lines
-----Original Message-----
From: dev <[email protected]> On Behalf Of Gaetan Rivet
Sent: Thursday, January 23, 2020 3:28 PM
To: [email protected]
Cc: Vamsi Krishna Attunuru <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>
Subject: [dpdk-dev] [PATCH v7] eal: add manual probing optionAdd a new EAL option enabling manual probing in the EAL. This command line option will configure the EAL so that buses will not trigger their probe step on their own.
Applications are then expected to hotplug devices as they see fit.
Devices declared on the command line by the user (using -w and -- vdev),
will be probed using the hotplug API, in the order they are declared.This has the effect of offering a way for users to control probe order of their devices, for drivers requiring it.
Signed-off-by Gaetan Rivet <[email protected]>Acked-by Vamsi Attunuru <[email protected]>Tested-by Vamsi Attunuru <[email protected]>Reviewed-by Jerin Jacob <[email protected]>---
haven't heard many opinions on the matter, please shout if you see an issue
with this approach.@Slava: I have tested rather quickly that it does not break anything,
and that it works as intended for basic cases.
Can you test it further for your use-case and tell me if it works fine?Beyond the obvious difference between both probe mode, something to keep in mind:
while using -w on invalid devices would not block (PCI) bus probing, it will stop manual
probing in its track. All devices need to exist and be valid device IDs.v2: fixed a few typos, map file (and used Travis to validate).
Slava, are you able to test this patch?
v3: properly fixed the map file (inherited 19.08 instead of 19.05).
Added a function to set the probe manual from the application, without having the user do it from the command line.
Stopped spamming Slava about it, Vamsi was actually the one
interested in it!Standing issue worth chiming in:
Currently manual-probe will cut off probing from all buses. It could be interesting to be able to only cut buses supporting hotplug, given that they are the one able to probe devices afterward.
No real use-case for this right now, so leaving as-is. Might be worth considering in the future.
v4: Rebased on master,
Moved implementation in common EAL,
Used public API within the EAL to set the option, Made the API experimentalv5: added note in the Getting Started Guide.
v6: Rebased on master
see https://urldefense.proofpoint.com/v2/url?u=http-
3A__mails.dpdk.org_archives_dev_2020- 2DJanuary_154178.html&d=DwIDAg&c=nKjWec2b6R0mOyPaz7xtfQ&r= 1cjuAHrGh745jHNmj2fD85sUMIJ2IPIDsIJzo6FN6Z0&m=BsQe7kO_de- Kb6YFvVZPpZgIhGnWWpW8Pou2KkH_Cjk&s=oFGrj6beplYlYJWodLDGY GCUExSbvb1iJEERsA18QnA&e=
for reference to this version, linking v7 to v5 thread.v7: Updated author and SoB.
doc/guides/linux_gsg/eal_args.include.rst | 13 ++++++ doc/guides/rel_notes/release_20_02.rst | 9 ++++ lib/librte_eal/common/eal_common_bus.c | 6 +++ lib/librte_eal/common/eal_common_dev.c | 54 ++++++++++++++++++++++ lib/librte_eal/common/eal_common_options.c | 8 ++++ lib/librte_eal/common/eal_internal_cfg.h | 1 + lib/librte_eal/common/eal_options.h | 2 + lib/librte_eal/common/eal_private.h | 9 ++++ lib/librte_eal/common/include/rte_eal.h | 36 +++++++++++++++ lib/librte_eal/rte_eal_version.map | 4 ++ 10 files changed, 142 insertions(+)doc/guides/linux_gsg/eal_args.include.rst +13
-0@@ -69,6 +69,19 @@ Device-related options --vdev 'net_pcap0,rx_pcap=input.pcap,tx_pcap=output.pcap' +* ``--manual-probe`` + + Switch the ``EAL`` probe mode to manual. The main bus probe step + is disabled and applications are expected to manually probe + devices using ``rte_dev_probe()``. + + Devices declared on the command-line using ``-w`` and ``-vdev`` + are interpreted as hotplug commands. They are thus probed in the + order they are declared. + + This makes this option useful to enforce a specific device probe + order, instead of relying on each bus scan implementation details. + * ``-d <path to shared object or directory>`` Load external drivers. An argument can be a single shared object file,or a
doc/guides/rel_notes/release_20_02.rst +0
-0@@ -56,6 +56,15 @@ New Features Also, make sure to start the actual text at the margin.======================================================
===doc/guides/rel_notes/release_20_02.rst +3
-0+* **EAL will now allow probing devices manually.** + + Previously, a user could not force an order when probing declareddevices.
+ This could cause issues for drivers depending on another device being present.doc/guides/rel_notes/release_20_02.rst +5
-0+ A new option ``--manual-probe`` is now available to do just that. + This new option relies on the device bus supporting hotplug. It can + also be used to disable automatic probing from the ``PCI`` bus without + having to disable the whole bus. +* **Added Wait Until Equal API.**
A new API has been added to wait for a memory location to be
updated with alib/librte_eal/common/eal_common_bus.c +6
-0@@ -6,6 +6,7 @@#include <string.h> #include <sys/queue.h> +#include <rte_eal.h> #include <rte_bus.h> #include <rte_debug.h> #include <rte_string_fns.h> @@ -63,6 +64,11 @@ rte_bus_probe(void)int ret;struct rte_bus *bus, *vbus =NULL ; +if (rte_eal_manual_probe ()) { +RTE_LOG (DEBUG ,EAL ,"Manual probing enabled.\n" ); +return rte_dev_probe_devargs_list (); + } + TAILQ_FOREACH(bus, &rte_bus_list, next) {if (!strcmp (bus->name,"vdev" )) { vbus = bus;lib/librte_eal/common/eal_common_dev.c +33
-0@@ -21,6 +21,7 @@#include <rte_malloc.h> #include <rte_string_fns.h> +#include "eal_internal_cfg.h" #include "eal_private.h" #include "hotplug_mp.h" @@ -83,6 +84,59 @@ rte_dev_is_probed(const struct rte_device *dev)return dev->driver !=NULL ; } +int +rte_eal_manual_probe (void ) +{ +return internal_config.manual_probe; +} + +void +rte_eal_manual_probe_set (int enabled) +{ + internal_config.manual_probe = !!enabled; +} + +int +rte_dev_probe_devargs_list (void ) +{ +struct rte_device *dev; +struct rte_devargs *da; +int ret; + + RTE_EAL_DEVARGS_FOREACH(NULL, da) { + dev = da->bus->find_device (NULL , cmp_dev_name, da->name); +if (dev ==NULL ) { + RTE_LOG(ERR, EAL, "Unable to find device %s on bus %s\n", + da->name, da->bus->name); +continue ; + } + +if (rte_dev_is_probed (dev)) +continue ; + +if (dev->bus->plug ==NULL ) { + RTE_LOG(ERR, EAL, "Manual probing (hotplug)not supported by bus %s, "
lib/librte_eal/common/eal_common_dev.c +8
-0+"required by device %s\n" , + dev->bus->name, dev->name); +continue ; + } + + ret = dev->bus->plug (dev); +/* Ignore positive return values, they are possibly +* triggered by blacklisted devices on the PCI bus. Probing
lib/librte_eal/common/eal_common_dev.c +4
-0+* should then continue. +*/ +if (ret <0 ) { + RTE_LOG(ERR, EAL, "Driver cannot attach device%s\n",
lib/librte_eal/common/eal_common_dev.c +9
-0+ dev->name); +/* Fail on first real probe error. */ +return ret; + } + } + +return 0 ; +} +/* helper function to build devargs, caller should free the memory */ static int
build_devargs(const char *busname, const char *devname,lib/librte_eal/common/eal_common_options.c +1
-0@@ -82,6 +82,7 @@ eal_long_options[] = { {OPT_LEGACY_MEM ,0 ,NULL ,OPT_LEGACY_MEM_NUM }, {OPT_SINGLE_FILE_SEGMENTS, 0, NULL, OPT_SINGLE_FILE_SEGMENTS_NUM}, {OPT_MATCH_ALLOCATIONS, 0, NULL, OPT_MATCH_ALLOCATIONS_NUM}, + {OPT_MANUAL_PROBE, 0, NULL,
OPT_MANUAL_PROBE_NUM},{0, 0, NULL, 0 } };lib/librte_eal/common/eal_common_options.c +3
-0@@ -1443,6 +1444,9 @@ eal_parse_common_option(int opt, constchar *optarg, return -1; }break ; +case OPT_MANUAL_PROBE_NUM : +rte_eal_manual_probe_set (1 ); +break ;/* don't know what to do, leave this to caller */ default:
lib/librte_eal/common/eal_common_options.c +0
-0@@ -1669,6 +1673,10 @@ eal_common_usage(void)" --" OPT_VDEV " Add a virtual device.\n" " The argument format is<driver><id>[,key=val,...]\n"
" (ex: --vdev=net_pcap0,iface=eth2).\n"lib/librte_eal/common/eal_common_options.c +3
-0+" --" OPT_MANUAL_PROBE " Enable manual probing.\n" +" Disable probe step for all buses.\n" + " Devices will need to be probed using the hotplugAPI.\n"
lib/librte_eal/common/eal_common_options.c +1
-0+ " PCI and vdev declarations will be treated inorder as hotplug commands.\n"
" --"OPT_IOVA_MODE" Set IOVA mode. 'pa' for
IOVA_PA\n"
" 'va' for IOVA_VA\n"
" -d LIB.so|DIR Add a driver or driver directory\n"lib/librte_eal/common/eal_internal_cfg.h +1
-0@@ -44,6 +44,7 @@ struct internal_config {unsigned hugepage_unlink;/**< true to unlink backing files */ volatile unsigned no_pci;/**< true to disable PCI */ volatile unsigned no_hpet;/**< true to disable HPET */ +volatile unsigned manual_probe;/**< true to enable manual device probing. */
volatile unsignedvmware_tsc_map; /**< true to use VMware
TSC mapping* instead of native TSC */
volatile unsignedno_shconf; /**< true if there is no shared
config */lib/librte_eal/common/eal_options.h +2
-0@@ -69,6 +69,8 @@ enum {OPT_IOVA_MODE_NUM , #define OPT_MATCH_ALLOCATIONS "match-allocations"OPT_MATCH_ALLOCATIONS_NUM , +#define OPT_MANUAL_PROBE "manual-probe" +OPT_MANUAL_PROBE_NUM , OPT_LONG_MAX_NUM };lib/librte_eal/common/eal_private.h +0
-0@@ -443,4 +443,13 @@ rte_option_usage(void);
uint64_teal_get_baseaddr(void);lib/librte_eal/common/eal_private.h +9
-0+/** +* Go through the devargs list and probe everything in order. +* +* @return +* 0 on success, negative on error. +*/ +int +rte_dev_probe_devargs_list (void ); +#endif /* _EAL_PRIVATE_H_ */
lib/librte_eal/common/include/rte_eal.h +36
-0@@ -421,6 +421,42 @@ int rte_eal_has_hugepages(void);*/ int rte_eal_has_pci (void ); +/** +* Whether EAL probe is manual. +* Enabled by the --manual-probe option or by +* using rte_eal_manual_probe_set(). +* +* When manual probing is enabled, batched bus probe of +* their devices is disabled. All devices need to be probed +* using the proper rte_dev API. +* +* In this mode, devices declared on the command line will +* be probed using the bus hotplug API. It is used to enforce +* a specific probe order. +* +* @return +* Nonzero if manual device probing is enabled. +* +* @see rte_eal_manual_probe_set +*/ +__rte_experimental +int rte_eal_manual_probe(void); + +/** +* Configure EAL probe mode -- manual or automatic. +* +* Enable or disable manual probe mode in EAL. +* This function can be called at any time, but must be used +* before calling rte_eal_init() to have any effect. +* +* @param enabled +* zero to disable manual probe, non-zero to enable it. +* +* @see rte_eal_manual_probe +*/ +__rte_experimental +void rte_eal_manual_probe_set(int enabled); +/** * Whether the EAL was asked to create UIO device. * lib/librte_eal/rte_eal_version.map +4
-1@@ -332,4 +332,8 @@ EXPERIMENTAL { # added in 19.11 rte_log_get_stream; rte_mcfg_get_single_file_segments; + + # added in 20.02 + rte_eal_manual_probe; + rte_eal_manual_probe_set; }; --2.25.0
Re: [dpdk-dev] [PATCH v7] eal: add manual probing option
03/02/2020 06:16, Pavan Nikhilesh Bhagavatula:
Quoted 5 lines
@David Marchand @[email protected]
Ping?
Are there any more changes required for this patch? It's been in queue since last October.
Sorry we have not decided whether it is a good idea or not.
All changes related to probing are very sensitive, and we know a big refactoring would be better than stacking more and more options and corner cases.
As we are busy with ABI stability stuff, we did not allocate enough time to properly think about this feature. Please accept our apologies, and let's consider it as a high priority for 20.05 cycle.
Re: [dpdk-dev] [PATCH v7] eal: add manual probing option
Quoted 9 lines · Thomas Monjalon
03/02/2020 06:16, Pavan Nikhilesh Bhagavatula:
Quoted 5 lines
@David Marchand @[email protected]
Ping?
Are there any more changes required for this patch? It's been in queue since last October.
Sorry we have not decided whether it is a good idea or not.
Quoted 11 lines
All changes related to probing are very sensitive, and we know a big refactoring would be better than stacking more and more options and corner cases.
As we are busy with ABI stability stuff, we did not allocate enough time to properly think about this feature. Please accept our apologies, and let's consider it as a high priority for 20.05 cycle.
Hello Thomas,
This is unfortunate. I pushed Pavan to accept an alternative implementation of this functionality that was less obtrusive, to make the integration smoother. I took care to alleviate those risks from the common path.
The big refactoring is needed yes, but considering the current path I'm not seeing it happen in 20.05. If that means taking this patch as-is in 20.05 for Marvell users, I'm not sure much is gained from waiting 3 months, except minimal risk avoidance.
Re: [dpdk-dev] [PATCH v7] eal: add manual probing option
04/02/2020 11:03, Gaetan Rivet:
Quoted 20 lines · Thomas Monjalon
Quoted 9 lines
03/02/2020 06:16, Pavan Nikhilesh Bhagavatula:
Quoted 5 lines
@David Marchand @[email protected]
Ping?
Are there any more changes required for this patch? It's been in queue since last October.
Sorry we have not decided whether it is a good idea or not.
Quoted 9 lines
All changes related to probing are very sensitive, and we know a big refactoring would be better than stacking more and more options and corner cases.
As we are busy with ABI stability stuff, we did not allocate enough time to properly think about this feature. Please accept our apologies, and let's consider it as a high priority for 20.05 cycle.
Quoted 5 lines
Hello Thomas,
This is unfortunate. I pushed Pavan to accept an alternative implementation of this functionality that was less obtrusive, to make the integration smoother. I took care to alleviate those risks from the common path.
The big refactoring is needed yes, but considering the current path I'm not seeing it happen in 20.05. If that means taking this patch as-is in 20.05 for Marvell users, I'm not sure much is gained from waiting 3 months, except minimal risk avoidance.
Yes, life is full of bad decisions and consequences.
I still think there is a risk in adding new user expectations, and maintaining some code to workaround unknown issues.
The real question here is to know why this patch? Is it to workaround a broken driver?
Or to workaround a broken design in EAL and bus drivers?
Re: [dpdk-dev] [PATCH v7] eal: add manual probing option
Quoted 21 lines · Thomas Monjalon
04/02/2020 11:03, Gaetan Rivet:
Quoted 20 lines · Thomas Monjalon
Quoted 18 lines
03/02/2020 06:16, Pavan Nikhilesh Bhagavatula:
Quoted 5 lines
@David Marchand @[email protected]
Ping?
Are there any more changes required for this patch? It's been in queue since last October.
Sorry we have not decided whether it is a good idea or not.
All changes related to probing are very sensitive, and we know a big refactoring would be better than stacking more and more options and corner cases.
As we are busy with ABI stability stuff, we did not allocate enough time to properly think about this feature. Please accept our apologies, and let's consider it as a high priority for 20.05 cycle.
Quoted 8 lines
Quoted 5 lines
Hello Thomas,
This is unfortunate. I pushed Pavan to accept an alternative implementation of this functionality that was less obtrusive, to make the integration smoother. I took care to alleviate those risks from the common path.
The big refactoring is needed yes, but considering the current path I'm not seeing it happen in 20.05. If that means taking this patch as-is in 20.05 for Marvell users, I'm not sure much is gained from waiting 3 months, except minimal risk avoidance.
Yes, life is full of bad decisions and consequences.
Ah, yes, but I stand by my initial opinion, the first implementation [1] was riskier and less useful.
Quoted 9 lines
I still think there is a risk in adding new user expectations, and maintaining some code to workaround unknown issues.
The real question here is to know why this patch? Is it to workaround a broken driver?
Or to workaround a broken design in EAL and bus drivers?
Two birds - one stone here: OVS needed a way to disable automatic probing cleanly (current workaround seen in multiple deployment is to add a dummy whitelisted device, which will be ignored by the PCI bus --> it sets the bus in whitelist mode but avoid probing anything), and as a bonus this option allows using devices that depends on other devices being probed already (LAG, representors, failsafe, etc).
I'm not sure having a dependent-probe by default is good, and that would be a big change.
If we are doing the genesis of this patch, the initial motivation should be asked for more details from Marvell people and David for the OVS side.
[1]: First proposal:
http://mails.dpdk.org/archives/dev/2019-September/144166.html My arguments: http://mails.dpdk.org/archives/dev/2019-September/144564.html
Re: [dpdk-dev] [PATCH v7] eal: add manual probing option
04/02/2020 13:43, Gaetan Rivet:
Quoted 44 lines · Thomas Monjalon
Quoted 21 lines
04/02/2020 11:03, Gaetan Rivet:
Quoted 20 lines · Thomas Monjalon
Quoted 18 lines
03/02/2020 06:16, Pavan Nikhilesh Bhagavatula:
Quoted 5 lines
@David Marchand @[email protected]
Ping?
Are there any more changes required for this patch? It's been in queue since last October.
Sorry we have not decided whether it is a good idea or not.
All changes related to probing are very sensitive, and we know a big refactoring would be better than stacking more and more options and corner cases.
As we are busy with ABI stability stuff, we did not allocate enough time to properly think about this feature. Please accept our apologies, and let's consider it as a high priority for 20.05 cycle.
Quoted 8 lines
Quoted 5 lines
Hello Thomas,
This is unfortunate. I pushed Pavan to accept an alternative implementation of this functionality that was less obtrusive, to make the integration smoother. I took care to alleviate those risks from the common path.
The big refactoring is needed yes, but considering the current path I'm not seeing it happen in 20.05. If that means taking this patch as-is in 20.05 for Marvell users, I'm not sure much is gained from waiting 3 months, except minimal risk avoidance.
Yes, life is full of bad decisions and consequences.
Ah, yes, but I stand by my initial opinion, the first implementation [1] was riskier and less useful.
Quoted 7 lines
I still think there is a risk in adding new user expectations, and maintaining some code to workaround unknown issues.
The real question here is to know why this patch? Is it to workaround a broken driver?
Or to workaround a broken design in EAL and bus drivers?Two birds - one stone here: OVS needed a way to disable automatic probing cleanly (current workaround seen in multiple deployment is to add a dummy whitelisted device, which will be ignored by the PCI bus --> it sets the bus in whitelist mode but avoid probing anything), and as a bonus this option allows using devices that depends on other devices being probed already (LAG, representors, failsafe, etc).
Quoted 8 lines
I'm not sure having a dependent-probe by default is good, and that would be a big change.
If we are doing the genesis of this patch, the initial motivation should be asked for more details from Marvell people and David for the OVS side.
[1]: First proposal:
http://mails.dpdk.org/archives/dev/2019-September/144166.html My arguments: http://mails.dpdk.org/archives/dev/2019-September/144564.html
OK so there are two needs:
1/ Better control whitelist/blacklist mode.
We already know that a rework is needed here. Unfortunately neither you or me had time to work on it, and others who were interested disappeared.
2/ Associate ports with equivalent properties in applications. This must be done in applications.
Tweaking the probe order is a hack.
Re: [dpdk-dev] [PATCH v7] eal: add manual probing option
Quoted 57 lines · Thomas Monjalon
04/02/2020 13:43, Gaetan Rivet:
Quoted 44 lines · Thomas Monjalon
Quoted 21 lines
04/02/2020 11:03, Gaetan Rivet:
Quoted 20 lines · Thomas Monjalon
Quoted 18 lines
03/02/2020 06:16, Pavan Nikhilesh Bhagavatula:
Quoted 5 lines
@David Marchand @[email protected]
Ping?
Are there any more changes required for this patch? It's been in queue since last October.
Sorry we have not decided whether it is a good idea or not.
All changes related to probing are very sensitive, and we know a big refactoring would be better than stacking more and more options and corner cases.
As we are busy with ABI stability stuff, we did not allocate enough time to properly think about this feature. Please accept our apologies, and let's consider it as a high priority for 20.05 cycle.
Quoted 8 lines
Quoted 5 lines
Hello Thomas,
This is unfortunate. I pushed Pavan to accept an alternative implementation of this functionality that was less obtrusive, to make the integration smoother. I took care to alleviate those risks from the common path.
The big refactoring is needed yes, but considering the current path I'm not seeing it happen in 20.05. If that means taking this patch as-is in 20.05 for Marvell users, I'm not sure much is gained from waiting 3 months, except minimal risk avoidance.
Yes, life is full of bad decisions and consequences.
Ah, yes, but I stand by my initial opinion, the first implementation [1] was riskier and less useful.
Quoted 7 lines
I still think there is a risk in adding new user expectations, and maintaining some code to workaround unknown issues.
The real question here is to know why this patch? Is it to workaround a broken driver?
Or to workaround a broken design in EAL and bus drivers?Two birds - one stone here: OVS needed a way to disable automatic probing cleanly (current workaround seen in multiple deployment is to add a dummy whitelisted device, which will be ignored by the PCI bus --> it sets the bus in whitelist mode but avoid probing anything), and as a bonus this option allows using devices that depends on other devices being probed already (LAG, representors, failsafe, etc).
Quoted 8 lines
I'm not sure having a dependent-probe by default is good, and that would be a big change.
If we are doing the genesis of this patch, the initial motivation should be asked for more details from Marvell people and David for the OVS side.
[1]: First proposal:
http://mails.dpdk.org/archives/dev/2019-September/144166.html My arguments: http://mails.dpdk.org/archives/dev/2019-September/144564.htmlOK so there are two needs:
Quoted 10 lines
1/ Better control whitelist/blacklist mode.
We already know that a rework is needed here. Unfortunately neither you or me had time to work on it, and others who were interested disappeared.2/ Associate ports with equivalent properties in applications. This must be done in applications.
Tweaking the probe order is a hack.
An application that want to tightly control the port init order, currently (by doing exactly like me here, hotpluging one by one the ports), would still need the bigger hack that consist in inserting a whitelist PCI devargs with a dummy address, depending on a undocumented PCI bus feature consisting in ignoring matching errors but keeping probing policy from failed devargs processing.
Instead, with this patch this app can do
rte_eal_manual_probe_set(1); rte_eal_init();
to have the same behavior and be able to hotplug ports as it sees fit.
You are worried about creating user expectations about this behavior (being forced to replicate in some way the functionality during the rewrite, as I understand it?), but then you are currently forcing users to expect this workaround to exist in the PCI bus, blocking devs from touching it as it will thus break current app configurations. I've seen systemd unit file using this -w dummy flag, as well as the programmatic equivalent. Which is better, to have to rework it cutting short these configs, or to propose beforehand a deprecation path?.
This rework won't happen in 20.05, nor in the medium term unless you decide to drive this change. This workaround serves three needs (PCI normalization, port congruence and port dependency) in a low-risk implementation.
Re: [dpdk-dev] [PATCH v7] eal: add manual probing option
Quoted 74 lines · Gaetan Rivet
Quoted 57 lines · Thomas Monjalon
04/02/2020 13:43, Gaetan Rivet:
Quoted 44 lines · Thomas Monjalon
Quoted 21 lines
04/02/2020 11:03, Gaetan Rivet:
Quoted 20 lines · Thomas Monjalon
Quoted 18 lines
03/02/2020 06:16, Pavan Nikhilesh Bhagavatula:
Quoted 5 lines
@David Marchand @[email protected]
Ping?
Are there any more changes required for this patch? It's been in queue since last October.
Sorry we have not decided whether it is a good idea or not.
All changes related to probing are very sensitive, and we know a big refactoring would be better than stacking more and more options and corner cases.
As we are busy with ABI stability stuff, we did not allocate enough time to properly think about this feature. Please accept our apologies, and let's consider it as a high priority for 20.05 cycle.
Quoted 8 lines
Quoted 5 lines
Hello Thomas,
This is unfortunate. I pushed Pavan to accept an alternative implementation of this functionality that was less obtrusive, to make the integration smoother. I took care to alleviate those risks from the common path.
The big refactoring is needed yes, but considering the current path I'm not seeing it happen in 20.05. If that means taking this patch as-is in 20.05 for Marvell users, I'm not sure much is gained from waiting 3 months, except minimal risk avoidance.
Yes, life is full of bad decisions and consequences.
Ah, yes, but I stand by my initial opinion, the first implementation [1] was riskier and less useful.
Quoted 7 lines
I still think there is a risk in adding new user expectations, and maintaining some code to workaround unknown issues.
The real question here is to know why this patch? Is it to workaround a broken driver?
Or to workaround a broken design in EAL and bus drivers?Two birds - one stone here: OVS needed a way to disable automatic probing cleanly (current workaround seen in multiple deployment is to add a dummy whitelisted device, which will be ignored by the PCI bus --> it sets the bus in whitelist mode but avoid probing anything), and as a bonus this option allows using devices that depends on other devices being probed already (LAG, representors, failsafe, etc).
Quoted 8 lines
I'm not sure having a dependent-probe by default is good, and that would be a big change.
If we are doing the genesis of this patch, the initial motivation should be asked for more details from Marvell people and David for the OVS side.
[1]: First proposal:
http://mails.dpdk.org/archives/dev/2019-September/144166.html My arguments: http://mails.dpdk.org/archives/dev/2019-September/144564.htmlOK so there are two needs:
Quoted 10 lines
1/ Better control whitelist/blacklist mode.
We already know that a rework is needed here. Unfortunately neither you or me had time to work on it, and others who were interested disappeared.2/ Associate ports with equivalent properties in applications. This must be done in applications.
Tweaking the probe order is a hack.An application that want to tightly control the port init order, currently (by doing exactly like me here, hotpluging one by one the ports), would still need the bigger hack that consist in inserting a whitelist PCI devargs with a dummy address, depending on a undocumented PCI bus feature consisting in ignoring matching errors but keeping probing policy from failed devargs processing.
Instead, with this patch this app can do
Quoted 8 lines
rte_eal_manual_probe_set(1); rte_eal_init();to have the same behavior and be able to hotplug ports as it sees fit.
You are worried about creating user expectations about this behavior (being forced to replicate in some way the functionality during the rewrite, as I understand it?), but then you are currently forcing users to expect this workaround to exist in the PCI bus, blocking devs from touching it as it will thus break current app configurations. I've seen systemd unit file using this -w dummy flag, as well as the programmatic equivalent. Which is better, to have to rework it cutting short these configs, or to propose beforehand a deprecation path?.
This rework won't happen in 20.05, nor in the medium term unless you decide to drive this change. This workaround serves three needs (PCI normalization, port congruence and port dependency) in a low-risk implementation.
Thomas,
What would be the resolution of this? What is your recommendation to fix the issue as you have the concern of this patch?
Issue:
1) When l2fwd does the forwarding for simplicity and performance reason it just xor the port to find the destination port to forward.
2) If the adjacent ports are not symmetrical(example: one is 40G and other 10G) then forwarding will drop the packets.
So, either
a) We need to control the probing order
b) Or Application need
1) To track the symmetrical ports and maintain the forwarding table OR
2) Have the command-line option to specify destination port like l3fwd.
We can fix it in the application, but do we need to complicate l2fwd? I am fine with that, if that is consensus.
Thoughts? If you think, there is a rework needed in eal then could you enumerate the items for the rework.
Re: [dpdk-dev] [PATCH v7] eal: add manual probing option
10/02/2020 15:51, Jerin Jacob:
Quoted 98 lines · Gaetan Rivet
Quoted 74 lines · Thomas Monjalon
Quoted 57 lines
04/02/2020 13:43, Gaetan Rivet:
Quoted 44 lines · Thomas Monjalon
Quoted 21 lines
04/02/2020 11:03, Gaetan Rivet:
Quoted 20 lines · Thomas Monjalon
Quoted 18 lines
03/02/2020 06:16, Pavan Nikhilesh Bhagavatula:
Quoted 5 lines
@David Marchand @[email protected]
Ping?
Are there any more changes required for this patch? It's been in queue since last October.
Sorry we have not decided whether it is a good idea or not.
All changes related to probing are very sensitive, and we know a big refactoring would be better than stacking more and more options and corner cases.
As we are busy with ABI stability stuff, we did not allocate enough time to properly think about this feature. Please accept our apologies, and let's consider it as a high priority for 20.05 cycle.
Quoted 8 lines
Quoted 5 lines
Hello Thomas,
This is unfortunate. I pushed Pavan to accept an alternative implementation of this functionality that was less obtrusive, to make the integration smoother. I took care to alleviate those risks from the common path.
The big refactoring is needed yes, but considering the current path I'm not seeing it happen in 20.05. If that means taking this patch as-is in 20.05 for Marvell users, I'm not sure much is gained from waiting 3 months, except minimal risk avoidance.
Yes, life is full of bad decisions and consequences.
Ah, yes, but I stand by my initial opinion, the first implementation [1] was riskier and less useful.
Quoted 7 lines
I still think there is a risk in adding new user expectations, and maintaining some code to workaround unknown issues.
The real question here is to know why this patch? Is it to workaround a broken driver?
Or to workaround a broken design in EAL and bus drivers?Two birds - one stone here: OVS needed a way to disable automatic probing cleanly (current workaround seen in multiple deployment is to add a dummy whitelisted device, which will be ignored by the PCI bus --> it sets the bus in whitelist mode but avoid probing anything), and as a bonus this option allows using devices that depends on other devices being probed already (LAG, representors, failsafe, etc).
Quoted 8 lines
I'm not sure having a dependent-probe by default is good, and that would be a big change.
If we are doing the genesis of this patch, the initial motivation should be asked for more details from Marvell people and David for the OVS side.
[1]: First proposal:
http://mails.dpdk.org/archives/dev/2019-September/144166.html My arguments: http://mails.dpdk.org/archives/dev/2019-September/144564.htmlOK so there are two needs:
Quoted 10 lines
1/ Better control whitelist/blacklist mode.
We already know that a rework is needed here. Unfortunately neither you or me had time to work on it, and others who were interested disappeared.2/ Associate ports with equivalent properties in applications. This must be done in applications.
Tweaking the probe order is a hack.An application that want to tightly control the port init order, currently (by doing exactly like me here, hotpluging one by one the ports), would still need the bigger hack that consist in inserting a whitelist PCI devargs with a dummy address, depending on a undocumented PCI bus feature consisting in ignoring matching errors but keeping probing policy from failed devargs processing.
Instead, with this patch this app can do
Quoted 8 lines
rte_eal_manual_probe_set(1); rte_eal_init();to have the same behavior and be able to hotplug ports as it sees fit.
You are worried about creating user expectations about this behavior (being forced to replicate in some way the functionality during the rewrite, as I understand it?), but then you are currently forcing users to expect this workaround to exist in the PCI bus, blocking devs from touching it as it will thus break current app configurations. I've seen systemd unit file using this -w dummy flag, as well as the programmatic equivalent. Which is better, to have to rework it cutting short these configs, or to propose beforehand a deprecation path?.
This rework won't happen in 20.05, nor in the medium term unless you decide to drive this change. This workaround serves three needs (PCI normalization, port congruence and port dependency) in a low-risk implementation.
Thomas,
What would be the resolution of this? What is your recommendation to fix the issue as you have the concern of this patch?
Issue:
1) When l2fwd does the forwarding for simplicity and performance reason it just xor the port to find the destination port to forward.
2) If the adjacent ports are not symmetrical(example: one is 40G and other 10G) then forwarding will drop the packets.So, either
a) We need to control the probing order
Quoted 6 lines
b) Or Application need
1) To track the symmetrical ports and maintain the forwarding table OR
2) Have the command-line option to specify destination port like l3fwd.We can fix it in the application, but do we need to complicate l2fwd? I am fine with that, if that is consensus.
You are describing an application issue,
that's why I believe it should be fixed in the application.
Should we have a command line option to configure the forwarding rules in the application (2)? I think yes.
Should we implement an application logic to automatically create the best forwarding rules (1)? It would be nice, but anyway, I think we need manual config (2) as a fallback.
Quoted 2 lines
Thoughts? If you think, there is a rework needed in eal then could you enumerate the items for the rework.
Sorry I don't have time to describe dive into EAL probing and
enumerate the items to rework.
The most important issues I remind are:
- white/blacklist policy is a mess and should be done in a higher layer
- devargs syntax should allow generic matching (thanks to class awareness and generic syntax)
Starting from these 2 items, we could imagine a generic path to disable automatic probing, but I think the l2fwd logic should not rely on probing order anyway.
Re: [dpdk-dev] [PATCH v7] eal: add manual probing option
Quoted 104 lines · Thomas Monjalon
10/02/2020 15:51, Jerin Jacob:
Quoted 102 lines · Gaetan Rivet
Quoted 74 lines · Thomas Monjalon
Quoted 57 lines
04/02/2020 13:43, Gaetan Rivet:
Quoted 44 lines · Thomas Monjalon
Quoted 21 lines
04/02/2020 11:03, Gaetan Rivet:
Quoted 20 lines · Thomas Monjalon
Quoted 18 lines
03/02/2020 06:16, Pavan Nikhilesh Bhagavatula:
Quoted 5 lines
@David Marchand @[email protected]
Ping?
Are there any more changes required for this patch? It's been in queue since last October.
Sorry we have not decided whether it is a good idea or not.
All changes related to probing are very sensitive, and we know a big refactoring would be better than stacking more and more options and corner cases.
As we are busy with ABI stability stuff, we did not allocate enough time to properly think about this feature. Please accept our apologies, and let's consider it as a high priority for 20.05 cycle.
Quoted 8 lines
Quoted 5 lines
Hello Thomas,
This is unfortunate. I pushed Pavan to accept an alternative implementation of this functionality that was less obtrusive, to make the integration smoother. I took care to alleviate those risks from the common path.
The big refactoring is needed yes, but considering the current path I'm not seeing it happen in 20.05. If that means taking this patch as-is in 20.05 for Marvell users, I'm not sure much is gained from waiting 3 months, except minimal risk avoidance.
Yes, life is full of bad decisions and consequences.
Ah, yes, but I stand by my initial opinion, the first implementation [1] was riskier and less useful.
Quoted 7 lines
I still think there is a risk in adding new user expectations, and maintaining some code to workaround unknown issues.
The real question here is to know why this patch? Is it to workaround a broken driver?
Or to workaround a broken design in EAL and bus drivers?Two birds - one stone here: OVS needed a way to disable automatic probing cleanly (current workaround seen in multiple deployment is to add a dummy whitelisted device, which will be ignored by the PCI bus --> it sets the bus in whitelist mode but avoid probing anything), and as a bonus this option allows using devices that depends on other devices being probed already (LAG, representors, failsafe, etc).
Quoted 8 lines
I'm not sure having a dependent-probe by default is good, and that would be a big change.
If we are doing the genesis of this patch, the initial motivation should be asked for more details from Marvell people and David for the OVS side.
[1]: First proposal:
http://mails.dpdk.org/archives/dev/2019-September/144166.html My arguments: http://mails.dpdk.org/archives/dev/2019-September/144564.htmlOK so there are two needs:
Quoted 10 lines
1/ Better control whitelist/blacklist mode.
We already know that a rework is needed here. Unfortunately neither you or me had time to work on it, and others who were interested disappeared.2/ Associate ports with equivalent properties in applications. This must be done in applications.
Tweaking the probe order is a hack.An application that want to tightly control the port init order, currently (by doing exactly like me here, hotpluging one by one the ports), would still need the bigger hack that consist in inserting a whitelist PCI devargs with a dummy address, depending on a undocumented PCI bus feature consisting in ignoring matching errors but keeping probing policy from failed devargs processing.
Instead, with this patch this app can do
Quoted 8 lines
rte_eal_manual_probe_set(1); rte_eal_init();to have the same behavior and be able to hotplug ports as it sees fit.
You are worried about creating user expectations about this behavior (being forced to replicate in some way the functionality during the rewrite, as I understand it?), but then you are currently forcing users to expect this workaround to exist in the PCI bus, blocking devs from touching it as it will thus break current app configurations. I've seen systemd unit file using this -w dummy flag, as well as the programmatic equivalent. Which is better, to have to rework it cutting short these configs, or to propose beforehand a deprecation path?.
This rework won't happen in 20.05, nor in the medium term unless you decide to drive this change. This workaround serves three needs (PCI normalization, port congruence and port dependency) in a low-risk implementation.
Thomas,
What would be the resolution of this? What is your recommendation to fix the issue as you have the concern of this patch?
Issue:
1) When l2fwd does the forwarding for simplicity and performance reason it just xor the port to find the destination port to forward.
2) If the adjacent ports are not symmetrical(example: one is 40G and other 10G) then forwarding will drop the packets.So, either
a) We need to control the probing orderb) Or Application need
1) To track the symmetrical ports and maintain the forwarding table OR
2) Have the command-line option to specify destination port like l3fwd.
Quoted 5 lines
Quoted 2 lines
We can fix it in the application, but do we need to complicate l2fwd? I am fine with that, if that is consensus.
You are describing an application issue,
that's why I believe it should be fixed in the application.
Thanks for the quick reply and I agree.
Quoted 17 lines
Should we have a command line option to configure the forwarding rules in the application (2)? I think yes.
Should we implement an application logic to automatically create the best forwarding rules (1)? It would be nice, but anyway, I think we need manual config (2) as a fallback.Quoted 2 lines
Thoughts? If you think, there is a rework needed in eal then could you enumerate the items for the rework.
Sorry I don't have time to describe dive into EAL probing and
enumerate the items to rework.
The most important issues I remind are:
- white/blacklist policy is a mess and should be done in a higher layer
- devargs syntax should allow generic matching (thanks to class awareness and generic syntax)
Quoted 3 lines
Starting from these 2 items, we could imagine a generic path to disable automatic probing, but I think the l2fwd logic should not rely on probing order anyway.
+ Bruce as l2fwd maintainer.
Since in any case, l2fwd needs to be updated, We will focus on l2fwd change for v20.05 and leaving the fate of this patch to EAL maintainers.
Let us know, Are we are OK with below change in l2fwd as
- Introduce an array-based port lookup table instead of hardcoding to xor based lookup.
- if no argument specified fill dest port as xor of source
- If argument is specified override the lookup table with a user-specified destination port.
Quoted 2 lines
PATCH 1/1 examples/l2fwd: add cmdline option for forwarding port info
From: Vamsi Attunuru <[email protected]>
Current l2fwd application configures adjacent ports as destination ports for forwarding the traffic which is a kind of static mapping that can not be altered by the command line options.
Patch adds a config option to pass the forwarding port pair mapping as a command line parameter which allows the user to pass required forwarding port mapping.
If no config argument is specified, destination port map is not changed and traffic gets forwarded with existing mapping.
When port pair mapping is passed in config option, destination port map is configured and traffic gets forwarded accordingly.
Ex: ./l2fwd -c 0xff -- -p 0x3f --config="(0,3)(1,4)(2,5)"
With above config option, traffic received from portid = 0 gets forwarded to port = 3 and vice versa, similarly traffic gets forwarded on other port pairs (1,4) and (2,5).
--- doc/guides/rel_notes/release_20_05.rst | 6 + .../sample_app_ug/l2_forward_real_virtual.rst | 18 ++- examples/l2fwd/main.c | 174 ++++++++++++++++++--- 3 files changed, 177 insertions(+), 21 deletions(-)
doc/guides/rel_notes/release_20_05.rst +6 -0
@@ -62,6 +62,12 @@ New Features * Added support for matching on IPv4 Time To Live and IPv6 Hop Limit. +* **Added --config command line parameter to l2fwd example.** + + Added new command line option ``--config(port, port)[,(port, port)]`` to + pass forwarding port details. + See the :doc:`doc/guides/sample_app_ug/l2_forward_real_virtual` for more + details of this parameter usage. Removed Items -------------
doc/guides/sample_app_ug/l2_forward_real_virtual.rst +16 -2
@@ -91,7 +91,10 @@ The application requires a number of command line options: .. code-block:: console - ./build/l2fwd [EAL options] -- -p PORTMASK [-q NQ] --[no-]mac-updating + ./build/l2fwd [EAL options] -- -p PORTMASK + [-q NQ] + --[no-]mac-updating + --config(port, port)[,(port, port)] where,
@@ -99,7 +102,10 @@ where, * q NQ: A number of queues (=ports) per lcore (default is 1) -* --[no-]mac-updating: Enable or disable MAC addresses updating (enabled by default). +* --[no-]mac-updating: Enable or disable MAC addresses updating (enabled by default) + +* --config(port,port)[,(port,port)]: Determines which ports are mapped to + which ports for packet forwarding. To run the application in linux environment with 4 lcores, 16 ports and 8 RX queues per lcore and MAC address updating enabled, issue the command:
@@ -108,6 +114,14 @@ updating enabled, issue the command: $ ./build/l2fwd -l 0-3 -n 4 -- -q 8 -p ffff +To run the application in linux environment with 4 lcores, 4 ports, 8 RX queues +per lcore and MAC address updating enabled, to forward RX traffic of ports 0 & 1 +on ports 2 & 3 respectively and vice versa, issue the command: + +.. code-block:: console + + $ ./build/l2fwd -l 0-3 -n 4 -- -q 8 -p f --config="(0,2)(1,3)" + Refer to the *DPDK Getting Started Guide* for general information on running applications and the Environment Abstraction Layer (EAL) options.
examples/l2fwd/main.c +155 -19
248 lines not on this page · raw message
Signature · 1 line
2.8.4
Re: [dpdk-dev] [PATCH] [v1 1/1] examples/l2fwd: add cmdline option for forwarding port info
Quoted 34 lines · On 4/3/20 5:30 AM, [email protected] wrote:
From: Vamsi Attunuru <[email protected]>
Current l2fwd application configures adjacent ports as destination ports for forwarding the traffic which is a kind of static mapping that can not be altered by the command line options.
Patch adds a config option to pass the forwarding port pair mapping as a command line parameter which allows the user to pass required forwarding port mapping.
If no config argument is specified, destination port map is not changed and traffic gets forwarded with existing mapping.
When port pair mapping is passed in config option, destination port map is configured and traffic gets forwarded accordingly.
Ex: ./l2fwd -c
0xff-- -p0x3f--config="(0,3)(1,4)(2,5)"With above config option, traffic received from portid = 0 gets forwarded to port = 3 and vice versa, similarly traffic gets forwarded on other port pairs (1,4) and (2,5).
Signed-off-by Vamsi Attunuru <[email protected]>--- doc/guides/rel_notes/release_20_05.rst | 6 + .../sample_app_ug/l2_forward_real_virtual.rst | 18 ++- examples/l2fwd/main.c | 174 ++++++++++++++++++--- 3 files changed, 177 insertions(+), 21 deletions(-)doc/guides/rel_notes/release_20_05.rst +0
-0
doc/guides/rel_notes/release_20_05.rst New Features · line 67
Quoted 6 lines
doc/guides/rel_notes/release_20_05.rst +3
-0@@ -62,6 +62,12 @@ New Features * Added support for matching on IPv4 Time To Live and IPv6 Hop Limit. +* **Added --config command line parameter to l2fwd example.** + + Added new command line option ``--config(port, port)[,(port, port)]`` to
Maybe use the actual syntax with '=' or space after 'config'?
Quoted 11 lines
doc/guides/rel_notes/release_20_05.rst +3
-0+ pass forwarding port details. + See the :doc:`doc/guides/sample_app_ug/l2_forward_real_virtual` for more + details of this parameter usage. Removed Items -------------doc/guides/sample_app_ug/l2_forward_real_virtual.rst +0
-0
doc/guides/sample_app_ug/l2_forward_real_virtual.rst The application requires a number of command line options: · line 97
Quoted 8 lines
doc/guides/sample_app_ug/l2_forward_real_virtual.rst +4
-1@@ -91,7 +91,10 @@ The application requires a number of command line options: .. code-block:: console - ./build/l2fwd [EAL options] -- -p PORTMASK [-q NQ] --[no-]mac-updating + ./build/l2fwd [EAL options] -- -p PORTMASK + [-q NQ] + --[no-]mac-updating + --config(port, port)[,(port, port)]
Ditto
doc/guides/sample_app_ug/l2_forward_real_virtual.rst where, · line 108
Quoted 12 lines
doc/guides/sample_app_ug/l2_forward_real_virtual.rst +4
-1where,@@ -99,7 +102,10 @@ where, * q NQ: A number of queues (=ports) per lcore (default is 1) -* --[no-]mac-updating: Enable or disable MAC addresses updating (enabled by default). +* --[no-]mac-updating: Enable or disable MAC addresses updating (enabled by default) + +* --config(port,port)[,(port,port)]: Determines which ports are mapped to + which ports for packet forwarding.
Ditto
Quoted 32 lines
doc/guides/sample_app_ug/l2_forward_real_virtual.rst +8
-0To run the application in linux environment with 4 lcores, 16 ports and 8 RX queues per lcore and MAC address updating enabled, issue the command:@@ -108,6 +114,14 @@ updating enabled, issue the command: $ ./build/l2fwd -l 0-3 -n 4 -- -q 8 -p ffff +To run the application in linux environment with 4 lcores, 4 ports, 8 RX queues +per lcore and MAC address updating enabled, to forward RX traffic of ports 0 & 1 +on ports 2 & 3 respectively and vice versa, issue the command: + +.. code-block:: console + + $ ./build/l2fwd -l 0-3 -n 4 -- -q 8 -p f --config="(0,2)(1,3)" + Refer to the *DPDK Getting Started Guide* for general information on running applications and the Environment Abstraction Layer (EAL) options.examples/l2fwd/main.c +1
-0@@ -38,6 +38,7 @@#include <rte_ethdev.h> #include <rte_mempool.h> #include <rte_mbuf.h> +#include <rte_string_fns.h> static volatile bool force_quit;@@ -67,6 +68,15 @@ static uint32_t l2fwd_enabled_port_mask = 0;/* list of enabled ports */
examples/l2fwd/main.c static uint32_t l2fwd_enabled_port_mask = 0; · line 78
Quoted 10 lines
examples/l2fwd/main.c +8
-0static uint32_t l2fwd_dst_ports[RTE_MAX_ETHPORTS ]; +struct port_pair_params { +#define NUM_PORTS 2 +uint16_t port[NUM_PORTS ]; +} __rte_cache_aligned; + +static struct port_pair_params port_pair_params_array[RTE_MAX_ETHPORTS ]; +static struct port_pair_params *port_pair_params; +static uint16_t nb_port_pair_params =1 ;
Why this initialization to 1? Below (during parsing) it is initialized to 0, so I would leave this default initialized.
Quoted 27 lines
examples/l2fwd/main.c +20
-0+static unsigned int l2fwd_rx_queue_per_lcore =1 ;#define MAX_RX_QUEUE_PER_LCORE 16@@ -319,6 +329,60 @@ l2fwd_parse_portmask(const char *portmask)return pm; } +static int +l2fwd_parse_port_pair_config (const char *q_arg) +{ +enum fieldnames { +FLD_PORT1 =0 , +FLD_PORT2 , + _NUM_FLD + }; +unsigned long int_fld[_NUM_FLD]; +const char *p, *p0 = q_arg; +char *str_fld[_NUM_FLD]; +unsigned int size; +char s[256 ]; +char *end; +int i; + + nb_port_pair_params =0 ; + +while ((p =strchr (p0,'(' )) !=NULL ) {
examples/l2fwd/main.c l2fwd_parse_portmask(const char *portmask) · line 360
Quoted 10 lines
examples/l2fwd/main.c +10
-0+ ++p; + p0 =strchr (p,')' ); +if (p0 ==NULL ) +return -1 ; + + size = p0 - p; +if (size >=sizeof (s)) +return -1 ; + +snprintf (s,sizeof (s),"%.*s" , size, p);
Same comment as for recent l2fwd-event - this can be simple memcpy.
examples/l2fwd/main.c l2fwd_parse_portmask(const char *portmask) · line 368
Quoted 8 lines
examples/l2fwd/main.c +8
-0+if (rte_strsplit (s,sizeof (s), str_fld, + _NUM_FLD,',' ) != _NUM_FLD) +return -1 ; +for (i =0 ; i < _NUM_FLD; i++) { + errno =0 ; + int_fld[i] =strtoul (str_fld[i], &end,0 ); +if (errno !=0 || end == str_fld[i] || + int_fld[i] >RTE_MAX_ETHPORTS )
I think this check should be >=, RTE_MAX_ETHPORTS is an invalid port id.
Quoted 16 lines
examples/l2fwd/main.c +16
-0+return -1 ; + } +if (nb_port_pair_params >=RTE_MAX_ETHPORTS /2 ) { +printf ("exceeded max number of port pair params: %hu\n" , + nb_port_pair_params); +return -1 ; + } + port_pair_params_array[nb_port_pair_params].port[0 ] = + (uint16_t )int_fld[FLD_PORT1 ]; + port_pair_params_array[nb_port_pair_params].port[1 ] = + (uint16_t )int_fld[FLD_PORT2 ]; + ++nb_port_pair_params; + } + port_pair_params = port_pair_params_array; +return 0 ; +}
[...]
Quoted 15 lines
examples/l2fwd/main.c +4
-7@@ -565,26 +686,40 @@ main(int argc, char **argv) l2fwd_dst_ports[portid] =0 ; last_port =0 ; -/* -* Each logical core is assigned a dedicated TX queue on each port. -*/ -RTE_ETH_FOREACH_DEV (portid) { -/* skip ports that are not enabled */ -if ((l2fwd_enabled_port_mask & (1 << portid)) ==0 ) -continue ; +/* populate destination port details */ +if (port_pair_params !=NULL ) { +uint16_t idx; +
examples/l2fwd/main.c main(int argc, char **argv) · line 702
Quoted 10 lines
examples/l2fwd/main.c +10
-0+for (idx =0 ; idx < (nb_port_pair_params <<1 ); idx++) { +if (idx %2 ==0 ) { + portid = port_pair_params[idx >>1 ].port[0 ]; + l2fwd_dst_ports[portid] = + port_pair_params[idx >>1 ].port[1 ]; + }else { + portid = port_pair_params[idx >>1 ].port[1 ]; + l2fwd_dst_ports[portid] = + port_pair_params[idx >>1 ].port[0 ]; + }
I think this conditional can be rephrased like:
uint16_t idx, p; ... p = idx & 1; port_id = port_pair_params[idx >> 1].port[p]; l2fwd_dst_ports[port_id] = port_pair_params[idx >> 1].port[p ^ 1];
but I'm not sure which one is more readable - I leave you the decision,
feel free to skip this comment altogether :).
Quoted 35 lines
examples/l2fwd/main.c +17
-10+ } + }else { +RTE_ETH_FOREACH_DEV (portid) { +/* skip ports that are not enabled */ +if ((l2fwd_enabled_port_mask & (1 << portid)) ==0 ) +continue ; + +if (nb_ports_in_mask %2 ) { + l2fwd_dst_ports[portid] = last_port; + l2fwd_dst_ports[last_port] = portid; + }else { + last_port = portid; + } + nb_ports_in_mask++; + }if (nb_ports_in_mask %2 ) { - l2fwd_dst_ports[portid] = last_port; - l2fwd_dst_ports[last_port] = portid; +printf ("Notice: odd number of ports in portmask.\n" ); + l2fwd_dst_ports[last_port] = last_port; } -else - last_port = portid; - - nb_ports_in_mask++; - } -if (nb_ports_in_mask %2 ) { -printf ("Notice: odd number of ports in portmask.\n" ); - l2fwd_dst_ports[last_port] = last_port; } rx_lcore_id =0 ;
examples/l2fwd/main.c main(int argc, char **argv) · line 755
Quoted 9 lines
examples/l2fwd/main.c +2
-1@@ -613,7 +748,8 @@ main(int argc, char **argv) qconf->rx_port_list[qconf->n_rx_port] = portid; qconf->n_rx_port++; -printf ("Lcore %u: RX port %u\n" , rx_lcore_id, portid); +printf ("Lcore %u: RX port %u TX port %u\n" , rx_lcore_id, + portid, l2fwd_dst_ports[portid]); } nb_mbufs = RTE_MAX(nb_ports * (nb_rxd + nb_txd + MAX_PKT_BURST +
Apart from these comments:
With regards
Andrzej Ostruszka
Re: [dpdk-dev] [PATCH] [v1 1/1] examples/l2fwd: add cmdline option for forwarding port info
Quoted 306 lines
-----Original Message-----
From: Andrzej Ostruszka [C] <[email protected]>
Sent: Friday, April 3, 2020 6:22 PM
To: Vamsi Krishna Attunuru <[email protected]>; [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; Jerin Jacob Kollanukkaran <[email protected]>; Pavan Nikhilesh Bhagavatula <[email protected]>; [email protected]
Subject: Re: [PATCH] [v1 1/1] examples/l2fwd: add cmdline option for forwarding port infoQuoted 35 lines · On 4/3/20 5:30 AM, [email protected] wrote:
From: Vamsi Attunuru <[email protected]>
Current l2fwd application configures adjacent ports as destination ports for forwarding the traffic which is a kind of static mapping that can not be altered by the command line options.
Patch adds a config option to pass the forwarding port pair mapping as a command line parameter which allows the user to pass required forwarding port mapping.
If no config argument is specified, destination port map is not changed and traffic gets forwarded with existing mapping.
When port pair mapping is passed in config option, destination port map is configured and traffic gets forwarded accordingly.
Ex: ./l2fwd -c
0xff-- -p0x3f--config="(0,3)(1,4)(2,5)"With above config option, traffic received from portid = 0 gets forwarded to port = 3 and vice versa, similarly traffic gets forwarded on other port pairs (1,4) and (2,5).
Signed-off-by Vamsi Attunuru <[email protected]>--- doc/guides/rel_notes/release_20_05.rst | 6 + .../sample_app_ug/l2_forward_real_virtual.rst | 18 ++- examples/l2fwd/main.c | 174 ++++++++++++++++++--- 3 files changed, 177 insertions(+), 21 deletions(-)doc/guides/rel_notes/release_20_05.rst +0
-0doc/guides/rel_notes/release_20_05.rst New Features · line 68
Quoted 7 lines
doc/guides/rel_notes/release_20_05.rst +4
-0@@ -62,6 +62,12 @@ New Features * Added support for matching on IPv4 Time To Live and IPv6 Hop Limit. +* **Added --config command line parameter to l2fwd example.** + + Added new command line option ``--config(port, port)[,(port, + port)]`` toMaybe use the actual syntax with '=' or space after 'config'?
doc/guides/sample_app_ug/l2_forward_real_virtual.rst The application requires a number of command line
Quoted 12 lines
doc/guides/rel_notes/release_20_05.rst +3
-0+ pass forwarding port details. + See the :doc:`doc/guides/sample_app_ug/l2_forward_real_virtual` for + more details of this parameter usage. Removed Items -------------doc/guides/sample_app_ug/l2_forward_real_virtual.rst +0
-0@@ -91,7 +91,10 @@ The application requires a number of command lineoptions:
doc/guides/sample_app_ug/l2_forward_real_virtual.rst The application requires a number of command line options:
Quoted 9 lines
doc/guides/sample_app_ug/l2_forward_real_virtual.rst +5
-1.. code-block:: console - ./build/l2fwd [EAL options] -- -p PORTMASK [-q NQ] --[no-]mac-updating + ./build/l2fwd [EAL options] -- -p PORTMASK + [-q NQ] + --[no-]mac-updating + --config(port, port)[,(port, + port)]Ditto
doc/guides/sample_app_ug/l2_forward_real_virtual.rst where, · line 102
Quoted 8 lines
doc/guides/sample_app_ug/l2_forward_real_virtual.rst +0
-1where,@@ -99,7 +102,10 @@ where, * q NQ: A number of queues (=ports) per lcore (default is 1) -* --[no-]mac-updating: Enable or disable MAC addresses updating(enabled by default).
doc/guides/sample_app_ug/l2_forward_real_virtual.rst where, · line 105
Quoted 1 line
doc/guides/sample_app_ug/l2_forward_real_virtual.rst +1
-0+* --[no-]mac-updating: Enable or disable MAC addresses updating(enabled by default)
doc/guides/sample_app_ug/l2_forward_real_virtual.rst where, · line 107
Quoted 2 lines
doc/guides/sample_app_ug/l2_forward_real_virtual.rst +2
-0+ +* --config(port,port)[,(port,port)]: Determines which ports are mappedto
doc/guides/sample_app_ug/l2_forward_real_virtual.rst where, · line 108
Quoted 1 line
doc/guides/sample_app_ug/l2_forward_real_virtual.rst +1
-0+ which ports for packet forwarding.Ditto
Quoted 3 lines
doc/guides/sample_app_ug/l2_forward_real_virtual.rst +0
-0To run the application in linux environment with 4 lcores, 16 portsand 8 RX queues per lcore and MAC address updating enabled, issue the
command:
doc/guides/sample_app_ug/l2_forward_real_virtual.rst updating enabled, issue the command: · line 119
Quoted 7 lines
doc/guides/sample_app_ug/l2_forward_real_virtual.rst +3
-0@@ -108,6 +114,14 @@ updating enabled, issue the command: $ ./build/l2fwd -l 0-3 -n 4 -- -q 8 -p ffff +To run the application in linux environment with 4 lcores, 4 ports, 8 +RX queues per lcore and MAC address updating enabled, to forward RX +traffic of ports 0 & 1 on ports 2 & 3 respectively and vice versa, issue thecommand:
Quoted 23 lines
doc/guides/sample_app_ug/l2_forward_real_virtual.rst +5
-0+ +.. code-block:: console + + $ ./build/l2fwd -l 0-3 -n 4 -- -q 8 -p f --config="(0,2)(1,3)" + Refer to the *DPDK Getting Started Guide* for general information onrunning applications and the Environment Abstraction Layer (EAL) options.
examples/l2fwd/main.c index renamed from examples/l2fwd/main.c +0
-009257aa..a8b98b9 100644
examples/l2fwd/main.c +1
-0@@ -38,6 +38,7 @@#include <rte_ethdev.h> #include <rte_mempool.h> #include <rte_mbuf.h> +#include <rte_string_fns.h> static volatile bool force_quit;@@ -67,6 +68,15 @@ static uint32_t l2fwd_enabled_port_mask = 0;/* list of enabled ports */ static uint32_t l2fwd_dst_ports[RTE_MAX_ETHPORTS ];examples/l2fwd/main.c static uint32_t l2fwd_enabled_port_mask = 0; · line 79
Quoted 10 lines
examples/l2fwd/main.c +9
-0+struct port_pair_params { +#define NUM_PORTS 2 +uint16_t port[NUM_PORTS ]; +} __rte_cache_aligned; + +static struct port_pair_params +port_pair_params_array[RTE_MAX_ETHPORTS ]; +static struct port_pair_params *port_pair_params;static uint16_t +nb_port_pair_params =1 ;Why this initialization to 1? Below (during parsing) it is initialized to 0, so I would leave this default initialized.
Quoted 26 lines
examples/l2fwd/main.c +19
-0+static unsigned int l2fwd_rx_queue_per_lcore =1 ;#define MAX_RX_QUEUE_PER_LCORE 16@@ -319,6 +329,60 @@ l2fwd_parse_portmask(const char *portmask)return pm; } +static int +l2fwd_parse_port_pair_config (const char *q_arg) { +enum fieldnames { +FLD_PORT1 =0 , +FLD_PORT2 , + _NUM_FLD + }; +unsigned long int_fld[_NUM_FLD]; +const char *p, *p0 = q_arg; +char *str_fld[_NUM_FLD]; +unsigned int size; +char s[256 ]; +char *end; +int i; + + nb_port_pair_params =0 ; + +while ((p =strchr (p0,'(' )) !=NULL ) {examples/l2fwd/main.c l2fwd_parse_portmask(const char *portmask) · line 359
Quoted 10 lines
examples/l2fwd/main.c +10
-0+ ++p; + p0 =strchr (p,')' ); +if (p0 ==NULL ) +return -1 ; + + size = p0 - p; +if (size >=sizeof (s)) +return -1 ; + +snprintf (s,sizeof (s),"%.*s" , size, p);Same comment as for recent l2fwd-event - this can be simple memcpy.
examples/l2fwd/main.c l2fwd_parse_portmask(const char *portmask) · line 368
Quoted 8 lines
examples/l2fwd/main.c +8
-0+if (rte_strsplit (s,sizeof (s), str_fld, + _NUM_FLD,',' ) != _NUM_FLD) +return -1 ; +for (i =0 ; i < _NUM_FLD; i++) { + errno =0 ; + int_fld[i] =strtoul (str_fld[i], &end,0 ); +if (errno !=0 || end == str_fld[i] || + int_fld[i] >RTE_MAX_ETHPORTS )I think this check should be >=,
RTE_MAX_ETHPORTSis an invalid port id.examples/l2fwd/main.c l2fwd_parse_portmask(const char *portmask)
Quoted 4 lines
examples/l2fwd/main.c +4
-0+return -1 ; + } +if (nb_port_pair_params >=RTE_MAX_ETHPORTS /2 ) { + printf("exceeded max number of port pair params:%hu\n",
Quoted 12 lines
examples/l2fwd/main.c +12
-0+ nb_port_pair_params); +return -1 ; + } + port_pair_params_array[nb_port_pair_params].port[0 ] = + (uint16_t )int_fld[FLD_PORT1 ]; + port_pair_params_array[nb_port_pair_params].port[1 ] = + (uint16_t )int_fld[FLD_PORT2 ]; + ++nb_port_pair_params; + } + port_pair_params = port_pair_params_array; +return 0 ; +}[...]
Quoted 15 lines
examples/l2fwd/main.c +4
-7@@ -565,26 +686,40 @@ main(int argc, char **argv) l2fwd_dst_ports[portid] =0 ; last_port =0 ; -/* -* Each logical core is assigned a dedicated TX queue on each port. -*/ -RTE_ETH_FOREACH_DEV (portid) { -/* skip ports that are not enabled */ -if ((l2fwd_enabled_port_mask & (1 << portid)) ==0 ) -continue ; +/* populate destination port details */ +if (port_pair_params !=NULL ) { +uint16_t idx; +examples/l2fwd/main.c main(int argc, char **argv) · line 702
Quoted 10 lines
examples/l2fwd/main.c +10
-0+for (idx =0 ; idx < (nb_port_pair_params <<1 ); idx++) { +if (idx %2 ==0 ) { + portid = port_pair_params[idx >>1 ].port[0 ]; + l2fwd_dst_ports[portid] = + port_pair_params[idx >>1 ].port[1 ]; + }else { + portid = port_pair_params[idx >>1 ].port[1 ]; + l2fwd_dst_ports[portid] = + port_pair_params[idx >>1 ].port[0 ]; + }I think this conditional can be rephrased like:
uint16_t idx, p; ... p = idx & 1; port_id = port_pair_params[idx >> 1].port[p]; l2fwd_dst_ports[port_id] = port_pair_params[idx >> 1].port[p ^ 1];but I'm not sure which one is more readable - I leave you the decision, feel free to skip this comment altogether :).
Quoted 10 lines
examples/l2fwd/main.c +10
-0+ } + }else { +RTE_ETH_FOREACH_DEV (portid) { +/* skip ports that are not enabled */ +if ((l2fwd_enabled_port_mask & (1 << portid)) ==0 ) +continue ; + +if (nb_ports_in_mask %2 ) { + l2fwd_dst_ports[portid] = last_port; + l2fwd_dst_ports[last_port] = portid;examples/l2fwd/main.c main(int argc, char **argv)
Quoted 10 lines
examples/l2fwd/main.c +6
-2+ }else { + last_port = portid; + } + nb_ports_in_mask++; + }if (nb_ports_in_mask %2 ) { - l2fwd_dst_ports[portid] = last_port; - l2fwd_dst_ports[last_port] = portid; + printf("Notice: odd number of ports inportmask.\n");
Quoted 14 lines
examples/l2fwd/main.c +1
-8+ l2fwd_dst_ports[last_port] = last_port; } -else - last_port = portid; - - nb_ports_in_mask++; - } -if (nb_ports_in_mask %2 ) { -printf ("Notice: odd number of ports in portmask.\n" ); - l2fwd_dst_ports[last_port] = last_port; } rx_lcore_id =0 ;examples/l2fwd/main.c main(int argc, char **argv) · line 755
Quoted 9 lines
examples/l2fwd/main.c +2
-1@@ -613,7 +748,8 @@ main(int argc, char **argv) qconf->rx_port_list[qconf->n_rx_port] = portid; qconf->n_rx_port++; -printf ("Lcore %u: RX port %u\n" , rx_lcore_id, portid); +printf ("Lcore %u: RX port %u TX port %u\n" , rx_lcore_id, + portid, l2fwd_dst_ports[portid]); } nb_mbufs = RTE_MAX(nb_ports * (nb_rxd + nb_txd +
MAX_PKT_BURST+Quoted 1 line
Quoted 6 lines
Apart from these comments:
Reviewed-by Andrzej Ostruszka <[email protected]>With regards
Andrzej Ostruszka
Thanks Andrzej,
Acked all the comments in V2.
PATCH v2 1/1 examples/l2fwd: add cmdline option for forwarding port info
In reply to [email protected]From: Vamsi Attunuru <[email protected]>
Current l2fwd application configures adjacent ports as destination ports for forwarding the traffic which is a kind of static mapping that can not be altered by the command line options.
Patch adds a config option to pass the forwarding port pair mapping as a command line parameter which allows the user to pass required forwarding port mapping.
If no config argument is specified, destination port map is not changed and traffic gets forwarded with existing mapping.
When port pair mapping is passed in config option, destination port map is configured and traffic gets forwarded accordingly.
Ex: ./l2fwd -c 0xff -- -p 0x3f --config="(0,3)(1,4)(2,5)"
With above config option, traffic received from portid = 0 gets forwarded to port = 3 and vice versa, similarly traffic gets forwarded on other port pairs (1,4) and (2,5).
---
V2 Changes:
* Fix command option format in docs.
* Use memcpy instead of snprintf.
* Rephrase if-else condition.
doc/guides/rel_notes/release_20_05.rst | 6 + .../sample_app_ug/l2_forward_real_virtual.rst | 18 ++- examples/l2fwd/main.c | 169 ++++++++++++++++++--- 3 files changed, 172 insertions(+), 21 deletions(-)
doc/guides/rel_notes/release_20_05.rst +6 -0
@@ -62,6 +62,12 @@ New Features * Added support for matching on IPv4 Time To Live and IPv6 Hop Limit. +* **Added --config command line parameter to l2fwd example.** + + Added new command line option ``--config="(port, port)[,(port, port)]"`` to + pass forwarding port details. + See the :doc:`doc/guides/sample_app_ug/l2_forward_real_virtual` for more + details of this parameter usage. Removed Items -------------
doc/guides/sample_app_ug/l2_forward_real_virtual.rst +16 -2
@@ -91,7 +91,10 @@ The application requires a number of command line options: .. code-block:: console - ./build/l2fwd [EAL options] -- -p PORTMASK [-q NQ] --[no-]mac-updating + ./build/l2fwd [EAL options] -- -p PORTMASK + [-q NQ] + --[no-]mac-updating + --config="(port, port)[,(port, port)]" where,
@@ -99,7 +102,10 @@ where, * q NQ: A number of queues (=ports) per lcore (default is 1) -* --[no-]mac-updating: Enable or disable MAC addresses updating (enabled by default). +* --[no-]mac-updating: Enable or disable MAC addresses updating (enabled by default) + +* --config="(port,port)[,(port,port)]": Determines which ports are mapped to + which ports for packet forwarding. To run the application in linux environment with 4 lcores, 16 ports and 8 RX queues per lcore and MAC address updating enabled, issue the command:
@@ -108,6 +114,14 @@ updating enabled, issue the command: $ ./build/l2fwd -l 0-3 -n 4 -- -q 8 -p ffff +To run the application in linux environment with 4 lcores, 4 ports, 8 RX queues +per lcore and MAC address updating enabled, to forward RX traffic of ports 0 & 1 +on ports 2 & 3 respectively and vice versa, issue the command: + +.. code-block:: console + + $ ./build/l2fwd -l 0-3 -n 4 -- -q 8 -p f --config="(0,2)(1,3)" + Refer to the *DPDK Getting Started Guide* for general information on running applications and the Environment Abstraction Layer (EAL) options.
examples/l2fwd/main.c +150 -19
243 lines not on this page · raw message
Signature · 1 line
2.8.4
Re: [dpdk-dev] [PATCH] [v2 1/1] examples/l2fwd: add cmdline option for forwarding port info
Quoted 26 lines · On 4/5/20 5:52 AM, [email protected] wrote:
From: Vamsi Attunuru <[email protected]>
Current l2fwd application configures adjacent ports as destination ports for forwarding the traffic which is a kind of static mapping that can not be altered by the command line options.
Patch adds a config option to pass the forwarding port pair mapping as a command line parameter which allows the user to pass required forwarding port mapping.
If no config argument is specified, destination port map is not changed and traffic gets forwarded with existing mapping.
When port pair mapping is passed in config option, destination port map is configured and traffic gets forwarded accordingly.
Ex: ./l2fwd -c
0xff-- -p0x3f--config="(0,3)(1,4)(2,5)"With above config option, traffic received from portid = 0 gets forwarded to port = 3 and vice versa, similarly traffic gets forwarded on other port pairs (1,4) and (2,5).
Signed-off-by Vamsi Attunuru <[email protected]>---
V2 Changes:
Quoted 8 lines
* Fix command option format in docs.
* Use memcpy instead of snprintf.
* Rephrase if-else condition.doc/guides/rel_notes/release_20_05.rst | 6 + .../sample_app_ug/l2_forward_real_virtual.rst | 18 ++- examples/l2fwd/main.c | 169 ++++++++++++++++++--- 3 files changed, 172 insertions(+), 21 deletions(-)
Re: [dpdk-dev] [PATCH] [v2 1/1] examples/l2fwd: add cmdline option for forwarding port info
In reply to [email protected]05/04/2020 05:52, [email protected]:
Quoted 10 lines
From: Vamsi Attunuru <[email protected]>
Current l2fwd application configures adjacent ports as destination ports for forwarding the traffic which is a kind of static mapping that can not be altered by the command line options.
Patch adds a config option to pass the forwarding port pair mapping as a command line parameter which allows the user to pass required forwarding port mapping.
Quoted 7 lines
If no config argument is specified, destination port map is not changed and traffic gets forwarded with existing mapping.
When port pair mapping is passed in config option, destination port map is configured and traffic gets forwarded accordingly.
Ex: ./l2fwd -c
0xff-- -p0x3f--config="(0,3)(1,4)(2,5)"
The option name "--config" is vague.
Could we call it "--portmap", or something similar?
PATCH v3 1/1 examples/l2fwd: add cmdline option for forwarding port info
In reply to [email protected]From: Vamsi Attunuru <[email protected]>
Current l2fwd application statically configures adjacent ports as destination ports for forwarding the traffic.
Add a portmap option to pass the forwarding port pair mapping which allows the user to configure forwarding port mapping.
If no portmap argument is specified, destination port map is not changed and traffic gets forwarded with existing mapping.
To align port/queue configuration of each lcore with destination port map, port/queue configuration of each lcore gets modified when portmap option is specified.
Ex: ./l2fwd -c 0xff -- -p 0x3f -q 2 --portmap="(0,3)(1,4)(2,5)"
With above portmap option, traffic received from portid = 0 gets forwarded to port = 3 and vice versa, similarly traffic gets forwarded on other port pairs (1,4) and (2,5)
---
v3 Changes:
* s/config/portmap/
* rebase on master
v2 Changes:
* Fix command option format in docs.
* Use memcpy instead of snprintf.
* Rephrase if-else condition.
doc/guides/rel_notes/release_20_05.rst | 7 + .../sample_app_ug/l2_forward_real_virtual.rst | 18 +- examples/l2fwd/main.c | 182 +++++++++++++++--- 3 files changed, 181 insertions(+), 26 deletions(-)
doc/guides/rel_notes/release_20_05.rst +7 -0
@@ -212,6 +212,13 @@ New Features * Added IPsec inbound load-distribution support for ipsec-secgw application using NIC load distribution feature(Flow Director). +* **Added --portmap command line parameter to l2fwd example.** + + Added new command line option ``--portmap="(port, port)[,(port, port)]"`` to + pass forwarding port details. + See the :doc:`doc/guides/sample_app_ug/l2_forward_real_virtual` for more + details of this parameter usage. + Removed Items -------------
doc/guides/sample_app_ug/l2_forward_real_virtual.rst +16 -2
@@ -91,7 +91,10 @@ The application requires a number of command line options: .. code-block:: console - ./build/l2fwd [EAL options] -- -p PORTMASK [-q NQ] --[no-]mac-updating + ./build/l2fwd [EAL options] -- -p PORTMASK + [-q NQ] + --[no-]mac-updating + --portmap="(port, port)[,(port, port)]" where,
@@ -99,7 +102,10 @@ where, * q NQ: A number of queues (=ports) per lcore (default is 1) -* --[no-]mac-updating: Enable or disable MAC addresses updating (enabled by default). +* --[no-]mac-updating: Enable or disable MAC addresses updating (enabled by default) + +* --portmap="(port,port)[,(port,port)]": Determines which ports are mapped to + which ports for packet forwarding. To run the application in linux environment with 4 lcores, 16 ports and 8 RX queues per lcore and MAC address updating enabled, issue the command:
@@ -108,6 +114,14 @@ updating enabled, issue the command: $ ./build/l2fwd -l 0-3 -n 4 -- -q 8 -p ffff +To run the application in linux environment with 4 lcores, 4 ports, 8 RX queues +per lcore and MAC address updating enabled, to forward RX traffic of ports 0 & 1 +on ports 2 & 3 respectively and vice versa, issue the command: + +.. code-block:: console + + $ ./build/l2fwd -l 0-3 -n 4 -- -q 8 -p f --portmap="(0,2)(1,3)" + Refer to the *DPDK Getting Started Guide* for general information on running applications and the Environment Abstraction Layer (EAL) options.
examples/l2fwd/main.c +158 -24
262 lines not on this page · raw message
--
2.17.1
Re: [dpdk-dev] [PATCH v3] examples/l2fwd: add cmdline option for forwarding port info
Quoted 84 lines
-----Original Message-----
From: [email protected] <[email protected]>
Sent: Monday, April 27, 2020 1:30 PM
To: Jerin Jacob Kollanukkaran <[email protected]>; [email protected]; John McNamara <[email protected]>; Marko Kovacevic <[email protected]>; Ori Kam <[email protected]>; Bruce Richardson <[email protected]>; Radu Nicolau <[email protected]>; Akhil Goyal <[email protected]>; Tomasz Kantecki <[email protected]>; Sunil Kumar Kori <[email protected]>; Pavan Nikhilesh Bhagavatula <[email protected]>
Cc: Andrzej Ostruszka [C] <[email protected]>; [email protected]; Vamsi Krishna Attunuru <[email protected]>
Subject: [dpdk-dev] [PATCH v3] examples/l2fwd: add cmdline option for forwarding port infoFrom: Vamsi Attunuru <[email protected]>
Current l2fwd application statically configures adjacent ports as destination ports for forwarding the traffic.
Add a portmap option to pass the forwarding port pair mapping which allows the user to configure forwarding port mapping.
If no portmap argument is specified, destination port map is not changed and traffic gets forwarded with existing mapping.
To align port/queue configuration of each lcore with destination port map, port/queue configuration of each lcore gets modified when portmap option is specified.
Ex: ./l2fwd -c
0xff-- -p0x3f-q 2 --portmap="(0,3)(1,4)(2,5)"With above portmap option, traffic received from portid = 0 gets forwarded to port = 3 and vice versa, similarly traffic gets forwarded on other port pairs (1,4) and (2,5)
Signed-off-by Vamsi Attunuru <[email protected]>Signed-off-by Pavan Nikhilesh <[email protected]>---
v3 Changes:
* s/config/portmap/
* rebase on masterv2 Changes:
* Fix command option format in docs.
* Use memcpy instead of snprintf.
* Rephrase if-else condition.doc/guides/rel_notes/release_20_05.rst | 7 + .../sample_app_ug/l2_forward_real_virtual.rst | 18 +- examples/l2fwd/main.c | 182 +++++++++++++++--- 3 files changed, 181 insertions(+), 26 deletions(-)doc/guides/rel_notes/release_20_05.rst +7
-0@@ -212,6 +212,13 @@ New Features * Added IPsec inbound load-distribution support for ipsec-secgw application using NIC load distribution feature(Flow Director). +* **Added --portmap command line parameter to l2fwd example.** + + Added new command line option ``--portmap="(port, port)[,(port, + port)]"`` to pass forwarding port details. + See the :doc:`doc/guides/sample_app_ug/l2_forward_real_virtual` for + more details of this parameter usage. +Removed Items
-------------doc/guides/sample_app_ug/l2_forward_real_virtual.rst +0
-0@@ -91,7 +91,10 @@ The application requires a number of command lineoptions:
.. code-block:: console
doc/guides/sample_app_ug/l2_forward_real_virtual.rst The application requires a number of command line options: · line 97
Quoted 5 lines
doc/guides/sample_app_ug/l2_forward_real_virtual.rst +4
-1- ./build/l2fwd [EAL options] -- -p PORTMASK [-q NQ] --[no-]mac-updating + ./build/l2fwd [EAL options] -- -p PORTMASK + [-q NQ] + --[no-]mac-updating + --portmap="(port, port)[,(port, port)]"
Will it be better to represent as [--portmap="(port, port)[,(port, port)]] as it is a optional parameter ?
Quoted 13 lines
where,
doc/guides/sample_app_ug/l2_forward_real_virtual.rst +0
-1@@ -99,7 +102,10 @@ where, * q NQ: A number of queues (=ports) per lcore (default is 1) -* --[no-]mac-updating: Enable or disable MAC addresses updating (enabledby default).
+* --[no-]mac-updating: Enable or disable MAC addresses updating (enabled
by default)doc/guides/sample_app_ug/l2_forward_real_virtual.rst +2
-0+ +* --portmap="(port,port)[,(port,port)]": Determines which ports are
Quoted 2 lines
mapped to
+ which ports for packet forwarding.
May be rephrased a bit shorter as "Determines forwarding ports mapping".
Quoted 11 lines
To run the application in linux environment with 4 lcores, 16 ports and 8 RX queues per lcore and MAC address updating enabled, issue the command:
doc/guides/sample_app_ug/l2_forward_real_virtual.rst +3
-0@@ -108,6 +114,14 @@ updating enabled, issue the command: $ ./build/l2fwd -l 0-3 -n 4 -- -q 8 -p ffff +To run the application in linux environment with 4 lcores, 4 ports, 8 +RX queues per lcore and MAC address updating enabled, to forward RX +traffic of ports 0 & 1 on ports 2 & 3 respectively and vice versa, issue thecommand:
IMO, No need to mentioned information about MAC address update enabled because it is enabled by default and no param is passed in below command. So information is not relevant here.
doc/guides/sample_app_ug/l2_forward_real_virtual.rst updating enabled, issue the command: · line 123
Quoted 4 lines
doc/guides/sample_app_ug/l2_forward_real_virtual.rst +4
-0+ +.. code-block:: console + + $ ./build/l2fwd -l 0-3 -n 4 -- -q 8 -p f --portmap="(0,2)(1,3)"
It looks like comma is missed in between two port map information.
Quoted 17 lines
+
Refer to the *DPDK Getting Started Guide* for general information on running applications and the Environment Abstraction Layer (EAL) options.examples/l2fwd/main.c index renamed from examples/l2fwd/main.c +0
-0
88ddfe589..81861a22a100644examples/l2fwd/main.c +1
-0@@ -38,6 +38,7 @@#include <rte_ethdev.h> #include <rte_mempool.h> #include <rte_mbuf.h> +#include <rte_string_fns.h> static volatile bool force_quit;
examples/l2fwd/main.c static uint32_t l2fwd_enabled_port_mask = 0; · line 74
Quoted 7 lines
examples/l2fwd/main.c +4
-0@@ -67,6 +68,15 @@ static uint32_t l2fwd_enabled_port_mask = 0;/* list of enabled ports */ static uint32_t l2fwd_dst_ports[RTE_MAX_ETHPORTS ]; +struct port_pair_params { +#define NUM_PORTS 2 +uint16_t port[NUM_PORTS ]; +} __rte_cache_aligned;
Is there any specific reason to use this syntax to declare two ports instead of following
struct port_pair_params {
uint16_t port1;
uint16_t port2;
}; examples/l2fwd/main.c static uint32_t l2fwd_enabled_port_mask = 0; · line 77
Quoted 3 lines
examples/l2fwd/main.c +3
-0+ +static struct port_pair_params +port_pair_params_array[RTE_MAX_ETHPORTS ];
Should not be RTE_MAX_ETHPORTS/2 if only 1:1 mapping is allowed ? As I understood it is used to store CLI port mapping, I think it is better to Use any other MACRO which define maximum number of port map parameter. If you consider it, then it can also be updated in documentation.
Quoted 28 lines
examples/l2fwd/main.c +3
-0+static struct port_pair_params *port_pair_params;static uint16_t +nb_port_pair_params; +static unsigned int l2fwd_rx_queue_per_lcore =1 ;#define
MAX_RX_QUEUE_PER_LCORE16examples/l2fwd/main.c +0
-1@@ -294,11 +304,13 @@ l2fwd_usage(const char *prgname)printf ("%s [EAL options] -- -p PORTMASK [-q NQ]\n" " -p PORTMASK: hexadecimal bitmask of ports to configure\n" " -q NQ: number of queue (=ports) per lcore (default is 1)\n" - " -T PERIOD: statistics will be refreshed each PERIODseconds (0 to disable, 10 default, 86400 maximum)\n"
examples/l2fwd/main.c +0
-1- " --[no-]mac-updating: Enable or disable MAC addressesupdating (enabled by default)\n"
examples/l2fwd/main.c +0
-2-" When enabled:\n" - " - The source MAC address is replaced by the TX portMAC address\n"
examples/l2fwd/main.c +0
-1- " - The destination MAC address is replaced by02:00:00:00:00:TX_PORT_ID\n",
examples/l2fwd/main.c +1
-0+ " -T PERIOD: statistics will be refreshed each PERIOD seconds (0 todisable, 10 default, 86400 maximum)\n"
examples/l2fwd/main.c +1
-0+ " --[no-]mac-updating: Enable or disable MAC addresses updating(enabled by default)\n"
examples/l2fwd/main.c +2
-0+" When enabled:\n" + " - The source MAC address is replaced by the TX port MACaddress\n"
examples/l2fwd/main.c +1
-0+ " - The destination MAC address is replaced by02:00:00:00:00:TX_PORT_ID\n"
examples/l2fwd/main.c l2fwd_usage(const char *prgname) · line 313
Quoted 2 lines
examples/l2fwd/main.c +2
-0+" --portmap: Configure forwarding port pair mapping\n" + " Default: alternate port pairs\n\n",
IMO, Indentation changes for other parameters should in separate patch.
Quoted 241 lines
prgname); }examples/l2fwd/main.c +54
-0@@ -319,6 +331,61 @@ l2fwd_parse_portmask(const char *portmask)return pm; } +static int +l2fwd_parse_port_pair_config (const char *q_arg) { +enum fieldnames { +FLD_PORT1 =0 , +FLD_PORT2 , + _NUM_FLD + }; +unsigned long int_fld[_NUM_FLD]; +const char *p, *p0 = q_arg; +char *str_fld[_NUM_FLD]; +unsigned int size; +char s[256 ]; +char *end; +int i; + + nb_port_pair_params =0 ; + +while ((p =strchr (p0,'(' )) !=NULL ) { + ++p; + p0 =strchr (p,')' ); +if (p0 ==NULL ) +return -1 ; + + size = p0 - p; +if (size >=sizeof (s)) +return -1 ; + +memcpy (s, p, size); + s[size] ='\0' ; +if (rte_strsplit (s,sizeof (s), str_fld, + _NUM_FLD,',' ) != _NUM_FLD) +return -1 ; +for (i =0 ; i < _NUM_FLD; i++) { + errno =0 ; + int_fld[i] =strtoul (str_fld[i], &end,0 ); +if (errno !=0 || end == str_fld[i] || + int_fld[i] >=RTE_MAX_ETHPORTS ) +return -1 ; + } +if (nb_port_pair_params >=RTE_MAX_ETHPORTS /2 ) { + printf("exceeded max number of port pair params: %hu\n", + nb_port_pair_params); +return -1 ; + } + port_pair_params_array[nb_port_pair_params].port[0 ] = + (uint16_t )int_fld[FLD_PORT1 ]; + port_pair_params_array[nb_port_pair_params].port[1 ] = + (uint16_t )int_fld[FLD_PORT2 ]; + ++nb_port_pair_params; + } + port_pair_params = port_pair_params_array; +return 0 ; +} +static unsigned int
l2fwd_parse_nqueue(const char *q_arg)
{examples/l2fwd/main.c +1
-0@@ -361,6 +428,7 @@ static const char short_options[] =#define CMD_LINE_OPT_MAC_UPDATING "mac-updating"#define CMD_LINE_OPT_NO_MAC_UPDATING "no-mac-updating" +#define CMD_LINE_OPT_PORTMAP_CONFIG "portmap"enum { /* long options mapped to a short option */ @@ -368,11 +436,13 @@enum { /* first long only option value must be >= 256, so that we won't * conflict with short options */ CMD_LINE_OPT_MIN_NUM = 256,examples/l2fwd/main.c +1
-0+ CMD_LINE_OPT_PORTMAP_NUM,};
static const struct option lgopts[] = {
{CMD_LINE_OPT_MAC_UPDATING,no_argument,&mac_updating,
1},
{CMD_LINE_OPT_NO_MAC_UPDATING,no_argument,&mac_updating, 0},examples/l2fwd/main.c +1
-0+ { CMD_LINE_OPT_PORTMAP_CONFIG, 1, 0,
CMD_LINE_OPT_PORTMAP_NUM},{NULL, 0, 0, 0} };examples/l2fwd/main.c +2
-0@@ -386,6 +456,7 @@ l2fwd_parse_args(int argc, char **argv)char *prgname = argv[0 ]; argvopt = argv; + port_pair_params =NULL ;while ((opt =getopt_long (argc, argvopt, short_options, lgopts, &option_index)) != EOF) { @@ -423,7 +494,13 @@ l2fwd_parse_args(int argc, char **argv)break; /* long options */examples/l2fwd/main.c +7
-1-case 0 : +case CMD_LINE_OPT_PORTMAP_NUM : + ret =l2fwd_parse_port_pair_config (optarg); +if (ret) { +fprintf (stderr,"Invalid config\n" ); +l2fwd_usage (prgname); +return -1 ; + }break; default:examples/l2fwd/main.c +42
-0@@ -440,6 +517,48 @@ l2fwd_parse_args(int argc, char **argv)return ret; } +/* +* Check port pair config with enabled port mask, +* and for valid port pair combinations. +*/ +static int +check_port_pair_config (void ) +{ +uint32_t port_pair_config_mask =0 ; +uint32_t port_pair_mask =0 ; +uint16_t index, i, portid; + +for (index =0 ; index < nb_port_pair_params; index++) { + port_pair_mask =0 ; + +for (i =0 ; i <NUM_PORTS ; i++) { + portid = port_pair_params[index].port[i]; +if ((l2fwd_enabled_port_mask & (1 << portid)) ==0 ) { +printf ("port %u is not enabled in port mask\n", + portid); +return -1 ; + } +if (!rte_eth_dev_is_valid_port (portid)) { +printf ("port %u is not present on the board\n", + portid); +return -1 ; + } + + port_pair_mask |=1 << portid; + } + +if (port_pair_config_mask & port_pair_mask) { +printf ("port %u is used in other port pairs\n" , portid); +return -1 ; + } + port_pair_config_mask |= port_pair_mask; + } + + l2fwd_enabled_port_mask &= port_pair_config_mask; + +return 0 ; +} + /* Check the link status of all ports in up to 9s, and print them finally */ staticvoid
check_all_ports_link_status(uint32_t port_mask)@@ -555,6 +674,11 @@main(int argc, char **argv)if (nb_ports == 0) rte_exit(EXIT_FAILURE, "No Ethernet ports - bye\n");examples/l2fwd/main.c +5
-0+if (port_pair_params !=NULL ) { +if (check_port_pair_config () <0 ) +rte_exit (EXIT_FAILURE ,"Invalid port pair config\n" ); + } +/* check port mask to possible port mask */ if (l2fwd_enabled_port_mask & ~((1 << nb_ports) - 1)) rte_exit(EXIT_FAILURE, "Invalid portmask; possible (0x%x)\n",examples/l2fwd/main.c +28
-18@@ -565,26 +689,35 @@ main(int argc, char **argv) l2fwd_dst_ports[portid] =0 ; last_port =0 ; -/* -* Each logical core is assigned a dedicated TX queue on each port. -*/ -RTE_ETH_FOREACH_DEV (portid) { -/* skip ports that are not enabled */ -if ((l2fwd_enabled_port_mask & (1 << portid)) ==0 ) -continue ; +/* populate destination port details */ +if (port_pair_params !=NULL ) { +uint16_t idx, p; -if (nb_ports_in_mask %2 ) { - l2fwd_dst_ports[portid] = last_port; - l2fwd_dst_ports[last_port] = portid; +for (idx =0 ; idx < (nb_port_pair_params <<1 ); idx++) { + p = idx &1 ; + portid = port_pair_params[idx >>1 ].port[p]; + l2fwd_dst_ports[portid] = + port_pair_params[idx >>1 ].port[p ^1 ]; } -else - last_port = portid; + }else { +RTE_ETH_FOREACH_DEV (portid) { +/* skip ports that are not enabled */ +if ((l2fwd_enabled_port_mask & (1 << portid)) ==0 ) +continue ; - nb_ports_in_mask++; - } -if (nb_ports_in_mask %2 ) { -printf ("Notice: odd number of ports in portmask.\n" ); - l2fwd_dst_ports[last_port] = last_port; +if (nb_ports_in_mask %2 ) { + l2fwd_dst_ports[portid] = last_port; + l2fwd_dst_ports[last_port] = portid; + }else { + last_port = portid; + } + + nb_ports_in_mask++; + } +if (nb_ports_in_mask %2 ) { +printf ("Notice: odd number of ports in portmask.\n" ); + l2fwd_dst_ports[last_port] = last_port; + } } rx_lcore_id =0 ;@@ -613,7 +746,8 @@ main(int argc, char **argv) qconf->rx_port_list[qconf->n_rx_port] = portid; qconf->n_rx_port++; -printf ("Lcore %u: RX port %u\n" , rx_lcore_id, portid); +printf ("Lcore %u: RX port %u TX port %u\n" , rx_lcore_id, + portid, l2fwd_dst_ports[portid]); } nb_mbufs = RTE_MAX(nb_ports * (nb_rxd + nb_txd +
MAX_PKT_BURST+
--
2.17.1
Re: [dpdk-dev] [PATCH v3] examples/l2fwd: add cmdline option for forwarding port info
Quoted 2 lines · Sunil Kumar Kori
Quoted 2 lines
-----Original Message-----
From: [email protected] <[email protected]>
[...]
Quoted 10 lines
examples/l2fwd/main.c static uint32_t l2fwd_enabled_port_mask = 0; · line 74
Quoted 8 lines
examples/l2fwd/main.c +4
-0@@ -67,6 +68,15 @@ static uint32_t l2fwd_enabled_port_mask = 0;/* list of enabled ports */ static uint32_t l2fwd_dst_ports[RTE_MAX_ETHPORTS ]; +struct port_pair_params { +#define NUM_PORTS 2 +uint16_t port[NUM_PORTS ]; +} __rte_cache_aligned;Is there any specific reason to use this syntax to declare two ports instead of following
Quoted 4 lines
struct port_pair_params {
uint16_t port1;
uint16_t port2;
};Initially it was so, but I made a comment that this leads to code duplication in check_port_pair_config() (same checks for port1 and port2
- now handled via loop). I still stand by this comment :)
With regards
Andrzej Ostruszka
Re: [dpdk-dev] [PATCH v3] examples/l2fwd: add cmdline option for forwarding port info
Quoted 34 lines
-----Original Message-----
From: Andrzej Ostruszka [C] <[email protected]>
Sent: Monday, April 27, 2020 3:06 PM
To: Sunil Kumar Kori <[email protected]>; Pavan Nikhilesh Bhagavatula <[email protected]>; Jerin Jacob Kollanukkaran <[email protected]>; [email protected]; John McNamara <[email protected]>; Marko Kovacevic <[email protected]>; Ori Kam <[email protected]>; Bruce Richardson <[email protected]>; Radu Nicolau <[email protected]>; Akhil Goyal <[email protected]>; Tomasz Kantecki <[email protected]>
Cc: [email protected]; Vamsi Krishna Attunuru <[email protected]>
Subject: Re: [dpdk-dev] [PATCH v3] examples/l2fwd: add cmdline option for forwarding port infoQuoted 2 lines · Sunil Kumar Kori
Quoted 2 lines
-----Original Message-----
From: [email protected] <[email protected]>[...]
Quoted 10 lines
examples/l2fwd/main.c static uint32_t l2fwd_enabled_port_mask = 0; · line 74
Quoted 8 lines
examples/l2fwd/main.c +4
-0@@ -67,6 +68,15 @@ static uint32_t l2fwd_enabled_port_mask = 0;/* list of enabled ports */ static uint32_t l2fwd_dst_ports[RTE_MAX_ETHPORTS ]; +struct port_pair_params { +#define NUM_PORTS 2 +uint16_t port[NUM_PORTS ]; +} __rte_cache_aligned;Is there any specific reason to use this syntax to declare two ports
Quoted 4 lines
instead of following struct port_pair_params { uint16_t port1; uint16_t port2; };
Quoted 4 lines
Initially it was so, but I made a comment that this leads to code duplication in
check_port_pair_config()(same checks for port1 and port2
- now handled via loop). I still stand by this comment :)
Okay, I think having for two variables only is not a big deal. if it is some higher number then it look fine. IMO, it is more readable when using suggested way. You can take on this.
Quoted 2 lines
With regards
Andrzej Ostruszka
Re: [dpdk-dev] [PATCH v3] examples/l2fwd: add cmdline option for forwarding port info
In reply to Sunil Kumar KoriQuoted 12 lines
doc/guides/sample_app_ug/l2_forward_real_virtual.rst The application requires a number of command
Quoted 6 lines
doc/guides/sample_app_ug/l2_forward_real_virtual.rst +0
-0@@ -91,7 +91,10 @@ The application requires a number of commandline
Quoted 5 lines
options:
.. code-block:: console
- ./build/l2fwd [EAL options] -- -p PORTMASK [-q NQ] --[no-]mac-
Quoted 9 lines
updating
doc/guides/sample_app_ug/l2_forward_real_virtual.rst The application requires a number of command line options: · line 97
Quoted 4 lines
doc/guides/sample_app_ug/l2_forward_real_virtual.rst +4
-0+ ./build/l2fwd [EAL options] -- -p PORTMASK + [-q NQ] + --[no-]mac-updating + --portmap="(port, port)[,(port, port)]"Will it be better to represent as [--portmap="(port, port)[,(port, port)]] as it is a optional parameter ?
Will change in v4.
Quoted 12 lines
doc/guides/sample_app_ug/l2_forward_real_virtual.rst where, · line 102
Quoted 8 lines
where,
doc/guides/sample_app_ug/l2_forward_real_virtual.rst +0
-1@@ -99,7 +102,10 @@ where, * q NQ: A number of queues (=ports) per lcore (default is 1) -* --[no-]mac-updating: Enable or disable MAC addresses updating(enabled
Quoted 2 lines
by default).
+* --[no-]mac-updating: Enable or disable MAC addresses updating(enabled
Quoted 8 lines
Quoted 5 lines
by default)
doc/guides/sample_app_ug/l2_forward_real_virtual.rst +2
-0+ +* --portmap="(port,port)[,(port,port)]": Determines which ports aremapped to
+ which ports for packet forwarding.May be rephrased a bit shorter as "Determines forwarding ports mapping".
Will change in v4.
Quoted 13 lines
Quoted 2 lines
To run the application in linux environment with 4 lcores, 16 ports and
8 RX
Quoted 1 line
queues per lcore and MAC address updating enabled, issue the
command:
doc/guides/sample_app_ug/l2_forward_real_virtual.rst updating enabled, issue the command: · line 118
Quoted 6 lines
doc/guides/sample_app_ug/l2_forward_real_virtual.rst +2
-0@@ -108,6 +114,14 @@ updating enabled, issue the command: $ ./build/l2fwd -l 0-3 -n 4 -- -q 8 -p ffff +To run the application in linux environment with 4 lcores, 4 ports, 8 +RX queues per lcore and MAC address updating enabled, to forwardRX
doc/guides/sample_app_ug/l2_forward_real_virtual.rst updating enabled, issue the command: · line 119
Quoted 1 line
doc/guides/sample_app_ug/l2_forward_real_virtual.rst +1
-0+traffic of ports 0 & 1 on ports 2 & 3 respectively and vice versa, issue
Quoted 9 lines
the
Quoted 1 line
command:
IMO, No need to mentioned information about MAC address update enabled because it
is enabled by default and no param is passed in below command. So information is not
relevant here.
Will change in v4.
Quoted 33 lines
doc/guides/sample_app_ug/l2_forward_real_virtual.rst updating enabled, issue the command: · line 123
Quoted 4 lines
doc/guides/sample_app_ug/l2_forward_real_virtual.rst +4
-0+ +.. code-block:: console + + $ ./build/l2fwd -l 0-3 -n 4 -- -q 8 -p f --portmap="(0,2)(1,3)"It looks like comma is missed in between two port map information.
Quoted 2 lines
+
Refer to the *DPDK Getting Started Guide* for general informationon running
Quoted 15 lines
applications and the Environment Abstraction Layer (EAL) options.
examples/l2fwd/main.c index renamed from examples/l2fwd/main.c +0
-0
88ddfe589..81861a22a100644examples/l2fwd/main.c +1
-0@@ -38,6 +38,7 @@#include <rte_ethdev.h> #include <rte_mempool.h> #include <rte_mbuf.h> +#include <rte_string_fns.h> static volatile bool force_quit;examples/l2fwd/main.c static uint32_t l2fwd_enabled_port_mask = 0; · line 74
Quoted 7 lines
examples/l2fwd/main.c +4
-0@@ -67,6 +68,15 @@ static uint32_t l2fwd_enabled_port_mask = 0;/* list of enabled ports */ static uint32_t l2fwd_dst_ports[RTE_MAX_ETHPORTS ]; +struct port_pair_params { +#define NUM_PORTS 2 +uint16_t port[NUM_PORTS ]; +} __rte_cache_aligned;Is there any specific reason to use this syntax to declare two ports instead of following
Quoted 5 lines
struct port_pair_params {
uint16_t port1;
uint16_t port2;
};It was done to reduce code duplication. Will leave as is.
Quoted 5 lines
examples/l2fwd/main.c static uint32_t l2fwd_enabled_port_mask = 0; · line 77
Quoted 3 lines
examples/l2fwd/main.c +3
-0+ +static struct port_pair_params +port_pair_params_array[RTE_MAX_ETHPORTS ];Should not be RTE_MAX_ETHPORTS/2 if only 1:1 mapping is allowed ? As I understood it is used to store CLI port mapping,
RTE_MAX_ETHPORTS/2 will be sufficient.
Quoted 37 lines
I think it is better to
Use any other MACRO which define maximum number of port map parameter.
If you consider it, then it can also be updated in documentation.examples/l2fwd/main.c l2fwd_usage(const char *prgname) · line 305
Quoted 9 lines
examples/l2fwd/main.c +3
-0+static struct port_pair_params *port_pair_params;static uint16_t +nb_port_pair_params; +static unsigned int l2fwd_rx_queue_per_lcore =1 ;#define
MAX_RX_QUEUE_PER_LCORE16examples/l2fwd/main.c +0
-0@@ -294,11 +304,13 @@ l2fwd_usage(const char *prgname) printf("%s [EAL options] -- -p PORTMASK [-q NQ]\n" " -p PORTMASK: hexadecimal bitmask of ports toconfigure\n"
examples/l2fwd/main.c l2fwd_usage(const char *prgname) · line 306
Quoted 1 line
examples/l2fwd/main.c +0
-0" -q NQ: number of queue (=ports) per lcore (default is1)\n"
examples/l2fwd/main.c
Quoted 3 lines
examples/l2fwd/main.c +0
-1- " -T PERIOD: statistics will be refreshed each PERIODseconds (0 to disable, 10 default, 86400 maximum)\n"
examples/l2fwd/main.c +0
-1- " --[no-]mac-updating: Enable or disable MACaddresses
examples/l2fwd/main.c
Quoted 3 lines
updating (enabled by default)\n"
examples/l2fwd/main.c +0
-2-" When enabled:\n" - " - The source MAC address is replaced by the TXport
examples/l2fwd/main.c
Quoted 4 lines
MAC address\n"
examples/l2fwd/main.c +0
-1- " - The destination MAC address is replaced by02:00:00:00:00:TX_PORT_ID\n",
examples/l2fwd/main.c +1
-0+ " -T PERIOD: statistics will be refreshed each PERIODseconds (0 to
examples/l2fwd/main.c
Quoted 2 lines
disable, 10 default, 86400 maximum)\n"
examples/l2fwd/main.c +1
-0+ " --[no-]mac-updating: Enable or disable MAC addressesupdating
examples/l2fwd/main.c
Quoted 3 lines
(enabled by default)\n"
examples/l2fwd/main.c +2
-0+" When enabled:\n" + " - The source MAC address is replaced by the TX portMAC
Quoted 8 lines
examples/l2fwd/main.c l2fwd_usage(const char *prgname) · line 313
Quoted 5 lines
address\n"
examples/l2fwd/main.c +1
-0+ " - The destination MAC address is replaced by02:00:00:00:00:TX_PORT_ID\n"
examples/l2fwd/main.c +2
-0+" --portmap: Configure forwarding port pair mapping\n" + " Default: alternate port pairs\n\n",IMO, Indentation changes for other parameters should in separate patch.
It's not a functionality change it just minor code formatting change and doesn't deserve a separate patch.
Quoted 8 lines
examples/l2fwd/main.c l2fwd_parse_portmask(const char
Quoted 4 lines
prgname); }examples/l2fwd/main.c +0
-0@@ -319,6 +331,61 @@ l2fwd_parse_portmask(const char*portmask)
Quoted 3 lines
examples/l2fwd/main.c +0
-0return pm; }
Re: [dpdk-dev] [PATCH v3] examples/l2fwd: add cmdline option for forwarding port info
Sent from Workspace ONE Boxer
Quoted 112 lines · Pavan Nikhilesh Bhagavatula
Quoted 13 lines
doc/guides/sample_app_ug/l2_forward_real_virtual.rst The application requires a number of command
Quoted 7 lines
doc/guides/sample_app_ug/l2_forward_real_virtual.rst +0
-0@@ -91,7 +91,10 @@ The application requires a number of commandline
Quoted 5 lines
options:
.. code-block:: console
- ./build/l2fwd [EAL options] -- -p PORTMASK [-q NQ] --[no-]mac-
Quoted 9 lines
updating
doc/guides/sample_app_ug/l2_forward_real_virtual.rst The application requires a number of command line options: · line 97
Quoted 4 lines
doc/guides/sample_app_ug/l2_forward_real_virtual.rst +4
-0+ ./build/l2fwd [EAL options] -- -p PORTMASK + [-q NQ] + --[no-]mac-updating + --portmap="(port, port)[,(port, port)]"Will it be better to represent as [--portmap="(port, port)[,(port, port)]] as it is a optional parameter ?
Will change in v4.
Quoted 12 lines
doc/guides/sample_app_ug/l2_forward_real_virtual.rst where, · line 102
Quoted 8 lines
where,
doc/guides/sample_app_ug/l2_forward_real_virtual.rst +0
-1@@ -99,7 +102,10 @@ where, * q NQ: A number of queues (=ports) per lcore (default is 1) -* --[no-]mac-updating: Enable or disable MAC addresses updating(enabled
Quoted 2 lines
by default).
+* --[no-]mac-updating: Enable or disable MAC addresses updating(enabled
Quoted 8 lines
Quoted 5 lines
by default)
doc/guides/sample_app_ug/l2_forward_real_virtual.rst +2
-0+ +* --portmap="(port,port)[,(port,port)]": Determines which ports aremapped to
+ which ports for packet forwarding.May be rephrased a bit shorter as "Determines forwarding ports mapping".
Will change in v4.
Quoted 13 lines
Quoted 2 lines
To run the application in linux environment with 4 lcores, 16 ports and
8 RX
Quoted 1 line
queues per lcore and MAC address updating enabled, issue the
command:
doc/guides/sample_app_ug/l2_forward_real_virtual.rst updating enabled, issue the command: · line 118
Quoted 6 lines
doc/guides/sample_app_ug/l2_forward_real_virtual.rst +2
-0@@ -108,6 +114,14 @@ updating enabled, issue the command: $ ./build/l2fwd -l 0-3 -n 4 -- -q 8 -p ffff +To run the application in linux environment with 4 lcores, 4 ports, 8 +RX queues per lcore and MAC address updating enabled, to forwardRX
doc/guides/sample_app_ug/l2_forward_real_virtual.rst updating enabled, issue the command: · line 119
Quoted 1 line
doc/guides/sample_app_ug/l2_forward_real_virtual.rst +1
-0+traffic of ports 0 & 1 on ports 2 & 3 respectively and vice versa, issueQuoted 9 lines
the
Quoted 1 line
command:
IMO, No need to mentioned information about MAC address update enabled because it
is enabled by default and no param is passed in below command. So information is not
relevant here.Will change in v4.
Quoted 33 lines
doc/guides/sample_app_ug/l2_forward_real_virtual.rst updating enabled, issue the command: · line 123
Quoted 4 lines
doc/guides/sample_app_ug/l2_forward_real_virtual.rst +4
-0+ +.. code-block:: console + + $ ./build/l2fwd -l 0-3 -n 4 -- -q 8 -p f --portmap="(0,2)(1,3)"It looks like comma is missed in between two port map information.
Quoted 2 lines
+
Refer to the *DPDK Getting Started Guide* for general informationon running
Quoted 15 lines
applications and the Environment Abstraction Layer (EAL) options.
examples/l2fwd/main.c index renamed from examples/l2fwd/main.c +0
-0
88ddfe589..81861a22a100644examples/l2fwd/main.c +1
-0@@ -38,6 +38,7 @@#include <rte_ethdev.h> #include <rte_mempool.h> #include <rte_mbuf.h> +#include <rte_string_fns.h> static volatile bool force_quit;examples/l2fwd/main.c static uint32_t l2fwd_enabled_port_mask = 0; · line 74
Quoted 7 lines
examples/l2fwd/main.c +4
-0@@ -67,6 +68,15 @@ static uint32_t l2fwd_enabled_port_mask = 0;/* list of enabled ports */ static uint32_t l2fwd_dst_ports[RTE_MAX_ETHPORTS ]; +struct port_pair_params { +#define NUM_PORTS 2 +uint16_t port[NUM_PORTS ]; +} __rte_cache_aligned;Is there any specific reason to use this syntax to declare two ports instead of following
Quoted 5 lines
struct port_pair_params { uint16_t port1; uint16_t port2; };
Quoted 1 line
It was done to reduce code duplication. Will leave as is.
Ack.
Quoted 8 lines
Quoted 6 lines
examples/l2fwd/main.c static uint32_t l2fwd_enabled_port_mask = 0; · line 77
Quoted 4 lines
examples/l2fwd/main.c +3
-0+ +static struct port_pair_params +port_pair_params_array[RTE_MAX_ETHPORTS ];Should not be RTE_MAX_ETHPORTS/2 if only 1:1 mapping is allowed ? As I understood it is used to store CLI port mapping,
RTE_MAX_ETHPORTS/2 will be sufficient.
Ack.
Quoted 44 lines
Quoted 44 lines
I think it is better to
Use any other MACRO which define maximum number of port map parameter.
If you consider it, then it can also be updated in documentation.examples/l2fwd/main.c l2fwd_usage(const char *prgname) · line 305
Quoted 9 lines
examples/l2fwd/main.c +3
-0+static struct port_pair_params *port_pair_params;static uint16_t +nb_port_pair_params; +static unsigned int l2fwd_rx_queue_per_lcore =1 ;#define
MAX_RX_QUEUE_PER_LCORE16examples/l2fwd/main.c +0
-0@@ -294,11 +304,13 @@ l2fwd_usage(const char *prgname) printf("%s [EAL options] -- -p PORTMASK [-q NQ]\n" " -p PORTMASK: hexadecimal bitmask of ports toconfigure\n"
examples/l2fwd/main.c l2fwd_usage(const char *prgname) · line 306
Quoted 1 line
examples/l2fwd/main.c +0
-0" -q NQ: number of queue (=ports) per lcore (default is1)\n"
Quoted 3 lines
examples/l2fwd/main.c +0
-1- " -T PERIOD: statistics will be refreshed each PERIODseconds (0 to disable, 10 default, 86400 maximum)\n"
- " --[no-]mac-updating: Enable or disable MACaddresses
examples/l2fwd/main.c
Quoted 3 lines
updating (enabled by default)\n"
examples/l2fwd/main.c +0
-2-" When enabled:\n" - " - The source MAC address is replaced by the TXport
Quoted 4 lines
MAC address\n"
- " - The destination MAC address is replaced by
02:00:00:00:00:TX_PORT_ID\n",
+ " -T PERIOD: statistics will be refreshed each PERIODseconds (0 to
Quoted 2 lines
disable, 10 default, 86400 maximum)\n"
+ " --[no-]mac-updating: Enable or disable MAC addressesupdating
examples/l2fwd/main.c
Quoted 3 lines
(enabled by default)\n"
examples/l2fwd/main.c +2
-0+" When enabled:\n" + " - The source MAC address is replaced by the TX portMAC
Quoted 5 lines
address\n"
+ " - The destination MAC address is replaced by
02:00:00:00:00:TX_PORT_ID\n"examples/l2fwd/main.c +2
-0+" --portmap: Configure forwarding port pair mapping\n" + " Default: alternate port pairs\n\n",
Quoted 5 lines
Quoted 2 lines
IMO, Indentation changes for other parameters should in separate patch.
It's not a functionality change it just minor code formatting change and doesn’t deserve a separate patch.
Ack.
Quoted 9 lines
Quoted 9 lines
examples/l2fwd/main.c l2fwd_parse_portmask(const char
Quoted 5 lines
prgname); }examples/l2fwd/main.c +0
-0@@ -319,6 +331,61 @@ l2fwd_parse_portmask(const char*portmask)
Quoted 3 lines
examples/l2fwd/main.c +0
-0return pm; }
PATCH v4 1/1 examples/l2fwd: add cmdline option for forwarding port info
In reply to [email protected]From: Vamsi Attunuru <[email protected]>
Current l2fwd application statically configures adjacent ports as destination ports for forwarding the traffic.
Add a portmap option to pass the forwarding port pair mapping which allows the user to configure forwarding port mapping.
If no portmap argument is specified, destination port map is not changed and traffic gets forwarded with existing mapping.
To align port/queue configuration of each lcore with destination port map, port/queue configuration of each lcore gets modified when portmap option is specified.
Ex: ./l2fwd -c 0xff -- -p 0x3f -q 2 --portmap="(0,3)(1,4)(2,5)"
With above portmap option, traffic received from portid = 0 gets forwarded to port = 3 and vice versa, similarly traffic gets forwarded on other port pairs (1,4) and (2,5)
---
v4 Changes:
* Documentation changes. (Sunil)
* reduce port_pair_params_array size. (Sunil)
v3 Changes:
* s/config/portmap/
* rebase on master
v2 Changes:
* Fix command option format in docs.
* Use memcpy instead of snprintf.
* Rephrase if-else condition.
doc/guides/rel_notes/release_20_05.rst | 7 + .../sample_app_ug/l2_forward_real_virtual.rst | 17 +- examples/l2fwd/main.c | 182 +++++++++++++++--- 3 files changed, 180 insertions(+), 26 deletions(-)
doc/guides/rel_notes/release_20_05.rst +7 -0
@@ -212,6 +212,13 @@ New Features * Added IPsec inbound load-distribution support for ipsec-secgw application using NIC load distribution feature(Flow Director). +* **Added --portmap command line parameter to l2fwd example.** + + Added new command line option ``--portmap="(port, port)[,(port, port)]"`` to + pass forwarding port details. + See the :doc:`doc/guides/sample_app_ug/l2_forward_real_virtual` for more + details of this parameter usage. + Removed Items -------------
doc/guides/sample_app_ug/l2_forward_real_virtual.rst +15 -2
@@ -91,7 +91,10 @@ The application requires a number of command line options: .. code-block:: console - ./build/l2fwd [EAL options] -- -p PORTMASK [-q NQ] --[no-]mac-updating + ./build/l2fwd [EAL options] -- -p PORTMASK + [-q NQ] + --[no-]mac-updating + [--portmap="(port, port)[,(port, port)]"] where,
@@ -99,7 +102,9 @@ where, * q NQ: A number of queues (=ports) per lcore (default is 1) -* --[no-]mac-updating: Enable or disable MAC addresses updating (enabled by default). +* --[no-]mac-updating: Enable or disable MAC addresses updating (enabled by default) + +* --portmap="(port,port)[,(port,port)]": Determines forwarding ports mapping. To run the application in linux environment with 4 lcores, 16 ports and 8 RX queues per lcore and MAC address updating enabled, issue the command:
@@ -108,6 +113,14 @@ updating enabled, issue the command: $ ./build/l2fwd -l 0-3 -n 4 -- -q 8 -p ffff +To run the application in linux environment with 4 lcores, 4 ports, 8 RX queues +per lcore, to forward RX traffic of ports 0 & 1 on ports 2 & 3 respectively and +vice versa, issue the command: + +.. code-block:: console + + $ ./build/l2fwd -l 0-3 -n 4 -- -q 8 -p f --portmap="(0,2)(1,3)" + Refer to the *DPDK Getting Started Guide* for general information on running applications and the Environment Abstraction Layer (EAL) options.
examples/l2fwd/main.c +158 -24
262 lines not on this page · raw message
--
2.17.1
Re: [dpdk-dev] [PATCH v4] examples/l2fwd: add cmdline option for forwarding port info
Looks okay.
Regards
Sunil Kumar Kori
Quoted 33 lines
-----Original Message-----
From: [email protected] <[email protected]>
Sent: Tuesday, April 28, 2020 12:01 AM
To: Jerin Jacob Kollanukkaran <[email protected]>; [email protected]; John McNamara <[email protected]>; Marko Kovacevic <[email protected]>; Ori Kam <[email protected]>; Bruce Richardson <[email protected]>; Radu Nicolau <[email protected]>; Akhil Goyal <[email protected]>; Tomasz Kantecki <[email protected]>; Sunil Kumar Kori <[email protected]>; Pavan Nikhilesh Bhagavatula <[email protected]>
Cc: Andrzej Ostruszka [C] <[email protected]>; [email protected]; Vamsi Krishna Attunuru <[email protected]>
Subject: [dpdk-dev] [PATCH v4] examples/l2fwd: add cmdline option for forwarding port infoFrom: Vamsi Attunuru <[email protected]>
Current l2fwd application statically configures adjacent ports as destination ports for forwarding the traffic.
Add a portmap option to pass the forwarding port pair mapping which allows the user to configure forwarding port mapping.
If no portmap argument is specified, destination port map is not changed and traffic gets forwarded with existing mapping.
To align port/queue configuration of each lcore with destination port map, port/queue configuration of each lcore gets modified when portmap option is specified.
Ex: ./l2fwd -c
0xff-- -p0x3f-q 2 --portmap="(0,3)(1,4)(2,5)"
Quoted 7 lines
With above portmap option, traffic received from portid = 0 gets forwarded to port = 3 and vice versa, similarly traffic gets forwarded on other port pairs (1,4) and (2,5)
Signed-off-by Vamsi Attunuru <[email protected]>Signed-off-by Pavan Nikhilesh <[email protected]>
Quoted 380 lines
---
v4 Changes:
* Documentation changes. (Sunil)
* reduceport_pair_params_arraysize. (Sunil)v3 Changes:
* s/config/portmap/
* rebase on masterv2 Changes:
* Fix command option format in docs.
* Use memcpy instead of snprintf.
* Rephrase if-else condition.doc/guides/rel_notes/release_20_05.rst | 7 + .../sample_app_ug/l2_forward_real_virtual.rst | 17 +- examples/l2fwd/main.c | 182 +++++++++++++++--- 3 files changed, 180 insertions(+), 26 deletions(-)doc/guides/rel_notes/release_20_05.rst +7
-0@@ -212,6 +212,13 @@ New Features * Added IPsec inbound load-distribution support for ipsec-secgw application using NIC load distribution feature(Flow Director). +* **Added --portmap command line parameter to l2fwd example.** + + Added new command line option ``--portmap="(port, port)[,(port, + port)]"`` to pass forwarding port details. + See the :doc:`doc/guides/sample_app_ug/l2_forward_real_virtual` for + more details of this parameter usage. +Removed Items
-------------doc/guides/sample_app_ug/l2_forward_real_virtual.rst +0
-0@@ -91,7 +91,10 @@ The application requires a number of command lineoptions:
.. code-block:: console
doc/guides/sample_app_ug/l2_forward_real_virtual.rst +5
-1- ./build/l2fwd [EAL options] -- -p PORTMASK [-q NQ] --[no-]mac-updating + ./build/l2fwd [EAL options] -- -p PORTMASK + [-q NQ] + --[no-]mac-updating + [--portmap="(port, port)[,(port, + port)]"]where,
doc/guides/sample_app_ug/l2_forward_real_virtual.rst +0
-1@@ -99,7 +102,9 @@ where, * q NQ: A number of queues (=ports) per lcore (default is 1) -* --[no-]mac-updating: Enable or disable MAC addresses updating (enabledby default).
+* --[no-]mac-updating: Enable or disable MAC addresses updating (enabled
by default)doc/guides/sample_app_ug/l2_forward_real_virtual.rst +2
-0+ +* --portmap="(port,port)[,(port,port)]": Determines forwarding portsmapping.
To run the application in linux environment with 4 lcores, 16 ports and 8 RX queues per lcore and MAC address updating enabled, issue the command:
doc/guides/sample_app_ug/l2_forward_real_virtual.rst +8
-0@@ -108,6 +113,14 @@ updating enabled, issue the command: $ ./build/l2fwd -l 0-3 -n 4 -- -q 8 -p ffff +To run the application in linux environment with 4 lcores, 4 ports, 8 +RX queues per lcore, to forward RX traffic of ports 0 & 1 on ports 2 & +3 respectively and vice versa, issue the command: + +.. code-block:: console + + $ ./build/l2fwd -l 0-3 -n 4 -- -q 8 -p f --portmap="(0,2)(1,3)" +Refer to the *DPDK Getting Started Guide* for general information on running applications and the Environment Abstraction Layer (EAL) options.
examples/l2fwd/main.c index renamed from examples/l2fwd/main.c +0
-0
88ddfe589..1d84cd789100644examples/l2fwd/main.c +7
-0@@ -38,6 +38,7 @@#include <rte_ethdev.h> #include <rte_mempool.h> #include <rte_mbuf.h> +#include <rte_string_fns.h> static volatile bool force_quit;@@ -67,6 +68,15 @@ static uint32_t l2fwd_enabled_port_mask = 0;/* list of enabled ports */ static uint32_t l2fwd_dst_ports[RTE_MAX_ETHPORTS ]; +struct port_pair_params { +#define NUM_PORTS 2 +uint16_t port[NUM_PORTS ]; +} __rte_cache_aligned; + +static struct port_pair_paramsport_pair_params_array[RTE_MAX_ETHPORTS
examples/l2fwd/main.c +3
-0+/ 2]; static struct port_pair_params *port_pair_params; static uint16_t +nb_port_pair_params; +static unsigned int
l2fwd_rx_queue_per_lcore= 1;#define
MAX_RX_QUEUE_PER_LCORE16examples/l2fwd/main.c +0
-1@@ -294,11 +304,13 @@ l2fwd_usage(const char *prgname)printf ("%s [EAL options] -- -p PORTMASK [-q NQ]\n" " -p PORTMASK: hexadecimal bitmask of ports to configure\n" " -q NQ: number of queue (=ports) per lcore (default is 1)\n" - " -T PERIOD: statistics will be refreshed each PERIODseconds (0 to disable, 10 default, 86400 maximum)\n"
examples/l2fwd/main.c +0
-1- " --[no-]mac-updating: Enable or disable MAC addressesupdating (enabled by default)\n"
examples/l2fwd/main.c +0
-2-" When enabled:\n" - " - The source MAC address is replaced by the TX portMAC address\n"
examples/l2fwd/main.c +0
-1- " - The destination MAC address is replaced by02:00:00:00:00:TX_PORT_ID\n",
examples/l2fwd/main.c +1
-0+ " -T PERIOD: statistics will be refreshed each PERIOD seconds (0 todisable, 10 default, 86400 maximum)\n"
examples/l2fwd/main.c +1
-0+ " --[no-]mac-updating: Enable or disable MAC addresses updating(enabled by default)\n"
examples/l2fwd/main.c +2
-0+" When enabled:\n" + " - The source MAC address is replaced by the TX port MACaddress\n"
examples/l2fwd/main.c +1
-0+ " - The destination MAC address is replaced by02:00:00:00:00:TX_PORT_ID\n"
examples/l2fwd/main.c +2
-0+" --portmap: Configure forwarding port pair mapping\n" + " Default: alternate port pairs\n\n",prgname); }examples/l2fwd/main.c +54
-0@@ -319,6 +331,61 @@ l2fwd_parse_portmask(const char *portmask)return pm; } +static int +l2fwd_parse_port_pair_config (const char *q_arg) { +enum fieldnames { +FLD_PORT1 =0 , +FLD_PORT2 , + _NUM_FLD + }; +unsigned long int_fld[_NUM_FLD]; +const char *p, *p0 = q_arg; +char *str_fld[_NUM_FLD]; +unsigned int size; +char s[256 ]; +char *end; +int i; + + nb_port_pair_params =0 ; + +while ((p =strchr (p0,'(' )) !=NULL ) { + ++p; + p0 =strchr (p,')' ); +if (p0 ==NULL ) +return -1 ; + + size = p0 - p; +if (size >=sizeof (s)) +return -1 ; + +memcpy (s, p, size); + s[size] ='\0' ; +if (rte_strsplit (s,sizeof (s), str_fld, + _NUM_FLD,',' ) != _NUM_FLD) +return -1 ; +for (i =0 ; i < _NUM_FLD; i++) { + errno =0 ; + int_fld[i] =strtoul (str_fld[i], &end,0 ); +if (errno !=0 || end == str_fld[i] || + int_fld[i] >=RTE_MAX_ETHPORTS ) +return -1 ; + } +if (nb_port_pair_params >=RTE_MAX_ETHPORTS /2 ) { + printf("exceeded max number of port pair params: %hu\n", + nb_port_pair_params); +return -1 ; + } + port_pair_params_array[nb_port_pair_params].port[0 ] = + (uint16_t )int_fld[FLD_PORT1 ]; + port_pair_params_array[nb_port_pair_params].port[1 ] = + (uint16_t )int_fld[FLD_PORT2 ]; + ++nb_port_pair_params; + } + port_pair_params = port_pair_params_array; +return 0 ; +} +static unsigned int
l2fwd_parse_nqueue(const char *q_arg)
{examples/l2fwd/main.c +1
-0@@ -361,6 +428,7 @@ static const char short_options[] =#define CMD_LINE_OPT_MAC_UPDATING "mac-updating"#define CMD_LINE_OPT_NO_MAC_UPDATING "no-mac-updating" +#define CMD_LINE_OPT_PORTMAP_CONFIG "portmap"enum { /* long options mapped to a short option */ @@ -368,11 +436,13 @@enum { /* first long only option value must be >= 256, so that we won't * conflict with short options */ CMD_LINE_OPT_MIN_NUM = 256,examples/l2fwd/main.c +1
-0+ CMD_LINE_OPT_PORTMAP_NUM,};
static const struct option lgopts[] = {
{CMD_LINE_OPT_MAC_UPDATING,no_argument,&mac_updating,
1},
{CMD_LINE_OPT_NO_MAC_UPDATING,no_argument,&mac_updating, 0},examples/l2fwd/main.c +1
-0+ { CMD_LINE_OPT_PORTMAP_CONFIG, 1, 0,
CMD_LINE_OPT_PORTMAP_NUM},{NULL, 0, 0, 0} };examples/l2fwd/main.c +2
-0@@ -386,6 +456,7 @@ l2fwd_parse_args(int argc, char **argv)char *prgname = argv[0 ]; argvopt = argv; + port_pair_params =NULL ;while ((opt =getopt_long (argc, argvopt, short_options, lgopts, &option_index)) != EOF) { @@ -423,7 +494,13 @@ l2fwd_parse_args(int argc, char **argv)break; /* long options */examples/l2fwd/main.c +7
-1-case 0 : +case CMD_LINE_OPT_PORTMAP_NUM : + ret =l2fwd_parse_port_pair_config (optarg); +if (ret) { +fprintf (stderr,"Invalid config\n" ); +l2fwd_usage (prgname); +return -1 ; + }break; default:examples/l2fwd/main.c +42
-0@@ -440,6 +517,48 @@ l2fwd_parse_args(int argc, char **argv)return ret; } +/* +* Check port pair config with enabled port mask, +* and for valid port pair combinations. +*/ +static int +check_port_pair_config (void ) +{ +uint32_t port_pair_config_mask =0 ; +uint32_t port_pair_mask =0 ; +uint16_t index, i, portid; + +for (index =0 ; index < nb_port_pair_params; index++) { + port_pair_mask =0 ; + +for (i =0 ; i <NUM_PORTS ; i++) { + portid = port_pair_params[index].port[i]; +if ((l2fwd_enabled_port_mask & (1 << portid)) ==0 ) { +printf ("port %u is not enabled in port mask\n", + portid); +return -1 ; + } +if (!rte_eth_dev_is_valid_port (portid)) { +printf ("port %u is not present on the board\n", + portid); +return -1 ; + } + + port_pair_mask |=1 << portid; + } + +if (port_pair_config_mask & port_pair_mask) { +printf ("port %u is used in other port pairs\n" , portid); +return -1 ; + } + port_pair_config_mask |= port_pair_mask; + } + + l2fwd_enabled_port_mask &= port_pair_config_mask; + +return 0 ; +} + /* Check the link status of all ports in up to 9s, and print them finally */ staticvoid
check_all_ports_link_status(uint32_t port_mask)@@ -555,6 +674,11 @@main(int argc, char **argv)if (nb_ports == 0) rte_exit(EXIT_FAILURE, "No Ethernet ports - bye\n");examples/l2fwd/main.c +5
-0+if (port_pair_params !=NULL ) { +if (check_port_pair_config () <0 ) +rte_exit (EXIT_FAILURE ,"Invalid port pair config\n" ); + } +/* check port mask to possible port mask */ if (l2fwd_enabled_port_mask & ~((1 << nb_ports) - 1)) rte_exit(EXIT_FAILURE, "Invalid portmask; possible (0x%x)\n",examples/l2fwd/main.c +28
-18@@ -565,26 +689,35 @@ main(int argc, char **argv) l2fwd_dst_ports[portid] =0 ; last_port =0 ; -/* -* Each logical core is assigned a dedicated TX queue on each port. -*/ -RTE_ETH_FOREACH_DEV (portid) { -/* skip ports that are not enabled */ -if ((l2fwd_enabled_port_mask & (1 << portid)) ==0 ) -continue ; +/* populate destination port details */ +if (port_pair_params !=NULL ) { +uint16_t idx, p; -if (nb_ports_in_mask %2 ) { - l2fwd_dst_ports[portid] = last_port; - l2fwd_dst_ports[last_port] = portid; +for (idx =0 ; idx < (nb_port_pair_params <<1 ); idx++) { + p = idx &1 ; + portid = port_pair_params[idx >>1 ].port[p]; + l2fwd_dst_ports[portid] = + port_pair_params[idx >>1 ].port[p ^1 ]; } -else - last_port = portid; + }else { +RTE_ETH_FOREACH_DEV (portid) { +/* skip ports that are not enabled */ +if ((l2fwd_enabled_port_mask & (1 << portid)) ==0 ) +continue ; - nb_ports_in_mask++; - } -if (nb_ports_in_mask %2 ) { -printf ("Notice: odd number of ports in portmask.\n" ); - l2fwd_dst_ports[last_port] = last_port; +if (nb_ports_in_mask %2 ) { + l2fwd_dst_ports[portid] = last_port; + l2fwd_dst_ports[last_port] = portid; + }else { + last_port = portid; + } + + nb_ports_in_mask++; + } +if (nb_ports_in_mask %2 ) { +printf ("Notice: odd number of ports in portmask.\n" ); + l2fwd_dst_ports[last_port] = last_port; + } } rx_lcore_id =0 ;@@ -613,7 +746,8 @@ main(int argc, char **argv) qconf->rx_port_list[qconf->n_rx_port] = portid; qconf->n_rx_port++; -printf ("Lcore %u: RX port %u\n" , rx_lcore_id, portid); +printf ("Lcore %u: RX port %u TX port %u\n" , rx_lcore_id, + portid, l2fwd_dst_ports[portid]); } nb_mbufs = RTE_MAX(nb_ports * (nb_rxd + nb_txd +
MAX_PKT_BURST+
--
2.17.1
Re: [dpdk-dev] [PATCH v4] examples/l2fwd: add cmdline option for forwarding port info
In reply to [email protected]Hi Vamsi & Pavan,
I like this idea, couple of queries
snipped
Quoted 21 lines
examples/l2fwd/main.c +14
-0+static int +check_port_pair_config (void ) +{ +uint32_t port_pair_config_mask =0 ; +uint32_t port_pair_mask =0 ; +uint16_t index, i, portid; + +for (index =0 ; index < nb_port_pair_params; index++) { + port_pair_mask =0 ; + +for (i =0 ; i <NUM_PORTS ; i++) { + portid = port_pair_params[index].port[i]; +if ((l2fwd_enabled_port_mask & (1 << portid)) ==0 ) { + printf("port %u is not enabled in portmask\n",
examples/l2fwd/main.c +5
-0+ portid); +return -1 ; + } +if (!rte_eth_dev_is_valid_port (portid)) { + printf("port %u is not present on theboard\n",
examples/l2fwd/main.c l2fwd_parse_args(int argc, char **argv) · line 546
Quoted 4 lines
examples/l2fwd/main.c +4
-0+ portid); +return -1 ; + } +
Should we check & warn the user if
1. port speed mismatch
2. on different NUMA
3. port pairs are physical and vdev like tap, and KNI (performance).
Quoted 3 lines
examples/l2fwd/main.c +3
-0+ port_pair_mask |=1 << portid; + } +
snipped
examples/l2fwd/main.c main(int argc, char **argv) · line 691
Quoted 13 lines
examples/l2fwd/main.c +5
-0+if (port_pair_params !=NULL ) { +if (check_port_pair_config () <0 ) +rte_exit (EXIT_FAILURE ,"Invalid port pair config\n" ); + } +/* check port mask to possible port mask */ if (l2fwd_enabled_port_mask & ~((1 << nb_ports) -1 )) rte_exit(EXIT_FAILURE, "Invalid portmask; possible (0x%x)\n",@@ -565,26 +689,35 @@ main(int argc, char **argv) l2fwd_dst_ports[portid] =0 ; last_port =0 ;
Should not the check_port_pair be after this? If the port is not enabled in port_mask will you skip that pair? or skip RX-TX from that port?
Quoted 37 lines
examples/l2fwd/main.c +17
-17-/* -* Each logical core is assigned a dedicated TX queue on each port. -*/ -RTE_ETH_FOREACH_DEV (portid) { -/* skip ports that are not enabled */ -if ((l2fwd_enabled_port_mask & (1 << portid)) ==0 ) -continue ; +/* populate destination port details */ +if (port_pair_params !=NULL ) { +uint16_t idx, p; -if (nb_ports_in_mask %2 ) { - l2fwd_dst_ports[portid] = last_port; - l2fwd_dst_ports[last_port] = portid; +for (idx =0 ; idx < (nb_port_pair_params <<1 ); idx++) { + p = idx &1 ; + portid = port_pair_params[idx >>1 ].port[p]; + l2fwd_dst_ports[portid] = + port_pair_params[idx >>1 ].port[p ^1 ]; } -else - last_port = portid; + }else { +RTE_ETH_FOREACH_DEV (portid) { +/* skip ports that are not enabled */ +if ((l2fwd_enabled_port_mask & (1 << portid)) ==0 ) +continue ; - nb_ports_in_mask++; - } -if (nb_ports_in_mask %2 ) { -printf ("Notice: odd number of ports in portmask.\n" ); - l2fwd_dst_ports[last_port] = last_port; +if (nb_ports_in_mask %2 ) { + l2fwd_dst_ports[portid] = last_port; + l2fwd_dst_ports[last_port] = portid; + }else {
examples/l2fwd/main.c main(int argc, char **argv) · line 721
Quoted 10 lines
examples/l2fwd/main.c +9
-0+ last_port = portid; + } + + nb_ports_in_mask++; + } +if (nb_ports_in_mask %2 ) { +printf ("Notice: odd number of ports in portmask.\n" ); + l2fwd_dst_ports[last_port] = last_port; + } }
As mentioned above there can ports in mask which might be disabled for port pair. Should not that be skipped rather than setting last port rx-tx loopback?
Quoted 15 lines
examples/l2fwd/main.c +2
-1rx_lcore_id =0 ;@@ -613,7 +746,8 @@ main(int argc, char **argv) qconf->rx_port_list[qconf->n_rx_port] = portid; qconf->n_rx_port++; -printf ("Lcore %u: RX port %u\n" , rx_lcore_id, portid); +printf ("Lcore %u: RX port %u TX port %u\n" , rx_lcore_id, + portid, l2fwd_dst_ports[portid]); } nb_mbufs = RTE_MAX(nb_ports * (nb_rxd + nb_txd +
MAX_PKT_BURST+
--
2.17.1
Re: [dpdk-dev] [PATCH v4] examples/l2fwd: add cmdline option for forwarding port info
Quoted 29 lines
Hi Vamsi & Pavan,
I like this idea, couple of queries
snipped
examples/l2fwd/main.c l2fwd_parse_args(int argc, char **argv)
Quoted 13 lines
examples/l2fwd/main.c +13
-0+static int +check_port_pair_config (void ) +{ +uint32_t port_pair_config_mask =0 ; +uint32_t port_pair_mask =0 ; +uint16_t index, i, portid; + +for (index =0 ; index < nb_port_pair_params; index++) { + port_pair_mask =0 ; + +for (i =0 ; i <NUM_PORTS ; i++) { + portid = port_pair_params[index].port[i]; + if ((l2fwd_enabled_port_mask & (1 << portid))== 0) {
Quoted 9 lines
(quoted diff) +1
-0+ printf("port %u is not enabled in portmask\n",
examples/l2fwd/main.c +5
-0+ portid); +return -1 ; + } +if (!rte_eth_dev_is_valid_port (portid)) { + printf("port %u is not present on theboard\n",
(quoted diff) +1
-0+ portid);
Quoted 9 lines
examples/l2fwd/main.c l2fwd_parse_portmask(const char *portmask) · line 388
Quoted 3 lines
examples/l2fwd/main.c +3
-0+return -1 ; + } +Should we check & warn the user if
1. port speed mismatch
2. on different NUMA
3. port pairs are physical and vdev like tap, and KNI (performance).
Sure, it can be a separate patch as it will be applicable for multiple examples.
Quoted 18 lines
Quoted 3 lines
examples/l2fwd/main.c +3
-0+ port_pair_mask |=1 << portid; + } +snipped
examples/l2fwd/main.c main(int argc, char **argv)
Quoted 4 lines
examples/l2fwd/main.c +3
-0+if (port_pair_params !=NULL ) { +if (check_port_pair_config () <0 ) + rte_exit(EXIT_FAILURE, "Invalid port pairconfig\n");
examples/l2fwd/main.c main(int argc, char **argv)
Quoted 5 lines
examples/l2fwd/main.c +2
-0+ } +/* check port mask to possible port mask */ if (l2fwd_enabled_port_mask & ~((1 << nb_ports) -1 )) rte_exit(EXIT_FAILURE, "Invalid portmask; possible(0x%x)\n",
Quoted 7 lines
examples/l2fwd/main.c main(int argc, char **argv) · line 691
Quoted 4 lines
examples/l2fwd/main.c +0
-0@@ -565,26 +689,35 @@ main(int argc, char **argv) l2fwd_dst_ports[portid] =0 ; last_port =0 ;Should not the
check_port_pairbe after this? If the port is not enabled inport_maskwill you skip that pair? or skip RX-TX from that port?
We check every port pair against l2fwd_enabled_port_mask in check_port_pair_config()
Quoted 48 lines
Review main(int argc, char **argv) · line 569
Quoted 3 lines
(quoted diff) +0
-2- /* - * Each logical core is assigned a dedicated TX queue on eachport.
Quoted 14 lines
examples/l2fwd/main.c +5
-8-*/ -RTE_ETH_FOREACH_DEV (portid) { -/* skip ports that are not enabled */ -if ((l2fwd_enabled_port_mask & (1 << portid)) ==0 ) -continue ; +/* populate destination port details */ +if (port_pair_params !=NULL ) { +uint16_t idx, p; -if (nb_ports_in_mask %2 ) { - l2fwd_dst_ports[portid] = last_port; - l2fwd_dst_ports[last_port] = portid; +for (idx =0 ; idx < (nb_port_pair_params <<1 ); idx++) { + p = idx &1 ;examples/l2fwd/main.c main(int argc, char **argv)
Quoted 10 lines
examples/l2fwd/main.c +7
-2+ portid = port_pair_params[idx >>1 ].port[p]; + l2fwd_dst_ports[portid] = + port_pair_params[idx >>1 ].port[p ^1 ]; } -else - last_port = portid; + }else { +RTE_ETH_FOREACH_DEV (portid) { +/* skip ports that are not enabled */ + if ((l2fwd_enabled_port_mask & (1 << portid))== 0)
Quoted 8 lines
examples/l2fwd/main.c +2
-5+continue ; - nb_ports_in_mask++; - } -if (nb_ports_in_mask %2 ) { -printf ("Notice: odd number of ports in portmask.\n" ); - l2fwd_dst_ports[last_port] = last_port; +if (nb_ports_in_mask %2 ) {examples/l2fwd/main.c main(int argc, char **argv)
Quoted 10 lines
examples/l2fwd/main.c +10
-0+ l2fwd_dst_ports[portid] = last_port; + l2fwd_dst_ports[last_port] = portid; + }else { + last_port = portid; + } + + nb_ports_in_mask++; + } +if (nb_ports_in_mask %2 ) { + printf("Notice: odd number of ports inportmask.\n");
Quoted 7 lines
examples/l2fwd/main.c main(int argc, char **argv) · line 721
Quoted 3 lines
examples/l2fwd/main.c +2
-0+ l2fwd_dst_ports[last_port] = last_port; + } }As mentioned above there can ports in mask which might be disabled for port pair. Should not that be skipped rather than setting last port rx- tx loopback?
There could be scenarios where user might want to test 2x10G and 1x40G Why force the user to explicitly mention 1x40G as port pair of itself in the portpair config?
Quoted 17 lines
examples/l2fwd/main.c main(int argc, char **argv) · line 749
Quoted 9 lines
examples/l2fwd/main.c +1
-1rx_lcore_id =0 ;@@ -613,7 +746,8 @@ main(int argc, char **argv) qconf->rx_port_list[qconf->n_rx_port] = portid; qconf->n_rx_port++; -printf ("Lcore %u: RX port %u\n" , rx_lcore_id, portid); + printf("Lcore %u: RX port %u TX port %u\n",
rx_lcore_id,Quoted 7 lines
examples/l2fwd/main.c +1
-0+ portid, l2fwd_dst_ports[portid]); } nb_mbufs = RTE_MAX(nb_ports * (nb_rxd + nb_txd +
MAX_PKT_BURST+
--
2.17.1
Re: [dpdk-dev] [PATCH v4] examples/l2fwd: add cmdline option for forwarding port info
Hi Pavan,
snipped
Quoted 8 lines
Quoted 6 lines
Should we check & warn the user if
1. port speed mismatch
2. on different NUMA
3. port pairs are physical and vdev like tap, and KNI (performance).Sure, it can be a separate patch as it will be applicable for multiple examples.
I believe this patch is for example l2fwd. But you would like to have to updated for all example. I am ok for this.
snipped
Quoted 9 lines
Quoted 3 lines
Should not the
check_port_pairbe after this? If the port is not enabled inport_maskwill you skip that pair? or skip RX-TX from that port?We check every port pair against
l2fwd_enabled_port_maskincheck_port_pair_config()
snipped
Quoted 8 lines
Quoted 4 lines
As mentioned above there can ports in mask which might be disabled for port pair. Should not that be skipped rather than setting last port rx- tx loopback?
There could be scenarios where user might want to test 2x10G and 1x40G Why force the user to explicitly mention 1x40G as port pair of itself in the portpair config?
I am not sure if I follow your thought, as your current port map only allows 1:1 mapping by struct port_pair_params. This can be to self like (port0:port0),(port1:port1) or (port-0:port-1).
1. But current l2fwd_parse_port_pair_config does not consider the same port mapping as we have hard check for if (nb_port_pair_params >= RTE_MAX_ETHPORTS/2).
2. l2fwd_enabled_port_mask is global variable of user port mask. This can contain both valid and invalid mask. Hence we check l2fwd_enabled_port_mask & ~((1 << nb_ports) - 1).
3. can these scenarios are true if we invoke check_port_pair_config before actual port_mask check.
a. there are only 4 ports, hence possible mask is 0xf.
b. user passes port argument as 0xe
c. check_port_pair_config gets masks for (1,3) as input and populates port_pair_config_mask.
d. As per the code, port 2 which is valid port and part of user port mask will have lastport (which is port 3)? May be I did understand the logic correct. Can you help me?
So my concerns are 1) there is no same port mapping, 2) my understanding on lastport logic is not clear and 3) as per the code there is 1:N but 1:1.
Hence there should be sufficient warning to user if port are of wrong speed and NUMA.
Note: current speed can be fetched only if the port are started too (in Fortville).
snipped
Re: [dpdk-dev] [PATCH v4] examples/l2fwd: add cmdline option for forwarding port info
Hi Vipin,
Quoted 50 lines
Hi Pavan,
snipped
Quoted 8 lines
Quoted 6 lines
Should we check & warn the user if
1. port speed mismatch
2. on different NUMA
3. port pairs are physical and vdev like tap, and KNI (performance).Sure, it can be a separate patch as it will be applicable for multiple
examples.
I believe this patch is for examplel2fwd. But you would like to have to updated for allexample. I am ok for this.snipped
Quoted 3 lines
Quoted 3 lines
Should not the
check_port_pairbe after this? If the port is not enabled inport_maskwill you skip that pair? or skip RX-TX from thatport?
Quoted 6 lines
We check every port pair against
l2fwd_enabled_port_maskincheck_port_pair_config()snipped
Quoted 2 lines
Quoted 2 lines
As mentioned above there can ports in mask which might be
disabled for
Quoted 4 lines
Quoted 2 lines
port pair. Should not that be skipped rather than setting last port rx- tx loopback?
There could be scenarios where user might want to test 2x10G and
1x40G Why
Quoted 1 line
force the user to explicitly mention 1x40G as port pair of itself in the
portpair
Quoted 1 line
config?
I am not sure if I follow your thought, as your current port map only allows
1:1mapping bystruct port_pair_params. This can be to self like(port0:port0),(port1:port1)or(port-0:port-1).1. But current
l2fwd_parse_port_pair_configdoes not consider the same port mapping as we have hard check for `if (nb_port_pair_paramsQuoted 1 line
= RTE_MAX_ETHPORTS/2)`.
2.
l2fwd_enabled_port_maskis global variable of user port mask. This can contain both valid and invalid mask. Hence we checkl2fwd_enabled_port_mask & ~((1 << nb_ports) - 1).
Quoted 9 lines
3. can these scenarios are true if we invoke
check_port_pair_configbefore actualport_maskcheck.
a. there are only 4 ports, hence possible mask is0xf.
b. user passes port argument as0xe
c.check_port_pair_configgets masks for(1,3)as input and populatesport_pair_config_mask.
d. As per the code, port 2 which is valid port and part of user port mask will have lastport (which is port 3)? May be I did understand the logic correct. Can you help me?
Here user needs to explicitly mention (2,2) for port 2 to be setup else it will be skipped.
If you see check_port_pair_config below we disable the ports that are not Mentioned in portmap.
"check_port_pair_config(void)
{
<snip>
port_pair_config_mask |= port_pair_mask; } l2fwd_enabled_port_mask &= port_pair_config_mask; return 0; }
"
Quoted 7 lines
So my concerns are 1) there is no same port mapping, 2) my understanding on lastport logic is not clear and 3) as per the code there is 1:N but 1:1.
Hence there should be sufficient warning to user if port are of wrong speed and NUMA.
Unless the user disables stats using -T 0 option all the prints will be skipped.
Quoted 5 lines
Note: current speed can be fetched only if the port are started too (in Fortville).
snipped
Re: [dpdk-dev] [PATCH v4] examples/l2fwd: add cmdline option for forwarding port info
Bruce, as maintainer of l2fwd example, any opinion about this change?
11/05/2020 02:23, Pavan Nikhilesh Bhagavatula:
Quoted 97 lines
Hi Vipin,
Quoted 50 lines
Hi Pavan,
snipped
Quoted 8 lines
Quoted 6 lines
Should we check & warn the user if
1. port speed mismatch
2. on different NUMA
3. port pairs are physical and vdev like tap, and KNI (performance).Sure, it can be a separate patch as it will be applicable for multiple
examples.
I believe this patch is for examplel2fwd. But you would like to have to updated for allexample. I am ok for this.snipped
Quoted 3 lines
Quoted 3 lines
Should not the
check_port_pairbe after this? If the port is not enabled inport_maskwill you skip that pair? or skip RX-TX from thatport?
Quoted 6 lines
We check every port pair against
l2fwd_enabled_port_maskincheck_port_pair_config()snipped
Quoted 2 lines
Quoted 2 lines
As mentioned above there can ports in mask which might be
disabled for
Quoted 4 lines
Quoted 2 lines
port pair. Should not that be skipped rather than setting last port rx- tx loopback?
There could be scenarios where user might want to test 2x10G and
1x40G Why
Quoted 1 line
force the user to explicitly mention 1x40G as port pair of itself in the
portpair
Quoted 1 line
config?
I am not sure if I follow your thought, as your current port map only allows
1:1mapping bystruct port_pair_params. This can be to self like(port0:port0),(port1:port1)or(port-0:port-1).1. But current
l2fwd_parse_port_pair_configdoes not consider the same port mapping as we have hard check for `if (nb_port_pair_paramsQuoted 1 line
= RTE_MAX_ETHPORTS/2)`.
2.
l2fwd_enabled_port_maskis global variable of user port mask. This can contain both valid and invalid mask. Hence we checkl2fwd_enabled_port_mask & ~((1 << nb_ports) - 1).Quoted 9 lines
3. can these scenarios are true if we invoke
check_port_pair_configbefore actualport_maskcheck.
a. there are only 4 ports, hence possible mask is0xf.
b. user passes port argument as0xe
c.check_port_pair_configgets masks for(1,3)as input and populatesport_pair_config_mask.
d. As per the code, port 2 which is valid port and part of user port mask will have lastport (which is port 3)? May be I did understand the logic correct. Can you help me?Here user needs to explicitly mention (2,2) for port 2 to be setup else it will be skipped.
If you seecheck_port_pair_configbelow we disable the ports that are not Mentioned in portmap."
check_port_pair_config(void)
{<snip>
port_pair_config_mask |= port_pair_mask; } l2fwd_enabled_port_mask &= port_pair_config_mask; return 0; }"
Quoted 7 lines
So my concerns are 1) there is no same port mapping, 2) my understanding on lastport logic is not clear and 3) as per the code there is 1:N but 1:1.
Hence there should be sufficient warning to user if port are of wrong speed and NUMA.
Unless the user disables stats using -T 0 option all the prints will be skipped.
Quoted 5 lines
Note: current speed can be fetched only if the port are started too (in Fortville).
snipped
Bruce Richardson
Acked-byRe: [dpdk-dev] [PATCH v4] examples/l2fwd: add cmdline option for forwarding port info
Quoted 2 lines · Thomas Monjalon
Bruce, as maintainer of l2fwd example, any opinion about this change?
Assuming all previous discussion on it is resolved, I'm fine with this patch, though I suspect it will only make 20.08 now.
Quoted 103 lines
11/05/2020 02:23, Pavan Nikhilesh Bhagavatula:
Quoted 97 lines
Hi Vipin,
Quoted 50 lines
Hi Pavan,
snipped
Quoted 8 lines
Quoted 6 lines
Should we check & warn the user if
1. port speed mismatch
2. on different NUMA
3. port pairs are physical and vdev like tap, and KNI (performance).Sure, it can be a separate patch as it will be applicable for multiple
examples.
I believe this patch is for examplel2fwd. But you would like to have to updated for allexample. I am ok for this.snipped
Quoted 3 lines
Quoted 3 lines
Should not the
check_port_pairbe after this? If the port is not enabled inport_maskwill you skip that pair? or skip RX-TX from thatport?
Quoted 6 lines
We check every port pair against
l2fwd_enabled_port_maskincheck_port_pair_config()snipped
Quoted 2 lines
Quoted 2 lines
As mentioned above there can ports in mask which might be
disabled for
Quoted 4 lines
Quoted 2 lines
port pair. Should not that be skipped rather than setting last port rx- tx loopback?
There could be scenarios where user might want to test 2x10G and
1x40G Why
Quoted 1 line
force the user to explicitly mention 1x40G as port pair of itself in the
portpair
Quoted 1 line
config?
I am not sure if I follow your thought, as your current port map only allows
1:1mapping bystruct port_pair_params. This can be to self like(port0:port0),(port1:port1)or(port-0:port-1).1. But current
l2fwd_parse_port_pair_configdoes not consider the same port mapping as we have hard check for `if (nb_port_pair_paramsQuoted 1 line
= RTE_MAX_ETHPORTS/2)`.
2.
l2fwd_enabled_port_maskis global variable of user port mask. This can contain both valid and invalid mask. Hence we checkl2fwd_enabled_port_mask & ~((1 << nb_ports) - 1).Quoted 9 lines
3. can these scenarios are true if we invoke
check_port_pair_configbefore actualport_maskcheck.
a. there are only 4 ports, hence possible mask is0xf.
b. user passes port argument as0xe
c.check_port_pair_configgets masks for(1,3)as input and populatesport_pair_config_mask.
d. As per the code, port 2 which is valid port and part of user port mask will have lastport (which is port 3)? May be I did understand the logic correct. Can you help me?Here user needs to explicitly mention (2,2) for port 2 to be setup else it will be skipped.
If you seecheck_port_pair_configbelow we disable the ports that are not Mentioned in portmap."
check_port_pair_config(void)
{<snip>
port_pair_config_mask |= port_pair_mask; } l2fwd_enabled_port_mask &= port_pair_config_mask; return 0; }"
Quoted 7 lines
So my concerns are 1) there is no same port mapping, 2) my understanding on lastport logic is not clear and 3) as per the code there is 1:N but 1:1.
Hence there should be sufficient warning to user if port are of wrong speed and NUMA.
Unless the user disables stats using -T 0 option all the prints will be skipped.
Quoted 5 lines
Note: current speed can be fetched only if the port are started too (in Fortville).
snipped
Re: [dpdk-dev] [PATCH v4] examples/l2fwd: add cmdline option for forwarding port info
Quoted 8 lines · Bruce Richardson
Quoted 2 lines · Thomas Monjalon
Bruce, as maintainer of l2fwd example, any opinion about this change?
Assuming all previous discussion on it is resolved, I'm fine with this patch, though I suspect it will only make 20.08 now.
Acked-by Bruce Richardson <[email protected]>
Ping for merge.
Quoted 104 lines
Quoted 104 lines
11/05/2020 02:23, Pavan Nikhilesh Bhagavatula:
Quoted 97 lines
Hi Vipin,
Quoted 50 lines
Hi Pavan,
snipped
Quoted 8 lines
Quoted 6 lines
Should we check & warn the user if
1. port speed mismatch
2. on different NUMA
3. port pairs are physical and vdev like tap, and KNI (performance).Sure, it can be a separate patch as it will be applicable for multiple
examples.
I believe this patch is for examplel2fwd. But you would like to have to updated for allexample. I am ok for this.snipped
Quoted 3 lines
Quoted 3 lines
Should not the
check_port_pairbe after this? If the port is not enabled inport_maskwill you skip that pair? or skip RX-TX from thatport?
Quoted 6 lines
We check every port pair against
l2fwd_enabled_port_maskincheck_port_pair_config()snipped
Quoted 2 lines
Quoted 2 lines
As mentioned above there can ports in mask which might be
disabled for
Quoted 4 lines
Quoted 2 lines
port pair. Should not that be skipped rather than setting last port rx- tx loopback?
There could be scenarios where user might want to test 2x10G and
1x40G Why
Quoted 1 line
force the user to explicitly mention 1x40G as port pair of itself in the
portpair
Quoted 1 line
config?
I am not sure if I follow your thought, as your current port map only allows
1:1mapping bystruct port_pair_params. This can be to self like(port0:port0),(port1:port1)or(port-0:port-1).1. But current
l2fwd_parse_port_pair_configdoes not consider the same port mapping as we have hard check for `if (nb_port_pair_paramsQuoted 1 line
= RTE_MAX_ETHPORTS/2)`.
2.
l2fwd_enabled_port_maskis global variable of user port mask. This can contain both valid and invalid mask. Hence we checkl2fwd_enabled_port_mask & ~((1 << nb_ports) - 1).Quoted 9 lines
3. can these scenarios are true if we invoke
check_port_pair_configbefore actualport_maskcheck.
a. there are only 4 ports, hence possible mask is0xf.
b. user passes port argument as0xe
c.check_port_pair_configgets masks for(1,3)as input and populatesport_pair_config_mask.
d. As per the code, port 2 which is valid port and part of user port mask will have lastport (which is port 3)? May be I did understand the logic correct. Can you help me?Here user needs to explicitly mention (2,2) for port 2 to be setup else it will be skipped.
If you seecheck_port_pair_configbelow we disable the ports that are not Mentioned in portmap."
check_port_pair_config(void)
{<snip>
port_pair_config_mask |= port_pair_mask; } l2fwd_enabled_port_mask &= port_pair_config_mask; return 0; }"
Quoted 7 lines
So my concerns are 1) there is no same port mapping, 2) my understanding on lastport logic is not clear and 3) as per the code there is 1:N but 1:1.
Hence there should be sufficient warning to user if port are of wrong speed and NUMA.
Unless the user disables stats using -T 0 option all the prints will be skipped.
Quoted 5 lines
Note: current speed can be fetched only if the port are started too (in Fortville).
snipped
Thomas Monjalon
AppliedRe: [dpdk-dev] [PATCH v4] examples/l2fwd: add cmdline option for forwarding port info
04/07/2020 15:36, Jerin Jacob:
Quoted 4 lines
Quoted 2 lines · Bruce Richardson
On Sun, May 24, 2020 at 06:13:22PM +0200, Thomas Monjalon wrote:
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
Bruce, as maintainer of l2fwd example, any opinion about this change?
Assuming all previous discussion on it is resolved, I'm fine with this patch, though I suspect it will only make 20.08 now.
Acked-by Bruce Richardson <[email protected]>Ping for merge.
Applied, thanks
Re: [dpdk-dev] [EXT] Re: [PATCH v7] eal: add manual probing option
In reply to Jerin JacobQuoted 24 lines
Quoted 18 lines
Should we have a command line option to configure the forwarding rules in the application (2)? I think yes.
Should we implement an application logic to automatically create the best forwarding rules (1)? It would be nice, but anyway, I think we need manual config (2) as a fallback.Quoted 2 lines
Thoughts? If you think, there is a rework needed in eal then could you enumerate the items for the rework.
Sorry I don't have time to describe dive into EAL probing and enumerate the items to rework.
The most important issues I remind are:
- white/blacklist policy is a mess and should be done in a higher layer
- devargs syntax should allow generic matching (thanks to
class awareness and generic syntax)Quoted 3 lines
Starting from these 2 items, we could imagine a generic path to disable automatic probing, but I think the l2fwd logic should not rely on probing order anyway.
+ Bruce as l2fwd maintainer.
Quoted 8 lines
Since in any case, l2fwd needs to be updated, We will focus on l2fwd change for v20.05 and leaving the fate of this patch to EAL maintainers. Let us know, Are we are OK with below change in l2fwd as
- Introduce an array-based port lookup table instead of hardcoding to xor based lookup.
- if no argument specified fill dest port as xor of source
- If argument is specified override the lookup table with a user-specified destination port.
I think, this thread can be closed here.
L2fwd change sent as different patch.
Please review if it is for interest.
http://patches.dpdk.org/patch/67722/
Quoted 5 lines
Quoted 5 lines
Re: [dpdk-dev] [PATCH v7] eal: add manual probing option
In reply to Gaetan RivetQuoted 63 lines · On Thu, 23 Jan 2020 10:58:13 +0100 Gaetan Rivet <[email protected]> wrote:
Add a new EAL option enabling manual probing in the EAL. This command line option will configure the EAL so that buses will not trigger their probe step on their own.
Applications are then expected to hotplug devices as they see fit.
Devices declared on the command line by the user (using -w and --vdev), will be probed using the hotplug API, in the order they are declared.
This has the effect of offering a way for users to control probe order of their devices, for drivers requiring it.
Signed-off-by Gaetan Rivet <[email protected]>Acked-by Vamsi Attunuru <[email protected]>Tested-by Vamsi Attunuru <[email protected]>Reviewed-by Jerin Jacob <[email protected]>---
haven't heard many opinions on the matter, please shout if you see an issue
with this approach.@Slava: I have tested rather quickly that it does not break anything,
and that it works as intended for basic cases.
Can you test it further for your use-case and tell me if it works fine?Beyond the obvious difference between both probe mode, something to keep in mind:
while using -w on invalid devices would not block (PCI) bus probing, it will stop manual probing in its track. All devices need to exist and be valid device IDs.v2: fixed a few typos, map file (and used Travis to validate).
Slava, are you able to test this patch?
v3: properly fixed the map file (inherited 19.08 instead of 19.05).
Added a function to set the probe manual from the application, without having the user do it from the command line.
Stopped spamming Slava about it, Vamsi was actually the one interested in it!
Standing issue worth chiming in:
Currently manual-probe will cut off probing from all buses. It could be interesting to be able to only cut buses supporting hotplug, given that they are the one able to probe devices afterward.
No real use-case for this right now, so leaving as-is. Might be worth considering in the future.
v4: Rebased on master,
Moved implementation in common EAL,
Used public API within the EAL to set the option, Made the API experimentalv5: added note in the Getting Started Guide.
v6: Rebased on master
see http://mails.dpdk.org/archives/dev/2020-January/154178.html
for reference to this version, linking v7 to v5 thread.v7: Updated author and SoB.
Quoted 11 lines
doc/guides/linux_gsg/eal_args.include.rst | 13 ++++++ doc/guides/rel_notes/release_20_02.rst | 9 ++++ lib/librte_eal/common/eal_common_bus.c | 6 +++ lib/librte_eal/common/eal_common_dev.c | 54 ++++++++++++++++++++++ lib/librte_eal/common/eal_common_options.c | 8 ++++ lib/librte_eal/common/eal_internal_cfg.h | 1 + lib/librte_eal/common/eal_options.h | 2 + lib/librte_eal/common/eal_private.h | 9 ++++ lib/librte_eal/common/include/rte_eal.h | 36 +++++++++++++++ lib/librte_eal/rte_eal_version.map | 4 ++ 10 files changed, 142 insertions(+)
This patch seems to have been held in limbo for 3 years.
For me, it is ok, but concerned that it opens up a whole scenario of possible usages that may not be tested, and probably don't work. Testing all the possible combinations of probe ordering is a geometric problem.
So if user submits bug then the response would have to be:
Manual probing is an experimental option which may not work.
Re: [dpdk-dev] [PATCH v7] eal: add manual probing option
Quoted 78 lines · Stephen Hemminger
Quoted 63 lines · On Thu, 23 Jan 2020 10:58:13 +0100 Gaetan Rivet <[email protected]> wrote:
Add a new EAL option enabling manual probing in the EAL. This command line option will configure the EAL so that buses will not trigger their probe step on their own.
Applications are then expected to hotplug devices as they see fit.
Devices declared on the command line by the user (using -w and --vdev), will be probed using the hotplug API, in the order they are declared.
This has the effect of offering a way for users to control probe order of their devices, for drivers requiring it.
Signed-off-by Gaetan Rivet <[email protected]>Acked-by Vamsi Attunuru <[email protected]>Tested-by Vamsi Attunuru <[email protected]>Reviewed-by Jerin Jacob <[email protected]>---
haven't heard many opinions on the matter, please shout if you see an issue
with this approach.@Slava: I have tested rather quickly that it does not break anything,
and that it works as intended for basic cases.
Can you test it further for your use-case and tell me if it works fine?Beyond the obvious difference between both probe mode, something to keep in mind:
while using -w on invalid devices would not block (PCI) bus probing, it will stop manual probing in its track. All devices need to exist and be valid device IDs.v2: fixed a few typos, map file (and used Travis to validate).
Slava, are you able to test this patch?
v3: properly fixed the map file (inherited 19.08 instead of 19.05).
Added a function to set the probe manual from the application, without having the user do it from the command line.
Stopped spamming Slava about it, Vamsi was actually the one interested in it!
Standing issue worth chiming in:
Currently manual-probe will cut off probing from all buses. It could be interesting to be able to only cut buses supporting hotplug, given that they are the one able to probe devices afterward.
No real use-case for this right now, so leaving as-is. Might be worth considering in the future.
v4: Rebased on master,
Moved implementation in common EAL,
Used public API within the EAL to set the option, Made the API experimentalv5: added note in the Getting Started Guide.
v6: Rebased on master
see http://mails.dpdk.org/archives/dev/2020-January/154178.html
for reference to this version, linking v7 to v5 thread.v7: Updated author and SoB.
Quoted 11 lines
doc/guides/linux_gsg/eal_args.include.rst | 13 ++++++ doc/guides/rel_notes/release_20_02.rst | 9 ++++ lib/librte_eal/common/eal_common_bus.c | 6 +++ lib/librte_eal/common/eal_common_dev.c | 54 ++++++++++++++++++++++ lib/librte_eal/common/eal_common_options.c | 8 ++++ lib/librte_eal/common/eal_internal_cfg.h | 1 + lib/librte_eal/common/eal_options.h | 2 + lib/librte_eal/common/eal_private.h | 9 ++++ lib/librte_eal/common/include/rte_eal.h | 36 +++++++++++++++ lib/librte_eal/rte_eal_version.map | 4 ++ 10 files changed, 142 insertions(+)
Quoted 8 lines
This patch seems to have been held in limbo for 3 years.
For me, it is ok, but concerned that it opens up a whole scenario of possible usages that may not be tested, and probably don't work. Testing all the possible combinations of probe ordering is a geometric problem.
So if user submits bug then the response would have to be:
Manual probing is an experimental option which may not work.
Hello Stephen,
I am not pushing for this series anymore.
I wrote it to help other people, I guess they used another way since. If someone needs it, I can take a moment to reanimate it.
I'm still using the PCI bus hack to force manual probing, as well as port hotplug to control strict ordering. I guess at this point this is the stable way of working with DPDK, instead of a proper documented option.
Best regards,
Signature · 1 line
Gaetan Rivet