BBDEV FPGA PF Config app
PATCH v2 1/1 BBDEV FPGA PF Config app
v2: Couple of typos in documentation and removed dependency on igb_uio and related PCIe VF creation feature option.
Addition of an stand alone application in PMD driver to be able to configure the HW from PF. Then the VF can be used from container/VM running BBDEV/DPDK.
The history of the previous discussions can be tracked on this previous patch https://patches.dpdk.org/patch/67379/
PMD documentation is updated to provides steps to build with related dependency
Nicolas Chautru (1):
baseband/fpga_5gnr_fec: add companion PF config App
doc/guides/bbdevs/fpga_5gnr_fec.rst | 80 +++-- .../baseband/fpga_5gnr_fec/pf_config_app/Makefile | 36 +++ .../fpga_5gnr_fec/pf_config_app/config_app.c | 333 +++++++++++++++++++ .../pf_config_app/fpga_5gnr_cfg_app.c | 351 +++++++++++++++++++++ .../pf_config_app/fpga_5gnr_cfg_app.h | 102 ++++++ .../pf_config_app/fpga_5gnr_cfg_parser.c | 187 +++++++++++ .../pf_config_app/fpga_5gnr_config.cfg | 18 ++ 7 files changed, 1087 insertions(+), 20 deletions(-) create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/Makefile create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/config_app.c create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_app.c create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_app.h create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_parser.c create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_config.cfg
Signature · 1 line
1.8.3.1
PATCH v2 1/1 baseband/fpga_5gnr_fec: add companion PF config App
Adding companion application to configure HW Device from the PF. Then the device can be accessed through BBDEV from VF (or PF).
--- doc/guides/bbdevs/fpga_5gnr_fec.rst | 80 +++-- .../baseband/fpga_5gnr_fec/pf_config_app/Makefile | 36 +++ .../fpga_5gnr_fec/pf_config_app/config_app.c | 333 +++++++++++++++++++ .../pf_config_app/fpga_5gnr_cfg_app.c | 351 +++++++++++++++++++++ .../pf_config_app/fpga_5gnr_cfg_app.h | 102 ++++++ .../pf_config_app/fpga_5gnr_cfg_parser.c | 187 +++++++++++ .../pf_config_app/fpga_5gnr_config.cfg | 18 ++ 7 files changed, 1087 insertions(+), 20 deletions(-) create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/Makefile create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/config_app.c create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_app.c create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_app.h create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_parser.c create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_config.cfg
doc/guides/bbdevs/fpga_5gnr_fec.rst +60 -20
@@ -218,33 +218,73 @@ parameters defined in ``fpga_5gnr_fec_conf`` structure: time_out = flr_time_out x 16.384us. For instance, if you want to set 10ms for the FLR time out then set this setting to 0x262=610. +A companion application pf_config_app is provided as a standalone application +described in next section. -An example configuration code calling the function ``fpga_5gnr_fec_configure()`` is shown -below: +PF Config App +------------- -.. code-block:: c +The PF Configuration Application ``pf_config_app`` provides a means to +configure the baseband device at the host-level. The program sets the various +parameters through memory-mapped IO read/writes. Then the BBDEV driver can be +used to run the workload. + +The parameters are parsed from a given configuration file (with .cfg extensions) +that is specific to particular BBDEV device, although they follow same format. + +External Dependencies +~~~~~~~~~~~~~~~~~~~~~ + +The third party .INI parser is a pre-requisite for building the application. +It can be downloaded from [github]: + +.. code-block:: console - struct fpga_5gnr_fec_conf conf; - unsigned int i; + git clone https://github.com/benhoyt/inih - memset(&conf, 0, sizeof(struct fpga_5gnr_fec_conf)); - conf.pf_mode_en = 1; +Use the version release 44 tracked by tag 'r44': + +.. code-block:: console + + git checkout r44 + +The application features a makefile in the `extra/` directory which generates +the library, `libinih.a`. To compile the inih library, run make as: + +.. code-block:: console + + make -f Makefile.static + +Building PF Config App +~~~~~~~~~~~~~~~~~~~~~~ + +Before building the application, set the following environment variables with +the location where the INI library is located: + +.. code-block:: console + + export INIH_PATH=<path-to-inih-lib> + +If not set, makefile will look into current folder. + +Next, build the program by typing ``make`` from the pf_Config_app subdirectory +to produce the binary ``pf_config_app_fpga_5gnr``. + +Usage +~~~~~ + +The application executes as the following: + +.. code-block:: console - for (i = 0; i < FPGA_5GNR_FEC_NUM_VFS; ++i) { - conf.vf_ul_queues_number[i] = 4; - conf.vf_dl_queues_number[i] = 4; - } - conf.ul_bandwidth = 12; - conf.dl_bandwidth = 5; - conf.dl_load_balance = 64; - conf.ul_load_balance = 64; + ./pf_config_app_fpga_5gnr [-h] [-a] [-c CFG_FILE] [-p PCI_ID] - /* setup FPGA PF */ - ret = fpga_5gnr_fec_configure(info->dev_name, &conf); - TEST_ASSERT_SUCCESS(ret, - "Failed to configure 4G FPGA PF for bbdev %s", - info->dev_name); +* ``-c CFG_FILE``: Specifies configuration file to use +* ``-p PCI_ID``: Specifies PCI ID of device to configure +* ``-a``: Configures all PCI devices +* ``-h``: Prints help +Default configure file is provided: ``fpga_5gnr_config.cfg``. Test Application ----------------
drivers/baseband/fpga_5gnr_fec/pf_config_app/Makefile new file +36 -0
@@ -0,0 +1,36 @@ + +CC=gcc +CFLAGS=-O0 -g -Wall +ODIR=build +DEPS= + +ifeq ($(INIH_PATH),) +INCLUDE=-I. +LDFLAGS=-L. +else +INCLUDE=-I. -I$(INIH_PATH) +LDFLAGS=-L. -L$(INIH_PATH) +endif + +LDLIBS=-linih + +SRC = config_app.c fpga_5gnr_cfg_app.c fpga_5gnr_cfg_parser.c +OBJ = $(patsubst %.c,$(ODIR)/%.o,$(SRC)) + +.PHONY: clean + +all: pf_config_app_fpga_5gnr + +$(ODIR): + mkdir -p $(ODIR) + +$(OBJ): $(ODIR)/%.o: ./%.c | $(DEPS) $(ODIR) + @mkdir -p $(@D) + $(CC) -c -o $@ $< $(CFLAGS) $(INCLUDE) + +pf_config_app_fpga_5gnr: $(OBJ) + $(CC) -o $@ $^ $(CFLAGS) $(LDFLAGS) $(LDLIBS) + +clean: + rm -rf $(ODIR) + rm -rf pf_config_app_fpga_5gnr
drivers/baseband/fpga_5gnr_fec/pf_config_app/config_app.c new file +333 -0
333 lines not on this page · raw message
drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_app.c new file +351 -0
351 lines not on this page · raw message
drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_app.h new file +102 -0
102 lines not on this page · raw message
drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_parser.c new file +187 -0
187 lines not on this page · raw message
drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_config.cfg new file +18 -0
@@ -0,0 +1,18 @@ +; SPDX-License-Identifier: BSD-3-Clause +; Copyright(c) 2020 Intel Corporation + +[MODE] +pf_mode_en = 1 + +[UL] +bandwidth = 3 +load_balance = 128 +vfqmap = 4,4,4,4,4,4,4,4 + +[DL] +bandwidth = 3 +load_balance = 128 +vfqmap = 4,4,4,4,4,4,4,4 + +[FLR] +flr_time_out = 610
Signature · 1 line
1.8.3.1
Re: [dpdk-dev] [20.11, PATCH v2] baseband/fpga_5gnr_fec: add companion PF config App
Hi Nicolas,
Quoted 4 lines · Nicolas Chautru
Adding companion application to configure HW Device from the PF. Then the device can be accessed through BBDEV from VF (or PF).
Signed-off-by Nicolas Chautru <[email protected]>
Quoted 15 lines
--- doc/guides/bbdevs/fpga_5gnr_fec.rst | 80 +++-- .../baseband/fpga_5gnr_fec/pf_config_app/Makefile | 36 +++ .../fpga_5gnr_fec/pf_config_app/config_app.c | 333 +++++++++++++++++++ .../pf_config_app/fpga_5gnr_cfg_app.c | 351 +++++++++++++++++++++ .../pf_config_app/fpga_5gnr_cfg_app.h | 102 ++++++ .../pf_config_app/fpga_5gnr_cfg_parser.c | 187 +++++++++++ .../pf_config_app/fpga_5gnr_config.cfg | 18 ++ 7 files changed, 1087 insertions(+), 20 deletions(-) create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/Makefile create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/config_app.c create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_app.c create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_app.h create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_parser.c create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_config.cfg
I think having the pf_config_app in the driver directory is not a good idea, this is not the place for applications.
Also, it is not integrated in the DPDK build system, so it cannot benefit from the CI. Having an external dependency that is not packaged in distributions will not help to have it integrated in the build system.
I see some alternatives:
1. Move it in another directory in the main DPDK repo, but it is not a DPDK example, not a dev tool and not a build tool, so it would need a new directory.
2. Create a BBDEV tools repository on dpdk.org (It would require techboard approval).
3. Host it in a dedicated repository on Intel's github
4. Move it into some Intel FPGA tools repository
I think option 3 would be the best to get it packaged into distributions as it has no build dependency with any DPDK library. You could maybe add inih library as a git sub-repository within this repo. Other advantage is you wouldn't depend on DPDK release cycles to get fixes merged.
Regarding the tool itself, I understand from the commit message that the tool has a dependency on the BBDEV PMD version, but the tool run on the host while the PMD driver is used in the guest/container. So having it in the driver directory will not really help, as host DPDK (if any) and guest DPDK may come from different parties.
One question I have is whether this is the tool itself that has a dependency on the PMD, or just the config file?
Regards,
Maxime
Re: [dpdk-dev] [20.11, PATCH v2] baseband/fpga_5gnr_fec: add companion PF config App
Hi Maxime,
Quoted 31 lines
Hi Nicolas,
Quoted 8 lines · Nicolas Chautru
Adding companion application to configure HW Device from the PF. Then the device can be accessed through BBDEV from VF (or PF).
Signed-off-by Nicolas Chautru <[email protected]>--- doc/guides/bbdevs/fpga_5gnr_fec.rst | 80 +++-- .../baseband/fpga_5gnr_fec/pf_config_app/Makefile | 36 +++ .../fpga_5gnr_fec/pf_config_app/config_app.c | 333+++++++++++++++++++
Quoted 1 line
.../pf_config_app/fpga_5gnr_cfg_app.c | 351+++++++++++++++++++++
Quoted 14 lines
.../pf_config_app/fpga_5gnr_cfg_app.h | 102 ++++++ .../pf_config_app/fpga_5gnr_cfg_parser.c | 187 +++++++++++ .../pf_config_app/fpga_5gnr_config.cfg | 18 ++ 7 files changed, 1087 insertions(+), 20 deletions(-) create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/Makefile create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/config_app.c create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_app.c create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_app.h create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_parser.c create mode 100644Quoted 1 line
drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_config.cfg
Quoted 7 lines
I think having the
pf_config_appin the driver directory is not a good idea, this is not the place for applications.Also, it is not integrated in the DPDK build system, so it cannot benefit from the CI. Having an external dependency that is not packaged in distributions will not help to have it integrated in the build system.
Thanks for sharing.
Note that all these points were raised openly explicitly earlier as you know, ie part of both pros and cons. Still happy to get feedback from others notably Thomas. It appears you had side conversations with him on this very topic.
Quoted 8 lines
I see some alternatives:
1. Move it in another directory in the main DPDK repo, but it is not a DPDK example, not a dev tool and not a build tool, so it would need a new directory.
2. Create a BBDEV tools repository on dpdk.org (It would require techboard approval).
3. Host it in a dedicated repository on Intel's github 4. Move it into some Intel FPGA tools repository
There are several others options which were indeed considered in case this option was not viable.
Still DPDK was considered best option so far to keep everything in one recognized place for BBDEV devices but happy to get further input from others.
Quoted 11 lines
I think option 3 would be the best to get it packaged into distributions as it has no build dependency with any DPDK library.
You could maybe add inih library as a git sub-repository within this repo. Other advantage is you wouldn't depend on DPDK release cycles to get fixes merged.Regarding the tool itself, I understand from the commit message that the tool has a dependency on the BBDEV PMD version, but the tool run on the host while the PMD driver is used in the guest/container. So having it in the driver directory will not really help, as host DPDK (if any) and guest DPDK may come from different parties.
Yes this was captured earlier, purely stored there as a companion application for a given
version of the PMD (ie. different subdirectories for each PMD directory).
They do no run in the same container for deployment and are not built at the same time indeed, their interface is the HW really and one being needed to be run prior to the other one to be functional.
Quoted 2 lines
One question I have is whether this is the tool itself that has a dependency on the PMD, or just the config file?
Each PMD directory would have its own version of the companion PF config application.
Ie. the patch above is only for baseband/fpga_5gnr_fec ie. Intel Vista Creek with 5G LDPC user image.
There will be different companion applications upstreamed for each other PMD directories (current and future) as they rely on different HW devices with independent MMIO access. Said otherwise both the config file (features exposed) and implementation (registers required for these features) are defined per HW device (+ user image for FPGA) hence per PMD version.
There 2 entities have no API between themselves, only indirectly through HW (no shared memory, VF2PF comms, etc..).
New features may have to be added concurrently though, hence splitting repos create room for version mismatch and complicate the ingredients line up. That was part of the pros and cons described earlier and I can totally see arguments both ways, and that's what I have been trying to share openly in this ticket history.
Basically I see nothing fundamentally new here in the discussion, but it is great to receive input and I am happy to hear further input from tech board or others towards a decision.
This started as an open discussion on this DPDK mailing list capturing explicitly both pros and cons of this approach which are arguable, and in case this is not deemed practical eventually then we can still totally come back internally to the drawing board with other options outside of DPDK.
Thanks,
Nic
Re: [dpdk-dev] [20.11, PATCH v2] baseband/fpga_5gnr_fec: add companion PF config App
Hi Nicolas,
Quoted 73 lines · Nicolas
Hi Maxime,
Quoted 31 lines
Hi Nicolas,
Quoted 8 lines · Nicolas Chautru
Adding companion application to configure HW Device from the PF. Then the device can be accessed through BBDEV from VF (or PF).
Signed-off-by Nicolas Chautru <[email protected]>--- doc/guides/bbdevs/fpga_5gnr_fec.rst | 80 +++-- .../baseband/fpga_5gnr_fec/pf_config_app/Makefile | 36 +++ .../fpga_5gnr_fec/pf_config_app/config_app.c | 333+++++++++++++++++++
Quoted 1 line
.../pf_config_app/fpga_5gnr_cfg_app.c | 351+++++++++++++++++++++
Quoted 14 lines
.../pf_config_app/fpga_5gnr_cfg_app.h | 102 ++++++ .../pf_config_app/fpga_5gnr_cfg_parser.c | 187 +++++++++++ .../pf_config_app/fpga_5gnr_config.cfg | 18 ++ 7 files changed, 1087 insertions(+), 20 deletions(-) create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/Makefile create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/config_app.c create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_app.c create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_app.h create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_parser.c create mode 100644Quoted 1 line
drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_config.cfgQuoted 7 lines
I think having the
pf_config_appin the driver directory is not a good idea, this is not the place for applications.Also, it is not integrated in the DPDK build system, so it cannot benefit from the CI. Having an external dependency that is not packaged in distributions will not help to have it integrated in the build system.
Thanks for sharing.
Note that all these points were raised openly explicitly earlier as you know, ie part of both pros and cons. Still happy to get feedback from others notably Thomas. It appears you had side conversations with him on this very topic.Quoted 8 lines
I see some alternatives:
1. Move it in another directory in the main DPDK repo, but it is not a DPDK example, not a dev tool and not a build tool, so it would need a new directory.
2. Create a BBDEV tools repository on dpdk.org (It would require techboard approval).
3. Host it in a dedicated repository on Intel's github 4. Move it into some Intel FPGA tools repositoryThere are several others options which were indeed considered in case this option was not viable.
Still DPDK was considered best option so far to keep everything in one recognized place for BBDEV devices but happy to get further input from others.Quoted 11 lines
I think option 3 would be the best to get it packaged into distributions as it has no build dependency with any DPDK library.
You could maybe add inih library as a git sub-repository within this repo. Other advantage is you wouldn't depend on DPDK release cycles to get fixes merged.Regarding the tool itself, I understand from the commit message that the tool has a dependency on the BBDEV PMD version, but the tool run on the host while the PMD driver is used in the guest/container. So having it in the driver directory will not really help, as host DPDK (if any) and guest DPDK may come from different parties.
Yes this was captured earlier, purely stored there as a companion application for a given
version of the PMD (ie. different subdirectories for each PMD directory).
They do no run in the same container for deployment and are not built at the same time indeed, their interface is the HW really and one being needed to be run prior to the other one to be functional.
Quoted 5 lines
Quoted 2 lines
One question I have is whether this is the tool itself that has a dependency on the PMD, or just the config file?
Each PMD directory would have its own version of the companion PF config application. Ie. the patch above is only for baseband/fpga_5gnr_fec ie. Intel Vista Creek with 5G LDPC user image.
OK. Does it mean the same application and configuration will work for baseband/fpga_5gnr_fec PMD versions v20.11, v21.02, v21.05, etc, ...?
If not, is there a way for the PMD driver to detect whether a wrong configuration was applied? Something like checking the FW version of a NIC is supported by the PMD driver.
Quoted 2 lines
There will be different companion applications upstreamed for each other PMD directories (current and future) as they rely on different HW devices with independent MMIO access. Said otherwise both the config file (features exposed) and implementation (registers required for these features) are defined per HW device (+ user image for FPGA) hence per PMD version.
Would it make sense to have a single application, with having the registers map and their values to apply in a configuration file? It would avoid code duplication between devices and so ease the maintenance.
Quoted 3 lines
There 2 entities have no API between themselves, only indirectly through HW (no shared memory, VF2PF comms, etc..).
New features may have to be added concurrently though, hence splitting repos create room for version mismatch and complicate the ingredients line up.
I am not sure that this argument holds, because one could argue in this case that we could place the FPGA bitstream in the PMD directory too to ensure there is no mismatch.
Quoted 1 line
That was part of the pros and cons described earlier and I can totally see arguments both ways, and that's what I have been trying to share openly in this ticket history.
Thanks, I appreciate that.
Quoted 7 lines
Basically I see nothing fundamentally new here in the discussion, but it is great to receive input and I am happy to hear further input from tech board or others towards a decision.
This started as an open discussion on this DPDK mailing list capturing explicitly both pros and cons of this approach which are arguable, and in case this is not deemed practical eventually then we can still totally come back internally to the drawing board with other options outside of DPDK.Thanks,
Nic
Regards,
Maxime
Re: [dpdk-dev] [20.11, PATCH v2] baseband/fpga_5gnr_fec: add companion PF config App
Hi Maxime, Thomas,
Quoted 93 lines
From: Maxime Coquelin <[email protected]> Hi Nicolas,
Quoted 30 lines · Nicolas
Hi Maxime,
Quoted 28 lines
Hi Nicolas,
Quoted 8 lines · Nicolas Chautru
Adding companion application to configure HW Device from the PF. Then the device can be accessed through BBDEV from VF (or PF).
Signed-off-by Nicolas Chautru <[email protected]>--- doc/guides/bbdevs/fpga_5gnr_fec.rst | 80 +++-- .../baseband/fpga_5gnr_fec/pf_config_app/Makefile | 36 +++ .../fpga_5gnr_fec/pf_config_app/config_app.c | 333+++++++++++++++++++
Quoted 1 line
.../pf_config_app/fpga_5gnr_cfg_app.c | 351+++++++++++++++++++++
Quoted 13 lines
.../pf_config_app/fpga_5gnr_cfg_app.h | 102 ++++++ .../pf_config_app/fpga_5gnr_cfg_parser.c | 187 +++++++++++ .../pf_config_app/fpga_5gnr_config.cfg | 18 ++ 7 files changed, 1087 insertions(+), 20 deletions(-) create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/Makefile create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/config_app.c create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_app.c create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_app.h create mode 100644
drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_parser.cQuoted 9 lines
Quoted 9 lines
Quoted 2 lines
create mode 100644
drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_config.cfgI think having the
pf_config_appin the driver directory is not a good idea, this is not the place for applications.Also, it is not integrated in the DPDK build system, so it cannot benefit from the CI. Having an external dependency that is not packaged in distributions will not help to have it integrated in the build
system.
Quoted 4 lines
Quoted 1 line
Thanks for sharing.
Note that all these points were raised openly explicitly earlier as you know,ie part of both pros and cons.
Quoted 1 line
Still happy to get feedback from others notably Thomas. It appears you had
side conversations with him on this very topic.
Quoted 11 lines
Quoted 9 lines
I see some alternatives:
1. Move it in another directory in the main DPDK repo, but it is not a DPDK example, not a dev tool and not a build tool, so it would need a new directory.
2. Create a BBDEV tools repository on dpdk.org (It would require techboard approval).
3. Host it in a dedicated repository on Intel's github 4. Move it into some Intel FPGA tools repositoryThere are several others options which were indeed considered in case this
option was not viable.
Quoted 1 line
Still DPDK was considered best option so far to keep everything in one
recognized place for BBDEV devices but happy to get further input from others.
Quoted 7 lines
Quoted 7 lines
I think option 3 would be the best to get it packaged into distributions as it has no build dependency with any DPDK library. You could maybe add inih library as a git sub-repository within this repo. Other advantage is you wouldn't depend on DPDK release cycles to get fixes merged.
Quoted 8 lines
Quoted 5 lines
Regarding the tool itself, I understand from the commit message that the tool has a dependency on the BBDEV PMD version, but the tool run on the host while the PMD driver is used in the guest/container. So having it in the driver directory will not really help, as host DPDK (if any) and guest DPDK may come from different parties.
Yes this was captured earlier, purely stored there as a companion application for a given version of the PMD (ie. different subdirectories for
each PMD directory).
Quoted 1 line
They do no run in the same container for deployment and are not built at
the same time indeed, their interface is the HW really and one being needed to be run prior to the other one to be functional.
Quoted 5 lines
Quoted 3 lines
One question I have is whether this is the tool itself that has a dependency on the PMD, or just the config file?
Each PMD directory would have its own version of the companion PF
config application.
Quoted 1 line
Ie. the patch above is only for baseband/fpga_5gnr_fec ie. Intel Vista Creek
Quoted 9 lines
with 5G LDPC user image.
OK. Does it mean the same application and configuration will work for baseband/fpga_5gnr_fec PMD versions v20.11, v21.02, v21.05, etc, ...?
If not, is there a way for the PMD driver to detect whether a wrong configuration was applied? Something like checking the FW version of a NIC is supported by the PMD driver.
As mentioned above there is no API between the 2 config-app companion and BBDEV/DPDK, as they belong logically to 2 different entities (possibly containers) without shared interface except indirectly through HW. There is no strict API which could be broken between the 2, but purely 2 SW ingredients that ideally should be aligned to avoid discrepancy during deployment or validation.
To answer your question I don't foresee a specific case right now causing a strict loss of functional compatibility.
Note that the configure function being used within config-app already exists right now within the DPDK PMD : ie. that function is the one being exposed by PMD and used when running bbdev-test in DPDK for PF. The limitation being that this is only used when running *within DPDK and from PF*, which is not the deployment use case ie. when DPDK is on the VF only, and PF only performs HW configuration without DPDK dependency (apologize for repeating). You can see 'fpga_write_config()' in the current patchset which does match line-for-line with 'fpga_5gnr_fec_configure()' already in the PMD on DPDK main branch. Virtually the same code really with same config input structure 'fpga_5gnr_fec_conf' and same registers being written to.
Difference being that the former implementation is formally integrated in DPDK while the other (config app companion in same directory) has no DPDK dependency. That's it.
There is concern in having these 2 different implementations of the same initialization sequence diverging (a fix to a register setting would need to be applied on both repositories with different versioning) and splitting the HW support into multiple repos, hence my personal preference to keep everything in one place.
As stated from the beginning I personally see pros and cons on both sides, all in all DPDK arguably being cleaner to me for the reasons above.
Still the most important is that it needs to make sense and be valuable to DPDK.
Based on latest feedback from Maxime and Thomas, I believe this is now trending towards being preferred outside of DPDK. If you can confirm by email then that is fine and I will just abandon this ticket so that we can consider alternate options and split this into different repos.
Quoted 4 lines
Quoted 1 line
There will be different companion applications upstreamed for each other
PMD directories (current and future) as they rely on different HW devices with independent MMIO access.
Quoted 1 line
Said otherwise both the config file (features exposed) and implementation
Quoted 9 lines
(registers required for these features) are defined per HW device (+ user image for FPGA) hence per PMD version.
Would it make sense to have a single application, with having the registers map and their values to apply in a configuration file?
It would avoid code duplication between devices and so ease the maintenance.
Some pros and cons as well, but if outside DPDK we may just do this.
Quoted 11 lines
Quoted 2 lines
There 2 entities have no API between themselves, only indirectly through
HW (no shared memory, VF2PF comms, etc..).
Quoted 1 line
New features may have to be added concurrently though, hence splitting
repos create room for version mismatch and complicate the ingredients line up.
I am not sure that this argument holds, because one could argue in this case that we could place the FPGA bitstream in the PMD directory too to ensure there is no mismatch.
There are obviously licenses differences between the DPDK code and FPGA user images preventing to do this.
But still for the sake of the argument logically a different user image may indeed not
work with a PMD version not matching the feature set (as happened prior to PRQ).
From now on any future user image on the FPGA would need to be strictly compatible with current PMD.
The main point being that splitting these SW ingredients means increasing the number of different versions to keep track of and different repos to use to create a working environment. Both options are viable with different pros and cons.
Quoted 15 lines
Quoted 1 line
That was part of the pros and cons described earlier and I can totally see
arguments both ways, and that's what I have been trying to share openly in this ticket history.
Thanks, I appreciate that.
Quoted 2 lines
Basically I see nothing fundamentally new here in the discussion, but it is
great to receive input and I am happy to hear further input from tech board or others towards a decision.
Quoted 1 line
This started as an open discussion on this DPDK mailing list capturing
explicitly both pros and cons of this approach which are arguable, and in case this is not deemed practical eventually then we can still totally come back internally to the drawing board with other options outside of DPDK.
Quoted 1 line
Quoted 6 lines
Quoted 3 lines
Thanks,
NicRegards,
Maxime
Thanks
Nic
Re: [dpdk-dev] [20.11, PATCH v2] baseband/fpga_5gnr_fec: add companion PF config App
Hi Nicolas,
Quoted 105 lines · Nicolas
Hi Maxime, Thomas,
Quoted 93 lines
From: Maxime Coquelin <[email protected]> Hi Nicolas,
Quoted 30 lines · Nicolas
Hi Maxime,
Quoted 28 lines
Hi Nicolas,
Quoted 8 lines · Nicolas Chautru
Adding companion application to configure HW Device from the PF. Then the device can be accessed through BBDEV from VF (or PF).
Signed-off-by Nicolas Chautru <[email protected]>--- doc/guides/bbdevs/fpga_5gnr_fec.rst | 80 +++-- .../baseband/fpga_5gnr_fec/pf_config_app/Makefile | 36 +++ .../fpga_5gnr_fec/pf_config_app/config_app.c | 333+++++++++++++++++++
Quoted 1 line
.../pf_config_app/fpga_5gnr_cfg_app.c | 351+++++++++++++++++++++
Quoted 13 lines
.../pf_config_app/fpga_5gnr_cfg_app.h | 102 ++++++ .../pf_config_app/fpga_5gnr_cfg_parser.c | 187 +++++++++++ .../pf_config_app/fpga_5gnr_config.cfg | 18 ++ 7 files changed, 1087 insertions(+), 20 deletions(-) create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/Makefile create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/config_app.c create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_app.c create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_app.h create mode 100644
drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_parser.cQuoted 9 lines
Quoted 9 lines
Quoted 2 lines
create mode 100644
drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_config.cfgI think having the
pf_config_appin the driver directory is not a good idea, this is not the place for applications.Also, it is not integrated in the DPDK build system, so it cannot benefit from the CI. Having an external dependency that is not packaged in distributions will not help to have it integrated in the build
system.
Quoted 4 lines
Quoted 1 line
Thanks for sharing.
Note that all these points were raised openly explicitly earlier as you know,ie part of both pros and cons.
Quoted 1 line
Still happy to get feedback from others notably Thomas. It appears you had
side conversations with him on this very topic.
Quoted 11 lines
Quoted 9 lines
I see some alternatives:
1. Move it in another directory in the main DPDK repo, but it is not a DPDK example, not a dev tool and not a build tool, so it would need a new directory.
2. Create a BBDEV tools repository on dpdk.org (It would require techboard approval).
3. Host it in a dedicated repository on Intel's github 4. Move it into some Intel FPGA tools repositoryThere are several others options which were indeed considered in case this
option was not viable.
Quoted 1 line
Still DPDK was considered best option so far to keep everything in one
recognized place for BBDEV devices but happy to get further input from others.
Quoted 7 lines
Quoted 7 lines
I think option 3 would be the best to get it packaged into distributions as it has no build dependency with any DPDK library. You could maybe add inih library as a git sub-repository within this repo. Other advantage is you wouldn't depend on DPDK release cycles to get fixes merged.
Quoted 8 lines
Quoted 5 lines
Regarding the tool itself, I understand from the commit message that the tool has a dependency on the BBDEV PMD version, but the tool run on the host while the PMD driver is used in the guest/container. So having it in the driver directory will not really help, as host DPDK (if any) and guest DPDK may come from different parties.
Yes this was captured earlier, purely stored there as a companion application for a given version of the PMD (ie. different subdirectories for
each PMD directory).
Quoted 1 line
They do no run in the same container for deployment and are not built at
the same time indeed, their interface is the HW really and one being needed to be run prior to the other one to be functional.
Quoted 5 lines
Quoted 3 lines
One question I have is whether this is the tool itself that has a dependency on the PMD, or just the config file?
Each PMD directory would have its own version of the companion PF
config application.
Quoted 1 line
Ie. the patch above is only for baseband/fpga_5gnr_fec ie. Intel Vista Creek
Quoted 9 lines
with 5G LDPC user image.
OK. Does it mean the same application and configuration will work for baseband/fpga_5gnr_fec PMD versions v20.11, v21.02, v21.05, etc, ...?
If not, is there a way for the PMD driver to detect whether a wrong configuration was applied? Something like checking the FW version of a NIC is supported by the PMD driver.
Quoted 5 lines
As mentioned above there is no API between the 2 config-app companion and BBDEV/DPDK, as they belong logically to 2 different entities (possibly containers) without shared interface except indirectly through HW. There is no strict API which could be broken between the 2, but purely 2 SW ingredients that ideally should be aligned to avoid discrepancy during deployment or validation.
To answer your question I don't foresee a specific case right now causing a strict loss of functional compatibility.Note that the configure function being used within config-app already exists right now within the DPDK PMD : ie. that function is the one being exposed by PMD and used when running bbdev-test in DPDK for PF. The limitation being that this is only used when running *within DPDK and from PF*, which is not the deployment use case ie. when DPDK is on the VF only, and PF only performs HW configuration without DPDK dependency (apologize for repeating).
OK, got it.
Quoted 3 lines
You can see '
fpga_write_config()' in the current patchset which does match line-for-line with 'fpga_5gnr_fec_configure()' already in the PMD on DPDK main branch. Virtually the same code really with same config input structure 'fpga_5gnr_fec_conf' and same registers being written to. Difference being that the former implementation is formally integrated in DPDK while the other (config app companion in same directory) has no DPDK dependency. That's it.
There is concern in having these 2 different implementations of the same initialization sequence diverging (a fix to a register setting would need to be applied on both repositories with different versioning) and splitting the HW support into multiple repos, hence my personal preference to keep everything in one place.
I understand it would be easier for you.
Maybe an alternative would be to have a simple DPDK application that uses the PMD driver on the PF just for the initialization. Doing that, you would not even have to duplicate the config part between the PMD driver and the application, and you would have a single config app for every cards and bitstreams. It would also mean you get rid off the external dependency on inih library.
Quoted 3 lines
As stated from the beginning I personally see pros and cons on both sides, all in all DPDK arguably being cleaner to me for the reasons above.
Still the most important is that it needs to make sense and be valuable to DPDK.
Based on latest feedback from Maxime and Thomas, I believe this is now trending towards being preferred outside of DPDK. If you can confirm by email then that is fine and I will just abandon this ticket so that we can consider alternate options and split this into different repos.
In its current form, this is a nack for me. I would prefer either having it outside of DPDK, or having a DPDK application calling the PMD driver init.
Quoted 28 lines
Quoted 4 lines
Quoted 1 line
There will be different companion applications upstreamed for each other
PMD directories (current and future) as they rely on different HW devices with independent MMIO access.
Quoted 1 line
Said otherwise both the config file (features exposed) and implementation
Quoted 9 lines
(registers required for these features) are defined per HW device (+ user image for FPGA) hence per PMD version.
Would it make sense to have a single application, with having the registers map and their values to apply in a configuration file?
It would avoid code duplication between devices and so ease the maintenance.Some pros and cons as well, but if outside DPDK we may just do this.
Quoted 11 lines
Quoted 2 lines
There 2 entities have no API between themselves, only indirectly through
HW (no shared memory, VF2PF comms, etc..).
Quoted 1 line
New features may have to be added concurrently though, hence splitting
repos create room for version mismatch and complicate the ingredients line up.
I am not sure that this argument holds, because one could argue in this case that we could place the FPGA bitstream in the PMD directory too to ensure there is no mismatch.
Quoted 5 lines
There are obviously licenses differences between the DPDK code and FPGA user images preventing to do this.
But still for the sake of the argument logically a different user image may indeed not
work with a PMD version not matching the feature set (as happened prior to PRQ).
From now on any future user image on the FPGA would need to be strictly compatible with current PMD.
The main point being that splitting these SW ingredients means increasing the number of different versions to keep track of and different repos to use to create a working environment.
Ok for prototyping, but for production host and guest SW are likely to come from different streams.
Thanks for the detailed explanations,
Maxime
Quoted 27 lines
Both options are viable with different pros and cons.
Quoted 15 lines
Quoted 1 line
That was part of the pros and cons described earlier and I can totally see
arguments both ways, and that's what I have been trying to share openly in this ticket history.
Thanks, I appreciate that.
Quoted 2 lines
Basically I see nothing fundamentally new here in the discussion, but it is
great to receive input and I am happy to hear further input from tech board or others towards a decision.
Quoted 1 line
This started as an open discussion on this DPDK mailing list capturing
explicitly both pros and cons of this approach which are arguable, and in case this is not deemed practical eventually then we can still totally come back internally to the drawing board with other options outside of DPDK.
Quoted 1 line
Quoted 6 lines
Quoted 3 lines
Thanks,
NicRegards,
MaximeThanks
Nic
Re: [dpdk-dev] [20.11, PATCH v2] baseband/fpga_5gnr_fec: add companion PF config App
Hello Nicolas, Maxime,
Quoted 16 lines · Maxime Coquelin
Quoted 16 lines
Quoted 15 lines
Quoted 4 lines
Quoted 2 lines
One question I have is whether this is the tool itself that has a dependency on the PMD, or just the config file?
Each PMD directory would have its own version of the companion PF
config application.
Quoted 1 line
Ie. the patch above is only for baseband/fpga_5gnr_fec ie. Intel Vista Creek
with 5G LDPC user image.
OK. Does it mean the same application and configuration will work for baseband/fpga_5gnr_fec PMD versions v20.11, v21.02, v21.05, etc, ...?
If not, is there a way for the PMD driver to detect whether a wrong configuration was applied? Something like checking the FW version of a NIC is supported by the PMD driver.
Quoted 5 lines
Quoted 5 lines
As mentioned above there is no API between the 2 config-app companion and BBDEV/DPDK, as they belong logically to 2 different entities (possibly containers) without shared interface except indirectly through HW. There is no strict API which could be broken between the 2, but purely 2 SW ingredients that ideally should be aligned to avoid discrepancy during deployment or validation.
To answer your question I don't foresee a specific case right now causing a strict loss of functional compatibility.Note that the configure function being used within config-app already exists right now within the DPDK PMD : ie. that function is the one being exposed by PMD and used when running bbdev-test in DPDK for PF. The limitation being that this is only used when running *within DPDK and from PF*, which is not the deployment use case ie. when DPDK is on the VF only, and PF only performs HW configuration without DPDK dependency (apologize for repeating).
I am naively thinking that the bitstream + this configuration are a whole for a given usecase and should be loaded as a single step. A bit like how vendors provide ready-to-use firmwares on their nics.
Leaving the users with the possibility to load whatever config they write is scary.
When hitting an init issue, application crash etc... in a customer env, how will we know the configuration is actually what it should be?
Quoted 15 lines
OK, got it.
Quoted 3 lines
You can see '
fpga_write_config()' in the current patchset which does match line-for-line with 'fpga_5gnr_fec_configure()' already in the PMD on DPDK main branch. Virtually the same code really with same config input structure 'fpga_5gnr_fec_conf' and same registers being written to. Difference being that the former implementation is formally integrated in DPDK while the other (config app companion in same directory) has no DPDK dependency. That's it.
There is concern in having these 2 different implementations of the same initialization sequence diverging (a fix to a register setting would need to be applied on both repositories with different versioning) and splitting the HW support into multiple repos, hence my personal preference to keep everything in one place.I understand it would be easier for you.
Maybe an alternative would be to have a simple DPDK application that uses the PMD driver on the PF just for the initialization. Doing that, you would not even have to duplicate the config part between the PMD driver and the application, and you would have a single config app for every cards and bitstreams. It would also mean you get rid off the external dependency on inih library.
Quoted 7 lines
Quoted 3 lines
As stated from the beginning I personally see pros and cons on both sides, all in all DPDK arguably being cleaner to me for the reasons above.
Still the most important is that it needs to make sense and be valuable to DPDK.
Based on latest feedback from Maxime and Thomas, I believe this is now trending towards being preferred outside of DPDK. If you can confirm by email then that is fine and I will just abandon this ticket so that we can consider alternate options and split this into different repos.In its current form, this is a nack for me. I would prefer either having it outside of DPDK, or having a DPDK application calling the PMD driver init.
This is a nack from me too.
About having a DPDK application calling the PMD driver init. This requires to have a DPDK application from version X touching the hw in the host, while a guest could be running DPDK version Y. This will result in ping/pong when the support on the host and guest sides is done by two different teams.
Thanks.
Signature · 1 line
David Marchand