Skip to contentSkip to conversation

BBDEV FPGA PF Config app

PATCH New Nicolas Chautru 3 participants 8 messages Jul 16, 2020 – Aug 5, 2020

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 BBDEV FPGA PF Config app

v2: Couple of typos in documentation and removed dependency on igb_uio and related PCIe VF creation feature option.

Addition of an stand alone application in PMD driver to be able to configure the HW from PF. Then the VF can be used from container/VM running BBDEV/DPDK.

The history of the previous discussions can be tracked on this previous patch https://patches.dpdk.org/patch/67379/

PMD documentation is updated to provides steps to build with related dependency

Nicolas Chautru (1):
baseband/fpga_5gnr_fec: add companion PF config App

 doc/guides/bbdevs/fpga_5gnr_fec.rst                |  80 +++--
 .../baseband/fpga_5gnr_fec/pf_config_app/Makefile  |  36 +++
 .../fpga_5gnr_fec/pf_config_app/config_app.c       | 333 +++++++++++++++++++
 .../pf_config_app/fpga_5gnr_cfg_app.c              | 351 +++++++++++++++++++++
 .../pf_config_app/fpga_5gnr_cfg_app.h              | 102 ++++++
 .../pf_config_app/fpga_5gnr_cfg_parser.c           | 187 +++++++++++
 .../pf_config_app/fpga_5gnr_config.cfg             |  18 ++
 7 files changed, 1087 insertions(+), 20 deletions(-)
 create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/Makefile
 create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/config_app.c
 create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_app.c
 create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_app.h
 create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_parser.c
 create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_config.cfg
Signature · 1 line
1.8.3.1
#1 lore raw

PATCH v2 1/1 baseband/fpga_5gnr_fec: add companion PF config App

Adding companion application to configure HW Device from the PF. Then the device can be accessed through BBDEV from VF (or PF).

---
 doc/guides/bbdevs/fpga_5gnr_fec.rst                |  80 +++--
 .../baseband/fpga_5gnr_fec/pf_config_app/Makefile  |  36 +++
 .../fpga_5gnr_fec/pf_config_app/config_app.c       | 333 +++++++++++++++++++
 .../pf_config_app/fpga_5gnr_cfg_app.c              | 351 +++++++++++++++++++++
 .../pf_config_app/fpga_5gnr_cfg_app.h              | 102 ++++++
 .../pf_config_app/fpga_5gnr_cfg_parser.c           | 187 +++++++++++
 .../pf_config_app/fpga_5gnr_config.cfg             |  18 ++
 7 files changed, 1087 insertions(+), 20 deletions(-)
 create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/Makefile
 create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/config_app.c
 create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_app.c
 create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_app.h
 create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_parser.c
 create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_config.cfg
doc/guides/bbdevs/fpga_5gnr_fec.rst +60 -20
index 19bba36..9674e4c 100644
@@ -218,33 +218,73 @@ parameters defined in ``fpga_5gnr_fec_conf`` structure:
   time_out = flr_time_out x 16.384us. For instance, if you want to set 10ms for
   the FLR time out then set this setting to 0x262=610.
 
+A companion application pf_config_app is provided as a standalone application
+described in next section.
 
-An example configuration code calling the function ``fpga_5gnr_fec_configure()`` is shown
-below:
+PF Config App
+-------------
 
-.. code-block:: c
+The PF Configuration Application ``pf_config_app`` provides a means to
+configure the baseband device at the host-level. The program sets the various
+parameters through memory-mapped IO read/writes. Then the BBDEV driver can be
+used to run the workload.
+
+The parameters are parsed from a given configuration file (with .cfg extensions)
+that is specific to particular BBDEV device, although they follow same format.
+
+External Dependencies
+~~~~~~~~~~~~~~~~~~~~~
+
+The third party .INI parser is a pre-requisite for building the application.
+It can be downloaded from [github]:
+
+.. code-block:: console
 
-  struct fpga_5gnr_fec_conf conf;
-  unsigned int i;
+    git clone https://github.com/benhoyt/inih
 
-  memset(&conf, 0, sizeof(struct fpga_5gnr_fec_conf));
-  conf.pf_mode_en = 1;
+Use the version release 44 tracked by tag 'r44':
+
+.. code-block:: console
+
+    git checkout r44
+
+The application features a makefile in the `extra/` directory which generates
+the library, `libinih.a`. To compile the inih library, run make as:
+
+.. code-block:: console
+
+    make -f Makefile.static
+
+Building PF Config App
+~~~~~~~~~~~~~~~~~~~~~~
+
+Before building the application, set the following environment variables with
+the location where the INI library is located:
+
+.. code-block:: console
+
+    export INIH_PATH=<path-to-inih-lib>
+
+If not set, makefile will look into current folder.
+
+Next, build the program by typing ``make`` from the pf_Config_app subdirectory
+to produce the binary ``pf_config_app_fpga_5gnr``.
+
+Usage
+~~~~~
+
+The application executes as the following:
+
+.. code-block:: console
 
-  for (i = 0; i < FPGA_5GNR_FEC_NUM_VFS; ++i) {
-      conf.vf_ul_queues_number[i] = 4;
-      conf.vf_dl_queues_number[i] = 4;
-  }
-  conf.ul_bandwidth = 12;
-  conf.dl_bandwidth = 5;
-  conf.dl_load_balance = 64;
-  conf.ul_load_balance = 64;
+    ./pf_config_app_fpga_5gnr [-h] [-a] [-c CFG_FILE] [-p PCI_ID]
 
-  /* setup FPGA PF */
-  ret = fpga_5gnr_fec_configure(info->dev_name, &conf);
-  TEST_ASSERT_SUCCESS(ret,
-      "Failed to configure 4G FPGA PF for bbdev %s",
-      info->dev_name);
+* ``-c CFG_FILE``: Specifies configuration file to use
+* ``-p PCI_ID``: Specifies PCI ID of device to configure
+* ``-a``: Configures all PCI devices
+* ``-h``: Prints help
 
+Default configure file is provided: ``fpga_5gnr_config.cfg``.
 
 Test Application
 ----------------
drivers/baseband/fpga_5gnr_fec/pf_config_app/Makefile new file +36 -0
new file mode 100644
index 0000000..4f7517d
@@ -0,0 +1,36 @@
+
+CC=gcc
+CFLAGS=-O0 -g -Wall
+ODIR=build
+DEPS=
+
+ifeq ($(INIH_PATH),)
+INCLUDE=-I.
+LDFLAGS=-L.
+else
+INCLUDE=-I. -I$(INIH_PATH)
+LDFLAGS=-L. -L$(INIH_PATH)
+endif
+
+LDLIBS=-linih
+
+SRC = config_app.c fpga_5gnr_cfg_app.c fpga_5gnr_cfg_parser.c
+OBJ = $(patsubst %.c,$(ODIR)/%.o,$(SRC))
+
+.PHONY: clean
+
+all: pf_config_app_fpga_5gnr
+
+$(ODIR):
+	mkdir -p $(ODIR)
+
+$(OBJ): $(ODIR)/%.o: ./%.c | $(DEPS) $(ODIR)
+	@mkdir -p $(@D)
+	$(CC) -c -o $@ $< $(CFLAGS) $(INCLUDE)
+
+pf_config_app_fpga_5gnr: $(OBJ)
+	$(CC) -o $@ $^ $(CFLAGS) $(LDFLAGS) $(LDLIBS)
+
+clean:
+	rm -rf $(ODIR)
+	rm -rf pf_config_app_fpga_5gnr
drivers/baseband/fpga_5gnr_fec/pf_config_app/config_app.c new file +333 -0

333 lines not on this page · raw message

drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_app.c new file +351 -0

351 lines not on this page · raw message

drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_app.h new file +102 -0

102 lines not on this page · raw message

drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_parser.c new file +187 -0

187 lines not on this page · raw message

drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_config.cfg new file +18 -0
new file mode 100644
index 0000000..0854672
@@ -0,0 +1,18 @@
+; SPDX-License-Identifier: BSD-3-Clause
+; Copyright(c) 2020 Intel Corporation
+
+[MODE]
+pf_mode_en = 1
+
+[UL]
+bandwidth = 3
+load_balance = 128
+vfqmap = 4,4,4,4,4,4,4,4
+
+[DL]
+bandwidth = 3
+load_balance = 128
+vfqmap = 4,4,4,4,4,4,4,4
+
+[FLR]
+flr_time_out = 610
Signature · 1 line
1.8.3.1
#2 lore raw

Re: [dpdk-dev] [20.11, PATCH v2] baseband/fpga_5gnr_fec: add companion PF config App

Hi Nicolas,

Quoted 4 lines · Nicolas Chautru

Adding companion application to configure HW Device from the PF. Then the device can be accessed through BBDEV from VF (or PF).

Quoted 15 lines
---
 doc/guides/bbdevs/fpga_5gnr_fec.rst                |  80 +++--
 .../baseband/fpga_5gnr_fec/pf_config_app/Makefile  |  36 +++
 .../fpga_5gnr_fec/pf_config_app/config_app.c       | 333 +++++++++++++++++++
 .../pf_config_app/fpga_5gnr_cfg_app.c              | 351 +++++++++++++++++++++
 .../pf_config_app/fpga_5gnr_cfg_app.h              | 102 ++++++
 .../pf_config_app/fpga_5gnr_cfg_parser.c           | 187 +++++++++++
 .../pf_config_app/fpga_5gnr_config.cfg             |  18 ++
 7 files changed, 1087 insertions(+), 20 deletions(-)
 create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/Makefile
 create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/config_app.c
 create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_app.c
 create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_app.h
 create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_parser.c
 create mode 100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_config.cfg

I think having the pf_config_app in the driver directory is not a good idea, this is not the place for applications.

Also, it is not integrated in the DPDK build system, so it cannot benefit from the CI. Having an external dependency that is not packaged in distributions will not help to have it integrated in the build system.

I see some alternatives:
1. Move it in another directory in the main DPDK repo, but it is not a DPDK example, not a dev tool and not a build tool, so it would need a new directory.
2. Create a BBDEV tools repository on dpdk.org (It would require techboard approval).
3. Host it in a dedicated repository on Intel's github
4. Move it into some Intel FPGA tools repository

I think option 3 would be the best to get it packaged into distributions as it has no build dependency with any DPDK library. You could maybe add inih library as a git sub-repository within this repo. Other advantage is you wouldn't depend on DPDK release cycles to get fixes merged.

Regarding the tool itself, I understand from the commit message that the tool has a dependency on the BBDEV PMD version, but the tool run on the host while the PMD driver is used in the guest/container. So having it in the driver directory will not really help, as host DPDK (if any) and guest DPDK may come from different parties.

One question I have is whether this is the tool itself that has a dependency on the PMD, or just the config file?

Regards,
Maxime

#3 lore raw

Re: [dpdk-dev] [20.11, PATCH v2] baseband/fpga_5gnr_fec: add companion PF config App

Hi Maxime,

Quoted 31 lines

Hi Nicolas,

Quoted 8 lines · Nicolas Chautru

Adding companion application to configure HW Device from the PF. Then the device can be accessed through BBDEV from VF (or PF).

---
 doc/guides/bbdevs/fpga_5gnr_fec.rst                |  80 +++--
 .../baseband/fpga_5gnr_fec/pf_config_app/Makefile  |  36 +++
 .../fpga_5gnr_fec/pf_config_app/config_app.c       | 333

+++++++++++++++++++

Quoted 1 line
 .../pf_config_app/fpga_5gnr_cfg_app.c              | 351

+++++++++++++++++++++

Quoted 14 lines
 .../pf_config_app/fpga_5gnr_cfg_app.h              | 102 ++++++
 .../pf_config_app/fpga_5gnr_cfg_parser.c           | 187 +++++++++++
 .../pf_config_app/fpga_5gnr_config.cfg             |  18 ++
 7 files changed, 1087 insertions(+), 20 deletions(-)  create mode
100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/Makefile
 create mode 100644
drivers/baseband/fpga_5gnr_fec/pf_config_app/config_app.c
 create mode 100644
drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_app.c
 create mode 100644
drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_app.h
 create mode 100644
drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_parser.c
 create mode 100644
Quoted 1 line

drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_config.cfg

Quoted 7 lines

I think having the pf_config_app in the driver directory is not a good idea, this is not the place for applications.

Also, it is not integrated in the DPDK build system, so it cannot benefit from the CI. Having an external dependency that is not packaged in distributions will not help to have it integrated in the build system.

Thanks for sharing.
Note that all these points were raised openly explicitly earlier as you know, ie part of both pros and cons. Still happy to get feedback from others notably Thomas. It appears you had side conversations with him on this very topic.

Quoted 8 lines

I see some alternatives:
1. Move it in another directory in the main DPDK repo, but it is not a DPDK example, not a dev tool and not a build tool, so it would need a new directory.
2. Create a BBDEV tools repository on dpdk.org (It would require techboard approval).
3. Host it in a dedicated repository on Intel's github 4. Move it into some Intel FPGA tools repository

There are several others options which were indeed considered in case this option was not viable.
Still DPDK was considered best option so far to keep everything in one recognized place for BBDEV devices but happy to get further input from others.

Quoted 11 lines

I think option 3 would be the best to get it packaged into distributions as it has no build dependency with any DPDK library.
You could maybe add inih library as a git sub-repository within this repo. Other advantage is you wouldn't depend on DPDK release cycles to get fixes merged.

Regarding the tool itself, I understand from the commit message that the tool has a dependency on the BBDEV PMD version, but the tool run on the host while the PMD driver is used in the guest/container. So having it in the driver directory will not really help, as host DPDK (if any) and guest DPDK may come from different parties.

Yes this was captured earlier, purely stored there as a companion application for a given
version of the PMD (ie. different subdirectories for each PMD directory).
They do no run in the same container for deployment and are not built at the same time indeed, their interface is the HW really and one being needed to be run prior to the other one to be functional.

Quoted 2 lines

One question I have is whether this is the tool itself that has a dependency on the PMD, or just the config file?

Each PMD directory would have its own version of the companion PF config application.
Ie. the patch above is only for baseband/fpga_5gnr_fec ie. Intel Vista Creek with 5G LDPC user image.
There will be different companion applications upstreamed for each other PMD directories (current and future) as they rely on different HW devices with independent MMIO access. Said otherwise both the config file (features exposed) and implementation (registers required for these features) are defined per HW device (+ user image for FPGA) hence per PMD version.

There 2 entities have no API between themselves, only indirectly through HW (no shared memory, VF2PF comms, etc..).
New features may have to be added concurrently though, hence splitting repos create room for version mismatch and complicate the ingredients line up. That was part of the pros and cons described earlier and I can totally see arguments both ways, and that's what I have been trying to share openly in this ticket history.

Basically I see nothing fundamentally new here in the discussion, but it is great to receive input and I am happy to hear further input from tech board or others towards a decision.
This started as an open discussion on this DPDK mailing list capturing explicitly both pros and cons of this approach which are arguable, and in case this is not deemed practical eventually then we can still totally come back internally to the drawing board with other options outside of DPDK.

Thanks,
Nic

#4 lore raw

Re: [dpdk-dev] [20.11, PATCH v2] baseband/fpga_5gnr_fec: add companion PF config App

Hi Nicolas,

Quoted 73 lines · Nicolas

Hi Maxime,

Quoted 31 lines

Hi Nicolas,

Quoted 8 lines · Nicolas Chautru

Adding companion application to configure HW Device from the PF. Then the device can be accessed through BBDEV from VF (or PF).

---
 doc/guides/bbdevs/fpga_5gnr_fec.rst                |  80 +++--
 .../baseband/fpga_5gnr_fec/pf_config_app/Makefile  |  36 +++
 .../fpga_5gnr_fec/pf_config_app/config_app.c       | 333

+++++++++++++++++++

Quoted 1 line
 .../pf_config_app/fpga_5gnr_cfg_app.c              | 351

+++++++++++++++++++++

Quoted 14 lines
 .../pf_config_app/fpga_5gnr_cfg_app.h              | 102 ++++++
 .../pf_config_app/fpga_5gnr_cfg_parser.c           | 187 +++++++++++
 .../pf_config_app/fpga_5gnr_config.cfg             |  18 ++
 7 files changed, 1087 insertions(+), 20 deletions(-)  create mode
100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/Makefile
 create mode 100644
drivers/baseband/fpga_5gnr_fec/pf_config_app/config_app.c
 create mode 100644
drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_app.c
 create mode 100644
drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_app.h
 create mode 100644
drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_parser.c
 create mode 100644
Quoted 1 line

drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_config.cfg

Quoted 7 lines

I think having the pf_config_app in the driver directory is not a good idea, this is not the place for applications.

Also, it is not integrated in the DPDK build system, so it cannot benefit from the CI. Having an external dependency that is not packaged in distributions will not help to have it integrated in the build system.

Thanks for sharing.
Note that all these points were raised openly explicitly earlier as you know, ie part of both pros and cons. Still happy to get feedback from others notably Thomas. It appears you had side conversations with him on this very topic.

Quoted 8 lines

I see some alternatives:
1. Move it in another directory in the main DPDK repo, but it is not a DPDK example, not a dev tool and not a build tool, so it would need a new directory.
2. Create a BBDEV tools repository on dpdk.org (It would require techboard approval).
3. Host it in a dedicated repository on Intel's github 4. Move it into some Intel FPGA tools repository

There are several others options which were indeed considered in case this option was not viable.
Still DPDK was considered best option so far to keep everything in one recognized place for BBDEV devices but happy to get further input from others.

Quoted 11 lines

I think option 3 would be the best to get it packaged into distributions as it has no build dependency with any DPDK library.
You could maybe add inih library as a git sub-repository within this repo. Other advantage is you wouldn't depend on DPDK release cycles to get fixes merged.

Regarding the tool itself, I understand from the commit message that the tool has a dependency on the BBDEV PMD version, but the tool run on the host while the PMD driver is used in the guest/container. So having it in the driver directory will not really help, as host DPDK (if any) and guest DPDK may come from different parties.

Yes this was captured earlier, purely stored there as a companion application for a given
version of the PMD (ie. different subdirectories for each PMD directory).
They do no run in the same container for deployment and are not built at the same time indeed, their interface is the HW really and one being needed to be run prior to the other one to be functional.

Quoted 5 lines
Quoted 2 lines

One question I have is whether this is the tool itself that has a dependency on the PMD, or just the config file?

Each PMD directory would have its own version of the companion PF config application. Ie. the patch above is only for baseband/fpga_5gnr_fec ie. Intel Vista Creek with 5G LDPC user image.

OK. Does it mean the same application and configuration will work for baseband/fpga_5gnr_fec PMD versions v20.11, v21.02, v21.05, etc, ...?

If not, is there a way for the PMD driver to detect whether a wrong configuration was applied? Something like checking the FW version of a NIC is supported by the PMD driver.

Quoted 2 lines

There will be different companion applications upstreamed for each other PMD directories (current and future) as they rely on different HW devices with independent MMIO access. Said otherwise both the config file (features exposed) and implementation (registers required for these features) are defined per HW device (+ user image for FPGA) hence per PMD version.

Would it make sense to have a single application, with having the registers map and their values to apply in a configuration file? It would avoid code duplication between devices and so ease the maintenance.

Quoted 3 lines

There 2 entities have no API between themselves, only indirectly through HW (no shared memory, VF2PF comms, etc..).
New features may have to be added concurrently though, hence splitting repos create room for version mismatch and complicate the ingredients line up.

I am not sure that this argument holds, because one could argue in this case that we could place the FPGA bitstream in the PMD directory too to ensure there is no mismatch.

Quoted 1 line

That was part of the pros and cons described earlier and I can totally see arguments both ways, and that's what I have been trying to share openly in this ticket history.

Thanks, I appreciate that.

Quoted 7 lines

Basically I see nothing fundamentally new here in the discussion, but it is great to receive input and I am happy to hear further input from tech board or others towards a decision.
This started as an open discussion on this DPDK mailing list capturing explicitly both pros and cons of this approach which are arguable, and in case this is not deemed practical eventually then we can still totally come back internally to the drawing board with other options outside of DPDK.

Thanks,
Nic

Regards,
Maxime

#5 lore raw

Re: [dpdk-dev] [20.11, PATCH v2] baseband/fpga_5gnr_fec: add companion PF config App

Hi Maxime, Thomas,

Quoted 93 lines

From: Maxime Coquelin <[email protected]> Hi Nicolas,

Quoted 30 lines · Nicolas

Hi Maxime,

Quoted 28 lines

Hi Nicolas,

Quoted 8 lines · Nicolas Chautru

Adding companion application to configure HW Device from the PF. Then the device can be accessed through BBDEV from VF (or PF).

---
 doc/guides/bbdevs/fpga_5gnr_fec.rst                |  80 +++--
 .../baseband/fpga_5gnr_fec/pf_config_app/Makefile  |  36 +++
 .../fpga_5gnr_fec/pf_config_app/config_app.c       | 333

+++++++++++++++++++

Quoted 1 line
 .../pf_config_app/fpga_5gnr_cfg_app.c              | 351

+++++++++++++++++++++

Quoted 13 lines
 .../pf_config_app/fpga_5gnr_cfg_app.h              | 102 ++++++
 .../pf_config_app/fpga_5gnr_cfg_parser.c           | 187 +++++++++++
 .../pf_config_app/fpga_5gnr_config.cfg             |  18 ++
 7 files changed, 1087 insertions(+), 20 deletions(-)  create mode
100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/Makefile
 create mode 100644
drivers/baseband/fpga_5gnr_fec/pf_config_app/config_app.c
 create mode 100644
drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_app.c
 create mode 100644
drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_app.h
 create mode 100644

drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_parser.c

Quoted 9 lines
Quoted 9 lines
Quoted 2 lines
 create mode 100644

drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_config.cfg

I think having the pf_config_app in the driver directory is not a good idea, this is not the place for applications.

Also, it is not integrated in the DPDK build system, so it cannot benefit from the CI. Having an external dependency that is not packaged in distributions will not help to have it integrated in the build

system.

Quoted 4 lines
Quoted 1 line

Thanks for sharing.
Note that all these points were raised openly explicitly earlier as you know,

ie part of both pros and cons.

Quoted 1 line

Still happy to get feedback from others notably Thomas. It appears you had

side conversations with him on this very topic.

Quoted 11 lines
Quoted 9 lines

I see some alternatives:
1. Move it in another directory in the main DPDK repo, but it is not a DPDK example, not a dev tool and not a build tool, so it would need a new directory.
2. Create a BBDEV tools repository on dpdk.org (It would require techboard approval).
3. Host it in a dedicated repository on Intel's github 4. Move it into some Intel FPGA tools repository

There are several others options which were indeed considered in case this

option was not viable.

Quoted 1 line

Still DPDK was considered best option so far to keep everything in one

recognized place for BBDEV devices but happy to get further input from others.

Quoted 7 lines
Quoted 7 lines

I think option 3 would be the best to get it packaged into distributions as it has no build dependency with any DPDK library. You could maybe add inih library as a git sub-repository within this repo. Other advantage is you wouldn't depend on DPDK release cycles to get fixes merged.

Quoted 8 lines
Quoted 5 lines

Regarding the tool itself, I understand from the commit message that the tool has a dependency on the BBDEV PMD version, but the tool run on the host while the PMD driver is used in the guest/container. So having it in the driver directory will not really help, as host DPDK (if any) and guest DPDK may come from different parties.

Yes this was captured earlier, purely stored there as a companion application for a given version of the PMD (ie. different subdirectories for

each PMD directory).

Quoted 1 line

They do no run in the same container for deployment and are not built at

the same time indeed, their interface is the HW really and one being needed to be run prior to the other one to be functional.

Quoted 5 lines
Quoted 3 lines

One question I have is whether this is the tool itself that has a dependency on the PMD, or just the config file?

Each PMD directory would have its own version of the companion PF

config application.

Quoted 1 line

Ie. the patch above is only for baseband/fpga_5gnr_fec ie. Intel Vista Creek

Quoted 9 lines

with 5G LDPC user image.

OK. Does it mean the same application and configuration will work for baseband/fpga_5gnr_fec PMD versions v20.11, v21.02, v21.05, etc, ...?

If not, is there a way for the PMD driver to detect whether a wrong configuration was applied? Something like checking the FW version of a NIC is supported by the PMD driver.

As mentioned above there is no API between the 2 config-app companion and BBDEV/DPDK, as they belong logically to 2 different entities (possibly containers) without shared interface except indirectly through HW. There is no strict API which could be broken between the 2, but purely 2 SW ingredients that ideally should be aligned to avoid discrepancy during deployment or validation.
To answer your question I don't foresee a specific case right now causing a strict loss of functional compatibility.

Note that the configure function being used within config-app already exists right now within the DPDK PMD : ie. that function is the one being exposed by PMD and used when running bbdev-test in DPDK for PF. The limitation being that this is only used when running *within DPDK and from PF*, which is not the deployment use case ie. when DPDK is on the VF only, and PF only performs HW configuration without DPDK dependency (apologize for repeating). You can see 'fpga_write_config()' in the current patchset which does match line-for-line with 'fpga_5gnr_fec_configure()' already in the PMD on DPDK main branch. Virtually the same code really with same config input structure 'fpga_5gnr_fec_conf' and same registers being written to.
Difference being that the former implementation is formally integrated in DPDK while the other (config app companion in same directory) has no DPDK dependency. That's it.
There is concern in having these 2 different implementations of the same initialization sequence diverging (a fix to a register setting would need to be applied on both repositories with different versioning) and splitting the HW support into multiple repos, hence my personal preference to keep everything in one place.

As stated from the beginning I personally see pros and cons on both sides, all in all DPDK arguably being cleaner to me for the reasons above.
Still the most important is that it needs to make sense and be valuable to DPDK.
Based on latest feedback from Maxime and Thomas, I believe this is now trending towards being preferred outside of DPDK. If you can confirm by email then that is fine and I will just abandon this ticket so that we can consider alternate options and split this into different repos.

Quoted 4 lines
Quoted 1 line

There will be different companion applications upstreamed for each other

PMD directories (current and future) as they rely on different HW devices with independent MMIO access.

Quoted 1 line

Said otherwise both the config file (features exposed) and implementation

Quoted 9 lines

(registers required for these features) are defined per HW device (+ user image for FPGA) hence per PMD version.

Would it make sense to have a single application, with having the registers map and their values to apply in a configuration file?
It would avoid code duplication between devices and so ease the maintenance.

Some pros and cons as well, but if outside DPDK we may just do this.

Quoted 11 lines
Quoted 2 lines

There 2 entities have no API between themselves, only indirectly through

HW (no shared memory, VF2PF comms, etc..).

Quoted 1 line

New features may have to be added concurrently though, hence splitting

repos create room for version mismatch and complicate the ingredients line up.

I am not sure that this argument holds, because one could argue in this case that we could place the FPGA bitstream in the PMD directory too to ensure there is no mismatch.

There are obviously licenses differences between the DPDK code and FPGA user images preventing to do this.
But still for the sake of the argument logically a different user image may indeed not
work with a PMD version not matching the feature set (as happened prior to PRQ).
From now on any future user image on the FPGA would need to be strictly compatible with current PMD.
The main point being that splitting these SW ingredients means increasing the number of different versions to keep track of and different repos to use to create a working environment. Both options are viable with different pros and cons.

Quoted 15 lines
Quoted 1 line

That was part of the pros and cons described earlier and I can totally see

arguments both ways, and that's what I have been trying to share openly in this ticket history.

Thanks, I appreciate that.

Quoted 2 lines

Basically I see nothing fundamentally new here in the discussion, but it is

great to receive input and I am happy to hear further input from tech board or others towards a decision.

Quoted 1 line

This started as an open discussion on this DPDK mailing list capturing

explicitly both pros and cons of this approach which are arguable, and in case this is not deemed practical eventually then we can still totally come back internally to the drawing board with other options outside of DPDK.

Quoted 1 line
Quoted 6 lines
Quoted 3 lines

Thanks,
Nic

Regards,
Maxime

Thanks
Nic

#6 lore raw

Re: [dpdk-dev] [20.11, PATCH v2] baseband/fpga_5gnr_fec: add companion PF config App

Hi Nicolas,

Quoted 105 lines · Nicolas

Hi Maxime, Thomas,

Quoted 93 lines

From: Maxime Coquelin <[email protected]> Hi Nicolas,

Quoted 30 lines · Nicolas

Hi Maxime,

Quoted 28 lines

Hi Nicolas,

Quoted 8 lines · Nicolas Chautru

Adding companion application to configure HW Device from the PF. Then the device can be accessed through BBDEV from VF (or PF).

---
 doc/guides/bbdevs/fpga_5gnr_fec.rst                |  80 +++--
 .../baseband/fpga_5gnr_fec/pf_config_app/Makefile  |  36 +++
 .../fpga_5gnr_fec/pf_config_app/config_app.c       | 333

+++++++++++++++++++

Quoted 1 line
 .../pf_config_app/fpga_5gnr_cfg_app.c              | 351

+++++++++++++++++++++

Quoted 13 lines
 .../pf_config_app/fpga_5gnr_cfg_app.h              | 102 ++++++
 .../pf_config_app/fpga_5gnr_cfg_parser.c           | 187 +++++++++++
 .../pf_config_app/fpga_5gnr_config.cfg             |  18 ++
 7 files changed, 1087 insertions(+), 20 deletions(-)  create mode
100644 drivers/baseband/fpga_5gnr_fec/pf_config_app/Makefile
 create mode 100644
drivers/baseband/fpga_5gnr_fec/pf_config_app/config_app.c
 create mode 100644
drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_app.c
 create mode 100644
drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_app.h
 create mode 100644

drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_cfg_parser.c

Quoted 9 lines
Quoted 9 lines
Quoted 2 lines
 create mode 100644

drivers/baseband/fpga_5gnr_fec/pf_config_app/fpga_5gnr_config.cfg

I think having the pf_config_app in the driver directory is not a good idea, this is not the place for applications.

Also, it is not integrated in the DPDK build system, so it cannot benefit from the CI. Having an external dependency that is not packaged in distributions will not help to have it integrated in the build

system.

Quoted 4 lines
Quoted 1 line

Thanks for sharing.
Note that all these points were raised openly explicitly earlier as you know,

ie part of both pros and cons.

Quoted 1 line

Still happy to get feedback from others notably Thomas. It appears you had

side conversations with him on this very topic.

Quoted 11 lines
Quoted 9 lines

I see some alternatives:
1. Move it in another directory in the main DPDK repo, but it is not a DPDK example, not a dev tool and not a build tool, so it would need a new directory.
2. Create a BBDEV tools repository on dpdk.org (It would require techboard approval).
3. Host it in a dedicated repository on Intel's github 4. Move it into some Intel FPGA tools repository

There are several others options which were indeed considered in case this

option was not viable.

Quoted 1 line

Still DPDK was considered best option so far to keep everything in one

recognized place for BBDEV devices but happy to get further input from others.

Quoted 7 lines
Quoted 7 lines

I think option 3 would be the best to get it packaged into distributions as it has no build dependency with any DPDK library. You could maybe add inih library as a git sub-repository within this repo. Other advantage is you wouldn't depend on DPDK release cycles to get fixes merged.

Quoted 8 lines
Quoted 5 lines

Regarding the tool itself, I understand from the commit message that the tool has a dependency on the BBDEV PMD version, but the tool run on the host while the PMD driver is used in the guest/container. So having it in the driver directory will not really help, as host DPDK (if any) and guest DPDK may come from different parties.

Yes this was captured earlier, purely stored there as a companion application for a given version of the PMD (ie. different subdirectories for

each PMD directory).

Quoted 1 line

They do no run in the same container for deployment and are not built at

the same time indeed, their interface is the HW really and one being needed to be run prior to the other one to be functional.

Quoted 5 lines
Quoted 3 lines

One question I have is whether this is the tool itself that has a dependency on the PMD, or just the config file?

Each PMD directory would have its own version of the companion PF

config application.

Quoted 1 line

Ie. the patch above is only for baseband/fpga_5gnr_fec ie. Intel Vista Creek

Quoted 9 lines

with 5G LDPC user image.

OK. Does it mean the same application and configuration will work for baseband/fpga_5gnr_fec PMD versions v20.11, v21.02, v21.05, etc, ...?

If not, is there a way for the PMD driver to detect whether a wrong configuration was applied? Something like checking the FW version of a NIC is supported by the PMD driver.

Quoted 5 lines

As mentioned above there is no API between the 2 config-app companion and BBDEV/DPDK, as they belong logically to 2 different entities (possibly containers) without shared interface except indirectly through HW. There is no strict API which could be broken between the 2, but purely 2 SW ingredients that ideally should be aligned to avoid discrepancy during deployment or validation.
To answer your question I don't foresee a specific case right now causing a strict loss of functional compatibility.

Note that the configure function being used within config-app already exists right now within the DPDK PMD : ie. that function is the one being exposed by PMD and used when running bbdev-test in DPDK for PF. The limitation being that this is only used when running *within DPDK and from PF*, which is not the deployment use case ie. when DPDK is on the VF only, and PF only performs HW configuration without DPDK dependency (apologize for repeating).

OK, got it.

Quoted 3 lines

You can see 'fpga_write_config()' in the current patchset which does match line-for-line with 'fpga_5gnr_fec_configure()' already in the PMD on DPDK main branch. Virtually the same code really with same config input structure 'fpga_5gnr_fec_conf' and same registers being written to. Difference being that the former implementation is formally integrated in DPDK while the other (config app companion in same directory) has no DPDK dependency. That's it.
There is concern in having these 2 different implementations of the same initialization sequence diverging (a fix to a register setting would need to be applied on both repositories with different versioning) and splitting the HW support into multiple repos, hence my personal preference to keep everything in one place.

I understand it would be easier for you.
Maybe an alternative would be to have a simple DPDK application that uses the PMD driver on the PF just for the initialization. Doing that, you would not even have to duplicate the config part between the PMD driver and the application, and you would have a single config app for every cards and bitstreams. It would also mean you get rid off the external dependency on inih library.

Quoted 3 lines

As stated from the beginning I personally see pros and cons on both sides, all in all DPDK arguably being cleaner to me for the reasons above.
Still the most important is that it needs to make sense and be valuable to DPDK.
Based on latest feedback from Maxime and Thomas, I believe this is now trending towards being preferred outside of DPDK. If you can confirm by email then that is fine and I will just abandon this ticket so that we can consider alternate options and split this into different repos.

In its current form, this is a nack for me. I would prefer either having it outside of DPDK, or having a DPDK application calling the PMD driver init.

Quoted 28 lines
Quoted 4 lines
Quoted 1 line

There will be different companion applications upstreamed for each other

PMD directories (current and future) as they rely on different HW devices with independent MMIO access.

Quoted 1 line

Said otherwise both the config file (features exposed) and implementation

Quoted 9 lines

(registers required for these features) are defined per HW device (+ user image for FPGA) hence per PMD version.

Would it make sense to have a single application, with having the registers map and their values to apply in a configuration file?
It would avoid code duplication between devices and so ease the maintenance.

Some pros and cons as well, but if outside DPDK we may just do this.

Quoted 11 lines
Quoted 2 lines

There 2 entities have no API between themselves, only indirectly through

HW (no shared memory, VF2PF comms, etc..).

Quoted 1 line

New features may have to be added concurrently though, hence splitting

repos create room for version mismatch and complicate the ingredients line up.

I am not sure that this argument holds, because one could argue in this case that we could place the FPGA bitstream in the PMD directory too to ensure there is no mismatch.

Quoted 5 lines

There are obviously licenses differences between the DPDK code and FPGA user images preventing to do this.
But still for the sake of the argument logically a different user image may indeed not
work with a PMD version not matching the feature set (as happened prior to PRQ).
From now on any future user image on the FPGA would need to be strictly compatible with current PMD.
The main point being that splitting these SW ingredients means increasing the number of different versions to keep track of and different repos to use to create a working environment.

Ok for prototyping, but for production host and guest SW are likely to come from different streams.

Thanks for the detailed explanations,
Maxime

Quoted 27 lines

Both options are viable with different pros and cons.

Quoted 15 lines
Quoted 1 line

That was part of the pros and cons described earlier and I can totally see

arguments both ways, and that's what I have been trying to share openly in this ticket history.

Thanks, I appreciate that.

Quoted 2 lines

Basically I see nothing fundamentally new here in the discussion, but it is

great to receive input and I am happy to hear further input from tech board or others towards a decision.

Quoted 1 line

This started as an open discussion on this DPDK mailing list capturing

explicitly both pros and cons of this approach which are arguable, and in case this is not deemed practical eventually then we can still totally come back internally to the drawing board with other options outside of DPDK.

Quoted 1 line
Quoted 6 lines
Quoted 3 lines

Thanks,
Nic

Regards,
Maxime

Thanks
Nic

#7 lore raw

Re: [dpdk-dev] [20.11, PATCH v2] baseband/fpga_5gnr_fec: add companion PF config App

Hello Nicolas, Maxime,

Quoted 16 lines · Maxime Coquelin
Quoted 16 lines
Quoted 15 lines
Quoted 4 lines
Quoted 2 lines

One question I have is whether this is the tool itself that has a dependency on the PMD, or just the config file?

Each PMD directory would have its own version of the companion PF

config application.

Quoted 1 line

Ie. the patch above is only for baseband/fpga_5gnr_fec ie. Intel Vista Creek

with 5G LDPC user image.

OK. Does it mean the same application and configuration will work for baseband/fpga_5gnr_fec PMD versions v20.11, v21.02, v21.05, etc, ...?

If not, is there a way for the PMD driver to detect whether a wrong configuration was applied? Something like checking the FW version of a NIC is supported by the PMD driver.

Quoted 5 lines
Quoted 5 lines

As mentioned above there is no API between the 2 config-app companion and BBDEV/DPDK, as they belong logically to 2 different entities (possibly containers) without shared interface except indirectly through HW. There is no strict API which could be broken between the 2, but purely 2 SW ingredients that ideally should be aligned to avoid discrepancy during deployment or validation.
To answer your question I don't foresee a specific case right now causing a strict loss of functional compatibility.

Note that the configure function being used within config-app already exists right now within the DPDK PMD : ie. that function is the one being exposed by PMD and used when running bbdev-test in DPDK for PF. The limitation being that this is only used when running *within DPDK and from PF*, which is not the deployment use case ie. when DPDK is on the VF only, and PF only performs HW configuration without DPDK dependency (apologize for repeating).

I am naively thinking that the bitstream + this configuration are a whole for a given usecase and should be loaded as a single step. A bit like how vendors provide ready-to-use firmwares on their nics.

Leaving the users with the possibility to load whatever config they write is scary.
When hitting an init issue, application crash etc... in a customer env, how will we know the configuration is actually what it should be?

Quoted 15 lines

OK, got it.

Quoted 3 lines

You can see 'fpga_write_config()' in the current patchset which does match line-for-line with 'fpga_5gnr_fec_configure()' already in the PMD on DPDK main branch. Virtually the same code really with same config input structure 'fpga_5gnr_fec_conf' and same registers being written to. Difference being that the former implementation is formally integrated in DPDK while the other (config app companion in same directory) has no DPDK dependency. That's it.
There is concern in having these 2 different implementations of the same initialization sequence diverging (a fix to a register setting would need to be applied on both repositories with different versioning) and splitting the HW support into multiple repos, hence my personal preference to keep everything in one place.

I understand it would be easier for you.
Maybe an alternative would be to have a simple DPDK application that uses the PMD driver on the PF just for the initialization. Doing that, you would not even have to duplicate the config part between the PMD driver and the application, and you would have a single config app for every cards and bitstreams. It would also mean you get rid off the external dependency on inih library.

Quoted 7 lines
Quoted 3 lines

As stated from the beginning I personally see pros and cons on both sides, all in all DPDK arguably being cleaner to me for the reasons above.
Still the most important is that it needs to make sense and be valuable to DPDK.
Based on latest feedback from Maxime and Thomas, I believe this is now trending towards being preferred outside of DPDK. If you can confirm by email then that is fine and I will just abandon this ticket so that we can consider alternate options and split this into different repos.

In its current form, this is a nack for me. I would prefer either having it outside of DPDK, or having a DPDK application calling the PMD driver init.

This is a nack from me too.

About having a DPDK application calling the PMD driver init. This requires to have a DPDK application from version X touching the hw in the host, while a guest could be running DPDK version Y. This will result in ping/pong when the support on the host and guest sides is done by two different teams.

Thanks.

Signature · 1 line
David Marchand