FPGA manager user space interface

Hi all,

I just wrote a FPGA manager driver for the TS-7300 board which features an Altera Cyclone II on board. While the finished solution is my case might be something like the MFD driver for the FPGA devices requesting the bitstream load and registering the different devices it exposes, during development it is nice to exercise the FPGA manager driver to load something:

- a quick way is to add a pair of sysfs attributes to define the bitstream filename and to trigger the load

- offer a more consistent and robust interface through e.g; a character device that you can write/read/poll to see the loading progress about

- have the driver exposing FPGA peripherals be exposing an user space interface to trigger a (re)load/(re)/configuration, although that's really something that belongs in the FPGA manager it seems

I am mostly curious if these were taken into account during the initial design and it is agreed upon that yes these are some of options and that userspace loading is just anecdotal we do not need any userspace interface, or if this is just missing and we want one? Either way, I don't mind submitting what I came up with for the TS-7300 board.

Thanks!
--
Florian

Hi all,

I just wrote a FPGA manager driver for the TS-7300 board which features an Altera Cyclone II on board. While the finished solution is my case might be something like the MFD driver for the FPGA devices requesting the bitstream load and registering the different devices it exposes, during development it is nice to exercise the FPGA manager driver to load something:

- a quick way is to add a pair of sysfs attributes to define the bitstream filename and to trigger the load

- offer a more consistent and robust interface through e.g; a character device that you can write/read/poll to see the loading progress about

- have the driver exposing FPGA peripherals be exposing an user space interface to trigger a (re)load/(re)/configuration, although that's really something that belongs in the FPGA manager it seems

I am mostly curious if these were taken into account during the initial design and it is agreed upon that yes these are some of options and that userspace loading is just anecdotal we do not need any userspace interface, or if this is just missing and we want one?

Either way, I
don't mind submitting what I came up with for the TS-7300 board.

Thanks!
--
Florian

On Sun, 10 Jul 2016, Florian Fainelli wrote:

Hi Florian,

I'm in the process of upstreaming FPGA Regions support which allows reprogramming by applying Device Tree Overlays. If you want to see a tree which has the latest patches for that, I have all that in our github tree.

https://github.com/altera-opensource/linux-socfpga/tree/socfpga-4.6

The documentation for the DT overlays is in
Documentation/devicetree/bindings/fpga/fpga-region.txt

Documentation for the DT configfs interface is in
Documentation/devicetree/configfs-overlays.txt

The idea here is explained at legnth in the doc, but basically the user can use the configfs interface to apply a DT overlay that causes the FPGA to be reprogrammed and child devices to get added and probed.

I like DT overlays a lot, but I do not expect that (or anything else) will be a good interface for every use case. The FPGA Manager Framework went through a legnthy discussion on the mailing list where several people wanted the interface to match what they were already using. Interfaces similar to what you suggest below were proposed and shot down. My conclusion from all that was to implement the kernel API functions such that people could add whatever interfaces they needed for their use case.

Quoted 10 lines

Hi all,

I just wrote a FPGA manager driver for the TS-7300 board which features an Altera Cyclone II on board. While the finished solution is my case might be something like the MFD driver for the FPGA devices requesting the bitstream load and registering the different devices it exposes, during development it is nice to exercise the FPGA manager driver to load something:

It sounds like your goal is to use the FPGA Manager API inside the kernel, but it would be helpful to have an easy userspace interface during development. Is that right?

Quoted 6 lines

- a quick way is to add a pair of sysfs attributes to define the bitstream filename and to trigger the load

- offer a more consistent and robust interface through e.g; a character device that you can write/read/poll to see the loading progress about

Sysfs and char driver interfaces have been proposed and shot down a few times already.

The first version of the FPGA Manager Framework was a char driver. To program the FPGA, the user had to do 'cat bitstream-file > /dev/fpga0'. Bridge support was a separate thing that was also controlled by sysfs. It was messy and if userspace could easily do the wrong thing and crash the system.

Quoted 4 lines

- have the driver exposing FPGA peripherals be exposing an user space interface to trigger a (re)load/(re)/configuration, although that's really something that belongs in the FPGA manager it seems

If you use the device tree configfs interface, you can do that. It might not be exactly what you are thinking of. And there is a learning curve in getting the overlays right.

Quoted 5 lines

I am mostly curious if these were taken into account during the initial design and it is agreed upon that yes these are some of options and that userspace loading is just anecdotal we do not need any userspace interface, or if this is just missing and we want one?

There will be use cases that will need various userspace interfaces. It's been hard to get people who are already using FPGAs to agree on using any one interface because they would have to change the code they already have running.

Quoted 2 lines

Either way, I
don't mind submitting what I came up with for the TS-7300 board.

Yes, that would be great.

Quoted 5 lines

Thanks!
--
Florian

Hi Alan,

Quoted 22 lines · atull

On Sun, 10 Jul 2016, Florian Fainelli wrote:

Hi Florian,

I'm in the process of upstreaming FPGA Regions support which allows reprogramming by applying Device Tree Overlays. If you want to see a tree which has the latest patches for that, I have all that in our github tree.

https://github.com/altera-opensource/linux-socfpga/tree/socfpga-4.6

The documentation for the DT overlays is in
Documentation/devicetree/bindings/fpga/fpga-region.txt

Documentation for the DT configfs interface is in
Documentation/devicetree/configfs-overlays.txt

The idea here is explained at legnth in the doc, but basically the user can use the configfs interface to apply a DT overlay that causes the FPGA to be reprogrammed and child devices to get added and probed.

Quoted 9 lines

I like DT overlays a lot, but I do not expect that (or anything else) will be a good interface for every use case. The FPGA Manager Framework went through a legnthy discussion on the mailing list where several people wanted the interface to match what they were already using. Interfaces similar to what you suggest below were proposed and shot down. My conclusion from all that was to implement the kernel API functions such that people could add whatever interfaces they needed for their use case.

While I agree that the current state of the FPGA manager is completely sane, and people making products including a FPGA will want something (kernel driver or user-space scripts) that load a given set of bistreams (under version control), it still feels like there is a small bit of debugging and or use cases where it is desireable to have a way, from user-space to load an arbitrary FPGA bitstream which is not set in stone from e.g: a Linux kernel driver. More than being able to change something on the filesystem, it's about being able to reload the bitstream from user-space that seems to be critically missing imho.

Do you have links to the original discussion so I could get an idea of what were the different points back then?

Thanks!

Quoted 62 lines

Quoted 11 lines

Hi all,

I just wrote a FPGA manager driver for the TS-7300 board which features an Altera Cyclone II on board. While the finished solution is my case might be something like the MFD driver for the FPGA devices requesting the bitstream load and registering the different devices it exposes, during development it is nice to exercise the FPGA manager driver to load something:

It sounds like your goal is to use the FPGA Manager API inside the kernel, but it would be helpful to have an easy userspace interface during development. Is that right?

Quoted 6 lines

- a quick way is to add a pair of sysfs attributes to define the bitstream filename and to trigger the load

- offer a more consistent and robust interface through e.g; a character device that you can write/read/poll to see the loading progress about

Sysfs and char driver interfaces have been proposed and shot down a few times already.

The first version of the FPGA Manager Framework was a char driver. To program the FPGA, the user had to do 'cat bitstream-file > /dev/fpga0'. Bridge support was a separate thing that was also controlled by sysfs. It was messy and if userspace could easily do the wrong thing and crash the system.

Quoted 4 lines

- have the driver exposing FPGA peripherals be exposing an user space interface to trigger a (re)load/(re)/configuration, although that's really something that belongs in the FPGA manager it seems

If you use the device tree configfs interface, you can do that. It might not be exactly what you are thinking of. And there is a learning curve in getting the overlays right.

Quoted 5 lines

I am mostly curious if these were taken into account during the initial design and it is agreed upon that yes these are some of options and that userspace loading is just anecdotal we do not need any userspace interface, or if this is just missing and we want one?

There will be use cases that will need various userspace interfaces. It's been hard to get people who are already using FPGAs to agree on using any one interface because they would have to change the code they already have running.

Quoted 2 lines

Either way, I
don't mind submitting what I came up with for the TS-7300 board.

Yes, that would be great.

Quoted 5 lines

Thanks!
--
Florian

Signature · 1 line

Florian

Quoted 35 lines · Florian Fainelli

Hi Alan,

Quoted 22 lines · atull

On Sun, 10 Jul 2016, Florian Fainelli wrote:

Hi Florian,

I'm in the process of upstreaming FPGA Regions support which allows reprogramming by applying Device Tree Overlays. If you want to see a tree which has the latest patches for that, I have all that in our github tree.

https://github.com/altera-opensource/linux-socfpga/tree/socfpga-4.6

The documentation for the DT overlays is in
Documentation/devicetree/bindings/fpga/fpga-region.txt

Documentation for the DT configfs interface is in
Documentation/devicetree/configfs-overlays.txt

The idea here is explained at legnth in the doc, but basically the user can use the configfs interface to apply a DT overlay that causes the FPGA to be reprogrammed and child devices to get added and probed.

Quoted 9 lines

I like DT overlays a lot, but I do not expect that (or anything else) will be a good interface for every use case. The FPGA Manager Framework went through a legnthy discussion on the mailing list where several people wanted the interface to match what they were already using. Interfaces similar to what you suggest below were proposed and shot down. My conclusion from all that was to implement the kernel API functions such that people could add whatever interfaces they needed for their use case.

Quoted 10 lines

While I agree that the current state of the FPGA manager is completely sane, and people making products including a FPGA will want something (kernel driver or user-space scripts) that load a given set of bistreams (under version control), it still feels like there is a small bit of debugging and or use cases where it is desireable to have a way, from user-space to load an arbitrary FPGA bitstream which is not set in stone from e.g: a Linux kernel driver. More than being able to change something on the filesystem, it's about being able to reload the bitstream from user-space that seems to be critically missing imho.

Did you look into DT overlays? DT overlays can be applied from userspace using the configfs interface. The FPGA Region code will handle bridges and loading the nodes for the child devices. You can also remove the overlay, the child devices will be unloaded and the bridges disabled so you can iterate during your development.

Quoted 5 lines

Do you have links to the original discussion so I could get an idea of what were the different points back then?

Thanks!

Exposing reprogramming in sysfs or as a char driver was rejected as:
* it makes it easy to do the wrong thing and crash your system
* it didn't give us a way of showing the heirarchy of what a system with multiple fpgas and bridges could have.

* It makes it difficult to know which fpga you are reprogramming if you are doing 'cat image > /dev/fpga0' since your system may have several fpgas.

* Your system may have bridges between the cpu and the fpga. Those will need to be mapped not only to the fpga they are connected to, but more finely than that. Some fpga images need different bridges to be enabled/disabled.

When dynamic device trees started to become a reality in the kernel, that was a natural solution to a lot of this.

Feel free to google the lkml archives you want more details. The reason I am opposed to it now is it that is begging for the system to be crashed by doing a simple operation from userspace. And it makes it impossible to export to userspace the kind of control that the user needs except perhaps in only the most simple system.

Here's a few easy ways to crash your system if you can reprogram from userspace:
1. Reprogram the FPGA without shutting down the bridges first. Spurious data goes out onto a processor bus and things get hosed.

2. Program the FGPA, load some drivers to support the hw in the FPGA, then reprogram the FPGA.
3. Program the FPGA using DT overlays and then use sysfs as a back door to program it again.
4. A system with multiple FPGAs will need to be brought up or reconfigured in a sensible order. DT overlays can be applied on top of dt overlays, bringing up the system in order.

Alan

Quoted 67 lines

Quoted 62 lines

Quoted 11 lines

Hi all,

I just wrote a FPGA manager driver for the TS-7300 board which features an Altera Cyclone II on board. While the finished solution is my case might be something like the MFD driver for the FPGA devices requesting the bitstream load and registering the different devices it exposes, during development it is nice to exercise the FPGA manager driver to load something:

It sounds like your goal is to use the FPGA Manager API inside the kernel, but it would be helpful to have an easy userspace interface during development. Is that right?

Quoted 6 lines

- a quick way is to add a pair of sysfs attributes to define the bitstream filename and to trigger the load

- offer a more consistent and robust interface through e.g; a character device that you can write/read/poll to see the loading progress about

Sysfs and char driver interfaces have been proposed and shot down a few times already.

The first version of the FPGA Manager Framework was a char driver. To program the FPGA, the user had to do 'cat bitstream-file > /dev/fpga0'. Bridge support was a separate thing that was also controlled by sysfs. It was messy and if userspace could easily do the wrong thing and crash the system.

Quoted 4 lines

- have the driver exposing FPGA peripherals be exposing an user space interface to trigger a (re)load/(re)/configuration, although that's really something that belongs in the FPGA manager it seems

If you use the device tree configfs interface, you can do that. It might not be exactly what you are thinking of. And there is a learning curve in getting the overlays right.

Quoted 5 lines

I am mostly curious if these were taken into account during the initial design and it is agreed upon that yes these are some of options and that userspace loading is just anecdotal we do not need any userspace interface, or if this is just missing and we want one?

There will be use cases that will need various userspace interfaces. It's been hard to get people who are already using FPGAs to agree on using any one interface because they would have to change the code they already have running.

Quoted 2 lines

Either way, I
don't mind submitting what I came up with for the TS-7300 board.

Yes, that would be great.

Quoted 5 lines

Thanks!
--
Florian

Signature · 1 line

Florian