pci_ids: Added FPGA-related entries
Subsystems (2)
PATCH 1/2 pci_ids: Added FPGA-related entries
These entries are referred to by the Xillybus driver.
--- include/linux/pci_ids.h | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-)
include/linux/pci_ids.h +6 -0
@@ -1320,6 +1320,12 @@#define PCI_DEVICE_ID_REALTEK_8139 0x8139#define PCI_VENDOR_ID_XILINX 0x10ee +#define PCI_VENDOR_ID_ALTERA 0x1172 +#define PCI_VENDOR_ID_ACTEL 0x11aa +#define PCI_VENDOR_ID_LATTICE 0x1204 + +#define PCI_DEVICE_ID_XILLYBUS 0xebeb +#define PCI_DEVICE_ID_RME_DIGI96 0x3fc0#define PCI_DEVICE_ID_RME_DIGI96_8 0x3fc1#define PCI_DEVICE_ID_RME_DIGI96_8_PRO 0x3fc2
Signature · 1 line
1.7.2.3
Greg KH
maintainerQuoted 5 lines · Eli Billauer
These entries are referred to by the Xillybus driver.
Signed-off-by Eli Billauer <[email protected]>--- include/linux/pci_ids.h | 6 ++++++
Please read the top of this file for why you shouldn't be adding these device ids to that file.
Care to redo these two patches please?
thanks,
greg k-h
PATCH 2/2 New driver: Xillybus generic interface for FPGA (programmable logic)
In reply to Eli BillauerXillybus is a general-purpose framework for communication between programmable logic (FPGA) and a host. It provides a simple connection between hardware FIFOs in the FPGA and their respective device files on the host. The user space programming model is like piping data from or to the FPGA.
The underlying transport between the host and FPGA is either PCIe or AXI (AMBA bus by ARM).
The Xillybus logic (IP core) is configurable in the number of pipes it presents and their nature. The driver autodetects these pipes, making it essentially forward-compatible to future configurations. The benefit of having this driver enabled in the kernel is that hardware vendors may release a new card, knowing that it will work out of the box on any future Linux machine, with the specific configuration defined for the FPGA part.
This driver has been available for download for over a year, and has been actively used on a wide variety of kernels versions and configurations.
Website: http://xillybus.com
--- drivers/misc/Kconfig | 32 + drivers/misc/Makefile | 1 + drivers/misc/xillybus.c | 2845 +++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 2878 insertions(+), 0 deletions(-) create mode 100644 drivers/misc/xillybus.c
drivers/misc/Kconfig +32 -0
@@ -499,6 +499,38 @@ config USB_SWITCH_FSA9480 stereo and mono audio, video, microphone and UART data to use a common connector port. +config XILLYBUS + tristate "Xillybus Support" + depends on PCI || (OF_ADDRESS && OF_DEVICE && OF_IRQ) + default m + help + Xillybus is a generic interface for peripherals designed on + programmable logic (FPGA). The driver probes the hardware for + its capabilities, and creates device files accordingly. + + If unsure, say M. + +config XILLYBUS_PCIE + bool "Xillybus over PCIe" + depends on XILLYBUS && PCI + default y + help + Set to Y if you want Xillybus to use PCI Express for communicating + with the FPGA. This option is harmless, but it requires PCI + support on the kernel. Say Y if the target processor supports + PCI and/or PCIe. + +config XILLYBUS_OF + bool "Xillybus over Device Tree" + depends on XILLYBUS && OF_ADDRESS && OF_DEVICE && OF_IRQ + default y + help + Set to Y if you want Xillybus to find its resources from the + Open Firmware Flattened Device Tree. If the target is an embedded + system, say Y. This option is harmless, but it requires Device + Tree support on the kernel, which is usually not the case for + kernels for fullblown computers. + source "drivers/misc/c2port/Kconfig" source "drivers/misc/eeprom/Kconfig" source "drivers/misc/cb710/Kconfig"
drivers/misc/Makefile +1 -0
@@ -49,3 +49,4 @@ obj-y += carma/ obj-$(CONFIG_USB_SWITCH_FSA9480) += fsa9480.o obj-$(CONFIG_ALTERA_STAPL) +=altera-stapl/ obj-$(CONFIG_INTEL_MEI) += mei/ +obj-$(CONFIG_XILLYBUS) += xillybus.o
drivers/misc/xillybus.c new file +2845 -0
2,845 lines not on this page · raw message
Signature · 1 line
1.7.2.3
Greg KH
maintainerRe: [PATCH 2/2] New driver: Xillybus generic interface for FPGA (programmable logic)
Quoted 8 lines · Eli Billauer
Xillybus is a general-purpose framework for communication between programmable logic (FPGA) and a host. It provides a simple connection between hardware FIFOs in the FPGA and their respective device files on the host. The user space programming model is like piping data from or to the FPGA.
The underlying transport between the host and FPGA is either PCIe or AXI (AMBA bus by ARM).
Quoted 6 lines
The Xillybus logic (IP core) is configurable in the number of pipes it presents and their nature. The driver autodetects these pipes, making it essentially forward-compatible to future configurations. The benefit of having this driver enabled in the kernel is that hardware vendors may release a new card, knowing that it will work out of the box on any future Linux machine, with the specific configuration defined for the FPGA part.
What is the user/kernel interface for this driver? Is it documented anywhere?
drivers/misc/Kconfig config USB_SWITCH_FSA9480 · line 505
Quoted 4 lines
drivers/misc/Kconfig +4
-0+config XILLYBUS + tristate "Xillybus Support" + depends on PCI || (OF_ADDRESS && OF_DEVICE && OF_IRQ) + default m
Never default to on, unless you can not boot a box without this option.
drivers/misc/Kconfig config USB_SWITCH_FSA9480 · line 516
Quoted 11 lines
drivers/misc/Kconfig +11
-0+ help + Xillybus is a generic interface for peripherals designed on + programmable logic (FPGA). The driver probes the hardware for + its capabilities, and creates device files accordingly. + + If unsure, say M. + +config XILLYBUS_PCIE + bool "Xillybus over PCIe" + depends on XILLYBUS && PCI + default y
Same here.
drivers/misc/Kconfig config USB_SWITCH_FSA9480 · line 526
Quoted 10 lines
drivers/misc/Kconfig +10
-0+ help + Set to Y if you want Xillybus to use PCI Express for communicating + with the FPGA. This option is harmless, but it requires PCI + support on the kernel. Say Y if the target processor supports + PCI and/or PCIe. + +config XILLYBUS_OF + bool "Xillybus over Device Tree" + depends on XILLYBUS && OF_ADDRESS && OF_DEVICE && OF_IRQ + default y
Same here.
drivers/misc/Kconfig config USB_SWITCH_FSA9480 · line 533
Quoted 7 lines
drivers/misc/Kconfig +7
-0+ help + Set to Y if you want Xillybus to find its resources from the + Open Firmware Flattened Device Tree. If the target is an embedded + system, say Y. This option is harmless, but it requires Device + Tree support on the kernel, which is usually not the case for + kernels for fullblown computers. +
Shouldn't both of these options be different modules with a shared core, instead of a single driver with lots of #ifdefs all over the place, making it more complex?
Quoted 18 lines
source "
drivers/misc/c2port/Kconfig"
source "drivers/misc/eeprom/Kconfig"
source "drivers/misc/cb710/Kconfig"drivers/misc/Makefile +1
-0@@ -49,3 +49,4 @@ obj-y += carma/ obj-$(CONFIG_USB_SWITCH_FSA9480) += fsa9480.o obj-$(CONFIG_ALTERA_STAPL) +=altera-stapl/ obj-$(CONFIG_INTEL_MEI) += mei/ +obj-$(CONFIG_XILLYBUS) += xillybus.odrivers/misc/xillybus.c new file +0
-0
drivers/misc/xillybus.c line 1
Quoted 1 line
drivers/misc/xillybus.c new file +1
-0@@ -0,0 +1,2845 @@ +#include <linux/version.h>
You don't need this include file.
No copyright notice at the top of the file?
Quoted 50 lines
drivers/misc/xillybus.c new file +50
-0+#include <linux/list.h> +#include <linux/device.h> +#include <linux/module.h> +#include <linux/io.h> +#include <linux/dma-mapping.h> +#include <linux/interrupt.h> +#include <linux/sched.h> +#include <linux/fs.h> +#include <linux/cdev.h> +#include <linux/spinlock.h> +#include <linux/mutex.h> +#include <linux/list.h> +#include <linux/crc32.h> +#include <linux/poll.h> +#include <linux/delay.h> +#include <linux/io.h> +#include <linux/slab.h> +#include <linux/workqueue.h> +#ifdef CONFIG_XILLYBUS_PCIE +#include <linux/pci.h> +#include <linux/pci-aspm.h> +#endif + +#ifdef CONFIG_XILLYBUS_OF +#include <linux/platform_device.h> +#include <linux/of.h> +#include <linux/of_irq.h> +#include <linux/of_address.h> +#include <linux/of_device.h> +#include <linux/of_platform.h> +#endif + +MODULE_DESCRIPTION ("Xillybus driver" ); +MODULE_AUTHOR ("Eli Billauer, Xillybus Ltd." ); +MODULE_VERSION ("1.06" ); +MODULE_ALIAS ("xillybus" ); +MODULE_LICENSE ("GPL v2" ); + +/* General timeout is 100 ms, rx timeout is 10 ms */ +#define XILLY_RX_TIMEOUT (10*HZ/1000) +#define XILLY_TIMEOUT (100*HZ/1000) + +#define fpga_msg_ctrl_reg 0x0002 +#define fpga_dma_control_reg 0x0008 +#define fpga_dma_bufno_reg 0x0009 +#define fpga_dma_bufaddr_lowaddr_reg 0x000a +#define fpga_dma_bufaddr_highaddr_reg 0x000b +#define fpga_buf_ctrl_reg 0x000c +#define fpga_buf_offset_reg 0x000d +#define fpga_endian_reg 0x0010
drivers/misc/xillybus.c line 61
Quoted 10 lines
drivers/misc/xillybus.c new file +10
-0+ +#define XILLYMSG_OPCODE_RELEASEBUF 1 +#define XILLYMSG_OPCODE_QUIESCEACK 2 +#define XILLYMSG_OPCODE_FIFOEOF 3 +#define XILLYMSG_OPCODE_FATAL_ERROR 4 +#define XILLYMSG_OPCODE_NONEMPTY 5 + +#if (PAGE_SIZE <4096 ) +#error Your processor architecture has a page size smaller than 4096 +#endif
That can never happen. Even if it does, you don't care about that in the driver.
drivers/misc/xillybus.c line 73
Quoted 12 lines
drivers/misc/xillybus.c new file +12
-0+ +#ifdef CONFIG_XILLYBUS_OF +/* Match table for of_platform binding */ +static struct of_device_id xillybus_of_match[]__devinitdata = { + { .compatible ="xlnx,xillybus-1.00.a" , }, + {} +}; + +MODULE_DEVICE_TABLE (of, xillybus_of_match); +#endif + +static struct class *xillybus_class;
Why not just use the misc interface instead of your own class?
drivers/misc/xillybus.c line 2544
Quoted 5 lines
drivers/misc/xillybus.c new file +5
-0+#ifdef CONFIG_XILLYBUS_PCIE +static int xilly_probe_or_remove (struct pci_dev *pdev, +const struct pci_device_id *ent, +const int what_to_do) +{
Why would you call the same function for both probe and release? That is very odd.
drivers/misc/xillybus.c line 2551
Quoted 7 lines
drivers/misc/xillybus.c new file +7
-0+struct xilly_endpoint *endpoint; +int rc =0 ; + +if (!what_to_do) { + endpoint =pci_get_drvdata (pdev); +endpoint_discovery_or_remove (endpoint,1 ); + goto release;
That's all you need for the remove function, don't try to put them both in the same function.
drivers/misc/xillybus.c line 2664
Quoted 7 lines
drivers/misc/xillybus.c new file +7
-0+static int __devinit xilly_probe (struct pci_dev *pdev, +const struct pci_device_id *ent) +{ +return xilly_probe_or_remove (pdev, ent,1 ); +} + +MODULE_DEVICE_TABLE (pci, xillyids);
You can use the pci_module_init() function instead.
drivers/misc/xillybus.c line 2667
Quoted 3 lines
drivers/misc/xillybus.c new file +3
-0+ +static struct pci_driver xillybus_driver = { + .name = (char *) xillyname,
Why are you casting this? That implies that the original variable isn't correct :)
drivers/misc/xillybus.c line 2670
Quoted 3 lines
drivers/misc/xillybus.c new file +3
-0+ .id_table = xillyids, + .probe = xilly_probe, + .remove = __devexit_p(xilly_remove),
__devexit_p is going away, please don't use it.
thanks,
greg k-h
Re: [PATCH 2/2] New driver: Xillybus generic interface for FPGA (programmable logic)
Thanks for the remarks.
I'm sending the updated patches in a minute. Basically, I divided the module into three (one core, one for PCIe and one for OF) and made several corrections.
Quoted 3 lines · Greg KH
What is the user/kernel interface for this driver? Is it documented anywhere?
There's a rather extensive documentation for download at the site. The docs for the host side mostly instruct common UNIX programming techniques: The device files are just data pipes to FIFOs in the FPGA, behaving like one would expect.
Quoted 8 lines
drivers/misc/xillybus.c line 61
Quoted 4 lines
drivers/misc/xillybus.c new file +3
-0+#if (PAGE_SIZE <4096 ) +#error Your processor architecture has a page size smaller than 4096 +#endif That can never happen. Even if it does, you don't care about that in the driver.
I removed this check because it can't happen. But the driver *does* care about this, since it creates a lot of buffers with different alignments, hence depending on the pages' alignment.
Quoted 5 lines
Quoted 3 lines
+static struct class
*xillybus_class;Why not just use the misc interface instead of your own class?
When Xillybus is used, the whole system's mission is usually around it (e.g. it's a computer doing data acquisition through the Xillybus pipes). So giving it a high profile makes sense, I believe. Besides, a dozen of device files are not rare.
Regards,
Eli
Greg KH
maintainerRe: [PATCH 2/2] New driver: Xillybus generic interface for FPGA (programmable logic)
Quoted 6 lines · Eli Billauer
Thanks for the remarks.
I'm sending the updated patches in a minute. Basically, I divided the module into three (one core, one for PCIe and one for OF) and made several corrections.
Quoted 7 lines
Quoted 2 lines · Greg KH
What is the user/kernel interface for this driver? Is it documented anywhere?
There's a rather extensive documentation for download at the site. The docs for the host side mostly instruct common UNIX programming
techniques: The device files are just data pipes to FIFOs in the FPGA, behaving like one would expect.
As we need to review the user/kernel api here, putting the docs as part of the driver submission is a good idea :)
I didn't know, nor do I trust, that a random web site would have the correct documentation for a kernel driver.
Quoted 9 lines
Quoted 6 lines
drivers/misc/xillybus.c line 61
Quoted 3 lines
drivers/misc/xillybus.c new file +3
-0+#if (PAGE_SIZE <4096 ) +#error Your processor architecture has a page size smaller than 4096 +#endif That can never happen. Even if it does, you don't care about that in the driver.
I removed this check because it can't happen. But the driver *does* care about this, since it creates a lot of buffers with different alignments, hence depending on the pages' alignment.
Alignment is different than the size of a page. What happens if your driver runs on a machine with a page size bigger than 4K? You need to be able to handle that properly, so perhaps you should check that?
Quoted 6 lines
Quoted 2 lines
Quoted 1 line
+static struct class
*xillybus_class;Why not just use the misc interface instead of your own class?
When Xillybus is used, the whole system's mission is usually around it (e.g. it's a computer doing data acquisition through the Xillybus pipes). So giving it a high profile makes sense, I believe. Besides, a dozen of device files are not rare.
It is no problem to create dozens of misc devices. It makes your driver smaller, contain less code that I have to audit and you have to ensure you got right, and it removes another user of 'struct class' which we are trying to get rid of anyway. So please, move to use a misc device.
thanks,
greg k-h
Re: [PATCH 2/2] New driver: Xillybus generic interface for FPGA (programmable logic)
Quoted 7 lines · Greg KH
As we need to review the user/kernel api here, putting the docs as part of the driver submission is a good idea :)
I didn't know, nor do I trust, that a random web site would have the correct documentation for a kernel driver.
OK. I'll add a file in Documentation/misc-devices/.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
drivers/misc/xillybus.c line 61
Quoted 4 lines
drivers/misc/xillybus.c new file +3
-0+#if (PAGE_SIZE <4096 ) +#error Your processor architecture has a page size smaller than 4096 +#endif That can never happen. Even if it does, you don't care about that in the driver.
Quoted 8 lines
Quoted 4 lines
I removed this check because it can't happen. But the driver *does* care about this, since it creates a lot of buffers with different alignments, hence depending on the pages' alignment.
Alignment is different than the size of a page. What happens if your driver runs on a machine with a page size bigger than 4K? You need to be able to handle that properly, so perhaps you should check that?
The problem is if the page size *smaller* than 4kB. The buffers allocated by the driver must not cross a 4kB boundary, and it's assumed that anything returned by __get_free_pages() is 4 kB-aligned. Otherwise the FPGA will generate illegal PCIe packets by crossing that boundary.
If the page boundary is bigger than 4k, the driver handles that well.
Quoted 6 lines
It is no problem to create dozens of misc devices. It makes your driver smaller, contain less code that I have to audit and you have to ensure you got right, and it removes another user of 'struct class' which we are trying to get rid of anyway. So please, move to use a misc device.
Very well. I'll remove that.
Thanks again for your comments. I'll prepare a v3.
Eli
Arnd Bergmann
maintainerRe: [PATCH 2/2] New driver: Xillybus generic interface for FPGA (programmable logic)
Quoted 7 lines · Eli Billauer
The problem is if the page size smaller than 4kB. The buffers allocated by the driver must not cross a 4kB boundary, and it's assumed that anything returned by
__get_free_pages()is 4 kB-aligned. Otherwise the FPGA will generate illegal PCIe packets by crossing that boundary.If the page boundary is bigger than 4k, the driver handles that well.
That should be fine then. We don't really support running Linux on systems with a page size smaller than 4KB anyway, and I think the only ones that ever hard smaller hardware page sizes (iirc m68k or vax) were emulating 4KB pages in software.
Arnd
Re: [PATCH 2/2] New driver: Xillybus generic interface for FPGA (programmable logic)
In reply to Greg KHQuoted 9 lines · Greg KH
Quoted 9 lines
Quoted 8 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
+static struct class
*xillybus_class;Why not just use the misc interface instead of your own class?
When Xillybus is used, the whole system's mission is usually around it (e.g. it's a computer doing data acquisition through the Xillybus pipes). So giving it a high profile makes sense, I believe. Besides, a dozen of device files are not rare.
Quoted 6 lines
It is no problem to create dozens of misc devices. It makes your driver smaller, contain less code that I have to audit and you have to ensure you got right, and it removes another user of 'struct class' which we are trying to get rid of anyway. So please, move to use a misc device.
It has just occurred to me that DYNAMIC_MINORS is 64 (drivers/char/misc.c), so I guess that limits the number of misc devices that can be generated, at least with dynamically allocated minors. I previously mentioned "a dozen" as the number of devices, but I've already run tests with 100+ devices, and I can also think of a sane application for that.
So if I understood the situation correctly, it looks like using misc devices will create a limitation which will be reached sooner or later.
Any suggestion what to do?
Thanks in advance,
Eli
Greg KH
maintainerRe: [PATCH 2/2] New driver: Xillybus generic interface for FPGA (programmable logic)
Quoted 21 lines · Eli Billauer
Quoted 11 lines · Greg KH
Quoted 6 lines
Quoted 6 lines
Quoted 2 lines
Quoted 2 lines
Quoted 1 line
Quoted 1 line
+static struct class
*xillybus_class;Why not just use the misc interface instead of your own class?
When Xillybus is used, the whole system's mission is usually around it (e.g. it's a computer doing data acquisition through the Xillybus pipes). So giving it a high profile makes sense, I believe. Besides, a dozen of device files are not rare.
It is no problem to create dozens of misc devices. It makes your driver smaller, contain less code that I have to audit and you have to ensure you got right, and it removes another user of 'struct class' which we are trying to get rid of anyway. So please, move to use a misc device.
It has just occurred to me that
DYNAMIC_MINORSis 64 (drivers/char/misc.c), so I guess that limits the number of misc devices that can be generated, at least with dynamically allocated minors. I previously mentioned "a dozen" as the number of devices, but I've already run tests with 100+ devices, and I can also think of a sane application for that.
Quoted 5 lines
So if I understood the situation correctly, it looks like using misc devices will create a limitation which will be reached sooner or later.
Any suggestion what to do?
Given that I don't really understand how you can have that many device nodes, because I don't know what they all seem to be needed for, I can't answer this question.
Again, any hints on the user/kernel api you use/need here? Does it really have to be device nodes? What's wrong with the simple firmware interface the kernel provides?
thanks,
greg k-h
Re: [PATCH 2/2] New driver: Xillybus generic interface for FPGA (programmable logic)
Quoted 33 lines · Greg KH
Quoted 25 lines · Eli Billauer
Quoted 9 lines · Greg KH
Quoted 9 lines
Quoted 9 lines
Quoted 4 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
+static struct class
*xillybus_class;Why not just use the misc interface instead of your own class?
When Xillybus is used, the whole system's mission is usually around it (e.g. it's a computer doing data acquisition through the Xillybus pipes). So giving it a high profile makes sense, I believe. Besides, a dozen of device files are not rare.
Quoted 6 lines
It is no problem to create dozens of misc devices. It makes your driver smaller, contain less code that I have to audit and you have to ensure you got right, and it removes another user of 'struct class' which we are trying to get rid of anyway. So please, move to use a misc device.
It has just occurred to me that
DYNAMIC_MINORSis 64 (drivers/char/misc.c), so I guess that limits the number of misc devices that can be generated, at least with dynamically allocated minors. I previously mentioned "a dozen" as the number of devices, but I've already run tests with 100+ devices, and I can also think of a sane application for that.Quoted 6 lines
So if I understood the situation correctly, it looks like using misc devices will create a limitation which will be reached sooner or later.
Any suggestion what to do?
Quoted 9 lines
Given that I don't really understand how you can have that many device nodes, because I don't know what they all seem to be needed for, I can't answer this question.
Again, any hints on the user/kernel api you use/need here? Does it really have to be device nodes? What's wrong with the simple firmware interface the kernel provides?
I'm currently writing some documentation which will cover the API and also help reading the code, I hope. It takes some time...
Until it's done, let's look at a usage example: Suppose that the FPGA's application is to receive a high-speed bitstream with time multiplexed data, demultiplex the bitstream into individual channel streams, and send each channel's data to the host. And let's say that there are 64 channels in original bitstream. So the FPGA has now 64 independent sources of data.
For that purpose, the Xillybus IP core (on the FPGA) is configured to create 64 pipes for FPGA to host communication. The names of these pipes (say, "chan00", "chan01", ...) are also stored in the FPGA.
When the driver starts, it queries the FPGA for its Xillybus configuration, and creates 64 device nodes: /dev/xillybus_chan00, /dev/xillybus_chan01, ... /dev/xillybus_chan63.
If the user wants to dump the data in channel 43 into a file, it's just:
$ cat /dev/xillybus_chan43 > mydump.dat
I hope this clarified things a bit.
I can't see how the firmware interface would help here.
Regards,
Eli
Quoted 5 lines
thanks,
greg k-h
Arnd Bergmann
maintainerRe: [PATCH 2/2] New driver: Xillybus generic interface for FPGA (programmable logic)
Quoted 18 lines · Eli Billauer
I'm currently writing some documentation which will cover the API and also help reading the code, I hope. It takes some time...
Until it's done, let's look at a usage example: Suppose that the FPGA's application is to receive a high-speed bitstream with time multiplexed data, demultiplex the bitstream into individual channel streams, and send each channel's data to the host. And let's say that there are 64 channels in original bitstream. So the FPGA has now 64 independent sources of data.
For that purpose, the Xillybus IP core (on the FPGA) is configured to create 64 pipes for FPGA to host communication. The names of these pipes (say, "chan00", "chan01", ...) are also stored in the FPGA.
When the driver starts, it queries the FPGA for its Xillybus configuration, and creates 64 device nodes:
/dev/xillybus_chan00,/dev/xillybus_chan01, .../dev/xillybus_chan63.
Quoted 7 lines
If the user wants to dump the data in channel 43 into a file, it's just:
$ cat /dev/xillybus_chan43 > mydump.datI hope this clarified things a bit.
I can't see how the firmware interface would help here.
I think a lot of us (including Greg and me) were confused about the purpose of the driver, since you did not include much documentation.
The request_firmware interface would be useful for loading a model into the FPGA, but that doesn't seem to be what your driver is concerned with. It's also a bit confusing because it doesn't appear to be a "bus" in the Linux sense of being something that provides an abstract interface between hardware and kernel device drivers.
Instead, you just have a user interface for those FPGA models that don't need a kernel level driver themselves. This is something that sits on a somewhat higher level -- if we want a generic FPGA interface, this would not be directly connected to a PCI or AMBA bus, but instead connect to an FPGA bus that still needs to be invented.
In the user interface side that you provide seems to be on the same interface level as the USB passthrough interface implemented in drivers/usb/core/devio.c, which has a complex set of ioctls but does serve a very similar purpose. Greg may want to comment on whether that is actually a good interface or not, since I assume he has some experience with how well it worked for USB.
My feeling for now is that we actually need both an in-kernel interface and a user interface, with the complication that the hardware should not care which of the two is used for a particular instance. For the user interface, something that is purely read/write based is really nice, though I wonder if using debugfs or sysfs for this would be more appropriate than having lots of character devices for a single piece of hardware.
Arnd
Re: [PATCH 2/2] New driver: Xillybus generic interface for FPGA (programmable logic)
Quoted 22 lines · Arnd Bergmann
Quoted 20 lines · Eli Billauer
I'm currently writing some documentation which will cover the API and also help reading the code, I hope. It takes some time...
Until it's done, let's look at a usage example: Suppose that the FPGA's application is to receive a high-speed bitstream with time multiplexed data, demultiplex the bitstream into individual channel streams, and send each channel's data to the host. And let's say that there are 64 channels in original bitstream. So the FPGA has now 64 independent sources of data.
For that purpose, the Xillybus IP core (on the FPGA) is configured to create 64 pipes for FPGA to host communication. The names of these pipes (say, "chan00", "chan01", ...) are also stored in the FPGA.
When the driver starts, it queries the FPGA for its Xillybus configuration, and creates 64 device nodes:
/dev/xillybus_chan00,/dev/xillybus_chan01, .../dev/xillybus_chan63.If the user wants to dump the data in channel 43 into a file, it's just:
Quoted 9 lines
Quoted 6 lines
$ cat /dev/xillybus_chan43> mydump.datI hope this clarified things a bit.
I can't see how the firmware interface would help here.
I think a lot of us (including Greg and me) were confused about the purpose of the driver, since you did not include much documentation.
I'm really sorry about this. I begin to realize the confusion now, and Xillybus is indeed not a bus.
Quoted 3 lines
The
request_firmwareinterface would be useful for loading a model into the FPGA, but that doesn't seem to be what your driver is concerned with.
Indeed, Xillybus is not about loading the configuration bitstream for the FPGA.
Quoted 6 lines
It's also a bit confusing because it doesn't appear
to be a "bus" in the Linux sense of being something that provides an abstract interface between hardware and kernel device drivers.Instead, you just have a user interface for those FPGA models that don't need a kernel level driver themselves.
I'm not sure I would agree on that. Xillybus consists of an IP core (sort-of library function for an FPGA), and a driver. At the OS level, it's no different than any PCI card and its driver. I call it "generic" because it's not tailored to transport a certain kind of data (say, audio samples or video frames).
In the FPGA world, passing data to or from a processor is a project in itself, in particular if the latter runs a fullblown operating system. What Xillybus does, is supplying a simple interface on both sides: A hardware FIFO on the logic side for the FPGA designer to interface with, and a plain device file on the host's side. The whole point of this project is to make everything simple and intuitive.
Quoted 6 lines
This is something
that sits on a somewhat higher level -- if we want a generic FPGA interface, this would not be directly connected to a PCI or AMBA bus, but instead connect to an FPGA bus that still needs to be invented.
For what it's worth, the driver is now divided into three parts: A xillybus_core, a module for PCIe and a module for Open Firmware interface. The two latter depend on the first, of course.
Quoted 11 lines
In the user interface side that you provide seems to be on the same interface level as the USB passthrough interface implemented in
drivers/usb/core/devio.c, which has a complex set of ioctls but does serve a very similar purpose. Greg may want to comment on whether that is actually a good interface or not, since I assume he has some experience with how well it worked for USB.My feeling for now is that we actually need both an in-kernel interface and a user interface, with the complication that the hardware should not care which of the two is used for a particular instance.
I'm not sure what you meant here, but I'll mention this: FPGA designers using the IP core don't need to care what the transport is, PCIe, AMBA or anything else. They just see a FIFO. Neither is the host influenced by this, except for loading a different front end module.
Quoted 5 lines
For the user interface, something that is purely read/write
based is really nice, though I wonder if using debugfs or sysfs for this would be more appropriate than having lots of character devices for a single piece of hardware.
And this is where the term "hardware" becomes elusive with an FPGA: One could look at the entire FPGA chip as a single piece of hardware, and expect everything to be packed into a few device nodes.
Or, one could look at each of the hardware FIFOs in the FPGA as something like a sound card, an independent piece of hardware, which is the way I chose to look at it. That's why I allocated a character device for each.
Since the project has been in use by others for about a year (academic users and in the industry), I know at this point that the user interface is convenient to work with (judging from feedback I received). So I would be quite reluctant to make radical changes in the user interface, in particular knowing that it works well and makes UNIX guys feel at home.
Regards,
Eli
Quoted 3 lines
Arnd
Arnd Bergmann
maintainerRe: [PATCH 2/2] New driver: Xillybus generic interface for FPGA (programmable logic)
Quoted 15 lines · Eli Billauer
Quoted 7 lines · Arnd Bergmann
On Tuesday 04 December 2012, Eli Billauer wrote:
It's also a bit confusing because it doesn't appear
to be a "bus" in the Linux sense of being something that provides an abstract interface between hardware and kernel device drivers.Instead, you just have a user interface for those FPGA models that don't need a kernel level driver themselves.
I'm not sure I would agree on that. Xillybus consists of an IP core (sort-of library function for an FPGA), and a driver. At the OS level, it's no different than any PCI card and its driver. I call it "generic" because it's not tailored to transport a certain kind of data (say, audio samples or video frames).
Quoted 6 lines
In the FPGA world, passing data to or from a processor is a project in itself, in particular if the latter runs a fullblown operating system. What Xillybus does, is supplying a simple interface on both sides: A hardware FIFO on the logic side for the FPGA designer to interface with, and a plain device file on the host's side. The whole point of this project is to make everything simple and intuitive.
The problem with this approach is that it cannot be used to provide standard OS interfaces: when you have an audio/video device implemented in an FPGA, all Linux applications expect to use the alsa and v4l interfaces, not xillybus, which means you need a kernel-level driver. For special-purpose applications, having a generic kernel-level driver and a custom user application works fine, but you don't save any complexity for a lot of other use cases, you just move it somewhere else by requiring a redesign of existing user applications, which is often not a reasonable approach.
Quoted 9 lines
Quoted 6 lines
This is something
that sits on a somewhat higher level -- if we want a generic FPGA interface, this would not be directly connected to a PCI or AMBA bus, but instead connect to an FPGA bus that still needs to be invented.For what it's worth, the driver is now divided into three parts: A
xillybus_core, a module for PCIe and a module for Open Firmware interface. The two latter depend on the first, of course.
Ok, that is certainly a good step in the right direction.
Quoted 12 lines
Quoted 11 lines
In the user interface side that you provide seems to be on the same interface level as the USB passthrough interface implemented in
drivers/usb/core/devio.c, which has a complex set of ioctls but does serve a very similar purpose. Greg may want to comment on whether that is actually a good interface or not, since I assume he has some experience with how well it worked for USB.My feeling for now is that we actually need both an in-kernel interface and a user interface, with the complication that the hardware should not care which of the two is used for a particular instance.
Quoted 4 lines
I'm not sure what you meant here, but I'll mention this: FPGA designers using the IP core don't need to care what the transport is, PCIe, AMBA or anything else. They just see a FIFO. Neither is the host influenced by this, except for loading a different front end module.
I mean some IP cores can use your driver just fine, while other IP cores require a driver that interfaces with a kernel subsystem (alsa, v4l, network, iio, etc). Whether xillybus is a good design choice for those IP cores is a different question, but for all I can tell, it would be entirely possible to implement an ethernet adapter based on this, as long as it can interface to the kernel.
Quoted 5 lines
Quoted 5 lines
For the user interface, something that is purely read/write
based is really nice, though I wonder if using debugfs or sysfs for this would be more appropriate than having lots of character devices for a single piece of hardware.
Quoted 8 lines
And this is where the term "hardware" becomes elusive with an FPGA: One could look at the entire FPGA chip as a single piece of hardware, and expect everything to be packed into a few device nodes.
Or, one could look at each of the hardware FIFOs in the FPGA as something like a sound card, an independent piece of hardware, which is the way I chose to look at it. That's why I allocated a character device for each.
Most interfaces we have in the kernel are on a larger scale. E.g. a network adapter is a single instance rather than an input and an output queue.
Quoted 5 lines
Since the project has been in use by others for about a year (academic users and in the industry), I know at this point that the user interface is convenient to work with (judging from feedback I received). So I would be quite reluctant to make radical changes in the user interface, in particular knowing that it works well and makes UNIX guys feel at home.
Changing to sysfs or debugfs is not a radical change: you would still have multiple nodes in a file system that each represent a queue, but rather than using a flat name space under /dev, they would be hierarchical with a directory per physical device (e.g. one FPGA).
Arnd
Re: [PATCH 2/2] New driver: Xillybus generic interface for FPGA (programmable logic)
Quoted 25 lines · Arnd Bergmann
Quoted 16 lines · Eli Billauer
Quoted 8 lines · Arnd Bergmann
On Tuesday 04 December 2012, Eli Billauer wrote:
It's also a bit confusing because it doesn't appear
to be a "bus" in the Linux sense of being something that provides an abstract interface between hardware and kernel device drivers.Instead, you just have a user interface for those FPGA models that don't need a kernel level driver themselves.
I'm not sure I would agree on that. Xillybus consists of an IP core (sort-of library function for an FPGA), and a driver. At the OS level, it's no different than any PCI card and its driver. I call it "generic" because it's not tailored to transport a certain kind of data (say, audio samples or video frames).
Quoted 7 lines
In the FPGA world, passing data to or from a processor is a project in itself, in particular if the latter runs a fullblown operating system. What Xillybus does, is supplying a simple interface on both sides: A hardware FIFO on the logic side for the FPGA designer to interface with, and a plain device file on the host's side. The whole point of this project is to make everything simple and intuitive.
Quoted 12 lines
The problem with this approach is that it cannot be used to provide standard OS interfaces: when you have an audio/video device implemented in an FPGA, all Linux applications expect to use the alsa and v4l interfaces, not xillybus, which means you need a kernel-level driver. For special-purpose applications, having a generic kernel-level driver and a custom user application works fine, but you don't save any complexity for a lot of other use cases, you just move it somewhere else by requiring a redesign of existing user applications, which is often not a reasonable approach.
Xillybus is there exactly for special-purpose applications. In fact, the main reason people turn to FPGAs is because there are no general-purpose chips to do the job.
Besides, if the FPGA implements a well-known function (e.g. a video card) there is no reason to treat it differently IMHO. For example, drivers/video/xilinxfb.c, drivers/tty/serial/xilinx_uartps.c work only with Xilinx' IP cores, and they're mixed with the "hardware" drivers.
It's when it doesn't make sense to represent the FPGA logic as something standard, that Xillybus comes in. Even if the reason is not being ready to spend the effort.
Quoted 6 lines
Quoted 6 lines
I'm not sure what you meant here, but I'll mention this: FPGA designers using the IP core don't need to care what the transport is, PCIe, AMBA or anything else. They just see a FIFO. Neither is the host influenced by this, except for loading a different front end module.
Quoted 9 lines
I mean some IP cores can use your driver just fine, while other IP cores require a driver that interfaces with a kernel subsystem (alsa, v4l, network, iio, etc). Whether xillybus is a good design choice for those IP cores is a different question, but for all I can tell, it would be entirely possible to implement an ethernet adapter based on this, as long as it can interface to the kernel.
Xillybus' strength is its simplicity in sending plain streams of data. If the data is looped back into the kernel to implement a network interface, that's indeed possible. As for dedicated interfaces, I'll say this: I wrote a simple video adapter for Zynq lately. To some extent, the logic is based upon things I took from Xillybus, but no more than some basic blocks. As for the driver, I started from a completely different one.
What I'm trying to say, is that it's possible to implement dedicated functions based upon Xillybus, but in practice it doesn't make much sense.
Quoted 25 lines
Quoted 6 lines
Quoted 6 lines
For the user interface, something that is purely read/write
based is really nice, though I wonder if using debugfs or sysfs for this would be more appropriate than having lots of character devices for a single piece of hardware.Quoted 9 lines
And this is where the term "hardware" becomes elusive with an FPGA: One could look at the entire FPGA chip as a single piece of hardware, and expect everything to be packed into a few device nodes.
Or, one could look at each of the hardware FIFOs in the FPGA as something like a sound card, an independent piece of hardware, which is the way I chose to look at it. That's why I allocated a character device for each.
Most interfaces we have in the kernel are on a larger scale. E.g. a network adapter is a single instance rather than an input and an output queue.
Quoted 6 lines
Since the project has been in use by others for about a year (academic users and in the industry), I know at this point that the user interface is convenient to work with (judging from feedback I received). So I would be quite reluctant to make radical changes in the user interface, in particular knowing that it works well and makes UNIX guys feel at home.
Quoted 5 lines
Changing to sysfs or debugfs is not a radical change: you would still have multiple nodes in a file system that each represent a queue, but rather than using a flat name space under /dev, they would be hierarchical with a directory per physical device (e.g. one FPGA).
Just to make sure we're on the same page: The Xillybus char devices need to pass bulks of data efficiently (not just attributes), and also support .poll and .llseek methods (I thought that sysfs was for small bites of info).
I understand that you suggest that instead of polluting /dev, I should relocate my device files to /sys/something...?
Could you please point at a driver in the kernel tree that does this correctly, so I can imitate it? And under what directory would it make sense to put it? I'm not so familiar with sysfs.
Thanks,
Eli
Greg KH
maintainerRe: [PATCH 2/2] New driver: Xillybus generic interface for FPGA (programmable logic)
In reply to Arnd BergmannQuoted 27 lines · Arnd Bergmann
Quoted 18 lines · Eli Billauer
I'm currently writing some documentation which will cover the API and also help reading the code, I hope. It takes some time...
Until it's done, let's look at a usage example: Suppose that the FPGA's application is to receive a high-speed bitstream with time multiplexed data, demultiplex the bitstream into individual channel streams, and send each channel's data to the host. And let's say that there are 64 channels in original bitstream. So the FPGA has now 64 independent sources of data.
For that purpose, the Xillybus IP core (on the FPGA) is configured to create 64 pipes for FPGA to host communication. The names of these pipes (say, "chan00", "chan01", ...) are also stored in the FPGA.
When the driver starts, it queries the FPGA for its Xillybus configuration, and creates 64 device nodes:
/dev/xillybus_chan00,/dev/xillybus_chan01, .../dev/xillybus_chan63.Quoted 7 lines
If the user wants to dump the data in channel 43 into a file, it's just:
$ cat /dev/xillybus_chan43 > mydump.datI hope this clarified things a bit.
I can't see how the firmware interface would help here.
Quoted 8 lines
I think a lot of us (including Greg and me) were confused about the purpose of the driver, since you did not include much documentation.
The
request_firmwareinterface would be useful for loading a model into the FPGA, but that doesn't seem to be what your driver is concerned with. It's also a bit confusing because it doesn't appear to be a "bus" in the Linux sense of being something that provides an abstract interface between hardware and kernel device drivers.
Yes, that's what I was confused about as well. I'll wait for a new patch with new documentation, before commenting further.
greg k-h
Arnd Bergmann
maintainerRe: [PATCH 2/2] New driver: Xillybus generic interface for FPGA (programmable logic)
In reply to Eli BillauerQuoted 16 lines · Eli Billauer
Xillybus is a general-purpose framework for communication between programmable logic (FPGA) and a host. It provides a simple connection between hardware FIFOs in the FPGA and their respective device files on the host. The user space programming model is like piping data from or to the FPGA.
The underlying transport between the host and FPGA is either PCIe or AXI (AMBA bus by ARM).
The Xillybus logic (IP core) is configurable in the number of pipes it presents and their nature. The driver autodetects these pipes, making it essentially forward-compatible to future configurations. The benefit of having this driver enabled in the kernel is that hardware vendors may release a new card, knowing that it will work out of the box on any future Linux machine, with the specific configuration defined for the FPGA part.
Quoted 2 lines
This driver has been available for download for over a year, and has been actively used on a wide variety of kernels versions and configurations.
I have a much higher-level comment on this driver: There seem to be a number of parties that are interested in having reprogrammable logic available in Linux and that will want to merge their drivers. I'm aware of these other people that must have some interest (and one person I can't mention here because of NDA):
Philip Balister <[email protected]> (OpenSDR) Dinh Nguyen <[email protected]> (ARM SOCFPGA maintainer) Pavel Machek <[email protected]> (SOCFPGA contributor) John Linn <[email protected]> (Zynq maintainer) Michal Simek <[email protected]> (Zynq maintainer) Ira W. Snyder <[email protected]> (Carma driver author)
I believe it is in everybody's interest to define a single kernel-to-user interface that can be used to install a payload in an any of the FPGA implementations, and a common way to probe logical devices that are implemented in that FPGA.
Your driver seems like an good start for such an interface, and I think it would be reasonable to put it into a new drivers/fpga/ subsystems rather than a "misc" driver, but that requires at least the user level interface to be generic enough to cover all the use cases. Ideally, the in-kernel interface would also be generic to allow plugging in additional hardware drivers and allowing to add independent back-ends for functionality based on the model that gets loaded into the FPGA. However, we don't have to do it right away, as we can always change in-kernel interfaces when needed, unlike the user interfaces that have to remain stable.
Arnd
Greg KH
maintainerRe: [PATCH 2/2] New driver: Xillybus generic interface for FPGA (programmable logic)
Quoted 26 lines · Arnd Bergmann
Quoted 16 lines · Eli Billauer
Xillybus is a general-purpose framework for communication between programmable logic (FPGA) and a host. It provides a simple connection between hardware FIFOs in the FPGA and their respective device files on the host. The user space programming model is like piping data from or to the FPGA.
The underlying transport between the host and FPGA is either PCIe or AXI (AMBA bus by ARM).
The Xillybus logic (IP core) is configurable in the number of pipes it presents and their nature. The driver autodetects these pipes, making it essentially forward-compatible to future configurations. The benefit of having this driver enabled in the kernel is that hardware vendors may release a new card, knowing that it will work out of the box on any future Linux machine, with the specific configuration defined for the FPGA part.
Quoted 2 lines
This driver has been available for download for over a year, and has been actively used on a wide variety of kernels versions and configurations.
I have a much higher-level comment on this driver: There seem to be a number of parties that are interested in having reprogrammable logic available in Linux and that will want to merge their drivers. I'm aware of these other people that must have some interest (and one person I can't mention here because of NDA):
Quoted 6 lines
Philip Balister <[email protected]> (OpenSDR) Dinh Nguyen <[email protected]> (ARM SOCFPGA maintainer) Pavel Machek <[email protected]> (SOCFPGA contributor) John Linn <[email protected]> (Zynq maintainer) Michal Simek <[email protected]> (Zynq maintainer) Ira W. Snyder <[email protected]> (Carma driver author)
Yes, I know of at least one more device other than the ones listed above that wants this type of functionality as well, so defining it in a standard user/kernel api manner would be very good to do.
thanks,
greg k-h
Re: [PATCH 2/2] New driver: Xillybus generic interface for FPGA (programmable logic)
Quoted 34 lines · Greg KH
Quoted 26 lines · Arnd Bergmann
Quoted 16 lines · Eli Billauer
Xillybus is a general-purpose framework for communication between programmable logic (FPGA) and a host. It provides a simple connection between hardware FIFOs in the FPGA and their respective device files on the host. The user space programming model is like piping data from or to the FPGA.
The underlying transport between the host and FPGA is either PCIe or AXI (AMBA bus by ARM).
The Xillybus logic (IP core) is configurable in the number of pipes it presents and their nature. The driver autodetects these pipes, making it essentially forward-compatible to future configurations. The benefit of having this driver enabled in the kernel is that hardware vendors may release a new card, knowing that it will work out of the box on any future Linux machine, with the specific configuration defined for the FPGA part.
Quoted 2 lines
This driver has been available for download for over a year, and has been actively used on a wide variety of kernels versions and configurations.
I have a much higher-level comment on this driver: There seem to be a number of parties that are interested in having reprogrammable logic available in Linux and that will want to merge their drivers. I'm aware of these other people that must have some interest (and one person I can't mention here because of NDA):
Quoted 6 lines
Philip Balister <[email protected]> (OpenSDR) Dinh Nguyen <[email protected]> (ARM SOCFPGA maintainer) Pavel Machek <[email protected]> (SOCFPGA contributor) John Linn <[email protected]> (Zynq maintainer) Michal Simek <[email protected]> (Zynq maintainer) Ira W. Snyder <[email protected]> (Carma driver author)
Quoted 3 lines
Yes, I know of at least one more device other than the ones listed above that wants this type of functionality as well, so defining it in a standard user/kernel api manner would be very good to do.
I'm concerned that a standard driver for FPGA's will be a very difficult problem.
The Xillybus driver looks interesting on several levels, however my first concern is depends on a FPGA IP block that is not open source. This is not a bad thing, just a potential obstacle for some people.
I've been engaged in design discussions today with my customer. Our target is the Xilinx Zynq hardware. The first pass at a driver focuses on creating the minimal amount of code in the kernel doing most of the logic in user space. So the driver code allocates a large chunk of RAM for the FPGA to read/write to, provides a mmap function so user space can see this RAM, also mmaps in the address space of an AXI slave so the user space can control the logic. This approach has no dependencies on what is loaded into the fpga.
This is a very different approach then the Xillybus driver, but should also be useful to a large class of people. Hopefully, we can converge on a set of useful drivers, and not end up with a million drivers all based on custom fpga configuration :)
Philip
Quoted 6 lines
thanks,
greg k-h
Greg KH
maintainerRe: [PATCH 2/2] New driver: Xillybus generic interface for FPGA (programmable logic)
Quoted 39 lines · Philip Balister
Quoted 34 lines · Greg KH
Quoted 26 lines · Arnd Bergmann
Quoted 16 lines · Eli Billauer
Xillybus is a general-purpose framework for communication between programmable logic (FPGA) and a host. It provides a simple connection between hardware FIFOs in the FPGA and their respective device files on the host. The user space programming model is like piping data from or to the FPGA.
The underlying transport between the host and FPGA is either PCIe or AXI (AMBA bus by ARM).
The Xillybus logic (IP core) is configurable in the number of pipes it presents and their nature. The driver autodetects these pipes, making it essentially forward-compatible to future configurations. The benefit of having this driver enabled in the kernel is that hardware vendors may release a new card, knowing that it will work out of the box on any future Linux machine, with the specific configuration defined for the FPGA part.
Quoted 2 lines
This driver has been available for download for over a year, and has been actively used on a wide variety of kernels versions and configurations.
I have a much higher-level comment on this driver: There seem to be a number of parties that are interested in having reprogrammable logic available in Linux and that will want to merge their drivers. I'm aware of these other people that must have some interest (and one person I can't mention here because of NDA):
Quoted 6 lines
Philip Balister <[email protected]> (OpenSDR) Dinh Nguyen <[email protected]> (ARM SOCFPGA maintainer) Pavel Machek <[email protected]> (SOCFPGA contributor) John Linn <[email protected]> (Zynq maintainer) Michal Simek <[email protected]> (Zynq maintainer) Ira W. Snyder <[email protected]> (Carma driver author)
Quoted 3 lines
Yes, I know of at least one more device other than the ones listed above that wants this type of functionality as well, so defining it in a standard user/kernel api manner would be very good to do.
Quoted 6 lines
I'm concerned that a standard driver for FPGA's will be a very difficult problem.
The Xillybus driver looks interesting on several levels, however my first concern is depends on a FPGA IP block that is not open source. This is not a bad thing, just a potential obstacle for some people.
As long as that doesn't affect the kernel code, I don't see the obstacle here. What am I missing?
Quoted 8 lines
I've been engaged in design discussions today with my customer. Our target is the Xilinx Zynq hardware. The first pass at a driver focuses on creating the minimal amount of code in the kernel doing most of the logic in user space. So the driver code allocates a large chunk of RAM for the FPGA to read/write to, provides a mmap function so user space can see this RAM, also mmaps in the address space of an AXI slave so the user space can control the logic. This approach has no dependencies on what is loaded into the fpga.
Would a simple UIO driver work best for this type of arrangement? Then those types of hardware wouldn't even need to mess with a fpga-type interface.
Quoted 4 lines
This is a very different approach then the Xillybus driver, but should also be useful to a large class of people. Hopefully, we can converge on a set of useful drivers, and not end up with a million drivers all based on custom fpga configuration :)
Odds are, this should look something like the firmware interface in the end, right? Userspace dumps a bunch of data to the device, and then needs the driver to toggle some bits somewhere to enable the device. Also, a few control calls like clearing the device, and other minor things should be all that is needed, right?
So, in the grand tradition of, "The first one there wins", why not base it all off of your driver, and how that works, and we can go from there :)
thanks,
greg k-h
Greg KH
maintainerRe: [PATCH 2/2] New driver: Xillybus generic interface for FPGA (programmable logic)
Quoted 2 lines · Greg KH
So, in the grand tradition of, "The first one there wins", why not base it all off of your driver, and how that works, and we can go from there :)
Oops, sorry s/your/Eli's/, my confusion.
greg k-h
Re: [PATCH 2/2] New driver: Xillybus generic interface for FPGA (programmable logic)
In reply to Greg KHQuoted 43 lines · Greg KH
Quoted 42 lines · Philip Balister
Quoted 34 lines · Greg KH
Quoted 26 lines · Arnd Bergmann
Quoted 16 lines · Eli Billauer
Xillybus is a general-purpose framework for communication between programmable logic (FPGA) and a host. It provides a simple connection between hardware FIFOs in the FPGA and their respective device files on the host. The user space programming model is like piping data from or to the FPGA.
The underlying transport between the host and FPGA is either PCIe or AXI (AMBA bus by ARM).
The Xillybus logic (IP core) is configurable in the number of pipes it presents and their nature. The driver autodetects these pipes, making it essentially forward-compatible to future configurations. The benefit of having this driver enabled in the kernel is that hardware vendors may release a new card, knowing that it will work out of the box on any future Linux machine, with the specific configuration defined for the FPGA part.
Quoted 2 lines
This driver has been available for download for over a year, and has been actively used on a wide variety of kernels versions and configurations.
I have a much higher-level comment on this driver: There seem to be a number of parties that are interested in having reprogrammable logic available in Linux and that will want to merge their drivers. I'm aware of these other people that must have some interest (and one person I can't mention here because of NDA):
Quoted 6 lines
Philip Balister <[email protected]> (OpenSDR) Dinh Nguyen <[email protected]> (ARM SOCFPGA maintainer) Pavel Machek <[email protected]> (SOCFPGA contributor) John Linn <[email protected]> (Zynq maintainer) Michal Simek <[email protected]> (Zynq maintainer) Ira W. Snyder <[email protected]> (Carma driver author)
Quoted 3 lines
Yes, I know of at least one more device other than the ones listed above that wants this type of functionality as well, so defining it in a standard user/kernel api manner would be very good to do.
I'm concerned that a standard driver for FPGA's will be a very difficult problem.
Quoted 6 lines
Quoted 3 lines
The Xillybus driver looks interesting on several levels, however my first concern is depends on a FPGA IP block that is not open source. This is not a bad thing, just a potential obstacle for some people.
As long as that doesn't affect the kernel code, I don't see the obstacle here. What am I missing?
Nothing. The Xillybus approach is valid. It just depends on their piece of fpga code. If you are not willing to license that, you will need to reverse engineer their fpga code, or sues a different (equally valid) method to communicate with the fpga. Think of their fpga blob as a specific implementation of a network card. Not all network cards expose the same hardware interface.
The important thing to realize is that the fpga is user configurable. It is possible to create an infinite number of possible "devices" in it that can use any number of device drivers in Linux.
Think of it a a soc, with a large part that the end user can customize with his own IP. In the Xilly case, they have created a custom piece of hardware in the fpga that needs a driver with a specific interface to users space.
Quoted 10 lines
Quoted 9 lines
I've been engaged in design discussions today with my customer. Our target is the Xilinx Zynq hardware. The first pass at a driver focuses on creating the minimal amount of code in the kernel doing most of the logic in user space. So the driver code allocates a large chunk of RAM for the FPGA to read/write to, provides a mmap function so user space can see this RAM, also mmaps in the address space of an AXI slave so the user space can control the logic. This approach has no dependencies on what is loaded into the fpga.
Quoted 4 lines
Would a simple UIO driver work best for this type of arrangement? Then those types of hardware wouldn't even need to mess with a fpga-type interface.
It is very close to a UIO approach. My key problem I am trying to solve is getting a "large" buffer space for the driver and a way to communicate the physical address and size of the buffer to the fpga. The fpga has several ports that allow it to directly read/write RAM, but it needs to know physical addresses.
Quoted 11 lines
Quoted 4 lines
This is a very different approach then the Xillybus driver, but should also be useful to a large class of people. Hopefully, we can converge on a set of useful drivers, and not end up with a million drivers all based on custom fpga configuration :)
Odds are, this should look something like the firmware interface in the end, right? Userspace dumps a bunch of data to the device, and then needs the driver to toggle some bits somewhere to enable the device. Also, a few control calls like clearing the device, and other minor things should be all that is needed, right?
For zynq, there is an out of tree ( :) ) driver that handles the firmware loading. This is an independent problem.
Quoted 3 lines
So, in the grand tradition of, "The first one there wins", why not base it all off of your driver, and how that works, and we can go from there :)
I do not think there will be any one driver that will work for all use cases due to the variety of devices that can be implemented in the fpga.
Philip
Quoted 5 lines
thanks,
greg k-h
Re: [PATCH 2/2] New driver: Xillybus generic interface for FPGA (programmable logic)
Quoted 2 lines · Philip Balister
Quoted 1 line · Greg KH
On Fri, Nov 30, 2012 at 07:19:16PM -0800, Philip Balister wrote:
[..]
Quoted 32 lines
Quoted 10 lines
Quoted 9 lines
I've been engaged in design discussions today with my customer. Our target is the Xilinx Zynq hardware. The first pass at a driver focuses on creating the minimal amount of code in the kernel doing most of the logic in user space. So the driver code allocates a large chunk of RAM for the FPGA to read/write to, provides a mmap function so user space can see this RAM, also mmaps in the address space of an AXI slave so the user space can control the logic. This approach has no dependencies on what is loaded into the fpga.
Quoted 3 lines
Would a simple UIO driver work best for this type of arrangement? Then those types of hardware wouldn't even need to mess with a fpga-type interface.
It is very close to a UIO approach. My key problem I am trying to solve is getting a "large" buffer space for the driver and a way to communicate the physical address and size of the buffer to the fpga. The fpga has several ports that allow it to directly read/write RAM, but it needs to know physical addresses.
Quoted 11 lines
Quoted 4 lines
This is a very different approach then the Xillybus driver, but should also be useful to a large class of people. Hopefully, we can converge on a set of useful drivers, and not end up with a million drivers all based on custom fpga configuration :)
Odds are, this should look something like the firmware interface in the end, right? Userspace dumps a bunch of data to the device, and then needs the driver to toggle some bits somewhere to enable the device. Also, a few control calls like clearing the device, and other minor things should be all that is needed, right?
Quoted 8 lines
For zynq, there is an out of tree ( :) ) driver that handles the firmware loading. This is an independent problem.
Quoted 2 lines
So, in the grand tradition of, "The first one there wins", why not base it all off of your driver, and how that works, and we can go from there :)
I do not think there will be any one driver that will work for all use cases due to the variety of devices that can be implemented in the fpga.
I see this working such that each IP block/independent logic unit in the FPGA bitfile would be reified as an independent device to the kernel, perhaps as a child of an 'fpga' bus (or similar). These devices would have their own drivers, and would come and go as the logic is reprogrammed.
This is really useful if you're taking and integrating IP that exposes a known interface with existing in-kernel Linux drivers. Think a soft Ethernet MAC IP or a 16550 or something.
For more application specific things, or lower level/less defined IO like you described, an instance of a UIO driver might fit well. (Anything more complicated then simple register accesses or interrupts might require something more.)
The key point here is that there is a need for a common reprogrammable logic subsystem that provides an abstraction over the various reprogrammable logic devices and exposes a common user interfaces for download and configuration.
This user interface would need to encompass a way to hand the kernel a blob and a description of the IP contained within. Grant's recent dynamic device tree work seems like would be a great fit for this (and infact, I believe this use case was explicitly listed as a userstory).
Josh
Arnd Bergmann
maintainerRe: [PATCH 2/2] New driver: Xillybus generic interface for FPGA (programmable logic)
In reply to Philip BalisterQuoted 11 lines · Philip Balister
Quoted 3 lines · Greg KH
Yes, I know of at least one more device other than the ones listed above that wants this type of functionality as well, so defining it in a standard user/kernel api manner would be very good to do.
I'm concerned that a standard driver for FPGA's will be a very difficult problem.
The Xillybus driver looks interesting on several levels, however my first concern is depends on a FPGA IP block that is not open source. This is not a bad thing, just a potential obstacle for some people.
I agree that is a concern, but for now, I'm mostly worried about the kernel-to-user interface. If we can agree on a driver interface that works for Xillybus as well as any of the others we know about, we can start using that as the generic kernel FPGA interface.
Once we get a second FPGA driver, that can use the same user interface but talk to the hardware in a different way, and then we can reorganise the code to keep the user interface bits in a common driver, away from the hardware specific parts.
If you see anything in the user interface that directly depends on the Xillybus IP block, then that would make the approach impossible and we should change that to be more generic.
Quoted 9 lines
I've been engaged in design discussions today with my customer. Our target is the Xilinx Zynq hardware. The first pass at a driver focuses on creating the minimal amount of code in the kernel doing most of the logic in user space. So the driver code allocates a large chunk of RAM for the FPGA to read/write to, provides a mmap function so user space can see this RAM, also mmaps in the address space of an AXI slave so the user space can control the logic. This approach has no dependencies on what is loaded into the fpga.
Quoted 4 lines
This is a very different approach then the Xillybus driver, but should also be useful to a large class of people. Hopefully, we can converge on a set of useful drivers, and not end up with a million drivers all based on custom fpga configuration :)
Agreed. If I understand you correctly though, your approach is specific to a particular hardware implementation (Zynq) on the user interface layer, which I think is exactly what we should avoid. Obviously, there is always a driver involved that is specific to the IP block you load into an FPGA at runtime, and that is ok. The two parts that I think we should agree on are:
a) How to get a payload into the FPGA
b) How to find a device driver that can make the payload interface to user space.
Arnd
Re: [PATCH 2/2] New driver: Xillybus generic interface for FPGA (programmable logic)
Quoted 5 lines · Arnd Bergmann
I agree that is a concern, but for now, I'm mostly worried about the kernel-to-user interface. If we can agree on a driver interface that works for Xillybus as well as any of the others we know about, we can start using that as the generic kernel FPGA interface.
Quoted 9 lines
Once we get a second FPGA driver, that can use the same user interface but talk to the hardware in a different way, and then we can reorganise the code to keep the user interface bits in a common driver, away from the hardware specific parts.
If you see anything in the user interface that directly depends on the Xillybus IP block, then that would make the approach impossible and we should change that to be more generic.
The whole idea about Xillybus was not to invent a new user space interface. A lot of effort has been put in to make the device files behave like named pipes. For example, it makes perfect sense to go
$ cat mydata > /dev/xillybus_my_data_sink
knowing that the data will arrive as one would expect to the FPGA.
On my next patch submission, I'll include a documentation file, which will describe the fine details. But don't expect much on the API side: All of the configuration is already done on the hardware (FPGA) side, and the driver merely detects the pipes and their attributes.
All that is left for the user space programmer is to exercise common UNIX programming practices, and things will work as expected. Or at least, this is the intention.
Eli
RE: [PATCH 2/2] New driver: Xillybus generic interface for FPGA (programmable logic)
In reply to Arnd BergmannQuoted 56 lines
-----Original Message-----
From: Arnd Bergmann [mailto:[email protected]]
Sent: Saturday, December 01, 2012 12:49 PM
To: Philip Balister
Cc: Greg KH; Eli Billauer; [email protected]; Pavel Machek; John Linn; Michal Simek; Ira W. Snyder; Josh Cartwright
Subject: Re: [PATCH 2/2] New driver: Xillybus generic interface for FPGA (programmable logic)Quoted 11 lines · Philip Balister
Quoted 3 lines · Greg KH
Yes, I know of at least one more device other than the ones listed above that wants this type of functionality as well, so defining it in a standard user/kernel api manner would be very good to do.
I'm concerned that a standard driver for FPGA's will be a very difficult problem.
The Xillybus driver looks interesting on several levels, however my first concern is depends on a FPGA IP block that is not open source. This is not a bad thing, just a potential obstacle for some people.
I agree that is a concern, but for now, I'm mostly worried about the kernel-to-user interface. If we can agree on a driver interface that works for Xillybus as well as any of the others we know about, we can start using that as the generic kernel FPGA interface.
Once we get a second FPGA driver, that can use the same user interface but talk to the hardware in a different way, and then we can reorganise the code to keep the user interface bits in a common driver, away from the hardware specific parts.
If you see anything in the user interface that directly depends on the Xillybus IP block, then that would make the approach impossible and we should change that to be more generic.
Quoted 9 lines
I've been engaged in design discussions today with my customer. Our target is the Xilinx Zynq hardware. The first pass at a driver focuses on creating the minimal amount of code in the kernel doing most of the logic in user space. So the driver code allocates a large chunk of RAM for the FPGA to read/write to, provides a mmap function so user space can see this RAM, also mmaps in the address space of an AXI slave so the user space can control the logic. This approach has no dependencies on what is loaded into the fpga.
Quoted 4 lines
This is a very different approach then the Xillybus driver, but should also be useful to a large class of people. Hopefully, we can converge on a set of useful drivers, and not end up with a million drivers all based on custom fpga configuration :)
Agreed. If I understand you correctly though, your approach is specific to a particular hardware implementation (Zynq) on the user interface layer, which I think is exactly what we should avoid. Obviously, there is always a driver involved that is specific to the IP block you load into an FPGA at runtime, and that is ok. The two parts that I think we should agree on are:
Quoted 4 lines
a) How to get a payload into the FPGA
b) How to find a device driver that can make the payload interface to user space.
Yes I agree with these 2 parts and keeping them separated.
Josh's comments were aligned with where we want to go also with the device tree and Grant's work.
Thanks
John
Quoted 2 lines
Arnd
Re: [PATCH 2/2] New driver: Xillybus generic interface for FPGA (programmable logic)
In reply to Arnd BergmannQuoted 18 lines · Arnd Bergmann
Quoted 11 lines · Philip Balister
Quoted 3 lines · Greg KH
Yes, I know of at least one more device other than the ones listed above that wants this type of functionality as well, so defining it in a standard user/kernel api manner would be very good to do.
I'm concerned that a standard driver for FPGA's will be a very difficult problem.
The Xillybus driver looks interesting on several levels, however my first concern is depends on a FPGA IP block that is not open source. This is not a bad thing, just a potential obstacle for some people.
I agree that is a concern, but for now, I'm mostly worried about the kernel-to-user interface. If we can agree on a driver interface that works for Xillybus as well as any of the others we know about, we can start using that as the generic kernel FPGA interface.
Quoted 4 lines
Once we get a second FPGA driver, that can use the same user interface but talk to the hardware in a different way, and then we can reorganise the code to keep the user interface bits in a common driver, away from the hardware specific parts.
Actually, the user interface will depend on what "code" is loaded into the FPGA (side note, people argue over FPGA's being hardware or software). So it is entirely possible to load an FPGA with a XillyBus device, and ethernet interface, I2S, and the "UIO" appraoch I am working on.
We can use device tree to tell the kernel what drivers are needed, and Josh mentioned some ideas on managing device tree entries combined with the fpga image.
So it is very possible for one fpga to have several device drivers, each using different user interfaces. It is entirely possible to create a device in the fpga that uses an existing hardware interface so you could use the existing Linux driver to control it.
Quoted 18 lines
If you see anything in the user interface that directly depends on the Xillybus IP block, then that would make the approach impossible and we should change that to be more generic.
Quoted 9 lines
I've been engaged in design discussions today with my customer. Our target is the Xilinx Zynq hardware. The first pass at a driver focuses on creating the minimal amount of code in the kernel doing most of the logic in user space. So the driver code allocates a large chunk of RAM for the FPGA to read/write to, provides a mmap function so user space can see this RAM, also mmaps in the address space of an AXI slave so the user space can control the logic. This approach has no dependencies on what is loaded into the fpga.
Quoted 4 lines
This is a very different approach then the Xillybus driver, but should also be useful to a large class of people. Hopefully, we can converge on a set of useful drivers, and not end up with a million drivers all based on custom fpga configuration :)
Quoted 7 lines
Agreed. If I understand you correctly though, your approach is specific to a particular hardware implementation (Zynq) on the user interface layer, which I think is exactly what we should avoid. Obviously, there is always a driver involved that is specific to the IP block you load into an FPGA at runtime, and that is ok. The two parts that I think we should agree on are:
The approach I am taking is not Zynq specific. The only Zynq specific bits will be range checking data from the device tree to make sure the user is mapping address space that is used by the FPGA and a similar validation of the IRQ numbers. I have not looked at the Altera part closely, but I suspect their fpga/processor interface may use similar concepts.
Quoted 2 lines
a) How to get a payload into the FPGA
Just to be clear, what are you calling payload here? Is it the data that is used to configure the fpga, or actual data going back and forth? I'm a little concerned that not everyone understand how flexible the FPGA is.
Philip
Quoted 6 lines
b) How to find a device driver that can make the payload interface to user space.
Arnd
Arnd Bergmann
maintainerRe: [PATCH 2/2] New driver: Xillybus generic interface for FPGA (programmable logic)
Quoted 24 lines · Philip Balister
Quoted 18 lines · Arnd Bergmann
Quoted 11 lines · Philip Balister
Quoted 3 lines · Greg KH
Yes, I know of at least one more device other than the ones listed above that wants this type of functionality as well, so defining it in a standard user/kernel api manner would be very good to do.
I'm concerned that a standard driver for FPGA's will be a very difficult problem.
The Xillybus driver looks interesting on several levels, however my first concern is depends on a FPGA IP block that is not open source. This is not a bad thing, just a potential obstacle for some people.
I agree that is a concern, but for now, I'm mostly worried about the kernel-to-user interface. If we can agree on a driver interface that works for Xillybus as well as any of the others we know about, we can start using that as the generic kernel FPGA interface.
Quoted 4 lines
Once we get a second FPGA driver, that can use the same user interface but talk to the hardware in a different way, and then we can reorganise the code to keep the user interface bits in a common driver, away from the hardware specific parts.
Quoted 4 lines
Actually, the user interface will depend on what "code" is loaded into the FPGA (side note, people argue over FPGA's being hardware or software). So it is entirely possible to load an FPGA with a XillyBus device, and ethernet interface, I2S, and the "UIO" appraoch I am working on.
Yes, absolutely. Or multiple instances each of them even. Or constantly reload the FPGA depending on the application running in user space.
Quoted 8 lines
We can use device tree to tell the kernel what drivers are needed, and Josh mentioned some ideas on managing device tree entries combined with the fpga image.
So it is very possible for one fpga to have several device drivers, each using different user interfaces. It is entirely possible to create a device in the fpga that uses an existing hardware interface so you could use the existing Linux driver to control it.
One implication of using the device tree though is that it's statically configured, and you can't just load something else into it if you want to run something else. This is certainly fine in many cases, but I think we also need to consider the case where it is not know at boot time which image will get loaded into the FPGA.
Note that you can actually put the byte stream for the FPGA image into a device tree property so you don't have to load it from disk, which may not be easy for technical (e.g. the FPGA implementing you storage interface) or legal (e.g. the patents preventing you from shipping the image with a standard Linux distro) reasons.
Quoted 3 lines
Just to be clear, what are you calling payload here? Is it the data that is used to configure the fpga, or actual data going back and forth? I'm a little concerned that not everyone understand how flexible the FPGA is.
I mean the configuration byte stream.
Arnd
Re: [PATCH 2/2] New driver: Xillybus generic interface for FPGA (programmable logic)
In reply to Philip BalisterHi!
Quoted 7 lines
Quoted 6 lines
Agreed. If I understand you correctly though, your approach is specific to a particular hardware implementation (Zynq) on the user interface layer, which I think is exactly what we should avoid. Obviously, there is always a driver involved that is specific to the IP block you load into an FPGA at runtime, and that is ok. The two parts that I think we should agree on are:
Quoted 6 lines
The approach I am taking is not Zynq specific. The only Zynq specific bits will be range checking data from the device tree to make sure the user is mapping address space that is used by the FPGA and a similar validation of the IRQ numbers. I have not looked at the Altera part closely, but I suspect their fpga/processor interface may use similar concepts.
So... the arch may be called socfpga, but I don't think we managed to merge actual fpga parts, yet ;-). In fact, I'm not sure if I actually seen those... Dinh should know more about status there.
Pavel
Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html