The Linux Staging tree, what it is and is not.
Hi all,
It's been many months since the Linux Kernel developers conference, where the linux-staging tree was discussed and role changed. It turns out that people are still a bit confused as to what the staging tree is for, and how it works.
So here's a short summary, I'm not going into the history or background here, that's a much longer writeup that I'd be glad to do if people are interested.
The Linux Staging Tree, what it is and is not.
What the Linux Staging tree is:
The Linux Staging tree (or just "staging" from now on) is used to hold stand-alone[1] drivers and filesystems that are not ready to be merged into the main portion of the Linux kernel tree at this point in time for various technical reasons. It is contained within the main Linux kernel tree so that users can get access to the drivers much easier than before, and to provide a common place for the development to happen, resolving the "hundreds of different download sites" problem that most out-of-tree drivers have had in the past.
What the Linux Staging tree is not:
The staging tree is not a place to dump code and run away, hoping that someone else will to the cleanup work for you. While there are developers available and willing to do this kind of work, you need to get them to agree to "babysit" the code in order for it to be accepted.
Location and Development:
The staging tree is now contained within the main Linux kernel source tree at the location drivers/staging/. All development happens within the main kernel source tree, like any other subsystem within the kernel. This means:
- the linux-next tree contains the latest version of the staging tree, with bugfixes that are about to be merged into Linus's tree, as well as the patches that are to be merged into the next major kernel release.
- if you wish to do work on the staging tree, checkout the linux-next tree and send patches based on that.
Runtime:
When code from the staging tree is loaded in the kernel, a warning message
will be printed to the kernel log saying:
MODULE_NAME: module is from the staging directory, the quality is unknown, you have been warned.
and the kernel will be tainted with the TAINT_CRAP flag. This flag shows up
in any kernel oops that might be produced after the driver has been loaded.
Note, most kernel developers have expressed the warning that they will not work on bugs for when this taint flag has happened, so if you run into a kernel problem after loading such a module, please work to reproduce the issue without a staging module loaded in order to be able to get help from the community.
If anyone has any questions that this summary doesn't answer, please let me know.
thanks,
greg k-h
[1] stand-alone means no changes needed in other places within the kernel. There are some exceptions to this:
- Documenation for the code can live in the Documentation/ directory although this isn't recommended.
- Firmware for the drivers can live in the firmware/ directory, this is recommended.
- Some symbols might need to be exported from the main portion of the kernel source tree. This is acceptable as long as the relevant subsystem maintainer agrees with this export.
Greg,
Quoted 2 lines · Greg KH
If anyone has any questions that this summary doesn't answer, please let me know.
Let me take this as an opportunity to discuss the epl (Ethernet Powerlink) driver in staging. Taken aside the eye-cancer thing while looking at the code (this could be fixed in the staging model), I suppose the whole design is questionable.
We have ported similar commercial EPL stacks to linux-rt in the past, and what we simply did is to implement the code completely in userspace, ontop of a raw socket. It worked out pretty well and with reasonable realtime performance. For the high level API, we used the process data model provided by libpv [1], which gives you an abstraction that follows both, automation-people's "process variable" concept and a modern object oriented desing (in C, modeled after the POSIX object model for example like pthread_create() & friends).
Doing this kind of network protocols in kernel space may be possible in general, but IMHO the first thing that has to be done for a sane design is:
"Invent proper APIs and abstractions"
Unfortunately, industry people have somehow missed the last 10 years of software engineering, so even recent ethernet fieldbus designs like PowerLink or EtherCAT use the CANopen Object Dictionary [1] as their "abstraction" between the stack and the application. So writing applications in the EPL/CANopen/EtherCAT world works by PEEK and POKE on a variable-length global variable array. Welcome to software design of the 80es. Nevertheless, "Object Dictionary" is a standard API for industry people which cannot be discussed away, because automation people are used to this terminology.
So if we want to do any kind of EPL/CANopen/EtherCAT work in the kernel, let's start with the question what it buys us in comparism with a pure userspace solution like outlined above.
One interesting thing could be shared APIs among this group of fieldbusses, for example one ObjectDictionary API that works for CANopen, EPL and EtherCAT. If we put that into the kernel, I'm wondering how this API could look like; the problem with objdict is that it is variable length, here's an example:
/-------------> 16 bit index
/ /---------> length
/--\ / /--/----> data, addressed with 8 bit subindex 0000 02 aa bb
0001 00
0002 05 aa bb cc dd ee
....
FFFF ..
Using that "API" more or less works by poking the right magic bits to several locations and things start happening.
So what would be the right kernel-userspace API for that? mmap() on a device node does not work because of the variable length nature. On the other hand, accessing the objdict shall be fast, because it will happen in the fast path of a control application. Another possibility could be sysfs, with the indices being the file names. That could at least be used for manual access (development, debugging) or initial setup. Is that fast enough? I don't know. Usually the initial objdict is worked out once during the design phase of an application, so you'll be able to load a whole objdict into the system on boot time. Later on, during operational phase of the stack, netlink may be used to propagate events and changes, at least until somebody is brave enough to write an in-kernel dbus server :-)
The objdict is IMHO the most complicated part, because other interactions like NMT, PDO and SDO could be easily done with sysfs + netlink.
However, I'm still not convinced that this is the right way to go. What you want to do for the main APIs is provide some nice userspace library API for your applications. Why not put the whole "stack" into userspace then and connect it to the kernel only via raw socket.
What do others think? Is it worth the effort to invent a proper objdict API for linux?
rsc
[1] http://www.pengutronix.de/software/libpv/index_en.html
[2] http://en.wikipedia.org/wiki/CANopen#Object_dictionary
Signature · 4 lines
Pengutronix e.K. | | Industrial Linux Solutions | http://www.pengutronix.de/ | Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0 | Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 |
Quoted 10 lines · Robert Schwebel
Greg,
Quoted 2 lines · Greg KH
If anyone has any questions that this summary doesn't answer, please let me know.
Let me take this as an opportunity to discuss the epl (Ethernet Powerlink) driver in staging. Taken aside the eye-cancer thing while looking at the code (this could be fixed in the staging model), I suppose the whole design is questionable.
Sure it's questionable, and it's horrid code, but it is being used by people and is the only public implementation of EPL on Linux that I can find.
Quoted 9 lines
We have ported similar commercial EPL stacks to linux-rt in the past, and what we simply did is to implement the code completely in userspace, ontop of a raw socket. It worked out pretty well and with reasonable realtime performance. For the high level API, we used the process data model provided by libpv [1], which gives you an abstraction that follows both, automation-people's "process variable" concept and a modern object oriented desing (in C, modeled after the POSIX object model for example like
pthread_create()& friends).
Quoted 5 lines
Doing this kind of network protocols in kernel space may be possible in general, but IMHO the first thing that has to be done for a sane design is:
"Invent proper APIs and abstractions"
Agreed.
Quoted 10 lines
Unfortunately, industry people have somehow missed the last 10 years of software engineering, so even recent ethernet fieldbus designs like PowerLink or EtherCAT use the CANopen Object Dictionary [1] as their "abstraction" between the stack and the application. So writing applications in the EPL/CANopen/EtherCAT world works by PEEK and POKE on a variable-length global variable array. Welcome to software design of the 80es. Nevertheless, "Object Dictionary" is a standard API for industry people which cannot be discussed away, because automation people are used to this terminology.
Quoted 3 lines
So if we want to do any kind of EPL/CANopen/EtherCAT work in the kernel, let's start with the question what it buys us in comparism with a pure userspace solution like outlined above.
Are userspace solutions for this opensource today?
<snip>
Quoted 2 lines
What do others think? Is it worth the effort to invent a proper objdict API for linux?
I suggest discussing this on netdev, that's the proper place for network protocol discussions like this, right?
thanks,
greg k-h
Quoted 46 lines · Greg KH
Hi all,
It's been many months since the Linux Kernel developers conference, where the linux-staging tree was discussed and role changed. It turns out that people are still a bit confused as to what the staging tree is for, and how it works.
So here's a short summary, I'm not going into the history or background here, that's a much longer writeup that I'd be glad to do if people are interested.
The Linux Staging Tree, what it is and is not.
What the Linux Staging tree is:
The Linux Staging tree (or just "staging" from now on) is used to hold stand-alone[1] drivers and filesystems that are not ready to be merged into the main portion of the Linux kernel tree at this point in time for various technical reasons. It is contained within the main Linux kernel tree so that users can get access to the drivers much easier than before, and to provide a common place for the development to happen, resolving the "hundreds of different download sites" problem that most out-of-tree drivers have had in the past.What the Linux Staging tree is not:
The staging tree is not a place to dump code and run away, hoping that someone else will to the cleanup work for you. While there are developers available and willing to do this kind of work, you need to get them to agree to "babysit" the code in order for it to be accepted.Location and Development:
The staging tree is now contained within the main Linux kernel source tree at the locationdrivers/staging/. All development happens within the main kernel source tree, like any other subsystem within the kernel. This means:
- the linux-next tree contains the latest version of the staging tree, with bugfixes that are about to be merged into Linus's tree, as well as the patches that are to be merged into the next major kernel release.
- if you wish to do work on the staging tree, checkout the linux-next tree and send patches based on that.Runtime:
When code from the staging tree is loaded in the kernel, a warning message
will be printed to the kernel log saying:
MODULE_NAME: module is from the staging directory, the quality is unknown, you have been warned. and the kernel will be tainted with theTAINT_CRAPflag. This flag shows up
in any kernel oops that might be produced after the driver has been loaded.
Quoted 8 lines
Note, most kernel developers have expressed the warning that they will not work on bugs for when this taint flag has happened, so if you run into a kernel problem after loading such a module, please work to reproduce the issue without a staging module loaded in order to be able to get help from the community.
If anyone has any questions that this summary doesn't answer, please let me know.
What is the target audience for staging?
It doesn't fill the out-of-tree modules void as far as I can tell.
The model most users are stuck with are
a) distro kernel
b) hw not supported.
Distro kernel could be any revision from 2.6.15 or something crazy up until now.
Generally you can pull the out of tree modules zip file or repo, build it against
the kernel you have installed and know works with all your other hw and your nvidia
binary driver, and the out of tree people have wrapped all the API changes so that you don't have to do much in theory for it to just build and install on your
kernel.
Staging doesn't fulfill this role from my POV:
a) what sane distro will enable staging drivers? do they care about their users?
b) if your distro doesn't enable staging drivers you now have two options:
1. distro kernel rebuild with staging drivers enabled if new enough distro kernel
2. kernel.org kernel build with staging drivers enabled.
For a normal user who just wants his hw to work, these options are possibly most unappealing. They still want do just download something from out-of-tree which does all the nice API break wrapping for them and is 1MB instead of 300MB.
So hence my target audience question.
Dave.
Quoted 53 lines · Dave Airlie
Quoted 52 lines · Greg KH
Hi all,
It's been many months since the Linux Kernel developers conference, where the linux-staging tree was discussed and role changed. It turns out that people are still a bit confused as to what the staging tree is for, and how it works.
So here's a short summary, I'm not going into the history or background here, that's a much longer writeup that I'd be glad to do if people are interested.
The Linux Staging Tree, what it is and is not.
What the Linux Staging tree is:
The Linux Staging tree (or just "staging" from now on) is used to hold stand-alone[1] drivers and filesystems that are not ready to be merged into the main portion of the Linux kernel tree at this point in time for various technical reasons. It is contained within the main Linux kernel tree so that users can get access to the drivers much easier than before, and to provide a common place for the development to happen, resolving the "hundreds of different download sites" problem that most out-of-tree drivers have had in the past.What the Linux Staging tree is not:
The staging tree is not a place to dump code and run away, hoping that someone else will to the cleanup work for you. While there are developers available and willing to do this kind of work, you need to get them to agree to "babysit" the code in order for it to be accepted.Location and Development:
The staging tree is now contained within the main Linux kernel source tree at the locationdrivers/staging/. All development happens within the main kernel source tree, like any other subsystem within the kernel. This means:
- the linux-next tree contains the latest version of the staging tree, with bugfixes that are about to be merged into Linus's tree, as well as the patches that are to be merged into the next major kernel release.
- if you wish to do work on the staging tree, checkout the linux-next tree and send patches based on that.Runtime:
When code from the staging tree is loaded in the kernel, a warning message
will be printed to the kernel log saying:
MODULE_NAME: module is from the staging directory, the quality is unknown, you have been warned. and the kernel will be tainted with theTAINT_CRAPflag. This flag shows up
in any kernel oops that might be produced after the driver has been loaded.Note, most kernel developers have expressed the warning that they will not work on bugs for when this taint flag has happened, so if you run into a kernel problem after loading such a module, please work to reproduce the issue without a staging module loaded in order to be able to get help from the community.
Quoted 4 lines
Quoted 2 lines
If anyone has any questions that this summary doesn't answer, please let me know.
What is the target audience for staging?
two different groups:
- users who want to use their hardware
- developers working on the code in a common area.
Quoted 1 line
It doesn't fill the out-of-tree modules void as far as I can tell.
Hm, I think it does.
Quoted 4 lines
The model most users are stuck with are
a) distro kernel
b) hw not supported.
Quoted 8 lines
Distro kernel could be any revision from 2.6.15 or something crazy up until now.
Generally you can pull the out of tree modules zip file or repo, build it against the kernel you have installed and know works with all your other hw and your nvidia binary driver, and the out of tree people have wrapped all the API changes so that you don't have to do much in theory for it to just build and install on your kernel.
Um, no, I don't think you have looked at these out-of-tree modules much.
Quoted 3 lines
Staging doesn't fulfill this role from my POV:
a) what sane distro will enable staging drivers? do they care about their users?
If the distros look, most of these drivers were already included in them, I've merged all of the Fedora and Ubuntu and openSUSE out-of-tree drivers that were previously in their kernels.
So I'm pretty sure they will enable them, just to keep parity with what they are currently shipping :)
openSUSE enables them for this very reason.
Quoted 5 lines
b) if your distro doesn't enable staging drivers you now have two options:
1. distro kernel rebuild with staging drivers enabled if new enough distro kernel
2. kernel.org kernel build with staging drivers enabled.
That's up to a distro as to what they do.
Quoted 4 lines
For a normal user who just wants his hw to work, these options are possibly most unappealing. They still want do just download something from out-of-tree which does all the nice API break wrapping for them and is 1MB instead of 300MB.
A lot of the work that I've done getting these drivers in to the tree, is just finding all of the proper patches for the drivers, and getting the code to build. A "normal" user wouldn't be able to do that at all.
Take the PSB video driver as an example, just finding the proper patches took weeks.
thanks,
greg k-h
Quoted 6 lines · Greg KH
Quoted 5 lines · Robert Schwebel
Greg,
Quoted 2 lines · Greg KH
If anyone has any questions that this summary doesn't answer, please let me know.
Quoted 8 lines
Quoted 4 lines
Let me take this as an opportunity to discuss the epl (Ethernet Powerlink) driver in staging. Taken aside the eye-cancer thing while looking at the code (this could be fixed in the staging model), I suppose the whole design is questionable.
Sure it's questionable, and it's horrid code, but it is being used by people and is the only public implementation of EPL on Linux that I can find.
BTW, the implementation does not follow the kernel style guide, because our company has its own code style guide. But what is that you don't like?
Quoted 9 lines
Quoted 9 lines
We have ported similar commercial EPL stacks to linux-rt in the past, and what we simply did is to implement the code completely in userspace, ontop of a raw socket. It worked out pretty well and with reasonable realtime performance. For the high level API, we used the process data model provided by libpv [1], which gives you an abstraction that follows both, automation-people's "process variable" concept and a modern object oriented desing (in C, modeled after the POSIX object model for example like
pthread_create()& friends).
Quoted 7 lines
Quoted 5 lines
Doing this kind of network protocols in kernel space may be possible in general, but IMHO the first thing that has to be done for a sane design is:
"Invent proper APIs and abstractions"
Agreed.
I can only second that. But it is no easy task to find a common API for all field busses.
Quoted 10 lines
Quoted 10 lines
Unfortunately, industry people have somehow missed the last 10 years of software engineering, so even recent ethernet fieldbus designs like PowerLink or EtherCAT use the CANopen Object Dictionary [1] as their "abstraction" between the stack and the application. So writing applications in the EPL/CANopen/EtherCAT world works by PEEK and POKE on a variable-length global variable array. Welcome to software design of the 80es. Nevertheless, "Object Dictionary" is a standard API for industry people which cannot be discussed away, because automation people are used to this terminology.
Quoted 5 lines
Quoted 3 lines
So if we want to do any kind of EPL/CANopen/EtherCAT work in the kernel, let's start with the question what it buys us in comparism with a pure userspace solution like outlined above.
Are userspace solutions for this opensource today?
No. But openPOWERLINK can be ported to userspace.
I would propose another solution.
Just leave the high priority tasks in the kernel which directly deal with the Ethernet frames and implement all other modules in userspace. Maybe we can enhance the standard network driver interface and directly connect the data link layer of POWERLINK to it. Additionally I would put the core NMT state machine and the PDO processing (encode and decode the frames with the process data from and into the process image) into the kernel. The process image can be accessed via mmap from userspace. All other modules like the object dictionary, SDO module, network management of the MN, etc can be put in userspace. I think the described kernel part can be done without the knowledge of an object dictionary at all. It's just a matter of "proper APIs".
Quoted 7 lines
<snip>
Quoted 2 lines
What do others think? Is it worth the effort to invent a proper objdict API for linux?
I suggest discussing this on netdev, that's the proper place for network protocol discussions like this, right?
Ok, I will subscribe it.
cu,
Daniel
--
SYS TEC electronic GmbH
August-Bebel-Str. 29
D-07973 Greiz
Telefon : +49 (0) 3661 6279 0
Fax : +49 (0) 3661 6279 99
Email : [email protected]
Internet : http://www.systec-electronic.com
Managing Director : Dipl.-Phys. Siegmar Schmidt
Commercial registry : Amtsgericht Jena, HRB 205563
Quoted 7 lines · Daniel Krüger
Quoted 3 lines
Sure it's questionable, and it's horrid code, but it is being used by people and is the only public implementation of EPL on Linux that I can find.
BTW, the implementation does not follow the kernel style guide, because our company has its own code style guide. But what is that you don't like?
If you do your private stuff, you can do whatever you like. If you want to bring something into the kernel, you have to follow the kernel style.
But it's not only a matter of coding style. Please let's not start with implementation details as long as the overall design is unclear. So I suggest that following Greg's advise and moving the discussion to netdev is what we should do.
Quoted 10 lines
Quoted 7 lines
Quoted 5 lines
Doing this kind of network protocols in kernel space may be possible in general, but IMHO the first thing that has to be done for a sane design is:
"Invent proper APIs and abstractions"
Agreed.
I can only second that. But it is no easy task to find a common API for all field busses.
Finding a common API for all field busses is somethign we'll do on a higher level in the OSADL Fieldbus Framework project, but in userspace.
What matters for the kernel is that, if we want to have things in mainline, we must answer the question "what have these things in common". It doesn't make any sense to invent infrastructure for things that don't have anything in common. Kernel frameworks are all about abstraction.
Quoted 3 lines
Quoted 1 line
Are userspace solutions for this opensource today?
No. But openPOWERLINK can be ported to userspace.
Sure.
Quoted 3 lines
I would propose another solution. Just leave the high priority tasks in the kernel which directly deal with the Ethernet frames and implement all other modules in userspace.
So why not work completely ontop of raw sockets? What else do you need in the kernel?
The only thing I can imagine is a fast packet switch that separates high priority realtime packets from normal non-rt traffic in the network stack. As far as I know, tglx has a patch for that.
Let's move the discussion to netdev.
rsc
Signature · 4 lines
Pengutronix e.K. | | Industrial Linux Solutions | http://www.pengutronix.de/ | Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0 | Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 |
Quoted 4 lines
However, I'm still not convinced that this is the right way to go. What you want to do for the main APIs is provide some nice userspace library API for your applications. Why not put the whole "stack" into userspace then and connect it to the kernel only via raw socket.
- Security
- Compartmentalisation
- Sharing the device/interconnect
- Real time prioritisation
and a few others.
Quoted 2 lines
What do others think? Is it worth the effort to invent a proper objdict API for linux?
Objdict appears to be a legacy wart and can stay where it belongs I suspect - you might need to have a objdict -> 21st century conversion library for the afflicted. We don't put specialised COBOL support in the kernel either we let user space figure out how to cope with the funnies.
Alan
Hi Alan,
Thanks for your feedback! :-)
Quoted 6 lines · Alan Cox
Quoted 4 lines
However, I'm still not convinced that this is the right way to go. What you want to do for the main APIs is provide some nice userspace library API for your applications. Why not put the whole "stack" into userspace then and connect it to the kernel only via raw socket.
- Security
For industrial I/O? We are talking about transportation of analog and digital I/O, relais, ADC input etc. The only part that I'd consider security related is tunnel traffic; EPL and EtherCAT have the possibiltiy to do ethernet-over-$TECHNOLOGY. In the userspace variant, tunnel trafic would be pushed back into the stack via tun/tap.
Quoted 1 line
- Compartmentalisation
Yes, that's a good and valid point, IMHO. Also, from a political point of view, it would be very good to have *the one and only* kernel way to do these things, instead of myriards of incompatible userspace hacks.
Quoted 1 line
- Sharing the device/interconnect
Sharing the device works also with userspace, because the device is a network card and the interface is raw ethernet frames. What do you mean with interconnect?
Quoted 1 line
- Real time prioritisation
Would be done by giving the kernel task (with threaded interrupts) or userspace task the right POSIX relatime priorities, so I don't see a difference here.
Quoted 4 lines
Quoted 2 lines
What do others think? Is it worth the effort to invent a proper objdict API for linux?
Objdict appears to be a legacy wart
Don't tell that to the industrial networking people - they are very proud of the fact that they use a "standardized application API", so even the fanciest and newest industrial ethernet technologies are based on that legacy wart objdict, and it is officially considered to be a *feature*.
Quoted 1 line
and can stay where it belongs I suspect -
So where does it belong, kernel or userspace? I'd say userspace, but then you can put the whole stack into userspace. If kernel, we would need an objdict API to userspace...?
One thing that has to be considered is that messages often have to be handled fast and under realtime constraints; for example, it must be possible to hook up the userspace to certain changes in the objdict. Something like:
------- --------- --------------
eth objdict process data
can layer
------- --------- --------------
| | |
| rx() | |
|--------->| |
| # write pdo into |
| # position in |
| # objdict |
| | |
| | pdo_changed() |
| |--------------->|
| | # pull process data out of certain
| | # position in the objdict and make
| | # proper process variable objects
| | #
| | |Note that what they are doing is to use the objdict as a binary config space (think registers) to configure the whole network workflow; so if for example somebody wants to send out a temperature every delta-t seconds, he configures that by setting bits in the objdict, pushing the process data (temperature) into other bits in the objdict and the stack sends out can/eth frames with that data. It's a complete mixture of ISO/OSI layers, because they have things like "what is to be sent when" in the same binary blob space as "payload".
Quoted 2 lines
you might need to have a objdict -> 21st century conversion library for the afflicted.
That's what we are working on in the OSADL Fieldbus Framework, in userspace. The basic idea is to bring the fieldbusses together with the mathematics developed in the Ptolemy II project [1], in a way that you can do classic soft-PLC [2] / IEC61131 stuff as well as modern heterogenous model based design (thing LabVIEW, Matlab/Simulink, Rhapsody, ...).
Quoted 2 lines
We don't put specialised COBOL support in the kernel either we let user space figure out how to cope with the funnies.
Industrial communication people are in some strange time loop that started looping in the 80es, but didn't notice yet :-)
rsc
[1] http://ptolemy.berkeley.edu/ptolemyII/
[2] http://en.wikipedia.org/wiki/Programmable_logic_controller
Signature · 4 lines
Pengutronix e.K. | | Industrial Linux Solutions | http://www.pengutronix.de/ | Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0 | Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 |
Quoted 5 lines
Quoted 1 line
- Security
For industrial I/O? We are talking about transportation of analog and digital I/O, relais, ADC input etc. The only part that I'd consider security related is tunnel traffic; EPL and EtherCAT have the
Maybe in your case but if that analogue I/O controls parts of a power plant or weapons system you really really don't want a worm or trojan getting involved with it.
Quoted 5 lines
Quoted 1 line
- Sharing the device/interconnect
Sharing the device works also with userspace, because the device is a network card and the interface is raw ethernet frames. What do you mean with interconnect?
Load two user space fieldbus stacks on the same network card (remembering this could be done by accident)
Quoted 5 lines
Quoted 1 line
- Real time prioritisation
Would be done by giving the kernel task (with threaded interrupts) or userspace task the right POSIX relatime priorities, so I don't see a difference here.
There may be multiple user space tasks with differing priorities beyond the message priority. That in turn means you want them to have different priorities to the scheduler which means you want them to be different processes really.
Quoted 5 lines
Don't tell that to the industrial networking people - they are very proud of the fact that they use a "standardized application API", so even the fanciest and newest industrial ethernet technologies are based on that legacy wart objdict, and it is officially considered to be a *feature*.
It's a standard yes, but so once was the horse and cart.
Quoted 2 lines
then you can put the whole stack into userspace. If kernel, we would need an objdict API to userspace...?
It depends where you split the stack between kernel and user space. There needs to be enough in the kernel to allow reasonably efficient multi-process models but that can be a small part of the stack. The Appletalk stack is a good example. Appletalk has a single address/port system for all higher level protocols and the higher level protocols don't interact much with each other. It's actually very elegant in that way. One result of this is that the AF_APPLETALK kernel stack deals only with low level packet routing and delivery. It doesn't have to care about higher stuff to provide the needed separation and sharing. IP by comparison has all sorts of interactions, multiple port addressing schemes and ICMP interactions which mean the kernel has to hold the entire TCP and UDP layers.
Quoted 4 lines
One thing that has to be considered is that messages often have to be handled fast and under realtime constraints; for example, it must be possible to hook up the userspace to certain changes in the objdict. Something like:
One model would be
kernel demux by objdict id -> forwards packet to the right user
a completely and gloriously insane one might be to have the objdict in mmap space as you suggest but with a futex on each objdict entry.
Quoted 2 lines
Note that what they are doing is to use the objdict as a binary config space (think registers) to configure the whole network workflow; so if
I tend to think of it as a rather peculiar shared event space - much like the device end of SNMP. I guess we should be thankful for small mercies that it wasn't invented today and using XML ;)
None of which is to say that an entirely userspace stack might not be the right model, but there are distinct appeals to the kernel doing some of the demux work and separation of stuff.
I shall be interested to see what turns out best as there are other busses where the design really expects to set up states and the stack to do regular repeating.
Quoted 9 lines · Alan Cox
Quoted 5 lines
Quoted 1 line
- Security
For industrial I/O? We are talking about transportation of analog and digital I/O, relais, ADC input etc. The only part that I'd consider security related is tunnel traffic; EPL and EtherCAT have the
Maybe in your case but if that analogue I/O controls parts of a power plant or weapons system you really really don't want a worm or trojan getting involved with it.
right :) But is there any difference between the stack and the application here? Applications must be able to set process variables, so if you compromize them and write the wrong ones to the stack, your power plant goes baaaang as well.
Quoted 8 lines
Quoted 5 lines
Quoted 1 line
- Sharing the device/interconnect
Sharing the device works also with userspace, because the device is a network card and the interface is raw ethernet frames. What do you mean with interconnect?
Load two user space fieldbus stacks on the same network card (remembering this could be done by accident)
ok
Quoted 10 lines
Quoted 5 lines
Quoted 1 line
- Real time prioritisation
Would be done by giving the kernel task (with threaded interrupts) or userspace task the right POSIX relatime priorities, so I don't see a difference here.
There may be multiple user space tasks with differing priorities beyond the message priority. That in turn means you want them to have different priorities to the scheduler which means you want them to be different processes really.
right, but that's the same, no matter if it runs in kernel or user space.
Quoted 7 lines
Quoted 5 lines
Don't tell that to the industrial networking people - they are very proud of the fact that they use a "standardized application API", so even the fanciest and newest industrial ethernet technologies are based on that legacy wart objdict, and it is officially considered to be a *feature*.
It's a standard yes, but so once was the horse and cart.
/me puts this into his cites file :)
Quoted 15 lines
Quoted 2 lines
then you can put the whole stack into userspace. If kernel, we would need an objdict API to userspace...?
It depends where you split the stack between kernel and user space. There needs to be enough in the kernel to allow reasonably efficient multi-process models but that can be a small part of the stack. The Appletalk stack is a good example. Appletalk has a single address/port system for all higher level protocols and the higher level protocols don't interact much with each other. It's actually very elegant in that way. One result of this is that the
AF_APPLETALKkernel stack deals only with low level packet routing and delivery. It doesn't have to care about higher stuff to provide the needed separation and sharing. IP by comparison has all sorts of interactions, multiple port addressing schemes and ICMP interactions which mean the kernel has to hold the entire TCP and UDP layers.
So what "layers" do we have for CANopen, EtherCAT and PowerLink?
- the lower api is getting ethernet frames
- the stack itself is a state machine that pushes things to objdict
- objdict needs to notify applications on certain changes
- userspace needs to trigger state machine on certain objcict changes by the application
We can either put the state machine into the kernel or into userspace; if userspace, the kernel API is already there (socket), if kernel we would need an objdict API.
Quoted 3 lines
One model would be
kernel demux by objdict id -> forwards packet to the right user
How would a user attach to the "right" thing? Sounds like some publish/subscribe pattern.
Quoted 2 lines
a completely and gloriously insane one might be to have the objdict in mmap space as you suggest but with a futex on each objdict entry.
uuuuh :)
The problem with mmap is that the objdict is not "rectangular", so you would need an mmapped blob + description of the variable length content, plus access serialization. Sounds indeed gloriously insane :)
Quoted 11 lines
Quoted 3 lines
Note that what they are doing is to use the objdict as a binary config space (think registers) to configure the whole network workflow; so if
I tend to think of it as a rather peculiar shared event space - much like the device end of SNMP. I guess we should be thankful for small mercies that it wasn't invented today and using XML ;)
None of which is to say that an entirely userspace stack might not be the right model, but there are distinct appeals to the kernel doing some of the demux work and separation of stuff.
seconded, we'll put some brain into that idea ...
Quoted 3 lines
I shall be interested to see what turns out best as there are other busses where the design really expects to set up states and the stack to do regular repeating.
That's not different here.
rsc
Signature · 4 lines
Pengutronix e.K. | | Industrial Linux Solutions | http://www.pengutronix.de/ | Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0 | Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 |
Quoted 22 lines · Robert Schwebel
Quoted 4 lines · Alan Cox
It depends where you split the stack between kernel and user space. There needs to be enough in the kernel to allow reasonably efficient multi-process models but that can be a small part of the stack. The Appletalk stack is a good example. Appletalk has a single address/port
Quoted 8 lines
system for all higher level protocols and the higher level protocols don't interact much with each other. It's actually very elegant in that way. One result of this is that the
AF_APPLETALKkernel stack deals only with low level packet routing and delivery. It doesn't have to care about higher stuff to provide the needed separation and sharing. IP by comparison has all sorts of interactions, multiple port addressing schemes and ICMP interactions which mean the kernel has to hold the entire TCP and UDP layers.So what "layers" do we have for CANopen, EtherCAT and PowerLink?
- the lower api is getting ethernet frames
- the stack itself is a state machine that pushes things to objdict
- objdict needs to notify applications on certain changes
- userspace needs to trigger state machine on certain objcict changes by the application
Quoted 3 lines
We can either put the state machine into the kernel or into userspace; if userspace, the kernel API is already there (socket), if kernel we would need an objdict API.
My understanding of an object dictionary is that it is only a structured view to a set of configuration variables, process variables and diagnostic information. It could also be an XML file or a simple C struct. In our implementation the object dictionary is not necessarily the backing store of the data. The process variables for example are just linked into the objdict. The application or the process image provides the backing store. In the end we can find any appropriate structure to pass the configuration from userspace to the data link layer and the PDO module in the kernel.
The PDO module is in fact a programmable scatter-gather copy machine (like DMA) between process image and Ethernet frames and vice-versa. The information of what to copy to where comes from the object dictionary. But it is possible to extract this information from the objdict and pass it via generic structures (e.g. something like IO vectors) to the PDO module in kernel. The information flow of the configuration data is one-way only, from userspace to kernel. This is what I plan to implement in openPOWERLINK, because the copy information can also be passed to a special hardware like a DMA controller.
So you can implement the objdict in userspace and access the process image via mmap (e.g. libpv).
cu,
Daniel
--
SYS TEC electronic GmbH
August-Bebel-Str. 29
D-07973 Greiz
Telefon : +49 (0) 3661 6279 0
Fax : +49 (0) 3661 6279 99
Email : [email protected]
Internet : http://www.systec-electronic.com
Managing Director : Dipl.-Phys. Siegmar Schmidt
Commercial registry : Amtsgericht Jena, HRB 205563
Quoted 5 lines
My understanding of an object dictionary is that it is only a structured view to a set of configuration variables, process variables and diagnostic information. It could also be an XML file or a simple C struct. In our implementation the object dictionary is not necessarily the backing store of the data. The process variables for example are
It's also possible I think to look at the object dictionary as no different from port numbers. The object identifiers tell you who owns the data and thus who needs to be told. I admit to being a bit rusty on this. My last near encounters with fieldbus type systems were some years ago when I²IT was involved in EMUG and OLCHFA and I managed to keep out of the blast radius of those ;)
Quoted 7 lines
information of what to copy to where comes from the object dictionary. But it is possible to extract this information from the objdict and pass it via generic structures (e.g. something like IO vectors) to the PDO module in kernel. The information flow of the configuration data is one-way only, from userspace to kernel. This is what I plan to implement in openPOWERLINK, because the copy information can also be passed to a special hardware like a DMA controller.
Ok so your hardware in fact really is oriented around an mmap type interface ?
Alan
Quoted 2 lines · Dave Airlie
Staging doesn't fulfill this role from my POV:
a) what sane distro will enable staging drivers? do they care about their users?
If the code there actually builds, I don't see a problem with packaging the staging drivers into a -crap subpackage, or maybe blacklisting them all by default, but saving the user who wants to cut his head off and watch himself bleed from having to recompile the kernel package for that.
Signature · 1 line
Lubomir Rintel <[email protected]>
Quoted 12 lines · Daniel Krüger
Quoted 11 lines · Greg KH
Quoted 10 lines · Robert Schwebel
Greg,
Quoted 3 lines · Greg KH
If anyone has any questions that this summary doesn't answer, please let me
know.Let me take this as an opportunity to discuss the epl (Ethernet Powerlink) driver in staging. Taken aside the eye-cancer thing while looking at the code (this could be fixed in the staging model), I suppose the whole design is questionable.
Quoted 6 lines
Quoted 3 lines
Sure it's questionable, and it's horrid code, but it is being used by people and is the only public implementation of EPL on Linux that I can find.
BTW, the implementation does not follow the kernel style guide, because our company has its own code style guide. But what is that you don't like?
It's not an issue of "not liking" it, it just doesn't match the kernel style guide.
I'll go make up a patch today to fix the layout and lots of the structure issues of the code to match our rules, so that people can then start working on the real functionality much easier.
thanks,
greg k-h
Quoted 10 lines · Alan Cox
Quoted 7 lines
information of what to copy to where comes from the object dictionary. But it is possible to extract this information from the objdict and pass it via generic structures (e.g. something like IO vectors) to the PDO module in kernel. The information flow of the configuration data is one-way only, from userspace to kernel. This is what I plan to implement in openPOWERLINK, because the copy information can also be passed to a special hardware like a DMA controller.
Ok so your hardware in fact really is oriented around an mmap type interface ?
what I understand from this thread is that the hardware is a standard ethernet card.
David Lang
Quoted 10 lines · [email protected] schrieb:
Quoted 8 lines · Alan Cox
Quoted 7 lines
information of what to copy to where comes from the object dictionary. But it is possible to extract this information from the objdict and pass it via generic structures (e.g. something like IO vectors) to the PDO module in kernel. The information flow of the configuration data is one-way only, from userspace to kernel. This is what I plan to implement in openPOWERLINK, because the copy information can also be passed to a special hardware like a DMA controller.
Quoted 4 lines
Quoted 2 lines
Ok so your hardware in fact really is oriented around an mmap type interface ?
what I understand from this thread is that the hardware is a standard ethernet card.
It may be a standard Ethernet controller. But it could also be a controller with a special hardware acceleration.
cu,
Daniel
--
SYS TEC electronic GmbH
August-Bebel-Str. 29
D-07973 Greiz
Telefon : +49 (0) 3661 6279 0
Fax : +49 (0) 3661 6279 99
Email : [email protected]
Internet : http://www.systec-electronic.com
Managing Director : Dipl.-Phys. Siegmar Schmidt
Commercial registry : Amtsgericht Jena, HRB 205563
Quoted 5 lines · Daniel Krüger
Quoted 2 lines
what I understand from this thread is that the hardware is a standard ethernet card.
It may be a standard Ethernet controller. But it could also be a controller with a special hardware acceleration.
Industrial Ethernets or CANopen can also be implemented with hardware solutions like a netX or an FPGA, which both would have shared memory to the CPU plus a part of the stack in userspace; however, this is nothing that should be abstracted with this driver infrastructure. It is an issue for a more higher level interface, which is what we will provide with the OSADL Fieldbus Framework:
+------------------------------------+ I Application I +------------------------------------+
+------------------------------------+ I Process Data Library I +------------------------------------+
+--------------------+ +-------------+ I $FIELDBUS Stack I I Userspace I I I I Driver I +--------------------+ +-------------+
+--------------------+ +-------------+
I AF_CAN / AF_PACKET I I UIO I
I Socket I I I
+--------------------+ +-------------+ Userspace
___^___
+--------------------+ +-------------+ v
I eth/can Card I I UIO driver I Kernel
I with normal driver I I I
+--------------------+ +-------------+rsc
Signature · 4 lines
Pengutronix e.K. | | Industrial Linux Solutions | http://www.pengutronix.de/ | Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0 | Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 |