Driver Model 2 Proposal - Linux Kernel Performance v Usability
Following my initial post yesterday please find attached my proposal for a binary 'plugin' interface:
This is not an attempt to have a Microkernel, or any move away from GNU/OSS software. I believe that sometimes the ultimate goals of stability and portability get lost in the debate on OSS and desire to allow anyone to contribute. It is worth remembering that for every Kernel hacker there must be 1000's of plain users. I believe this proposal would lead to better software and more people using it.
Proposal
-----------
1. Implement binary kernel 'plugin' interface
2. Over time remove most existing kernel 'drivers' to use new interface - This is NOT a Microkernel.
3. Design 'plugin' interface to be extensible as possible and then rarely remove support from interface. Extending interface is fine but should be done in a considered way to avoid interface bloat. Suggest interface supports dependant 'plugins'
4. Allow 'plugins' to be bypassed at boot - perhaps a minimal 'known good' list
5. Ultimately, even FS 'plugins' could be created although IPL would be required to load these.
6. Code for Kernel, Interface and 'plugins' would still be GPL. This would not prevent the 'tainted' system idea.
Expected Outcomes
------------------------
1. Make Linux easier to use
2. The ability to replace even very core Kernel components without a restart.
3. Allow faulty 'plugins' to be fixed/replaced in isolation. No other system impact.
4. 'Plugins' could create their own interfaces as load time. This would remove the need to pre-populate /dev.
5. Remove need for joe soap user to often recompile Kernel.
6. Remove link between specific module versions and Kernel versions.
7. Reduce need for major Kernel releases. Allow effort to concentrate on improving Kernel not maintaining ever increasing Kernel source that includes support for the 'Kitchen Sink'
8. Make core Kernel more stable. Less releases and less changes mean less bugs. It would be easy to identify offending 'plugin' by simply starting up the Kernel with it disabled.
9. Remove need for modules to be maintained in sync with each Kernel thus freeing 'module' developers to add improved features or work on new projects.
James
James,
You missed a key point, the kernel people do not give a damn about binary modules and think they are evil. They do not care if a binary module works at all. If they can break it they will. You are wasting time and electrons, or just pee'ng in the wind.
The only solution is to created a GPL pre-loading module with all the GPL_ONLY needed extentions re-exported or externed as to bypass the horse sh*t.
What I am shocked to see is how people are being nice to you and not blasting you with a carpet bomb attack. I guess they are waiting to line up on me.
Cheers,
Andre
Quoted 55 lines · James Clark
Following my initial post yesterday please find attached my proposal for a binary 'plugin' interface:
This is not an attempt to have a Microkernel, or any move away from GNU/OSS software. I believe that sometimes the ultimate goals of stability and portability get lost in the debate on OSS and desire to allow anyone to contribute. It is worth remembering that for every Kernel hacker there must be 1000's of plain users. I believe this proposal would lead to better software and more people using it.
Proposal
-----------
1. Implement binary kernel 'plugin' interface
2. Over time remove most existing kernel 'drivers' to use new interface - This is NOT a Microkernel.
3. Design 'plugin' interface to be extensible as possible and then rarely remove support from interface. Extending interface is fine but should be done in a considered way to avoid interface bloat. Suggest interface supports dependant 'plugins'
4. Allow 'plugins' to be bypassed at boot - perhaps a minimal 'known good' list
5. Ultimately, even FS 'plugins' could be created although IPL would be required to load these.
6. Code for Kernel, Interface and 'plugins' would still be GPL. This would not prevent the 'tainted' system idea.Expected Outcomes
------------------------1. Make Linux easier to use
2. The ability to replace even very core Kernel components without a restart.
3. Allow faulty 'plugins' to be fixed/replaced in isolation. No other system impact.
4. 'Plugins' could create their own interfaces as load time. This would remove the need to pre-populate /dev.
5. Remove need for joe soap user to often recompile Kernel.
6. Remove link between specific module versions and Kernel versions.
7. Reduce need for major Kernel releases. Allow effort to concentrate on improving Kernel not maintaining ever increasing Kernel source that includes support for the 'Kitchen Sink'
8. Make core Kernel more stable. Less releases and less changes mean less bugs. It would be easy to identify offending 'plugin' by simply starting up the Kernel with it disabled.
9. Remove need for modules to be maintained in sync with each Kernel thus freeing 'module' developers to add improved features or work on new projects.James
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Dans un message du 03 Sep à 10:49, Andre Hedrick écrivait :
Quoted 3 lines
The only solution is to created a GPL pre-loading module with all the
GPL_ONLYneeded extentions re-exported or externed as to bypass the horse sh*t.
If you do that, you
Signature · 4 lines
Guillaume Morin <[email protected]> If you cause problems that only you can fix, you'll never be fired. http://www.copyleft.net/
Quoted 6 lines · Guillaume Morin
Dans un message du 03 Sep à 10:49, Andre Hedrick écrivait :
Quoted 3 lines
The only solution is to created a GPL pre-loading module with all the
GPL_ONLYneeded extentions re-exported or externed as to bypass the horse sh*t.If you do that, you
I what ?
Create and original work that happens to be open sourced and free and given away for any binary module vendors to use and operate in linux. Allow people the choice/option to go open source or closed source in their product selections.
All I see is a WIN -- WIN for the people using Linux.
Then again, in the beginning it was about having a choice.
Cheers,
Andre
Dans un message du 03 Sep à 10:49, Andre Hedrick écrivait :
Quoted 3 lines
The only solution is to created a GPL pre-loading module with all the
GPL_ONLYneeded extentions re-exported or externed as to bypass the horse sh*t.
Which would be a violation of the GPL :
What is the difference between "mere aggregation" and "combining two modules into one program"?
Mere aggregation of two programs means putting them side by side on
the same CD-ROM or hard disk. We use this term in the case where they are separate programs, not parts of a single program. In this case, if one of the programs is covered by the GPL, it has no effect on the other program.
Combining two modules means connecting them together so that they form a
single larger program. If either part is covered by the GPL, the whole combination must also be released under the GPL--if you can't, or won't, do that, you may not combine them.
What constitutes combining two parts into one program? This is a legal
question, which ultimately judges will decide. We believe that a proper criterion depends both on the mechanism of communication (exec, pipes, rpc, function calls within a shared address space, etc.) and the semantics of the communication (what kinds of information are interchanged).
If the modules are included in the same executable file, they are
definitely combined in one program. If modules are designed to run linked together in a shared address space, that almost surely means combining them into one program.
By contrast, pipes, sockets and command-line arguments are communication
mechanisms normally used between two separate programs. So when they are used for communication, the modules normally are separate programs. But if the semantics of the communication are intimate enough, exchanging complex internal data structures, that too could be a basis to consider the two parts as combined into a larger program.
Signature · 3 lines
Guillaume Morin <[email protected]> Tu veux que les gens réagissent ? Alors commence par réagir (Lofofora)
Quoted 21 lines · Guillaume Morin
Dans un message du 03 Sep à 10:49, Andre Hedrick écrivait :
Quoted 3 lines
The only solution is to created a GPL pre-loading module with all the
GPL_ONLYneeded extentions re-exported or externed as to bypass the horse sh*t.Which would be a violation of the GPL :
What is the difference between "mere aggregation" and "combining two modules into one program"?
Mere aggregation of two programs means putting them side by side on
the same CD-ROM or hard disk. We use this term in the case where they are separate programs, not parts of a single program. In this case, if one of the programs is covered by the GPL, it has no effect on the other program.Combining two modules means connecting them together so that they form a
single larger program. If either part is covered by the GPL, the whole combination must also be released under the GPL--if you can't, or won't, do that, you may not combine them.
Quoted 5 lines
What constitutes combining two parts into one program? This is a legal
question, which ultimately judges will decide. We believe that a proper criterion depends both on the mechanism of communication (exec, pipes, rpc, function calls within a shared address space, etc.) and the semantics of the communication (what kinds of information are interchanged).
You assume wrong.
The Pre-Loader or "freed-symbols" is to return symbols that were once EXPORT_SYMBOL which are now openly renamed EXPORT_SYMBOL_GPL.
This is open thieft of the kernel API.
So I welcome you and your lawyers to sue me for making a GPL module which allows free use of the stolen symbols. Please come to California and lets party. Since it is an original work as a new module, regardless if it is derived or not, being GPL and source available you lose ... Have a nice day.
Cheers,
Andre
Quoted 10 lines · James Clark
Following my initial post yesterday please find attached my proposal for a binary 'plugin' interface:
This is not an attempt to have a Microkernel, or any move away from GNU/OSS software. I believe that sometimes the ultimate goals of stability and portability get lost in the debate on OSS and desire to allow anyone to contribute. It is worth remembering that for every Kernel hacker there must be 1000's of plain users. I believe this proposal would lead to better software and more people using it.
Quoted 3 lines
Proposal
-----------
1. Implement binary kernel 'plugin' interface
And this interface will look like what?
What is wrong with the current kernel API interface?
Quoted 2 lines
2. Over time remove most existing kernel 'drivers' to use new interface - This is NOT a Microkernel.
"remove"???
Quoted 10 lines
3. Design 'plugin' interface to be extensible as possible and then rarely remove support from interface. Extending interface is fine but should be done in a considered way to avoid interface bloat. Suggest interface supports dependant 'plugins'
4. Allow 'plugins' to be bypassed at boot - perhaps a minimal 'known good' list
5. Ultimately, even FS 'plugins' could be created although IPL would be required to load these.
6. Code for Kernel, Interface and 'plugins' would still be GPL. This would not prevent the 'tainted' system idea.
So "drivers" are a third class citizen? They don't need to be under the GPL for some reason?
Quoted 4 lines
Expected Outcomes
------------------------1. Make Linux easier to use
How would this help this? The kernel would get bigger somehow, right?
Quoted 1 line
2. The ability to replace even very core Kernel components without a restart.
We can do that today with modules.
Quoted 2 lines
3. Allow faulty 'plugins' to be fixed/replaced in isolation. No other system impact.
How are you going to isolate parts of the kernel from itself?
Quoted 2 lines
4. 'Plugins' could create their own interfaces as load time. This would remove the need to pre-populate /dev.
What?
Ok, I'm just giving up now.
But remember, patches are always welcome. Please post your code to implement this system if you come up with some.
Good luck,
greg k-h
Quoted 3 lines · James Clark
Following my initial post yesterday please find attached my proposal for a binary 'plugin' interface:
But we already have a well-defined modular interface. It allows (driver) modules to not only be installed into a running kernel, but also to be removed from a running kernel. This makes driver development quite easy compared to others where one needs to re-boot to make the .... "mouse movement detected, reboot to make the changes take effect.."
The well-defined interface for the modules is "struct file_operations". It corresponds to the Unix I/O model for devices. This interface is not broken and is, therefore, not necessary to be changed.
There is a Microsoft NT model where software is put somewhere and its functions are 'registered' with the kernel. This is an abortion designed totally to hide the inner workings of the kernel. Since our source-code is visible, we don't need layers of abstraction to high it from developers.
So, to your proposal, no thanks. Also, there is no way that I will __ever__ allow software provided by others to execute within the kernel of anything I am responsible for, unless that software's source-code is available for inspection.
The recent problems with the Microsoft worms should certainly convince any sane person that we can't allow some hidden software to run our industry no matter how well we recognize the name.
There are two main reasons for using Linux in embedded systems:
(1) I know what's in there.
(2) I can make hardware drivers without having to reboot the target.
Before Linux, we had to roll our own Operating Systems. And, yes, we made ARTOS (Analogic Realtime Operating System). We created a whole 'C' runtime library environment. It was a bitch. We don't need to do this anymore because there is Linux. If you, or any other well-wisher ever succeeds in converting Linux to a binary, no-source system, then Linux will get forked and your interface will get fu*k^M^Mmorked also.
Cheers,
Dick Johnson
Penguin : Linux version 2.4.22 on an i686 machine (794.73 BogoMips).
Note 96.31% of all statistics are fiction.
Quoted 3 lines
Following my initial post yesterday please find attached my proposal for a binary 'plugin' interface:
Whoa, whoa, whoa! Hang on there, cowboy.
It's always easier to observe the behavior of a system from the outside and make suggestions from that objective point of view (especially when you have experience in another similar system). However, change can ultimately only come from within.
We've always been open to outside suggestions, but they're really only useful when they contain implementation details (usualy in the form of working code) that build on top of our existing system.
Linux is an evolutionary process. While it's tempting to project the behavior or ideals of another similar system on to it -- simply because you can -- it really doesn't work. We evolve and mutate and change. And, that's where we feel we are really superior.
I've not read your proposal -- nor do I intend to -- even as maintainer of the new driver model. I do however encourage you to pick up a copy of "Linux Device Drivers", read it, and start contributing to the development process. After getting a feel for how we work, reconsider your proposal, and submit some patches to implement your ideas.
Until then, I'm not interested in discussing license issues or binary modules, so please remove me from the CC list.
Thanks,
Pat
Quoted 2 lines · James Clark
Following my initial post yesterday please find attached my proposal for a binary 'plugin' interface:
Plugin? Ugly word. Let's say modules instead. And modules ARE implemented now with proper interface, so what are we talking about?
Quoted 2 lines
This is not an attempt to have a Microkernel, or any move away from GNU/OSS software. I believe that sometimes the ultimate goals of stability and
I can't follow. So you assumed that microkernel can't be implemented in open source environment? Let's check out Hurd (btw, afaik Hurd is a collection of 'server processes' running on the top of the Mach microkernel).
Quoted 2 lines
portability get lost in the debate on OSS and desire to allow anyone to contribute. It is worth remembering that for every Kernel hacker there must
? Anyone can write a kernel module even now, I don't see the problem here.
Quoted 6 lines
be 1000's of plain users. I believe this proposal would lead to better software and more people using it.
Proposal
-----------
1. Implement binary kernel 'plugin' interface
binary? You can't. For being precise: you can't do it with keeping performance in mind. Newer major kernel releases use more or less different kind of structures, algorithms, etc. There are SMP systems and UP and so on. For maximum performance, kernel module SHOULD be compiled from source with target configuration keeping in mind and matching with kernel's configuration of course.
Sure, you can create a 'glue' layer between the actual kernel module API and something you're calling 'binary interface', but it will be a performance bottleneck, also this would make kernel larger in memory. You will end at a situation where win NT also sucks: several abstracion layers should be implemented which wastes performance just "translating" things between software and/or hardware resources, while it can be possible to do this directly. Sure, direct method breaks compatibility, eg you can't load easily a 2.0.x kernel modules into 2.6.0-testX kernel, but exactly this is the why where you're not limited by the bad meaning of compatibility just for wasting several percent of your resources to keep everything working with VERY obsolated binary-only drivers ...
Quoted 2 lines
2. Over time remove most existing kernel 'drivers' to use new interface - This is NOT a Microkernel.
I can't understand, let's say:
A) "Over time remove most existing kernel 'drivers' to use new interface"
B) "This is NOT a Microkernel"
I can't see any logical relationship between A and B, so I don't understand this sentence at all.
Quoted 4 lines
3. Design 'plugin' interface to be extensible as possible and then rarely remove support from interface. Extending interface is fine but should be done in a considered way to avoid interface bloat. Suggest interface supports dependant 'plugins'
So if someone 'invents' a brand-new algorithm for Linux you SHOULD keep the binary interface the same, where you can gain (let's say) 1000% of performance with reconstructing the module interface as well? Ehhh. Or you should keep up an 'abstraction layer' to translate between 'good-old-braken' API and the real "internal" kernel API? It would be HORRIBLE, SLOW, HUGE, and UNSTABLE. This reminds me m$ win at the first sight ;-)
Quoted 6 lines
4. Allow 'plugins' to be bypassed at boot - perhaps a minimal 'known good' list
5. Ultimately, even FS 'plugins' could be created although IPL would be required to load these.
6. Code for Kernel, Interface and 'plugins' would still be GPL. This would not prevent the 'tainted' system idea.
? Please, say only ONE advantage of this binary plugins thing then! Because I don't see even ONE, just disadvantages, I've described.
Quoted 6 lines
Expected Outcomes
------------------------1. Make Linux easier to use
2. The ability to replace even very core Kernel components without a restart.
Modules. It's only a question to modularize the current kernel even more, not something 'we need some very different model'. But this question would lead to the fact that you can't modularize at this 'brutal' way, since between versions of Linux, core changes can be done as well. So to do this either you need to a very-complex-and-almost-imposible-to-implement 'translator' which would save internal stage of the whole system, and loads it to the probably very different new data structures, or linux should keep binary compatibility between versions with slow, huge, and unstable 'glue' layers, hiding internal things from other parts.
Quoted 4 lines
3. Allow faulty 'plugins' to be fixed/replaced in isolation. No other system impact.
4. 'Plugins' could create their own interfaces as load time. This would remove the need to pre-populate /dev.
??
Quoted 1 line
5. Remove need for joe soap user to often recompile Kernel.
User should not recompile the kernel even now. I have several 'windows-style' friends using Linux without even know the meaning of 'kernel' at all.
Quoted 1 line
6. Remove link between specific module versions and Kernel versions.
You CAN'T do this, I've described. Sure, you can, but there would be only disadvantages.
Quoted 8 lines
7. Reduce need for major Kernel releases. Allow effort to concentrate on improving Kernel not maintaining ever increasing Kernel source that includes support for the 'Kitchen Sink'
8. Make core Kernel more stable. Less releases and less changes mean less bugs. It would be easy to identify offending 'plugin' by simply starting up the Kernel with it disabled.
9. Remove need for modules to be maintained in sync with each Kernel thus freeing 'module' developers to add improved features or work on new projects.
So again, you can't gain the maximum of your hardware because the complexity of that 'binary API' would be something like a low bandwidth network between two supercomputers, eg with creating a single on-time binary API, you can't gain the maxmimum of new invents later without breaking it.
Quoted 8 lines
James
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Signature · 1 line
- Gábor (larta'H)
Quoted 9 lines · James Clark
Following my initial post yesterday please find attached my proposal for a binary 'plugin' interface:
This is not an attempt to have a Microkernel, or any move away from GNU/OSS software. I believe that sometimes the ultimate goals of stability and portability get lost in the debate on OSS and desire to allow anyone to contribute. It is worth remembering that for every Kernel hacker there must be 1000's of plain users. I believe this proposal would lead to better software and more people using it.
Looks like you're looking for UDI. Most people on this list don't like it for various reasons (me included) but you seem to like it. Since SCO was the driving factor development probably has slowed down even more, go and help them!
Don't expect it to ever be merged though..