Idea: Automatic binary driver compiling system
Hi all,
First off: I'm new on this list. I'm also new at starting projects. Anyways, I've decided on this idea for my first year computer science project. I have three months to set it off and definitely the whole thing would be released into the world afterwards.
Second: I don't want opinions on the issue of making it easier for companies to create binary (pre-compiled) drivers. Yes, we all want more open source drivers. Of course.
OK, so here goes: According to Distrowatch there's currently 377 different distributions out. We have multiple architectures and fresh kernel patches every week. If we multiply these numbers, we'd find that a company wanting to release a closed-source driver module for Linux would face - say - 5,000 compilations a month. Impossible. So most companies just release largely incompatible binary drivers in a frustratingly limited variety.
I've been looking around for material about the topic of binary Linux drivers and creating such in a broad-scale batch mode. But I haven't found any. Oh yes, except for Documentation/stable_api_nonsens.txt which I found very helpful. But the issue is not having a stable interface - rather it is about having a good debugging and building tool that handles very very large amounts of kernel versions and patches.
I would be very grateful for pointers to any helpful resource and also thoughts on what problems I'm facing. Also I'm aware that this is probably not the first time "batch module compiling" has been mentioned... ?
Sincerely,
Benjamin Bach
Quoted 3 lines · Benjamin Bach
I would be very grateful for pointers to any helpful resource and also thoughts on what problems I'm facing. Also I'm aware that this is probably not the first time "batch module compiling" has been mentioned... ?
"make modules" build all modules. You can batch it with nohup?
Quoted 10 lines · Benjamin Bach
Hi all,
First off: I'm new on this list. I'm also new at starting projects. Anyways, I've decided on this idea for my first year computer science project. I have three months to set it off and definitely the whole thing would be released into the world afterwards.
Second: I don't want opinions on the issue of making it easier for companies to create binary (pre-compiled) drivers. Yes, we all want more open source drivers. Of course.
Then let's end the thread here because this is the crux of ther issue - no one WANTS it to be easier for vendors to ship binary drivers.
Linux will not change to accomodate vendors desire for binary drivers. The vendors must adapt to Linux. End of story.
Lee
Quoted 6 lines · Lee Revell
Then let's end the thread here because this is the crux of ther issue - no one WANTS it to be easier for vendors to ship binary drivers.
Linux will not change to accomodate vendors desire for binary drivers. The vendors must adapt to Linux. End of story.
I see two pictures:
1) Open source does not accommodate binary drivers. In that way hardware support is weakened but an increasing number of companies should adopt open source drivers because demand increases even without super-duper hardware support.
2) Open source does in fact accommodate binary drivers because some idiot (me) just took the liberty of starting this project just to see what happens. It's exactly the same as when some people go and fork a project and everybody says it'll weaken the movement. But something could happen: It could be beneficial! Linux get's better hardware support, more people adopt it and business begins to realize that maybe they should try to co-operate a bit more software-wise. That's what we all want them to do, yes?
As of now Linux won't force vendors into creating open source drivers. That'd be like a small dog barking.
I here every word when people talk about the benefits of open source drivers. But really, think: If all these benefits why isn't there anymore open source drivers? And the answer is not "because companies are stupid". If we can't make it attractive enough to release driver source code then maybe the world just isn't matured yet. And we'll wait. But while waiting I want my 3d card to perform better and my mp3-player to work.
...
Bugga, here we went and turned it into opinions =)
/ Benjamin
Quoted 2 lines
As of now Linux won't force vendors into creating open source drivers.
it has happened several times already. So your claim is false.
Quoted 2 lines
But really, think: If all these benefits why isn't there anymore open source drivers?
there are over a thousand open source drivers, and at most a handful binary ones. Please go do your math.
Quoted 3 lines · Arjan van de Ven
there are over a thousand open source drivers, and at most a handful binary ones. Please go do your math.
You're doing the wrong comparison. How many drivers are missing or lacking in ability? And if you add to your handful of binary drivers those thousands that exist for xp...
well, numbers do change. Also, most open source drivers aren't made by the vendors themselves.
We're doing subjective math here. It doesn't change the fact that Linux would be better off with improved hardware support, right?
/ Benjamin
Quoted 5 lines · Benjamin Bach
Quoted 3 lines · Arjan van de Ven
there are over a thousand open source drivers, and at most a handful binary ones. Please go do your math.
You're doing the wrong comparison. How many drivers are missing
not too many. This is largely because hardware interfaces are getting increasingly standardized (it's cheaper for the hw vendors to not have to create a new driver for Windows XP)
Quoted 2 lines
or
lacking in ability?
some. But the vast majority is "good enough" by any standard.
Quoted 2 lines
And if you add to your handful of binary drivers
those thousands that exist for xp...
then it's clear that linux is better off ;)
(and yes while XP has more drivers, in linux a driver would generally drive the hardware that in the windows world uses 10 to 20 drivers)
Quoted 2 lines
well, numbers do change. Also, most open source drivers aren't made by the vendors themselves.
and? For standard interfaces... no big deal.
And for non-standard interfaces.. it's increasingly done with the vendor help
Quoted 3 lines
We're doing subjective math here. It doesn't change the fact that Linux would be better off with improved hardware support, right?
yes. But "more binary drivers" is absolutely not "better off"; but that's going towards the usual bimonthly troll topic so lets not go there and stop here.
Quoted 4 lines · Benjamin Bach
Quoted 3 lines · Arjan van de Ven
there are over a thousand open source drivers, and at most a handful binary ones. Please go do your math.
Quoted 9 lines
You're doing the wrong comparison. How many drivers are missing or lacking in ability? And if you add to your handful of binary drivers those thousands that exist for xp...
well, numbers do change. Also, most open source drivers aren't made by the vendors themselves.We're doing subjective math here. It doesn't change the fact that Linux would be better off with improved hardware support, right?
I strongly disagree.
Linux will be better off with improved hardware support by Open Source drivers - yes.
With improved hardware support by closed source, binary only drivers, Linux could very well end up being a *lot* worse off.
Once companies have the abillity to easily create closed drivers it seems very likely that more of them will start withholding specs, making it impossible to create open drivers.
This in turn will lead to less and less hardware supported by open drivers. The result of this will be more and more useless bugreports on LKML making it increasingly hard to maintain and improve the kernel. Users will report a bug that may or may not be in an open part of the kernel, but it will be impossible to tell due to all the binary only drivers for the users hardware she has loaded. Binary drivers also put the stability and security of the users system at the mercy of a commercial entity outside the users control. A binary blob loaded into the kernel could do *anything* to compromise the system or cause it to become unstable and noone would be able to tell if it's the driver or something else - Linux stability will suffer and users will in many cases blame Linux, not the driver vendors.
There's also the issue of companies going out of business, taking their closed drivers with them. Now the hardware that used to be supported by Linux is no longer useful since noone can keep the driver updated. One could then opt to keep the binary interfaces static to keep old vendor drivers working, but then we'd quickly accumulate a bunch of backwards compatibility cruft that would bloat the kernel and turn it into an unmaintainable mess.
For users there will be a short term bennefit of having more new and shiny toys work on Linux, but in the longer term it could easily destroy the usefulness of Linux as a free Operating System.
So please, let's *not* do work to make binary only closed source drivers easier to make or maintain, it's counterproductive in the long run.
It's a lot like peeing in your pants to try and stay warm in a blizzard. It may feel nice initially, but a little later on when your legs freeze up you'll be a lot worse off than you were initially.
--
Jesper Juhl <[email protected]>
Don't top-post http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please http://www.expita.com/nomime.html
Quoted 3 lines · Benjamin Bach
...
We're doing subjective math here. It doesn't change the fact that Linux would be better off with improved hardware support, right?
This is only true for the short-term future.
Whether Linux is better off with improved hardware support through binary modules for the long-term future is a more difficult question. Arjan recently posted an extreme example of the dangers of hardware support through binary modules [1].
You might agree or disagree with such opinions. But it has already been discussed to death on this list, and unless you are a troll you should read the past flamewars^Wdiscussions regarding this topic in the mailing list archives instead of restarting the same old discussion.
If you can accept the fact that less people consider your project useful than you might have hoped, there is no reason for you to not implement your project.
Quoted 1 line
/ Benjamin
cu
Adrian
[1] http://lwn.net/Articles/162686/
Signature · 4 lines
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon SeedWow, kudos to your work with the consequences of binary drivers. I certainly do not wish to add any redundant remarks of trolling sentences to this discussion. I've read +50 posts about binary drivers on this mailing list and in conclusion to that, I'd like to only add the following:
My idea was not to compromise the structure of the kernel. Nothing should be changed here. I also see a very notable resistance to binary drivers from distributions. Looking at the way ATI and NVidia drivers are treated by Fedora, SUSE and Ubuntu, I actually think they too have an agenda on this matter, and somehow it resembles their agenda on av-codecs. It's a sneaky-sneaky thing - if the user doesn't know a binary driver exists, we won't tell him. FC5 recently released made this huge "oops... we banned non-GPL modules in the kernel".
Anyways, I'm very happy with the combination of intelligence and idealism on this list, and suddenly I feel more attracted to writing a driver instead. For my Rio Karma mp3 player. It's a USB thing.. should be do-able in 3 months even though I've never written a driver.
Cheers everybody, and thanks for sharing! =)
/ Benjamin
Quoted 56 lines · Arjan van de Ven
Quoted 8 lines · Benjamin Bach
Quoted 4 lines · Arjan van de Ven
there are over a thousand open source drivers, and at most a handful binary ones. Please go do your math.
You're doing the wrong comparison. How many drivers are missing
not too many. This is largely because hardware interfaces are getting increasingly standardized (it's cheaper for the hw vendors to not have to create a new driver for Windows XP)
Quoted 3 lines
or
lacking in ability?some. But the vast majority is "good enough" by any standard.
Quoted 3 lines
And if you add to your handful of binary drivers
those thousands that exist for xp...then it's clear that linux is better off ;)
(and yes while XP has more drivers, in linux a driver would generally drive the hardware that in the windows world uses 10 to 20 drivers)Quoted 3 lines
well, numbers do change. Also, most open source drivers aren't made by the vendors themselves.
and? For standard interfaces... no big deal.
And for non-standard interfaces.. it's increasingly done with the vendor helpQuoted 3 lines
We're doing subjective math here. It doesn't change the fact that Linux would be better off with improved hardware support, right?
yes. But "more binary drivers" is absolutely not "better off"; but that's going towards the usual bimonthly troll topic so lets not go there and stop here.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
On 3/19/06, Benjamin Bach <[email protected]> wrote:
[snip]
Quoted 6 lines
Anyways, I'm very happy with the combination of intelligence and idealism on this list, and suddenly I feel more attracted to writing a driver instead. For my Rio Karma mp3 player. It's a USB thing.. should be do-able in 3 months even though I've never written a driver.
Writing a GPL'ed driver sounds like a great project.
For a USB device I'd suggest you take a look at libusb for writing a userspace driver as opposed to a kernelspace one. Userspace drivers for USB devices are often preferable.
Also take a look at a few documents in the Documentation/ dir in the kernel source:
HOWTO
SubmittingDrivers
CodingStyle
If you can get docs for the device (or are able to reverse engineer it) and get a driver done, more power to you. :-)
Quoted 7 lines
Cheers everybody, and thanks for sharing! =)
/ Benjamin
Arjan van de Ven wrote:
[snip]
Top-posting is generally frawned upon on the list as it's annoying and makes threads hard to read. Please consider changing that habbit.
--
Jesper Juhl <[email protected]>
Don't top-post http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please http://www.expita.com/nomime.html
Quoted 5 lines
Anyways, I'm very happy with the combination of intelligence and idealism on this list, and suddenly I feel more attracted to writing a driver instead. For my Rio Karma mp3 player. It's a USB thing.. should be do-able in 3 months even though I've never written a driver.
I've already done this and it is in 2.6.16. There's still some work to be done on the filesystem; check out
http://linux-karma.sourceforge.net to help out.
-Bob
Quoted 7 lines · Bob Copeland
Quoted 6 lines
Anyways, I'm very happy with the combination of intelligence and idealism on this list, and suddenly I feel more attracted to writing a driver instead. For my Rio Karma mp3 player. It's a USB thing.. should be do-able in 3 months even though I've never written a driver.
Quoted 5 lines
I've already done this and it is in 2.6.16. There's still some work to be done on the filesystem; check out
http://linux-karma.sourceforge.net to help out.
Yeah, I immediately found out =) Some very good explanation on your site (http://bobcopeland.com/karma/). I see that you've implemented most of the work, and I can't really figure out any isolated development suitable for a school project. But please, if you have an idea, say so.
Otherwise I'll probably dig up something. Just needs to be a small kernel-whatever project.
Is there someone maintaining a list of non-implemented ideas for kernel features/drivers?
/ Benjamin
Benjamin Bach wrote (ao):
Quoted 1 line
Linux get's better hardware support
With the help of binary drivers? Not likely. The smart people on this list can't help for example.
Quoted 2 lines
As of now Linux won't force vendors into creating open source drivers. That'd be like a small dog barking.
Well, as a customer you can put your money where your mouth is, and simply buy other hardware. That is a way of forcing, or at least stimulating the market.
As a kernel hacker you can choose to reverse engineer.
Btw, read this thread and tell me if you still agree that binary drivers and depending on a company are a good thing:
http://forum.matrox.com/mga/viewtopic.php?t=19208&sid=724d14d42714800f9d13ee6286f37f0b
Kind regards, Sander
Signature · 2 lines
Humilis IT Services and Solutions http://www.humilis.net
On 17 Mar 2006, Lee Revell murmured woefully:
Quoted 6 lines · Benjamin Bach
Quoted 5 lines
First off: I'm new on this list. I'm also new at starting projects. Anyways, I've decided on this idea for my first year computer science project. I have three months to set it off and definitely the whole thing would be released into the world afterwards.
Quoted 6 lines
Quoted 3 lines
Second: I don't want opinions on the issue of making it easier for companies to create binary (pre-compiled) drivers. Yes, we all want more open source drivers. Of course.
Then let's end the thread here because this is the crux of ther issue - no one WANTS it to be easier for vendors to ship binary drivers.
Well, this could be useful for distributors, too, except that distributors already have mechanisms for autobuilding kernels and modules.
Signature · 3 lines
`Come now, you should know that whenever you plan the duration of your unplanned downtime, you should add in padding for random management freakouts.'
Quoted 18 lines · Benjamin Bach
Quoted 7 lines · Bob Copeland
Quoted 6 lines
Anyways, I'm very happy with the combination of intelligence and idealism on this list, and suddenly I feel more attracted to writing a driver instead. For my Rio Karma mp3 player. It's a USB thing.. should be do-able in 3 months even though I've never written a driver.
Quoted 5 lines
I've already done this and it is in 2.6.16. There's still some work to be done on the filesystem; check out
http://linux-karma.sourceforge.net to help out.Yeah, I immediately found out =) Some very good explanation on your site
(http://bobcopeland.com/karma/). I see that you've implemented most of the work, and I can't really figure out any isolated development suitable for a school project. But please, if you have an idea, say so.
Quoted 5 lines
Otherwise I'll probably dig up something. Just needs to be a small kernel-whatever project.
Is there someone maintaining a list of non-implemented ideas for kernel features/drivers?
Wireless stack and drivers.
http://bcm43xx.berlios.de/
http://195.66.192.167/linux/acx_patches/current_sm/README
(you will also have to checkout wireless-git tree, above README explains how to do it)
--
vda
Quoted 5 lines · Benjamin Bach
Otherwise I'll probably dig up something. Just needs to be a small kernel-whatever project.
Is there someone maintaining a list of non-implemented ideas for kernel features/drivers?
Although neither of these are easy and you very well might not get anything done in three months, a couple of bits of hardware that I have for which there are incomplete/no drivers, and where the manufacturer refuses to give out specs are:
- Ricoh MMC/SD controllers. The project to figure those out is at:
http://mmc.drzeus.cx/wiki/Controllers/Ricoh/Frontreport
- 3D for NVidia. I know many people would take an open but basic 3D driver over the fully featured binary one - many people already use the 2D 'nv' driver for that reason. Rudolf Cornelissen has reverse engineered various bits of it (though it may apply only to the geforce-1 era cards) over here:
http://web.inter.nl.net/users/be-hold/BeOS/NVdriver/3dnews.html
You will find it's a whole lot easier to write drivers when you have specs though, and the resulting drivers will also be better. But depending on the scope of your project, you could definitely learn something either way.
Another thing that would be a lot easier to accomplish in 3 months would be to write a userspace filesystem using FUSE for something that isn't ordinarily accessed by filesystems; for example currently you can mount remote machines over ssh, cameras that can talk to gphoto, tar archives, gmail, etc.
Quoted 15 lines · Bob Copeland
Quoted 5 lines · Benjamin Bach
Otherwise I'll probably dig up something. Just needs to be a small kernel-whatever project.
Is there someone maintaining a list of non-implemented ideas for kernel features/drivers?
Although neither of these are easy and you very well might not get anything done in three months, a couple of bits of hardware that I have for which there are incomplete/no drivers, and where the manufacturer refuses to give out specs are:
- Ricoh MMC/SD controllers. The project to figure those out is at:
http://mmc.drzeus.cx/wiki/Controllers/Ricoh/Frontreport
Quoted 7 lines
- 3D for NVidia. I know many people would take an open but basic 3D driver over the fully featured binary one - many people already use the 2D 'nv' driver for that reason. Rudolf Cornelissen has reverse engineered various bits of it (though it may apply only to the geforce-1 era cards) over here:
http://web.inter.nl.net/users/be-hold/BeOS/NVdriver/3dnews.html
Lots of people don't even need 3D but have to run nvidia's driver to get multihead support, it seems this would be much easier than full 3D implementation.
Quoted 10 lines
You will find it's a whole lot easier to write drivers when you have specs though, and the resulting drivers will also be better. But depending on the scope of your project, you could definitely learn something either way.
Another thing that would be a lot easier to accomplish in 3 months would be to write a userspace filesystem using FUSE for something that isn't ordinarily accessed by filesystems; for example currently you can mount remote machines over ssh, cameras that can talk to gphoto, tar archives, gmail, etc.
Another easy project if you have old sound cards lying around is to port some of the old OSS drivers to ALSA (a list was posted on LKML a while back). This probably will take from a weekend to a few weeks.
Lee
Hi!
Quoted 14 lines
Quoted 5 lines
Otherwise I'll probably dig up something. Just needs to be a small kernel-whatever project.
Is there someone maintaining a list of non-implemented ideas for kernel features/drivers?
Although neither of these are easy and you very well might not get anything done in three months, a couple of bits of hardware that I have for which there are incomplete/no drivers, and where the manufacturer refuses to give out specs are:
- Ricoh MMC/SD controllers. The project to figure those out is at:
http://mmc.drzeus.cx/wiki/Controllers/Ricoh/Frontreport
Quoted 6 lines
- 3D for NVidia. I know many people would take an open but basic 3D driver over the fully featured binary one - many people already use the 2D 'nv' driver for that reason. Rudolf Cornelissen has reverse engineered various bits of it (though it may apply only to the geforce-1 era cards) over here:
http://web.inter.nl.net/users/be-hold/BeOS/NVdriver/3dnews.html
Or help openzaurus project. I'd certainly need help with collie :-).
Pavel
Signature · 1 line
Picture of sleeping (Linux) penguin wanted...