Some discussion points open source friendly graphics [was: HARDWARE: Open-Source-Friendly Graphics Cards -- Viable?]
I'm still trying to digest all the feedback I've been getting. It's overwhelming and gratifying, and I want to offer my gratitude for all the discussion and ideas.
It's been pointed out that some of this discussion may be getting off-topic for LKML. My addition to that is that soon, it will get very specialized in graphics math, and it would be best to move to a list where people actually talk about this on a regular basis.
To that end, I'd appreciate it people could point me to some other appropriate mailing lists. Not just the names, but URL's to the FAQ's which explain exactly how to subscribe, please.
Also, I'm thinking of starting my own yahoo groups list specifically for this chip. Is that a good idea?
Next, I'm getting lots of ideas from people. Some of them are core to the product, and some of them would be nice for follow-on products. For instance, dual-video would not be on the first model released. However, it is important that analog output always have crisp rise and fall times and be free of noise in order to maximize display quality.
The reprogramability of the FPGA has many advantages, but reprogramability is not its primary purpose. The primary reason to use an FPGA is to minimize NRE for manufacturing. However, as a result, users will be able to download updates. Additionally, those who are dedicated enough to reprogram it completely will find the necessary documentation to do so. Finally, it is my desire that we would release the source code to the FPGA for obsoleted products, however, it's too early to make promises.
Ok, now on to some design stuff:
The picture I have in my head at this time expands on the idea of the setup engine seen in most GPU's. What I'm thinking is that the setup engine will be general-purpose-ish CPU with special vector and matrix instructions. This way, the transformation stage will occur in "software" executed by a specialized processor. Additionally, the lighting phase might be done here as well.
The setup engine would produce triangle parameters which are fed to a rasterizer which does Gouraud shading and texture-mapping. That feeds pixels into something that handles antialiasing and alpha blending, etc.
The advantages are:
- The community can customize the setup engine as they please, just by writing code.
- This also includes the 2D emulation
- Anything "missing" can be emulated.
The disadvantages are:
- Triangle rate limited by speed of processor
- T&L is serialized, rather than being parallelized in dedicated hardware
- Phong shading and bump mapping may be impossible or too slow
It's been a while since I read about phong shading and bump mapping. As I understand it, some of the lighting phase is pushed into the rasterizer. With gouraud shading, colors are interpolated between the virtexes. With phong shading, the surface normals are interpolated across the triangle, and that's used to compute lighting. Bump mapping is like phong shading where the normals are specified in a bitmap.
What I don't know is how important bump-mapping and phong shading are.
Re: Some discussion points open source friendly graphics
Quoted 8 lines
It's been pointed out that some of this discussion may be getting off-topic for LKML. My addition to that is that soon, it will get very specialized in graphics math, and it would be best to move to a list where people actually talk about this on a regular basis.
To that end, I'd appreciate it people could point me to some other appropriate mailing lists. Not just the names, but URL's to the FAQ's which explain exactly how to subscribe, please.
Something like this?
http://www.opencores.org/forums.cgi/video/
Quoted 1 line · Timothy Miller
Also, I'm thinking of starting my own yahoo groups list specifically for this chip. Is that a good idea?
In theory it's useful; in reality at you'll have to balance what's public and what's not, and it's easy to allow yourself to get bogged down into "digesting", so much so that the "doing" is put off.
My only request is that you (a) make the graphics interface as simple and high level as feasible and (b) you publish the hardware interface specification as soon as possible, so that driver work can occur in parallel.
Quoted 1 line
Next, I'm getting lots of ideas from people. Some of them are core to the product, and some of them would be nice for follow-on products. For instance, dual-video would not be on the first model released. However, it is important that analog output always have crisp rise and fall times and be free of noise in order to maximize display quality.
Once you have a core design, it's easier to dicker about specific features. I would put this stuff on the "worry about it later" list.
Quoted 1 line
The reprogramability of the FPGA has many advantages, but reprogramability is not its primary purpose. The primary reason to use an FPGA is to minimize NRE for manufacturing. However, as a result, users will be able to download updates. Additionally, those who are
Will the capability to apply these updates be included with the base card? Will users need to purchase additional "update FPGA" hardware to do the reprogramming?
Quoted 13 lines
Ok, now on to some design stuff:
The picture I have in my head at this time expands on the idea of the setup engine seen in most GPU's. What I'm thinking is that the setup engine will be general-purpose-ish CPU with special vector and matrix instructions. This way, the transformation stage will occur in "software" executed by a specialized processor. Additionally, the lighting phase might be done here as well.
The setup engine would produce triangle parameters which are fed to a rasterizer which does Gouraud shading and texture-mapping. That feeds pixels into something that handles antialiasing and alpha blending, etc.
The advantages are:
- The community can customize the setup engine as they please, just by writing code.
- This also includes the 2D emulation
- Anything "missing" can be emulated.
Quoted 5 lines
The disadvantages are:
- Triangle rate limited by speed of processor
- T&L is serialized, rather than being parallelized in dedicated hardware
- Phong shading and bump mapping may be impossible or too slow
Well, I certainly like it :)
I think that a more generic approach allows you to recognize performance bottlenecks, and update the IP core to support instructions specific to (say) triangles.
Random closing notes:
* data movement will be an everpresent issue
* in graphics you really have a number of data types (16-bit float, etc.) that are specific to graphics. Supporting these natively should be quite helpful, if not an already-obvious prerequisite
Jeff
Quoted 6 lines · Jeff Garzik
Quoted 1 line · Timothy Miller
Also, I'm thinking of starting my own yahoo groups list specifically for this chip. Is that a good idea?
In theory it's useful; in reality at you'll have to balance what's public and what's not, and it's easy to allow yourself to get bogged down into "digesting", so much so that the "doing" is put off.
Right. I was just discussing this with the CEO, and he pointed out that I would get bogged down with too much online discussion. At FIRST, I might be able to handle an open list, but pretty quickly, I'd have to create an invite-only list where a select group who reads the other list would digest things for me.
Yeah, if I'm going to design a large chunk of the chip and write a large chunk of the software, I can't spend all day reading a mailing list. :)
Quoted 2 lines
My only request is that you (a) make the graphics interface as simple and high level as feasible and (b) you publish the hardware interface specification as soon as possible, so that driver work can occur in parallel.
Yes, and what I need to do is find the right developers to talk to so that I can given them interfaces that they actually NEED, because right now, I'm just guessing.
Quoted 6 lines
Quoted 3 lines
Next, I'm getting lots of ideas from people. Some of them are core to the product, and some of them would be nice for follow-on products. For instance, dual-video would not be on the first model released. However, it is important that analog output always have crisp rise and fall times and be free of noise in order to maximize display quality.
Once you have a core design, it's easier to dicker about specific features. I would put this stuff on the "worry about it later" list.
Definately. There's loads of features that are NICE, but not useful to most people. I need to make the lowest common demoninator first.
Quoted 6 lines
Quoted 2 lines
The reprogramability of the FPGA has many advantages, but reprogramability is not its primary purpose. The primary reason to use an FPGA is to minimize NRE for manufacturing. However, as a result, users will be able to download updates. Additionally, those who are
Will the capability to apply these updates be included with the base card? Will users need to purchase additional "update FPGA" hardware to do the reprogramming?
My intention is to include a bit of logic in the FPGA which can be used to reprogram the prom. You would then cycle power to get the FPGA to load the new code. Only if it gets corrupted would you have to solder on a header and reprogram it another way.
Also, if someone decides to reprogram it completely, and they haven't included a host interface and prom programmer in their design, then, well, they're on their own. :)
Quoted 23 lines
Quoted 14 lines
Ok, now on to some design stuff:
The picture I have in my head at this time expands on the idea of the setup engine seen in most GPU's. What I'm thinking is that the setup engine will be general-purpose-ish CPU with special vector and matrix instructions. This way, the transformation stage will occur in "software" executed by a specialized processor. Additionally, the lighting phase might be done here as well.
The setup engine would produce triangle parameters which are fed to a rasterizer which does Gouraud shading and texture-mapping. That feeds pixels into something that handles antialiasing and alpha blending, etc.
The advantages are:
- The community can customize the setup engine as they please, just by writing code.
- This also includes the 2D emulation
- Anything "missing" can be emulated.Quoted 5 lines
The disadvantages are:
- Triangle rate limited by speed of processor
- T&L is serialized, rather than being parallelized in dedicated hardware
- Phong shading and bump mapping may be impossible or too slowWell, I certainly like it :)
Quoted 7 lines
I think that a more generic approach allows you to recognize performance bottlenecks, and update the IP core to support instructions specific to (say) triangles.
Random closing notes:
* data movement will be an everpresent issue
* in graphics you really have a number of data types (16-bit float, etc.) that are specific to graphics. Supporting these natively should be quite helpful, if not an already-obvious prerequisite
Yeah, for the transformation phase, you need 32-bit floating point. After that, you can switch to 16.16 fixed point and get perfect results.
Quoted 3 lines · Timothy Miller
My intention is to include a bit of logic in the FPGA which can be used to reprogram the prom. You would then cycle power to get the FPGA to load the new code.
Power cycle the whole machine, or software-reset the card?
Regards,
Daniel
Quoted 10 lines · Daniel Phillips
Quoted 3 lines · Timothy Miller
My intention is to include a bit of logic in the FPGA which can be used to reprogram the prom. You would then cycle power to get the FPGA to load the new code.
Power cycle the whole machine, or software-reset the card?
Regards,
Daniel
The FPGA requires a physical reset-pin toggle to load new bits into the gate-array. This could be software-toggled, but that would require at least one external gate. This is because the power-reset needs to reset the chip before its bits are loaded plus some pin, after loading, needs to be an output to its reset pin also. Therefore, you need the external gate that is always present.
Cheers,
Dick Johnson
Penguin : Linux version 2.6.9 on an i686 machine (5537.79 BogoMips).
Notice : All mail here is now cached and reviewed by John Ashcroft.
98.36% of all statistics are fiction.
Quoted 16 lines · linux-os
Quoted 7 lines · Daniel Phillips
Quoted 3 lines · Timothy Miller
My intention is to include a bit of logic in the FPGA which can be used to reprogram the prom. You would then cycle power to get the FPGA to load the new code.
Quoted 5 lines
Power cycle the whole machine, or software-reset the card?
Regards,
Daniel
Quoted 7 lines
The FPGA requires a physical reset-pin toggle to load new bits into the gate-array. This could be software-toggled, but that would require at least one external gate. This is because the power-reset needs to reset the chip before its bits are loaded plus some pin, after loading, needs to be an output to its reset pin also. Therefore, you need the external gate that is always present.
Generally speaking, more chips == bad. If reprogramming the FPGA were a regular event, I'd see the point, but I hope that MOST users would NEVER have to reprogram it.
Quoted 8 lines · Daniel Phillips
Quoted 3 lines · Timothy Miller
My intention is to include a bit of logic in the FPGA which can be used to reprogram the prom. You would then cycle power to get the FPGA to load the new code.
Power cycle the whole machine, or software-reset the card?
I think you'll have to power-cycle the whole machine.
1) You have to get the FPGA's loader logic to reload the bitstream. While there are external signals to get the FPGA to reload, there may be a chicken&egg problem with getting soft logic in the FPGA to tell the FPGA to reload.
2) When you have reloaded the FPGA, registers are in an unknown state. One important thing that is lost if the PCI config space info. That would have to be restored by software.
Is it impossible to soft-reset it? Maybe not. But so much is lost and broken in the process that it may be highly impractical.
Nevertheless, if there's a way to do it, and the community wants to develop it, I'm all in favor.
Quoted 4 lines · Timothy Miller
Quoted 4 lines
The reprogramability of the FPGA has many advantages, but reprogramability is not its primary purpose. The primary reason to use an FPGA is to minimize NRE for manufacturing. However, as a result, users will be able to download updates. Additionally, those who are
Quoted 3 lines · Jeff Garzik
Will the capability to apply these updates be included with the base card? Will users need to purchase additional "update FPGA" hardware to do the reprogramming?
Also, what if the reprogramming goes wrong? Do I just throw the card away or will there be some form of recovery possible?
Signature · 7 lines
Giuseppe "Oblomov" Bilotta
Can't you see
It all makes perfect sense
Expressed in dollar and cents
Pounds shillings and pence
(Roger Waters)Quoted 5 lines · Giuseppe Bilotta
Quoted 1 line · Timothy Miller
Quoted 1 line
The reprogramability of the FPGA has many advantages, but reprogramability is not its primary purpose. The primary reason to use an FPGA is to minimize NRE for manufacturing. However, as a result, users will be able to download updates. Additionally, those who are
Quoted 8 lines
Quoted 2 lines · Jeff Garzik
Will the capability to apply these updates be included with the base card? Will users need to purchase additional "update FPGA" hardware to do the reprogramming?
Also, what if the reprogramming goes wrong? Do I just throw the card away or will there be some form of recovery possible?
For those who are taking the risk of reprogramming it completely, they'll already have read the schematics and instructions for using an external device to program the PROM.
For everyone else, it's the same problem you get when programming a motherboard goes awry. When the BIOS is hosed, you can't use the MB until you replace the chip.
For cost reasons, we likely wouldn't socket the chip, so you'd probably have to send it in for an RMA. We'd reprogram it, and send it back. Or if you have a friend with the right tools, they can do it.
Quoted 27 lines · Timothy Miller
Quoted 10 lines · Giuseppe Bilotta
Quoted 4 lines · Timothy Miller
Quoted 4 lines
The reprogramability of the FPGA has many advantages, but reprogramability is not its primary purpose. The primary reason to use an FPGA is to minimize NRE for manufacturing. However, as a result, users will be able to download updates. Additionally, those who are
Jeff Garzik wrote:
Quoted 8 lines
Quoted 3 lines
Will the capability to apply these updates be included with the base card? Will users need to purchase additional "update FPGA" hardware to do the reprogramming?
Also, what if the reprogramming goes wrong? Do I just throw the card away or will there be some form of recovery possible?
For those who are taking the risk of reprogramming it completely, they'll already have read the schematics and instructions for using an external device to program the PROM.
Quoted 8 lines
For everyone else, it's the same problem you get when programming a motherboard goes awry. When the BIOS is hosed, you can't use the MB until you replace the chip.
For cost reasons, we likely wouldn't socket the chip, so you'd probably have to send it in for an RMA. We'd reprogram it, and send it back. Or if you have a friend with the right tools, they can do it.
Normally you use a boundary-scan (JTAG) serial header so you can program, reprogram, debug the chip. FPGA development tools expect (require) this.
Check out http:/www.macraigor.com/full_gnu.htm for their GNU tools and devices, designed for Linux (and M$).
Cheers,
Dick Johnson
Penguin : Linux version 2.6.9 on an i686 machine (5537.79 GrumpyMips).
98.36% of all statistics are fiction.
Quoted 9 lines · linux-os
Quoted 4 lines · Timothy Miller
For cost reasons, we likely wouldn't socket the chip, so you'd probably have to send it in for an RMA. We'd reprogram it, and send it back. Or if you have a friend with the right tools, they can do it.
Quoted 6 lines
Normally you use a boundary-scan (JTAG) serial header so you can program, reprogram, debug the chip. FPGA development tools expect (require) this.
Check out http:/www.macraigor.com/full_gnu.htm for their GNU tools and devices, designed for Linux (and M$).
Sounds cool. The easier we can make it, the better.
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 1 line · pbecke
In many of the FPGA designs I have done, I have always tried to use some kind of parallel host interface port, that are built into many FPGAs to program the FPGA. This allows the FPGA to be loaded as if it is a memory device, Yes, it does mean that some external glue logic is usually required to decode address lines and such, but, it means that you can eliminate the eeprom, and usually the loading to the devices it self if much faster. I have more experience with the ram based FPGAs from Xilinx, then the eprom based ones from Actel, so it may not be feasible with the eprom based ones. But I really think that in system configuration is an advantage, that opens some possibilities that are much more difficult to implement if the design relies on a serial prom for the source of the FPGA configuration data.
My first desire is to minimize chip count, which means one FPGA with as much of the logic in it as possible. If we do multi-core designs, it would be logical to be able to reprogram each of the chips separately. I would bundle the PCI controller with the PROM reprogrammer and any other logic which doesn't change much and put the rest into the other chip(s). This way, you could reprogram the core without risking the whole thing.
Quoted 2 lines
By the way have you considered opencores.org's VGA controller as a starting point for a design?
I am aware of it, but I've hesitated to even look at it. I respect the author's wish to put it under a GPL-like license, and I don't want to "cheat". (The issue here isn't what's legal but what's ethical. Companies do bad things all the time which are perfectly legal, and I want to avoid that.) The author of that VGA core posted to the kerneltrap thread. Perhaps I can make some sort of arrangement with him. For instance, a friend of mine sold a commercial program which had some GPL'd components, but he first asked the authors to relicense under LGPL. I'd be happy to release my improvements to any PIECES that I got from elsewhere, but I want to have the option of not releasing the WHOLE thing.
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
Need help and advice... [was: HARDWARE: Open-Source-Friendly Graphics Cards -- Viable?]
I'm probably doing this too early, but it's been pointed out in so many words that I won't scale. I've spent two full work days digesting all of the discussions on LKML, kerneltrap, and slashdot. That's been as fun as you can imagine, but when the time comes for me to start working on the design, I can't do that at all anymore.
What I'll need are representatives of the community who work with me to make sure that my design does what everyone needs it to do, and I need representatives who engage in mailing lists and extract for me information that I should see.
I'm sure there are lots of people who would love to have that kind of influence over a company whose hardware they want to buy. I need the subset of those people who have expertise in 3D graphics, most of whom I'm sure are on OpenGL and X.org mailing lists right now. If there's anyone here who can get me connected with those people, I would appreciate it very much.
I feel like I'm being presumptuous. Although everyone at Tech Source is excited by the idea, and there seems to be a ground-swell of interest, the economic feasibility of this is still in the air. I don't want to waste anyone's time. However, a lot of excellent ideas have been put forth, and I'd like discussion to continue, but in a way that I can handle.
I humbly request advice on how I might go about handling all of this in the best way.
I also think the discussion should transition off LKML and onto its own forum.
Also, I want to thank everyone for talking to me about this. Members of this list and people posting to kerneltrap and slashdot have been very informative and helpful. I also want to mention Jeremy Andrews of kerneltrap.org who first turned my posting into news. I've wanted to do a project like this for a long time, but the past few days of online discussion have really helped push it towards becoming reality.
Quoted 2 lines · Timothy Miller
The reprogramability of the FPGA has many advantages, but reprogramability is not its primary purpose.
But it might turn out to be a reason for it turning into a geek trophy, if the price is not enormously higher than closed-spec cards. You could for example, program real-time sound effects processing into the FPGA and output the samples through a standard sound card.
The enthusiast market is a big market these days.
Quoted 10 lines
The picture I have in my head at this time expands on the idea of the setup engine seen in most GPU's. What I'm thinking is that the setup engine will be general-purpose-ish CPU with special vector and matrix instructions. This way, the transformation stage will occur in "software" executed by a specialized processor. Additionally, the lighting phase might be done here as well.
The setup engine would produce triangle parameters which are fed to a rasterizer which does Gouraud shading and texture-mapping. That feeds pixels into something that handles antialiasing and alpha blending, etc.
I hope you're planning to have a divider available to the rasterizer for perspective interpolation, particularly of textures.
Regards,
Daniel
Quoted 18 lines · Daniel Phillips
Quoted 2 lines · Timothy Miller
The reprogramability of the FPGA has many advantages, but reprogramability is not its primary purpose.
But it might turn out to be a reason for it turning into a geek trophy, if the price is not enormously higher than closed-spec cards. You could for example, program real-time sound effects processing into the FPGA and output the samples through a standard sound card.
The enthusiast market is a big market these days.
Quoted 7 lines
The picture I have in my head at this time expands on the idea of the setup engine seen in most GPU's. What I'm thinking is that the setup engine will be general-purpose-ish CPU with special vector and matrix instructions. This way, the transformation stage will occur in "software" executed by a specialized processor. Additionally, the lighting phase might be done here as well.
Quoted 6 lines
Quoted 3 lines
The setup engine would produce triangle parameters which are fed to a rasterizer which does Gouraud shading and texture-mapping. That feeds pixels into something that handles antialiasing and alpha blending, etc.
I hope you're planning to have a divider available to the rasterizer for perspective interpolation, particularly of textures.
My plans are not quite that specific at this time. For instance, I'm not entirely sure, yet, how perspective correction is done.