Machine config files
It's come up in other contexts recently, so I think it's worth mentioning that I am currently working on adding a machine config file support to qemu.
I am focusing on the board setup side of things, rather than the user-level config side. My goal is to come up with a system that will allow e.g. the entirety of realview.c and gumstix.c to be eliminated. Currently this is based this round Flattened Device Trees (as used by ppc-linux).
I have looked at the bits that Fabrice did a while ago. While that contains some good ideas (which I will probably steal!) it is approaching the problem from a somewhat different direction. FTDs are a much better fit for some of my requirements (e.g. being able to pass the config through to the guest OS).
I haven't entirely figured out how this will interact with user config (in particular user-pluggable devices like PCI NICs), but I think I can come up with something fairly sensible.
The code isn't even close to being functional at this point, and I can't give definite timescales, but with a bit of luck I'll have something by the end of the year.
Paul
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 8 lines · Paul Brook
It's come up in other contexts recently, so I think it's worth mentioning that I am currently working on adding a machine config file support to qemu.
I am focusing on the board setup side of things, rather than the user-level config side. My goal is to come up with a system that will allow e.g. the entirety of
realview.candgumstix.cto be eliminated. Currently this is based this round Flattened Device Trees (as used by ppc-linux).
Quoted 8 lines
I have looked at the bits that Fabrice did a while ago. While that contains some good ideas (which I will probably steal!) it is approaching the problem from a somewhat different direction. FTDs are a much better fit for some of my requirements (e.g. being able to pass the config through to the guest OS).
I haven't entirely figured out how this will interact with user config (in particular user-pluggable devices like PCI NICs), but I think I can come up with something fairly sensible.
Sounds great! For what it's worth, in the PowerPC KVM patches that I'll be posting here soon, we also use a flat device tree and pass it to the guest. We start with a template .dtb file in pc-bios/, then modify things like memory nodes at runtime based on qemu's command line options.
One practical consideration is that libfdt isn't generally available as a standalone package. I requested that Debian and Fedora create packages for it, and they agreed, but those obviously aren't available in older versions or other Linux distributions.
In the meantime, we could either include a libfdt snapshot in qemu (which is how it was originally intended to be used), or require users to build and install the package from source.
-Hollis
A message that is not in the archive (sent off-list or not yet received): fb412d760811140954j3252216cj226805ecfc002ffd-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org
Quoted 24 lines · Hollis Blanchard
Quoted 8 lines · Paul Brook
It's come up in other contexts recently, so I think it's worth mentioning that I am currently working on adding a machine config file support to qemu.
I am focusing on the board setup side of things, rather than the user-level config side. My goal is to come up with a system that will allow e.g. the entirety of
realview.candgumstix.cto be eliminated. Currently this is based this round Flattened Device Trees (as used by ppc-linux).Quoted 8 lines
I have looked at the bits that Fabrice did a while ago. While that contains some good ideas (which I will probably steal!) it is approaching the problem from a somewhat different direction. FTDs are a much better fit for some of my requirements (e.g. being able to pass the config through to the guest OS).
I haven't entirely figured out how this will interact with user config (in particular user-pluggable devices like PCI NICs), but I think I can come up with something fairly sensible.
Sounds great! For what it's worth, in the PowerPC KVM patches that I'll be posting here soon, we also use a flat device tree and pass it to the guest. We start with a template .dtb file in pc-bios/, then modify things like memory nodes at runtime based on qemu's command line options.
Quoted 4 lines
One practical consideration is that libfdt isn't generally available as a standalone package. I requested that Debian and Fedora create packages for it, and they agreed, but those obviously aren't available in older versions or other Linux distributions.
Yeah, I probably should have been a bit more active to try to get dtc and libfdt packaged for the various distros. Oh well, they'll get it eventually.
Quoted 3 lines
In the meantime, we could either include a libfdt snapshot in qemu (which is how it was originally intended to be used), or require users to build and install the package from source.
Well.. libfdt is designed to be easy to embed in other projects so that it can be incorporated into things like embedded bootloaders which will usually have interesting linking requirements. For a userspace application like this it would be nicer to use a system version, still, merging a snapshot is a reasonable approach for the interim until libfdt is more widely packaged.
It would be good to make sure, though, that you merge it in such a way that it's easy to update from upstream. Again, libfdt is laid out so that this should be easy.
If you want extra things in libfdt that would be convenient for you, just ask. The idea is that we build libfdt's features based on what people want my way of fdt manipulation.
Signature · 4 lines
David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson
Quoted 8 lines · Paul Brook
It's come up in other contexts recently, so I think it's worth mentioning that
I am currently working on adding a machine config file support to qemu.I am focusing on the board setup side of things, rather than the user-level config side. My goal is to come up with a system that will allow e.g. the entirety of
realview.candgumstix.cto be eliminated. Currently this is based this round Flattened Device Trees (as used by ppc-linux).
Quoted 4 lines
I have looked at the bits that Fabrice did a while ago. While that contains some good ideas (which I will probably steal!) it is approaching the problem from a somewhat different direction. FTDs are a much better fit for some of my requirements (e.g. being able to pass the config through to the guest OS).
libfdt looks like an excellent choice for both Sparc32 and Sparc64, I think the whole OF tree can be represented with it.
The config could be passed using the firmware configuration device.
Quoted 16 lines · Blue Swirl
Quoted 8 lines · Paul Brook
It's come up in other contexts recently, so I think it's worth mentioning that
I am currently working on adding a machine config file support to qemu.I am focusing on the board setup side of things, rather than the user-level config side. My goal is to come up with a system that will allow e.g. the entirety of
realview.candgumstix.cto be eliminated. Currently this is based this round Flattened Device Trees (as used by ppc-linux).Quoted 5 lines
I have looked at the bits that Fabrice did a while ago. While that contains some good ideas (which I will probably steal!) it is approaching the problem from a somewhat different direction. FTDs are a much better fit for some of my requirements (e.g. being able to pass the config through to the guest OS).
Quoted 5 lines
libfdt looks like an excellent choice for both Sparc32 and Sparc64, I think the whole OF tree can be represented with it.
The config could be passed using the firmware configuration device.
I was looking through libfdt too. I think it could also be used for x86. Hollis, I was looking through the Bamboo code, any reason you don't describe the bus/device layouts in the DTS and then use that to determine what bus/nic to add?
Regards,
Anthony Liguori
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 4 lines · On Friday 14 November 2008 13:29:12 Anthony Liguori wrote:
Quoted 3 lines · Blue Swirl
Quoted 1 line · Paul Brook
It's come up in other contexts recently, so I think it's worth mentioning
that
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
I am currently working on adding a machine config file support to qemu.
I am focusing on the board setup side of things, rather than the
user-level
Quoted 1 line
Quoted 1 line
Quoted 1 line
config side. My goal is to come up with a system that will allow e.g.
the
Quoted 1 line
Quoted 1 line
Quoted 1 line
entirety of
realview.candgumstix.cto be eliminated. Currently this
is
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
based this round Flattened Device Trees (as used by ppc-linux).
I have looked at the bits that Fabrice did a while ago. While that
contains
Quoted 1 line
Quoted 1 line
Quoted 1 line
some good ideas (which I will probably steal!) it is approaching the
problem
Quoted 1 line
Quoted 1 line
Quoted 1 line
from a somewhat different direction. FTDs are a much better fit for
some of
Quoted 1 line
Quoted 1 line
Quoted 1 line
my requirements (e.g. being able to pass the config through to the guest
OS).
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
libfdt looks like an excellent choice for both Sparc32 and Sparc64, I think the whole OF tree can be represented with it.
Quoted 7 lines
Quoted 2 lines
The config could be passed using the firmware configuration device.
I was looking through libfdt too. I think it could also be used for x86. Hollis, I was looking through the Bamboo code, any reason you don't describe the bus/device layouts in the DTS and then use that to determine what bus/nic to add?
We do; PLB, OPB, and EBC are all on-chip busses found on IBM PowerPC SoCs. For example, you can see devices representing the UARTs and PCI controller in our bamboo.dts file. Qemu only modifies the device tree at runtime with information not known until then (memory size, clock frequency, etc).
You may mean why don't we enumerate *PCI* devices in the device tree. Since libfdt is used for embedded systems, this is a tradeoff that allows us to keep the device tree smaller -- why put lots of PCI nodes in the device tree, when the OS can easily discover them itself? That said, PCI devices are indeed found in the device tree in IBM Power servers; Linux just doesn't require them.
The Bamboo board in particular doesn't have any on-board PCI devices, but it does have several PCI slots. Therefore, a Bamboo device tree in qemu wouldn't contain any PCI devices anyways, but if the user says "I have an e1000", qemu could add a device node to represent the e1000 (though again, this is not a Linux requirement).
(Actually, I don't think the above addresses your specific question, but I went to the trouble of typing it so I figured I'd leave it in this mail for additional background. :)
Getting back to your original question, we could include an e1000 PCI device node in the DTS file, and have qemu walk the tree, find the node, and initialize the e1000 emulation driver. We didn't do this mostly to maintain compatibility with existing qemu practice, where users are required to specify all configuration on the command line.
Signature · 2 lines
Hollis Blanchard IBM Linux Technology Center
Quoted 2 lines · Hollis Blanchard
We do; PLB, OPB, and EBC are all on-chip busses found on IBM PowerPC SoCs. For example, you can see devices representing the UARTs and PCI controller in our
bamboo.dtsfile. Qemu only modifies the device tree at runtime with information not known until then (memory size, clock frequency, etc).
Yes, but you don't build the machine based on the device tree. What would be useful would be to be able to simply write a device tree description and that would fully represent the board such that you didn't need a ppc440_bamboo.c file at all.
So what I'm getting at is, what is preventing us from being able to do this today?
Regards,
Anthony Liguori
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 2 lines · On Friday 14 November 2008 14:11:32 Anthony Liguori wrote:
Quoted 1 line · Hollis Blanchard
We do; PLB, OPB, and EBC are all on-chip busses found on IBM PowerPC SoCs.
For
Quoted 1 line
Quoted 1 line
example, you can see devices representing the UARTs and PCI controller in
our
Quoted 11 lines
Quoted 3 lines
bamboo.dtsfile. Qemu only modifies the device tree at runtime with information not known until then (memory size, clock frequency, etc).Yes, but you don't build the machine based on the device tree. What would be useful would be to be able to simply write a device tree description and that would fully represent the board such that you didn't need a
ppc440_bamboo.cfile at all.So what I'm getting at is, what is preventing us from being able to do this today?
Theoretically, nothing. If qemu were a clean and modular place, it would probably be pretty straightforward.
Some translation code would be required for compatibility. For example, when the DTS has 128MB of memory, and the user invokes qemu with -m 256, now you need to update the device tree in order to pass it into the guest. That's pretty easy, but once you start talking about adding PCI devices it gets a little more difficult. I seem to recall this conversation was had on qemu-devel a little while back.
Of course, *some* code would still be needed *somewhere* to load the kernel, initrd, set initial register state to point to those memory locations, etc. In the case of KVM on 440, we also need to override the DTS with the real host clock frequency. (This probably isn't necessary for qemu+TCG.)
But yes, replacing all the pci_nic_init(), isa_mmio_init(), cpu_register_physical_memory(), etc could be automated by walking the device tree.
Signature · 2 lines
Hollis Blanchard IBM Linux Technology Center
Quoted 6 lines · Hollis Blanchard
On Friday 14 November 2008 14:11:32 Anthony Liguori wrote:
Theoretically, nothing. If qemu were a clean and modular place, it would probably be pretty straightforward.
Some translation code would be required for compatibility. For example, when the DTS has 128MB of memory, and the user invokes qemu with -m 256, now you need to update the device tree in order to pass it into the guest. That's pretty easy, but once you start talking about adding PCI devices it gets a little more difficult. I seem to recall this conversation was had on qemu-devel a little while back.
The way I would expect that it would work is that you could invoke QEMU with a device tree or it would by default use a device tree "template". The options like '-m' would modify the device tree template. There's probably no need to differentiate between a user supplied device tree and a device tree template.
Quoted 4 lines
Of course, *some* code would still be needed *somewhere* to load the kernel, initrd, set initial register state to point to those memory locations, etc. In the case of KVM on 440, we also need to override the DTS with the real host clock frequency. (This probably isn't necessary for qemu+TCG.)
But yes, replacing all the
pci_nic_init(),isa_mmio_init(),cpu_register_physical_memory(), etc could be automated by walking the device tree.
Ok. This is starting to look interesting. Paul, what do you think of DTS?
Regards,
Anthony Liguori
Quoted 13 lines
Quoted 6 lines
Some translation code would be required for compatibility. For example, when the DTS has 128MB of memory, and the user invokes qemu with -m 256, now you need to update the device tree in order to pass it into the guest. That's pretty easy, but once you start talking about adding PCI devices it gets a little more difficult. I seem to recall this conversation was had on qemu-devel a little while back.
...
Quoted 6 lines
Of course, *some* code would still be needed *somewhere* to load the kernel, initrd, set initial register state to point to those memory locations, etc. In the case of KVM on 440, we also need to override the DTS with the real host clock frequency. (This probably isn't necessary for qemu+TCG.)
Quoted 5 lines
Quoted 3 lines
But yes, replacing all the
pci_nic_init(),isa_mmio_init(),cpu_register_physical_memory(), etc could be automated by walking the device tree.Ok. This is starting to look interesting. Paul, what do you think of DTS?
Things like kernel loading are easy, you just add a special device (which linux will ignore) specifying the relevant parameters. IMO it should be possible to use the same device tree for both linux and qemu, but it's ok to require some additional information on top of what linux currently requires.
I agree with Hollis that the tricky bits are when you want to augment a board with use supplied options (e.g. RAM size or additional PCI devices). That either requires qemu to modify the supplied device tree, merging multiple trees, or some other magic. I'm ignoring this problem for now. Worst case we remove the existing options, provide a reasonable set of "standard" configs and anyone who wants exotic setups can modify the machine description.
I'm mainly interested in emulation, where any host information leaking through to the emulated machine is a bug :-)
Paul
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 17 lines · Paul Brook
Quoted 17 lines
Quoted 6 lines
Some translation code would be required for compatibility. For example, when the DTS has 128MB of memory, and the user invokes qemu with -m 256, now you need to update the device tree in order to pass it into the guest. That's pretty easy, but once you start talking about adding PCI devices it gets a little more difficult. I seem to recall this conversation was had on qemu-devel a little while back.
...
Quoted 9 lines
Of course, *some* code would still be needed *somewhere* to load the kernel, initrd, set initial register state to point to those memory locations, etc. In the case of KVM on 440, we also need to override the DTS with the real host clock frequency. (This probably isn't necessary for qemu+TCG.)
But yes, replacing all the
pci_nic_init(),isa_mmio_init(),cpu_register_physical_memory(), etc could be automated by walking the device tree.
Quoted 6 lines
Quoted 1 line
Ok. This is starting to look interesting. Paul, what do you think of DTS?
Things like kernel loading are easy, you just add a special device (which linux will ignore) specifying the relevant parameters. IMO it should be possible to use the same device tree for both linux and qemu, but it's ok to require some additional information on top of what linux currently requires.
Depending on exactly what you want to pass through, it may be appropriate to use properties in /chosen for this.
Signature · 4 lines
David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson
A message that is not in the archive (sent off-list or not yet received): 200811141351.40613.hollisb-r/[email protected]
Quoted 42 lines · Hollis Blanchard
Quoted 4 lines · On Friday 14 November 2008 13:29:12 Anthony Liguori wrote:
Quoted 3 lines · Blue Swirl
Quoted 1 line · Paul Brook
It's come up in other contexts recently, so I think it's worth mentioning
that
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
I am currently working on adding a machine config file support to qemu.
I am focusing on the board setup side of things, rather than the
user-level
Quoted 1 line
Quoted 1 line
Quoted 1 line
config side. My goal is to come up with a system that will allow e.g.
the
Quoted 1 line
Quoted 1 line
Quoted 1 line
entirety of
realview.candgumstix.cto be eliminated. Currently thisis
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
based this round Flattened Device Trees (as used by ppc-linux).
I have looked at the bits that Fabrice did a while ago. While that
contains
Quoted 1 line
Quoted 1 line
Quoted 1 line
some good ideas (which I will probably steal!) it is approaching the
problem
Quoted 1 line
Quoted 1 line
Quoted 1 line
from a somewhat different direction. FTDs are a much better fit for
some of
Quoted 1 line
Quoted 1 line
Quoted 1 line
my requirements (e.g. being able to pass the config through to the guest
OS).
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
libfdt looks like an excellent choice for both Sparc32 and Sparc64, I think the whole OF tree can be represented with it.
Quoted 7 lines
Quoted 2 lines
The config could be passed using the firmware configuration device.
I was looking through libfdt too. I think it could also be used for x86. Hollis, I was looking through the Bamboo code, any reason you don't describe the bus/device layouts in the DTS and then use that to determine what bus/nic to add?
We do; PLB, OPB, and EBC are all on-chip busses found on IBM PowerPC SoCs. For example, you can see devices representing the UARTs and PCI controller in our
bamboo.dtsfile. Qemu only modifies the device tree at runtime with information not known until then (memory size, clock frequency, etc).
Quoted 6 lines
You may mean why don't we enumerate *PCI* devices in the device tree. Since libfdt is used for embedded systems, this is a tradeoff that allows us to keep the device tree smaller -- why put lots of PCI nodes in the device tree, when the OS can easily discover them itself? That said, PCI devices are indeed found in the device tree in IBM Power servers; Linux just doesn't require them.
Yes, in general the rule is that devices on busses that can be runtime probed - PCI, USB and the like - don't typically need to appear in a flattened device tree (though they would traditionally appear in a full Open Firmware device tree). Only the things the OS has to "just know" are there need appear - e.g. on-chip devices, hardwarired on-board devices, unprobeable legacy ISA devices, usually the PCI host bridge, and so forth.
Nodes for probeable devices may optionally appear, and in certain cases they must appear to supply bits of information that can't be probed. For example, on-board PCI devices often have interrupts routed directly to the system PIC, rather than going across the PCI bus. In this case a device node would need to appear with the interrupt routing information.
Signature · 4 lines
David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson