Release plan for 0.12.0

Now that 0.11.0 is behind us, it's time to start thinking about 0.12.0.

I'd like to do a few things different this time around.  I don't think the -rc process went very well as I don't think we got more testing out of it. I'd like to shorten the timeline for 0.12.0 a good bit.  The 0.10 stable tree got pretty difficult to maintain toward the end of the cycle.  We also had a pretty huge amount of change between 0.10 and 0.11 so I think a shorter cycle is warranted.

I think aiming for early to mid-December would give us roughly a 3 month cycle and would align well with some of the Linux distribution cycles.  I'd like to limit things to a single -rc that lasted only for about a week. This is enough time to fix most of the obvious issues I think.

Quoted 9 lines · Anthony Liguori

Now that 0.11.0 is behind us, it's time to start thinking about 0.12.0.

I'd like to do a few things different this time around. I don't think the -rc process went very well as I don't think we got more testing out of it.
I'd like to shorten the timeline for 0.12.0 a good bit. The 0.10 stable
tree got pretty difficult to maintain toward the end of the cycle. We also had a pretty huge amount of change between 0.10 and 0.11 so I think a shorter cycle is warranted.

Quoted 5 lines

I think aiming for early to mid-December would give us roughly a 3 month cycle and would align well with some of the Linux distribution cycles. I'd like to limit things to a single -rc that lasted only for about a week.
This is enough time to fix most of the obvious issues I think.

As a downstream packager of qemu-kvm, I thought I'd mention that the next Ubuntu cycle is now public:
* https://wiki.ubuntu.com/LucidReleaseSchedule

The key date here is Feature Freeze, which is February 25, 2010. That's the point by which we'd need to have a new qemu-kvm (which of course is downstream of qemu) package in Ubuntu for the LTS 10.04 release in April 2010.

Hi,

Now that 0.11.0 is behind us, it's time to start thinking about 0.12.0.

I'd like to do a few things different this time around. I don't think the -rc process went very well as I don't think we got more testing out of it. I'd like to shorten the timeline for 0.12.0 a good bit. The 0.10 stable tree got pretty difficult to maintain toward the end of the cycle. We also had a pretty huge amount of change between 0.10 and 0.11 so I think a shorter cycle is warranted.

I think aiming for early to mid-December would give us roughly a 3 month cycle and would align well with some of the Linux distribution cycles. I'd like to limit things to a single -rc that lasted only for about a week. This is enough time to fix most of the obvious issues I think.

I'd also like to try to enumerate some features for this release. Here's a short list of things I expect to see for this release (target-i386 centric). Please add or comment on items that you'd either like to see in the release or are planning on working on.

o VMState conversion -- I expect most of the pc target to be completed
o qdev conversion -- I hope that we'll get most of the pc target completely converted to qdev
o storage live migration
o switch to SeaBIOS (need to finish porting features from Bochs)
o switch to gPXE (need to resolve slirp tftp server issue)
o KSM integration
o in-kernel APIC support for KVM
o guest SMP support for KVM
o updates to the default pc machine type

Please add to this list and I'll collect it all and post it somewhere.

o newer chipset (which is based on Q35 chipset)
o multiple pci bus
o PCI express (MMCONFIG)
o PCI express hot plug (not acpi based)
o PCI express switch emulator

Although there is no PCIe emulated device at the moment, this will be a fundamental infrastructure for PCI express native direct attach.

Hi Isaku,

Quoted 9 lines · Isaku Yamahata

o newer chipset (which is based on Q35 chipset)
o multiple pci bus o PCI express (MMCONFIG)
o PCI express hot plug (not acpi based)
o PCI express switch emulator

Although there is no PCIe emulated device at the moment, this will be a fundamental infrastructure for PCI express native
direct attach.

Your patches definitely deserve review/commit. I'll make sure that happens for the 0.12 time frame.

Michael, could you help review some of the PCI patches?

Thanks,

Signature · 3 lines

Regards,

Anthony Liguori

Hi,

Now that 0.11.0 is behind us, it's time to start thinking about 0.12.0.

I'd like to do a few things different this time around. I don't think the -rc process went very well as I don't think we got more testing out of it. I'd like to shorten the timeline for 0.12.0 a good bit. The 0.10 stable tree got pretty difficult to maintain toward the end of the cycle. We also had a pretty huge amount of change between 0.10 and 0.11 so I think a shorter cycle is warranted.

I think aiming for early to mid-December would give us roughly a 3 month cycle and would align well with some of the Linux distribution cycles. I'd like to limit things to a single -rc that lasted only for about a week. This is enough time to fix most of the obvious issues I think.

I'd also like to try to enumerate some features for this release. Here's a short list of things I expect to see for this release (target-i386 centric). Please add or comment on items that you'd either like to see in the release or are planning on working on.

o VMState conversion -- I expect most of the pc target to be completed
o qdev conversion -- I hope that we'll get most of the pc target completely converted to qdev
o storage live migration
o switch to SeaBIOS (need to finish porting features from Bochs)
o switch to gPXE (need to resolve slirp tftp server issue)
o KSM integration
o in-kernel APIC support for KVM
o guest SMP support for KVM
o updates to the default pc machine type

Please add to this list and I'll collect it all and post it somewhere.

On (Tue) Sep 29 2009 [18:54:53], Anthony Liguori wrote:

o multiport virtio-console support

Quoted 3 lines · Amit Shah

On (Tue) Sep 29 2009 [18:54:53], Anthony Liguori wrote:
o multiport virtio-console support

Assuming we can get the kernel drivers straightened out, I think it's certainly reasonable for 0.12.

Quoted 2 lines · Anthony Liguori

Amit Shah wrote:

Quoted 6 lines

Quoted 4 lines · Anthony Liguori

o multiport virtio-console support

Assuming we can get the kernel drivers straightened out, I think it's certainly reasonable for 0.12.

The kernel drivers are in fine shape.

Amit

Quoted 7 lines · Amit Shah

Quoted 5 lines · Anthony Liguori

Quoted 3 lines · Amit Shah

On (Tue) Sep 29 2009 [18:54:53], Anthony Liguori wrote:
o multiport virtio-console support

Quoted 4 lines

Quoted 2 lines

Assuming we can get the kernel drivers straightened out, I think it's
certainly reasonable for 0.12.

The kernel drivers are in fine shape.

I meant on track for including into the appropriate tree. Looking for an Ack/Nack from Rusty. That's been the general policy for all virtio changes btw. Nothing specific to virtio-console.

Hi,

Now that 0.11.0 is behind us, it's time to start thinking about 0.12.0.

I'd like to do a few things different this time around. I don't think the -rc process went very well as I don't think we got more testing out of it. I'd like to shorten the timeline for 0.12.0 a good bit. The 0.10 stable tree got pretty difficult to maintain toward the end of the cycle. We also had a pretty huge amount of change between 0.10 and 0.11 so I think a shorter cycle is warranted.

I think aiming for early to mid-December would give us roughly a 3 month cycle and would align well with some of the Linux distribution cycles. I'd like to limit things to a single -rc that lasted only for about a week. This is enough time to fix most of the obvious issues I think.

I'd also like to try to enumerate some features for this release. Here's a short list of things I expect to see for this release (target-i386 centric). Please add or comment on items that you'd either like to see in the release or are planning on working on.

o VMState conversion -- I expect most of the pc target to be completed
o qdev conversion -- I hope that we'll get most of the pc target completely converted to qdev
o storage live migration
o switch to SeaBIOS (need to finish porting features from Bochs)
o switch to gPXE (need to resolve slirp tftp server issue)
o KSM integration
o in-kernel APIC support for KVM
o guest SMP support for KVM
o updates to the default pc machine type

Quoted 10 lines · Anthony Liguori

Hi,

Now that 0.11.0 is behind us, it's time to start thinking about 0.12.0.

I'd like to do a few things different this time around. I don't think the -rc process went very well as I don't think we got more testing out of it. I'd like to shorten the timeline for 0.12.0 a good bit. The 0.10 stable tree got pretty difficult to maintain toward the end of the cycle. We also had a pretty huge amount of change between 0.10 and 0.11 so I think a shorter cycle is warranted.

I think aiming for early to mid-December would give us roughly a 3 month cycle and would align well with some of the Linux distribution cycles. I'd like to limit things to a single -rc that lasted only for about a week. This is enough time to fix most of the obvious issues I think.

I'd also like to try to enumerate some features for this release. Here's a short list of things I expect to see for this release (target-i386 centric). Please add or comment on items that you'd either like to see in the release or are planning on working on.

Quoted 9 lines

o VMState conversion -- I expect most of the pc target to be completed
o qdev conversion -- I hope that we'll get most of the pc target completely converted to qdev
o storage live migration
o switch to SeaBIOS (need to finish porting features from Bochs)
o switch to gPXE (need to resolve slirp tftp server issue)
o KSM integration
o in-kernel APIC support for KVM
o guest SMP support for KVM
o updates to the default pc machine type

Machine monitor protocol.

Quoted 34 lines · Avi Kivity

Quoted 32 lines · Anthony Liguori

Hi,

Now that 0.11.0 is behind us, it's time to start thinking about 0.12.0.

I'd like to do a few things different this time around. I don't think the -rc process went very well as I don't think we got more testing out of it. I'd like to shorten the timeline for 0.12.0 a good bit. The 0.10 stable tree got pretty difficult to maintain toward the end of the cycle. We also had a pretty huge amount of change between 0.10 and 0.11 so I think a shorter cycle is warranted.

I think aiming for early to mid-December would give us roughly a 3 month cycle and would align well with some of the Linux distribution cycles. I'd like to limit things to a single -rc that lasted only for about a week. This is enough time to fix most of the obvious issues I think.

I'd also like to try to enumerate some features for this release. Here's a short list of things I expect to see for this release (target-i386 centric). Please add or comment on items that you'd either like to see in the release or are planning on working on.

o VMState conversion -- I expect most of the pc target to be completed
o qdev conversion -- I hope that we'll get most of the pc target completely converted to qdev
o storage live migration
o switch to SeaBIOS (need to finish porting features from Bochs)
o switch to gPXE (need to resolve slirp tftp server issue)
o KSM integration
o in-kernel APIC support for KVM
o guest SMP support for KVM
o updates to the default pc machine type

Quoted 1 line

Machine monitor protocol.

If we're going to support the protocol for 0.12, I'd like to most of the code merged by the end of October.

Quoted 3 lines

If we're going to support the protocol for 0.12, I'd like to most of the code merged by the end of October.

Four weeks.. Not so much time, but let's try.

There are two major issues that may delay QMP.

Firstly, we are still on the infrastructure/design phase, which
is natural to take time. Maybe when handlers start getting converted en masse things will be faster.

Secondly: testing. I have a very ugly python script to test the
already converted handlers. The problem is not only the ugliness, the right way to do this would be to use kvm-autotest. So, I was planning to take a detailed look at it and perhaps start writing tests for QMP right when each handler is converted. Right Thing, but takes time.

Quoted 5 lines · Luiz Capitulino

Quoted 3 lines

If we're going to support the protocol for 0.12, I'd like to most of the code merged by the end of October.

Four weeks.. Not so much time, but let's try.

Quoted 6 lines

There are two major issues that may delay QMP.

Firstly, we are still on the infrastructure/design phase, which
is natural to take time. Maybe when handlers start getting converted en masse things will be faster.

I sure hope so. Maybe someone can pitch in if not.

Quoted 7 lines

Secondly: testing. I have a very ugly python script to test the
already converted handlers. The problem is not only the ugliness, the right way to do this would be to use kvm-autotest. So, I was planning to take a detailed look at it and perhaps start writing tests for QMP right when each handler is converted. Right Thing, but takes time.

I think this could be done by having autotest use two monitors, one with the machine protocol and one with the human protocol, trying first the machine protocol and falling back if the command is not supported.

Hopefully we can get the autotest people to work on it so we parallelize development. They'll also give user-oriented feedback which can be valuable.

Are you using a standard json parser with your test script? That's an additional validation.

Quoted 3 lines

Are you using a standard json parser with your test script? That's an additional validation.

I'm using Python's json module, but I could run one of the checkers
listed in the json's page for each test, before the Python's module kicks in.

Quoted 22 lines · Anthony Liguori

Hi,

Now that 0.11.0 is behind us, it's time to start thinking about 0.12.0.

I'd like to do a few things different this time around. I don't think the -rc process went very well as I don't think we got more testing out of it. I'd like to shorten the timeline for 0.12.0 a good bit. The 0.10 stable tree got pretty difficult to maintain toward the end of the cycle. We also had a pretty huge amount of change between 0.10 and 0.11 so I think a shorter cycle is warranted.

I think aiming for early to mid-December would give us roughly a 3 month cycle and would align well with some of the Linux distribution cycles. I'd like to limit things to a single -rc that lasted only for about a week. This is enough time to fix most of the obvious issues I think.

I'd also like to try to enumerate some features for this release. Here's a short list of things I expect to see for this release (target-i386 centric). Please add or comment on items that you'd either like to see in the release or are planning on working on.

Quoted 10 lines

o VMState conversion -- I expect most of the pc target to be completed
o qdev conversion -- I hope that we'll get most of the pc target completely converted to qdev
o storage live migration
o switch to SeaBIOS (need to finish porting features from Bochs)
o switch to gPXE (need to resolve slirp tftp server issue)
o KSM integration
o in-kernel APIC support for KVM
o guest SMP support for KVM
o updates to the default pc machine type

Machine monitor protocol.

Here's a short list of things I expect to see for this release (target-i386 centric). Please add or comment on items that you'd either like to see in the release or are planning on working on.

o guest SMP support for KVM

Anthony Liguori wrote:
[]

Quoted 1 line

Here's a short list of things I expect to see for this release (target-i386 centric). Please add or comment on items that you'd either like to see in the release or are planning on working on.

[..]

Quoted 1 line

o guest SMP support for KVM

Hmm. What is this, can you elaborate a bit more please? -smp nn is already here, no?

Now that 0.11.0 is behind us, it's time to start thinking about 0.12.0.

I'd also like to try to enumerate some features for this release. Here's a short list of things I expect to see for this release (target-i386 centric).

o switch to SeaBIOS (need to finish porting features from Bochs)

Hi,

Quoted 7 lines · Anthony Liguori

Now that 0.11.0 is behind us, it's time to start thinking about 0.12.0.

I'd also like to try to enumerate some features for this release. Here's a short list of things I expect to see for this release (target-i386 centric).

o switch to SeaBIOS (need to finish porting features from Bochs)

That switch is much appreciated because it also reduces the testing matrix of those coreboot developers who boot test every commit with Qemu.

However, to run coreboot on Qemu with the same init sequence as on simplified real hardware, we need Cache-as-RAM (CAR) support. This is basically a mode where sizeof(cacheable area) <= sizeof (L2 cache) and causes the processor to lock the cache and not pass any reads/writes through to the RAM behind the cached area. The easiest way to implement this would be to check the cache size criterion upon every MTRR manipulation and either map a chunk of fresh memory on top of the existing memory (which may be RAM, ROM or unmapped) for every cacheable area, and if the cacheable area starts to exceed the L2 cache size, discard all memory contents of the memory mapped on top.
For additional correctness, the memory shoud not be discarded and written back to the lower layer of memory if WBINVD (instead of INVD) or CLFLUSH are called. That one is mostly sugar, though, and coreboot can do without.

Right now coreboot sets up the MTRRs correctly, but then (conditional on Qemu) only uses areas which are known to be backed by RAM instead of the areas designated by CAR.

I'd like to implement CAR support which builds on top of my MTRR code which was merged some months ago (and I already have code to check for total cacheable area size), but I need help with the memory mapping stuff. How do I proceed? Clean up what I have and insert "FIXME" comments where I don't know how to implement stuff so others can see the code and comment on it?

Regards,
Carl-Daniel

Quoted 2 lines · Carl-Daniel Hailfinger

However, to run coreboot on Qemu with the same init sequence as on simplified real hardware, we need Cache-as-RAM (CAR) support. [...]

Do we really need coreboot to use the same init sequence? coreboot
is firmware and we don't necessarily run real firmware under QEMU. It's a short cut that lets us avoid a lot of complexity.

Quoted 10 lines

Right now coreboot sets up the MTRRs correctly, but then (conditional on Qemu) only uses areas which are known to be backed by RAM instead of the areas designated by CAR.

I'd like to implement CAR support which builds on top of my MTRR code which was merged some months ago (and I already have code to check for total cacheable area size), but I need help with the memory mapping stuff. How do I proceed? Clean up what I have and insert "FIXME" comments where I don't know how to implement stuff so others can see the code and comment on it?

You could start there. But from a higher level, I'm not sure I think a partial implementation of something like CAR is all that valuable since coreboot already runs under QEMU.

Quoted 7 lines · Anthony Liguori

Quoted 2 lines · Carl-Daniel Hailfinger

However, to run coreboot on Qemu with the same init sequence as on simplified real hardware, we need Cache-as-RAM (CAR) support. [...]

Do we really need coreboot to use the same init sequence?   coreboot is firmware and we don't necessarily run real firmware under QEMU. It's a short cut that lets us avoid a lot of complexity.

I know that some people were running 440BX BIOS images for real hardware on Qemu and they got pretty far.

The complexity would be limited to the MTRR code and unless there were major architectural changes in mapping RAM to address ranges, no other code (except VM save and VM restore) should get even a single line changed.

Quoted 11 lines

Quoted 10 lines

Right now coreboot sets up the MTRRs correctly, but then (conditional on Qemu) only uses areas which are known to be backed by RAM instead of the areas designated by CAR.

I'd like to implement CAR support which builds on top of my MTRR code which was merged some months ago (and I already have code to check for total cacheable area size), but I need help with the memory mapping stuff. How do I proceed? Clean up what I have and insert "FIXME" comments where I don't know how to implement stuff so others can see the code and comment on it?

Quoted 3 lines

You could start there.  But from a higher level, I'm not sure I think a partial implementation of something like CAR is all that valuable since coreboot already runs under QEMU.

It only runs if WORKAROUND_QEMU is defined (maybe not exactly that name, but you get the point). The code in coreboot calculates MTRR settings to cover the place where the stack will be. To workaround missing CAR in Qemu, it then has to recalculate the stack location to be able to actually use the stack. That forces coreboot to keep two stack base variables and to completely replace the generic logic which switches off CAR.

I hope the explanation above didn't offend you, I just tried to clarify why working CAR is such a big deal for coreboot.

If you want either a full CAR implementation or no CAR implementation, I can write a patch which implements full CAR, but then I need to hook WBINVD, INVD and CLFLUSH. Neither instruction is executed often enough to show up in any profile. Besides that, for anything not using CAR (everything after the firmware), the penalty is a simple test of a boolean variable per WBINVD/INVD/CLFLUSH.

I think aiming for early to mid-December would give us roughly a 3 month cycle and would align well with some of the Linux distribution cycles. I'd like to limit things to a single -rc that lasted only for about a week. This is enough time to fix most of the obvious issues I think.

Quoted 2 lines · On Tue, 29 Sep 2009 18:54:53 -0500 Anthony Liguori <[email protected]> wrote:

I think aiming for early to mid-December would give us roughly a 3 month cycle and would align well with some of the Linux distribution cycles. I'd like to limit things to a single -rc that lasted only for about a week. This is enough time to fix most of the obvious issues I think.

How do you plan to do it? I mean, are you going to create a separate branch
or make master the -rc?

Creating a separate branch (which is what we do today, iiuc) makes it
get less attention, freezing master for a certain period is the best way to stabilize.

Is this what you had in mind?

Quoted 16 lines · Luiz Capitulino

Quoted 6 lines · On Tue, 29 Sep 2009 18:54:53 -0500 Anthony Liguori <[email protected]> wrote:

I think aiming for early to mid-December would give us roughly a 3 month cycle and would align well with some of the Linux distribution cycles. I'd like to limit things to a single -rc that lasted only for about a week.
This is enough time to fix most of the obvious issues I think.

How do you plan to do it? I mean, are you going to create a separate
branch
or make master the -rc?

Quoted 7 lines

Creating a separate branch (which is what we do today, iiuc) makes it
get less attention, freezing master for a certain period is the best way to stabilize.

Is this what you had in mind?

What do people think?

One reason I branch is because some people care a bit less about releases so it makes the process non-disruptive to them. If the other maintainers agreed though, I would certainly like to have the master branch essentially frozen for the week before the release.

2009/9/30 Anthony Liguori <[email protected]>

Quoted 27 lines

Quoted 16 lines · Luiz Capitulino

Quoted 6 lines · On Tue, 29 Sep 2009 18:54:53 -0500 Anthony Liguori <[email protected]> wrote:

I think aiming for early to mid-December would give us roughly a 3 month cycle and would align well with some of the Linux distribution cycles. I'd like to limit things to a single -rc that lasted only for about a week.
This is enough time to fix most of the obvious issues I think.

How do you plan to do it? I mean, are you going to create a separate
branch
or make master the -rc?

Quoted 7 lines

Creating a separate branch (which is what we do today, iiuc) makes it
get less attention, freezing master for a certain period is the best way to stabilize.

Is this what you had in mind?

What do people think?

Quoted 5 lines

One reason I branch is because some people care a bit less about releases so it makes the process non-disruptive to them. If the other maintainers agreed though, I would certainly like to have the master branch essentially frozen for the week before the release.

freezing is only neccesary if you need time to gather all the patches, build and test them together etc.

If you don't feel you or the developers need to
do that to get a reliable release out I think it only halts developers without any clear reason to do so. Calling 'attention' to a release is not a clear reason IMO.

Quoted 18 lines · Luiz Capitulino

Quoted 6 lines · On Tue, 29 Sep 2009 18:54:53 -0500 Anthony Liguori <[email protected]> wrote:

I think aiming for early to mid-December would give us roughly a 3 month cycle and would align well with some of the Linux distribution cycles. I'd like to limit things to a single -rc that lasted only for about a week. This is enough time to fix most of the obvious issues I think.

How do you plan to do it? I mean, are you going to create a separate branch
or make master the -rc?

Creating a separate branch (which is what we do today, iiuc) makes it
get less attention, freezing master for a certain period is the best way to stabilize.

Quoted 2 lines

Is this what you had in mind?

What do people think?

One reason I branch is because some people care a bit less about releases so it makes the process non-disruptive to them. If the other maintainers agreed though, I would certainly like to have the master branch essentially frozen for the week before the release.

One reason I branch is because some people care a bit less about releases so it makes the process non-disruptive to them. If the other maintainers agreed though, I would certainly like to have the master branch essentially frozen for the week before the release.

Hi,

Now that 0.11.0 is behind us, it's time to start thinking about 0.12.0.

I'd like to do a few things different this time around. I don't think the -rc process went very well as I don't think we got more testing out of it. I'd like to shorten the timeline for 0.12.0 a good bit. The 0.10 stable tree got pretty difficult to maintain toward the end of the cycle. We also had a pretty huge amount of change between 0.10 and 0.11 so I think a shorter cycle is warranted.

I think aiming for early to mid-December would give us roughly a 3 month cycle and would align well with some of the Linux distribution cycles. I'd like to limit things to a single -rc that lasted only for about a week. This is enough time to fix most of the obvious issues I think.

I'd also like to try to enumerate some features for this release. Here's a short list of things I expect to see for this release (target-i386 centric). Please add or comment on items that you'd either like to see in the release or are planning on working on.

o VMState conversion -- I expect most of the pc target to be completed
o qdev conversion -- I hope that we'll get most of the pc target completely converted to qdev
o storage live migration
o switch to SeaBIOS (need to finish porting features from Bochs)
o switch to gPXE (need to resolve slirp tftp server issue)
o KSM integration
o in-kernel APIC support for KVM
o guest SMP support for KVM
o updates to the default pc machine type

Please add to this list and I'll collect it all and post it somewhere.

Thanks!

Signature · 3 lines

Regards,

Anthony Liguori

for target-i386 and target-x86_64:

o EFI firmware emulation (as a machine or command line option)

Quoted 3 lines · Natalia Portillo

for target-i386 and target-x86_64:

o EFI firmware emulation (as a machine or command line option)

Are there any opensource EFI implementations with hardware init? I know that an opensource EFI glue layer exists, but AFAIK hardware init is still closed source. That leaves coreboot+UEFI (works somewhat) or SeaBIOS+UEFI (no idea if this is possible). If I missed an announcement about opensource hw init for EFI, I wish to apologize.

Regards,
Carl-Daniel

--
http://www.hailfinger.org/

Quoted 3 lines · Natalia Portillo

for target-i386 and target-x86_64:

o EFI firmware emulation (as a machine or command line option)

Are there any opensource EFI implementations with hardware init? I know that an opensource EFI glue layer exists, but AFAIK hardware init is still closed source. That leaves coreboot+UEFI (works somewhat) or SeaBIOS+UEFI (no idea if this is possible). If I missed an announcement about opensource hw init for EFI, I wish to apologize.

Regards,
Carl-Daniel

Signature · 1 line

http://www.hailfinger.org/

The only opensource EFI implementation I know about is the Intel EFI Reference.

Dunno if it includes hardware initialization.

Quoted 19 lines · Carl-Daniel Hailfinger

Quoted 4 lines · Natalia Portillo

for target-i386 and target-x86_64:

o EFI firmware emulation (as a machine or command line option)

Are there any opensource EFI implementations with hardware init? I know that an opensource EFI glue layer exists, but AFAIK hardware init is still closed source. That leaves coreboot+UEFI (works somewhat) or SeaBIOS+UEFI (no idea if this is possible). If I missed an announcement about opensource hw init for EFI, I wish to apologize.

Regards,
Carl-Daniel

--
http://www.hailfinger.org/

I work for Intel on the edk2.tianocore.org project. (Compared to the original edk) edk2 may be of interest to those on this list since it supports building on several OS's with several different toolchains. In other words, it also supports building under Linux and OS X with the GNU compiler and binutils.

Within our edk2 tree, we have two projects that relate to QEMU.

The DUET platform is a UEFI emulator that boots like a legacy OS on top of a legacy BIOS. It contains various hardware initialization drivers for several legacy hardware devices, but it also will call into the legacy BIOS and make use of certain items from the legacy BIOS. DUET can boot from the QEMU legacy BIOS, but it does require a disk to be setup to have the DUET image on it. I am not sure if all of DUET's code is currently safe for a UEFI OS to be able to access UEFI runtime services.

The OVMF platform is a project to build a (mostly) UEFI compatible firmware for virtual machines. QEMU support is one of the main goals for OVMF. The OVMF rom image completely replaces QEMU's standard bios.bin, and therefore we must have hardware drivers for any hardware within the QEMU VM that we want to make use of. The project also has the goal to support UEFI OS's at runtime.

for target-i386 and target-x86_64:

o EFI firmware emulation (as a machine or command line option)
o OS/2 1.x support

Quoted 5 lines · Natalia Portillo

for target-i386 and target-x86_64:

o EFI firmware emulation (as a machine or command line option)
o OS/2 1.x support

AFAIK, no one is actually working on these.  EFI is also not terribly useful without a CSM and I suspect porting SeaBIOS to tiano core is going to be a big effort.

--
Regards,

Anthony Liguori

Quoted 7 lines · Anthony Liguori

Quoted 5 lines · Natalia Portillo

for target-i386 and target-x86_64:

o EFI firmware emulation (as a machine or command line option)
o OS/2 1.x support

Quoted 4 lines

AFAIK, no one is actually working on these. EFI is also not terribly useful without a CSM and I suspect porting SeaBIOS to tiano core is going to be a big effort.

Several Linux distributions are getting UEFI support ironed out now, so UEFI can be used without a CSM. QEMU support for UEFI could help the Linux distributions to enable UEFI support...

Quoted 20 lines · Jordan Justen

Quoted 10 lines · Anthony Liguori

Quoted 6 lines · Natalia Portillo

for target-i386 and target-x86_64:

o EFI firmware emulation (as a machine or command line option)
o OS/2 1.x support

Quoted 6 lines

AFAIK, no one is actually working on these.  EFI is also not terribly useful
without a CSM and I suspect porting SeaBIOS to tiano core is going to be a
big effort.

Quoted 4 lines

Several Linux distributions are getting UEFI support ironed out now, so UEFI can be used without a CSM.  QEMU support for UEFI could help the Linux distributions to enable UEFI support...

I see a CSM as a pre-requisite for merging a uefi rom.  A user can already use a uefi rom by simply using the -bios parameter so there's nothing inhibiting testing.  My concern about introducing a new machine type is that it would require duplicate testing and force the selection of uefi up through the management tool stack.

OTH, uefi + a CSM based on SeaBIOS could be used as the default firmware.  I don't think it's a reasonable target for 0.12 but I think if someone actively worked on it, it would be possible for future releases.

Regards,

Anthony Liguori

Quoted 6 lines · Anthony Liguori

I see a CSM as a pre-requisite for merging a uefi rom. A user can already use a uefi rom by simply using the -bios parameter so there's nothing inhibiting testing. My concern about introducing a new machine type is that it would require duplicate testing and force the selection of uefi up through the management tool stack.

I understand your points. You want to keep a single machine type and try to support UEFI and legacy boots with that firmware. Unfortunately, I don't know that we will be able to provide that via OVMF. There is no open source CSM for us to make use of.

And, it is true that -bios can be used for OVMF by those with a particular interest in UEFI. But for a more general audience, I think without the -M switch a Linux distribution couldn't package qemu in such a way that both a legacy bios and the OVMF firmware would be available.

Quoted 4 lines

OTH, uefi + a CSM based on SeaBIOS could be used as the default firmware. I don't think it's a reasonable target for 0.12 but I think if someone actively worked on it, it would be possible for future releases.

I don't know of any efforts to create an open source CSM. Is someone working on converting SeaBIOS to act as a CSM?

For tianocore.org & OVMF we would further be restricted by requiring a BSD licensed CSM. Of course, if there was a SeaBIOS based CSM, then I'm sure OVMF could be modified to make use of it easily enough. We just wouldn't be able to have the SeaBIOS CSM on tianocore.org and part of the normal tianocore.org OVMF releases.

Quoted 19 lines · Jordan Justen

Quoted 9 lines · Anthony Liguori

I see a CSM as a pre-requisite for merging a uefi rom.  A user can already
use a uefi rom by simply using the -bios parameter so there's nothing inhibiting testing.  My concern about introducing a new machine type is that
it would require duplicate testing and force the selection of uefi up through the management tool stack.

I understand your points.  You want to keep a single machine type and try to support UEFI and legacy boots with that firmware. Unfortunately, I don't know that we will be able to provide that via OVMF.  There is no open source CSM for us to make use of.

Quoted 6 lines

And, it is true that -bios can be used for OVMF by those with a particular interest in UEFI.  But for a more general audience, I think without the -M switch a Linux distribution couldn't package qemu in such a way that both a legacy bios and the OVMF firmware would be available.

My concern is that if we provide separate machine types, it doubles our testing since we have to test guests with both the BIOS firmware and the UEFI firmware.  It also hurts usability since a user now has to make the decision as to whether they want to use UEFI or not.  A user should not have to even know what EFI is.

So I think the best way forward is to hold off on UEFI in mainline until we can provide a single unified stack.

Quoted 9 lines

Quoted 5 lines

OTH, uefi + a CSM based on SeaBIOS could be used as the default firmware. I
don't think it's a reasonable target for 0.12 but I think if someone actively worked on it, it would be possible for future releases.

I don't know of any efforts to create an open source CSM.  Is someone working on converting SeaBIOS to act as a CSM?

There are multiple parties that seem to be interested in uefi for guests.  I expect that one of those parties will end up doing the work if that's the requirement that we put down.

Quoted 6 lines

For tianocore.org & OVMF we would further be restricted by requiring a BSD licensed CSM.  Of course, if there was a SeaBIOS based CSM, then I'm sure OVMF could be modified to make use of it easily enough.  We just wouldn't be able to have the SeaBIOS CSM on tianocore.org and part of the normal tianocore.org OVMF releases.

I don't think that's a big problem for us.

Quoted 3 lines · Anthony Liguori

So I think the best way forward is to hold off on UEFI in mainline until we can provide a single unified stack.

While it is true that a separate machine type could potentially be viewed as increasing the testing requirements, I am not so sure. If QEMU has a UEFI+CSM solution, wouldn't we still have to test both UEFI based OS boot and the CSM based legacy OS boot?

Quoted 3 lines

Quoted 3 lines

I don't know of any efforts to create an open source CSM. Is someone working on converting SeaBIOS to act as a CSM?

I do know that we don't intend to start an open source CSM project on tianocore.org at this time.

But, if such a project was started, I think we would support development via email on our edk2 dev list, and potentially even host the project on tianocore.org.

So, I would invite anyone thinking of tackling such a task to join the edk2 dev list to discuss your work. Furthermore, if you'd like to request the project to be hosted on tianocore.org, feel free to ask our administrators. (As mentioned previously, we usually deal with the BSD license on tianocore.org, but please don't let this hold you back from asking questions, or requesting hosting of your project.)

Quoted 13 lines · Jordan Justen

Quoted 4 lines · Anthony Liguori

So I think the best way forward is to hold off on UEFI in mainline until we can provide a single unified stack.

While it is true that a separate machine type could potentially be viewed as increasing the testing requirements, I am not so sure. If QEMU has a UEFI+CSM solution, wouldn't we still have to test both UEFI based OS boot and the CSM based legacy OS boot?

Given that you can easily pack coreboot+SeaBIOS+UEFI into one ROM and have coreboot boot either SeaBIOS (BIOS interface) or UEFI (EFI interface), wouldn't that solve all problems in one go? You get an unified stack and don't have to mess around with different firmware files because coreboot can read in a Qemu machine config option (or NVRAM/CMOS) and select the interface based on that.

The hardware init part would be identical for all variants, only the interface would differ. coreboot works now and has the benefit of supporting real hardware as well, so the difference between a setup inside Qemu and a setup on real hardware is minimal.

The coreboot solution would also avoid converting SeaBIOS because SeaBIOS already works as coreboot payload (that's how coreboot developers call the CSM).

Quoted 7 lines · Carl-Daniel Hailfinger

Given that you can easily pack coreboot+SeaBIOS+UEFI into one ROM and have coreboot boot either SeaBIOS (BIOS interface) or UEFI (EFI interface), wouldn't that solve all problems in one go? You get an unified stack and don't have to mess around with different firmware files because coreboot can read in a Qemu machine config option (or NVRAM/CMOS) and select the interface based on that.

If you want to to work seamlessly, you need to check for the EFI system partition to see whether it's an EFI capable OS and then fallback gracefully to SeaBIOS boot.

Quoted 4 lines

The hardware init part would be identical for all variants, only the interface would differ. coreboot works now and has the benefit of supporting real hardware as well, so the difference between a setup inside Qemu and a setup on real hardware is minimal.

Tianocore's OVMF project should provide all the required initialization for EFI on QEMU.

Quoted 4 lines

The coreboot solution would also avoid converting SeaBIOS because SeaBIOS already works as coreboot payload (that's how coreboot developers call the CSM).

I'll bite, what's the advantage of doing coreboot + SeaBIOS vs. SeaBIOS alone? Forget about EFI for the moment, should be considering switching to coreboot + SeaBIOS for 0.12?

[Adding the coreboot list to CC. Please ignore the moderation messages, your addresses will be whitelisted ASAP.]

Quoted 9 lines · Anthony Liguori

Quoted 7 lines · Carl-Daniel Hailfinger

Given that you can easily pack coreboot+SeaBIOS+UEFI into one ROM and have coreboot boot either SeaBIOS (BIOS interface) or UEFI (EFI interface), wouldn't that solve all problems in one go? You get an unified stack and don't have to mess around with different firmware files because coreboot can read in a Qemu machine config option (or NVRAM/CMOS) and select the interface based on that.

Quoted 3 lines

If you want to to work seamlessly, you need to check for the EFI system partition to see whether it's an EFI capable OS and then fallback gracefully to SeaBIOS boot.

Hm. Wouldn't that be a layering violation (hw init != reading the disk) and also cause problems if you want to boot an EFI capable OS from SeaBIOS? I can think of someone having an EFI bootable disk image who wants to boot that disk image with EFI and BIOS without having to repartition.

Quoted 7 lines

Quoted 4 lines

The hardware init part would be identical for all variants, only the interface would differ. coreboot works now and has the benefit of supporting real hardware as well, so the difference between a setup inside Qemu and a setup on real hardware is minimal.

Tianocore's OVMF project should provide all the required initialization for EFI on QEMU.

Yes, but then we'd have two sets of hardware init: OVMF for EFI, SeaBIOS for SeaBIOS. That also means each hw init codebase has to support the new Q35 chipset proposed for Qemu.

It's not any better if SeaBIOS gets changed into a CSM for EFI. Then you have SeaBIOS on top of EFI on top of OVMF. As I understand it, OVMF had less testing than the Bochs BIOS or SeaBIOS on Qemu.

Jordan, I have to admit that I'm surprised OVMF was apparently created from scratch although quite a few (established) hardware init solutions already exist for Qemu: SeaBIOS, Bochs BIOS, coreboot, and some old hobbyist projects.

I'd like to understand the reasons for that and fix
them in coreboot (Kevin O'Connor will probably fix them in SeaBIOS).

If
you ported/modified existing code, I'd be interested in the original codebase to learn from it (especially what you had to change).

Quoted 9 lines

Quoted 5 lines

The coreboot solution would also avoid converting SeaBIOS because SeaBIOS already works as coreboot payload (that's how coreboot developers call the CSM).

I'll bite, what's the advantage of doing coreboot + SeaBIOS vs. SeaBIOS alone?  Forget about EFI for the moment, should be considering switching to coreboot + SeaBIOS for 0.12?

Advantages:
- Code coverage increase: SeaBIOS is used with coreboot on real hardware, so the BIOS interface part of SeaBIOS gets a lot more testing than the Qemu hardware init part of SeaBIOS.

- coreboot supports real 440BX hardware besides Qemu, so the coreboot init code is exercised more (and there is still a sizable number of such machines around (clusters), many of them running coreboot).

- Only one ROM image needed. A coreboot ROM can pack the VGA BIOS into the ROM image and SeaBIOS will automatically load it from there. Same for network card ROMs (with gPXE etc.).

- coreboot ROMs (including those with SeaBIOS and/or EFI and/or VGABIOS) are archives and can be listed/edited with cbfstool if you want to know what's in there.

- coreboot ROMs use compression, so you can stuff more code (and PCI option ROMs) into smaller ROMs.

- coreboot has pretty verbose hardware init messages (if you want that) and can be totally silent as well. The messages end up in a log buffer which can be read by the OS (experimental feature, not available by default).

There are a lot more advantages, but I want to give other coreboot developers a chance to chime in. If you add EFI to the mix, the synergies increase.

Regards,
Carl-Daniel

Quoted 5 lines · Carl-Daniel Hailfinger

Jordan, I have to admit that I'm surprised OVMF was apparently created from scratch although quite a few (established) hardware init solutions already exist for Qemu: SeaBIOS, Bochs BIOS, coreboot, and some old hobbyist projects.

The edk2 project provides an infrastructure for building complete UEFI firmware solutions. OVMF is a sample platform which demonstrates how edk2 can be used to build firmware to boot a UEFI platform.

Aside from just this, OVMF is an effort to support VMs on edk2. (Ie, more than just QEMU.) Ideally the project, and code of OVMF could be used by any VM vendor as a sample for building UEFI compatible firmware.

Most of the code required to support QEMU was already open sourced on edk2 before OVMF was launched. At the time that we started OVMF, it did not seem like any project was targeting UEFI support on QEMU. Tristan Gingold had done a port for UEFI QEMU support, but at that time it did not seem to be functional with the current QEMU, nor actively developed.

Quoted 3 lines

I'd like to understand the reasons for that and fix
them in coreboot (Kevin O'Connor will probably fix them in SeaBIOS).

If you want to remove any need for OVMF on QEMU, then I think all that is needed is to support booting UEFI OS's for both i386 and x86-64. Of course, we may still have some use for the OVMF project on edk2 as a sample platform.

Quoted 4 lines

If
you ported/modified existing code, I'd be interested in the original codebase to learn from it (especially what you had to change).

As I mentioned above, a good portion of the code was already available in edk2.tianocore.org.

Edk2 is BSD licensed, and therefore from a
licensing perspective should be easy for your project to utilize.

Quoted 9 lines · Jordan Justen

Quoted 5 lines · Carl-Daniel Hailfinger

Jordan, I have to admit that I'm surprised OVMF was apparently created from scratch although quite a few (established) hardware init solutions already exist for Qemu: SeaBIOS, Bochs BIOS, coreboot, and some old hobbyist projects.

Quoted 4 lines

The edk2 project provides an infrastructure for building complete UEFI firmware solutions.  OVMF is a sample platform which demonstrates how edk2 can be used to build firmware to boot a UEFI platform.

I wish to apologize for the misunderstanding. I thought OVMF was only hardware init. Thanks for correcting me.

Quoted 5 lines

Aside from just this, OVMF is an effort to support VMs on edk2.  (Ie, more than just QEMU.)  Ideally the project, and code of OVMF could be used by any VM vendor as a sample for building UEFI compatible firmware.

How does it relate to real hardware? You mention OVMF as an effort to support VMs on edk2. Does this mean that VMs need special support or that real hardware has different needs?

Quoted 7 lines

Most of the code required to support QEMU was already open sourced on edk2 before OVMF was launched.  At the time that we started OVMF, it did not seem like any project was targeting UEFI support on QEMU. Tristan Gingold had done a port for UEFI QEMU support, but at that time it did not seem to be functional with the current QEMU, nor actively developed.

I see.

Quoted 9 lines

Quoted 3 lines

I'd like to understand the reasons for that and fix
them in coreboot (Kevin O'Connor will probably fix them in SeaBIOS).

If you want to remove any need for OVMF on QEMU, then I think all that is needed is to support booting UEFI OS's for both i386 and x86-64. Of course, we may still have some use for the OVMF project on edk2 as a sample platform.

Given that my original statement incorrectly assumed OVMF was only about hardware init, please let me try to express what I originally meant (and which came across with unintended meaning).
I hope to use all EFI support code from OVMF, keep SeaBIOS, and have coreboot as common hardware init base.

Quoted 7 lines

Quoted 4 lines

If
you ported/modified existing code, I'd be interested in the original codebase to learn from it (especially what you had to change).

As I mentioned above, a good portion of the code was already available in edk2.tianocore.org.

Thanks for the pointer.

Quoted 3 lines

Edk2 is BSD licensed, and therefore from a
licensing perspective should be easy for your project to utilize.

(Speaking with my coreboot hat on.)
The goal of coreboot is not to assimilate and change other projects, but to provide hardware init for any code which needs it.

After hardware init, it passes control to a payload (SeaBIOS, UEFI, GRUB2, FILO, Linux, Plan9...). There are no callbacks to coreboot, each payload is expected to talk to the hardware on its own. Except for ACPI tables (and a memory map), nothing of coreboot stays in memory after passing control to that payload. If you want some basic hardware drivers for your favourite payload, you can use the BSD-licensed libpayload library in your code, but most payloads (SeaBIOS, GRUB2, ...) have their own drivers anyway.
Operating systems like Linux and Plan9 which do not need any BIOS/EFI interface can be stored in the ROM and will be booted directly by coreboot (if in ROM). Boot loaders like GRUB2 or FILO don't need BIOS/EFI either, can be stored in ROM and will then be booted directly. BIOS and EFI code can be used as a coreboot payload in ROM as well. Some people are even working on making U-Boot available as coreboot payload.

The idea is to have coreboot handle all the hardware-specific initialization and allow all other code to be mostly hardware-agnostic (unless said code wants to implement drivers). The clean interface (well, no interface, coreboot just passes control to the payload) allows payloads to do whatever they want as long as they provide a single 32-bit entry point coreboot can jump to.

Regards,
Carl-Daniel

Anyway, it sounds like a useful project might be to develop a UEFI coreboot payload based on the tianocore.org code.

Quoted 2 lines · Jordan Justen

Anyway, it sounds like a useful project might be to develop a UEFI coreboot payload based on the tianocore.org code.

I believe it might have been done already.

http://www.coreboot.org/File:Tianocoreboot.png

Quoted 4 lines

Quoted 2 lines · Jordan Justen

Anyway, it sounds like a useful project might be to develop a UEFI coreboot payload based on the tianocore.org code.

Quoted 3 lines

I believe it might have been done already.

http://www.coreboot.org/File:Tianocoreboot.png

That screenshot mentions DUET which is the tianocore.org UEFI emulator that boots on top of a legacy BIOS. But, it's unclear if it was just DUET, or something based modified specifically for coreboot based on DUET.

be able to boot UEFI OS's at this time.) However, we thought a project such as OVMF was a more direct approach to achieve UEFI compatibility for QEMU.

With coreboot, seabios and Duet, it should be reasonably simple to provide a single BIOS image that selects (based on nvram - ie. configuration) which interface to provide: PCBIOS or UEFI.

If that is just as simple with OVMF, and easier to maintain, then it's hard to argue with that.

Quoted 4 lines · Patrick Georgi

With coreboot, seabios and Duet, it should be reasonably simple to provide a single BIOS image that selects (based on nvram - ie. configuration) which interface to provide: PCBIOS or UEFI.

That isn't terribly useful for QEMU because it implies that QEMU needs to make the decision about which one to use. If QEMU was capable of making that decision without user intervention, then we could just select to load PCBIOS or UEFI.

I don't know enough about EFI, but my concern is that down the road, hardware devices will start to exist that require EFI because they implement EFI modules and not legacy option ROMs. In order to support PCI passthrough of such devices, we will need to provide an EFI capable firmware.

It's really just a matter of how long before this becomes a problem for us.

Quoted 7 lines · Anthony Liguori

Quoted 5 lines · Patrick Georgi

With coreboot, seabios and Duet, it should be reasonably simple to provide a single BIOS image that selects (based on nvram - ie. configuration) which interface to provide: PCBIOS or UEFI.

Quoted 5 lines

That isn't terribly useful for QEMU because it implies that QEMU needs to make the decision about which one to use. If QEMU was capable of making that decision without user intervention, then we could just select to load PCBIOS or UEFI.

Are you guys sure you want to go down that road rather than using an UEFI with a CSM?

This was attempted for Open Firmware and never happened during its life time. I doubt there is any incentive for vendors to ship devices without legacy option roms, especially since there is no advantage in doing so.

Quoted 13 lines · Stefan Reinauer

Quoted 11 lines · Anthony Liguori

Quoted 4 lines · Patrick Georgi

With coreboot, seabios and Duet, it should be reasonably simple to provide a single BIOS image that selects (based on nvram - ie. configuration) which interface to provide: PCBIOS or UEFI.

That isn't terribly useful for QEMU because it implies that QEMU needs to make the decision about which one to use. If QEMU was capable of making that decision without user intervention, then we could just select to load PCBIOS or UEFI.

Quoted 2 lines

Are you guys sure you want to go down that road rather than using an UEFI with a CSM?

We'll I've been advocating UEFI + CSM (based on SeaBIOS) so I'm not sure what you meant by 'that road'.

Quoted 4 lines

This was attempted for Open Firmware and never happened during its life time. I doubt there is any incentive for vendors to ship devices without legacy option roms, especially since there is no advantage in doing so.

We'll be stuck with legacy option roms for a long, long time. But I also expect there will be a few devices out there that only provide EFI modules.

What about SeaBIOS + CSM (based on DUET)?

I can't speak for Patrick, but he probably was concerned about making EFI the default with BIOS as fallback instead of the other way round. Forcing any EFI capable (or semi-capable) OS to be booted with EFI instead of leaving the choice in the hand of the user (NVRAM) or picking the sane default (what almost all boards out there are doing) sounds like a non-sustainable way for Qemu.

Quoted 4 lines

We'll be stuck with legacy option roms for a long, long time. But I also expect there will be a few devices out there that only provide EFI modules.

I expect that it will be some time before we see such devices (maybe only at trade show demos if at all). It will start to get interesting once such EFI modules have to interact with classic option ROMs.

Quoted 1 line · Carl-Daniel Hailfinger

What about SeaBIOS + CSM (based on DUET)?

That's not quite the same thing.

In EFI, CSM is a specification that defines how to port a legacy BIOS such that it runs as basically an EFI module providing the old legacy BIOS interfaces that OSes support. If you have a set of legacy option roms and efi modules, it defines how all of those things interact with each other to provide a consistent experience.

It's is not at all the same as just switching between EFI and BIOS. It's much more tightly integrated than that.

Quoted 6 lines

I can't speak for Patrick, but he probably was concerned about making EFI the default with BIOS as fallback instead of the other way round. Forcing any EFI capable (or semi-capable) OS to be booted with EFI instead of leaving the choice in the hand of the user (NVRAM) or picking the sane default (what almost all boards out there are doing) sounds like a non-sustainable way for Qemu.

Why? As long as it Just Works, I don't think it will ever even cross a users mind.

Quoted 9 lines

Quoted 4 lines

We'll be stuck with legacy option roms for a long, long time. But I also expect there will be a few devices out there that only provide EFI modules.

I expect that it will be some time before we see such devices (maybe only at trade show demos if at all). It will start to get interesting once such EFI modules have to interact with classic option ROMs.

I think at the high end, we'll see these sooner than you think.

I don't know enough about EFI, but my concern is that down the road, hardware devices will start to exist that require EFI because they implement EFI modules and not legacy option ROMs. In order to support PCI passthrough of such devices, we will need to provide an EFI capable firmware.

Am Sonntag, den 04.10.2009, 14:31 -0500 schrieb Anthony Liguori:

Quoted 2 lines

I don't know enough about EFI, but my concern is that down the road, hardware devices will start to exist that require EFI because they implement EFI modules and not legacy option ROMs. In order to support PCI passthrough of such devices, we will need to provide an EFI capable firmware.

Why? They're initialized at boot time (by the host's EFI firmware, if necessary), and are usable by the OS then.

Beyond boot loaders and toy kernels, firmware level drivers hurt more than they help. The world changed since the invention of BIOS in CP/M.

Quoted 9 lines · Peter Stuge

Quoted 3 lines · Jordan Justen

Anyway, it sounds like a useful project might be to develop a UEFI coreboot payload based on the tianocore.org code.

I believe it might have been done already.

http://www.coreboot.org/File:Tianocoreboot.png

That screenshot mentions DUET which is the tianocore.org UEFI emulator that boots on top of a legacy BIOS. But, it's unclear if it was just DUET, or something based modified specifically for coreboot based on DUET.

I will not dispute that DUET might be a potential solution to achieve UEFI compatibility for QEMU. (I'm not sure, but I think DUET may not be able to boot UEFI OS's at this time.) However, we thought a project such as OVMF was a more direct approach to achieve UEFI compatibility for QEMU.

Quoted 17 lines · Jordan Justen

Quoted 9 lines · Peter Stuge

Quoted 3 lines · Jordan Justen

Anyway, it sounds like a useful project might be to develop a UEFI coreboot payload based on the tianocore.org code.

I believe it might have been done already.

http://www.coreboot.org/File:Tianocoreboot.png

That screenshot mentions DUET which is the tianocore.org UEFI emulator that boots on top of a legacy BIOS.  But, it's unclear if it was just DUET, or something based modified specifically for coreboot based on DUET.

Quoted 6 lines

I will not dispute that DUET might be a potential solution to achieve UEFI compatibility for QEMU.  (I'm not sure, but I think DUET may not be able to boot UEFI OS's at this time.)  However, we thought a project such as OVMF was a more direct approach to achieve UEFI compatibility for QEMU.

We have DUET running as a coreboot payload with a small coreboot specific PE payload loader.

DUET is, however, not an emulator, it is executing much of the same code as all other TianoCore based UEFI implementations.

It is possible to boot an OS just fine with DUET.

Can you explain what you think would be more direct about OVMF than about DUET? As far as I understand it's another build target of EDK2 but besides that shares exactly the same design and even 99% of the code.

Quoted 29 lines · Stefan Reinauer

Quoted 27 lines · Jordan Justen

Quoted 12 lines · Peter Stuge

Quoted 4 lines · Jordan Justen

Anyway, it sounds like a useful project might be to develop a UEFI coreboot payload based on the tianocore.org code.

I believe it might have been done already.

http://www.coreboot.org/File:Tianocoreboot.png

That screenshot mentions DUET which is the tianocore.org UEFI emulator that boots on top of a legacy BIOS. But, it's unclear if it was just DUET, or something based modified specifically for coreboot based on DUET.

I will not dispute that DUET might be a potential solution to achieve UEFI compatibility for QEMU. (I'm not sure, but I think DUET may not be able to boot UEFI OS's at this time.) However, we thought a project such as OVMF was a more direct approach to achieve UEFI compatibility for QEMU.

Quoted 3 lines

We have DUET running as a coreboot payload with a small coreboot specific PE payload loader.

Meaning you bring up a legacy BIOS compatible interface before starting DUET?

DUET depends on a legacy BIOS.

My point is that a tianocore.org based coreboot payload ought be able to do away with this legacy BIOS dependency.

Quoted 4 lines

Can you explain what you think would be more direct about OVMF than about DUET? As far as I understand it's another build target of EDK2 but besides that shares exactly the same design and even 99% of the code.

DUET expects that you boot a legacy BIOS, and then you load DUET off the disk.

Once DUET is loaded, there is a (mostly) UEFI compatible
environment.

Both DUET and OVMF have some slight issues with providing a fully compatible UEFI variable interface.

Quoted 31 lines · Jordan Justen

Quoted 29 lines · Stefan Reinauer

Quoted 27 lines · Jordan Justen

Quoted 12 lines · Peter Stuge

Quoted 4 lines · Jordan Justen

Anyway, it sounds like a useful project might be to develop a UEFI coreboot payload based on the tianocore.org code.

I believe it might have been done already.

http://www.coreboot.org/File:Tianocoreboot.png

That screenshot mentions DUET which is the tianocore.org UEFI emulator that boots on top of a legacy BIOS.  But, it's unclear if it was just DUET, or something based modified specifically for coreboot based on DUET.

I will not dispute that DUET might be a potential solution to achieve UEFI compatibility for QEMU.  (I'm not sure, but I think DUET may not be able to boot UEFI OS's at this time.)  However, we thought a project such as OVMF was a more direct approach to achieve UEFI compatibility for QEMU.

Quoted 6 lines

Quoted 3 lines

We have DUET running as a coreboot payload with a small coreboot specific PE payload loader.

Meaning you bring up a legacy BIOS compatible interface before starting DUET?

No.

Quoted 1 line

 DUET depends on a legacy BIOS.

It did not for us, except for the loader which was replaced. We might have been lucky though...

Quoted 3 lines

My point is that a tianocore.org based coreboot payload ought be able to do away with this legacy BIOS dependency.

Absolutely agreed.

At some point it might make sense to have a coreboot specific target next to OVMF and DUET, some corebootPkg with specific adaptions and the loader integrated.
LegacyBiosInt86
The requirements for a coreboot target are very similar to those of OVMF and/or Duet, I guess. No hardware specific code is required, but in addition to what OVMF provides, we feature an in-flash filesystem and we export a coreboot table which contains memory information, cmos layout among other things...

What are the chances to get something like this integrated upstream TianoCore.org?

Quoted 7 lines

Quoted 4 lines

Can you explain what you think would be more direct about OVMF than about DUET? As far as I understand it's another build target of EDK2 but besides that shares exactly the same design and even 99% of the code.

DUET expects that you boot a legacy BIOS, and then you load DUET off the disk.

It does expect a few tables, but does not seem to make any 16bit calls once loaded.

Quoted 3 lines

 Once DUET is loaded, there is a (mostly) UEFI compatible environment.

I'm curious on the "mostly" here... what's missing? We certainly want to make sure what we do is fully UEFI compatible.

Quoted 2 lines

Both DUET and OVMF have some slight issues with providing a fully compatible UEFI variable interface.

Is that about saving settings in an NVRAM/flash memory?

Quoted 3 lines

I'll bite, what's the advantage of doing coreboot + SeaBIOS vs. SeaBIOS alone? Forget about EFI for the moment, should be considering switching to coreboot + SeaBIOS for 0.12?

Advantages:
- Code coverage increase: SeaBIOS is used with coreboot on real hardware, so the BIOS interface part of SeaBIOS gets a lot more testing than the Qemu hardware init part of SeaBIOS.

- coreboot supports real 440BX hardware besides Qemu, so the coreboot init code is exercised more (and there is still a sizable number of such machines around (clusters), many of them running coreboot).

- Only one ROM image needed. A coreboot ROM can pack the VGA BIOS into the ROM image and SeaBIOS will automatically load it from there. Same for network card ROMs (with gPXE etc.).

- coreboot ROMs (including those with SeaBIOS and/or EFI and/or VGABIOS) are archives and can be listed/edited with cbfstool if you want to know what's in there.

- coreboot ROMs use compression, so you can stuff more code (and PCI option ROMs) into smaller ROMs.

- coreboot has pretty verbose hardware init messages (if you want that) and can be totally silent as well. The messages end up in a log buffer which can be read by the OS (experimental feature, not available by default).

There are a lot more advantages, but I want to give other coreboot developers a chance to chime in. If you add EFI to the mix, the synergies increase.

Regards,
Carl-Daniel

So for me real 440BX hardware support of coreboot is actually disadvantage. QEMU don't have real 440BX hardware and there is not point in having one. It is possible to implement 440BX-qemu support in coreboot of course if there are other advantages worth having.

I use qemu for a lot of coreboot work. I really depend on qemu for many things I do, not just coreboot related. The qemu target in coreboot has been very heavily used by us to test out new ideas.

That said, I don't see a compelling need to augment seabios with coreboot on qemu *in the standard distribution*. If seabios gets the job done, and gets OSes booted, I think that's sufficient. I don't see a need to complicate anyone's life with something that is, after all, a sideshow for qemu users.

Conversely, I don't see the need to add the huge pile of stuff that comes with UEFI/OVMF/whatever to qemu either. One might argue that having any BIOS callbacks in the OS is a huge mistake, and certainly I've learned in practice that this argument is true.

thanks

ron

Exactly. I am glad to hear that coreboot has support for QEMU, but seabios does the job already, so why add more layers?

Quoted 2 lines · Gleb Natapov

Exactly. I am glad to hear that coreboot has support for QEMU, but seabios does the job already, so why add more layers?

If SeaBIOS does not need any code at all for QEMU machine init I agree there's no point in considering coreboot.

Quoted 2 lines · Gleb Natapov

Exactly. I am glad to hear that coreboot has support for QEMU, but seabios does the job already, so why add more layers?

If SeaBIOS does not need any code at all for QEMU machine init I agree there's no point in considering coreboot.

If QEMU machine specific init is in fact needed in that SeaBIOS, then the answer isn't as obvious..

I use qemu for a lot of coreboot work. I really depend on qemu for many things I do, not just coreboot related. The qemu target in coreboot has been very heavily used by us to test out new ideas.

That said, I don't see a compelling need to augment seabios with coreboot on qemu *in the standard distribution*. If seabios gets the job done, and gets OSes booted, I think that's sufficient. I don't see a need to complicate anyone's life with something that is, after all, a sideshow for qemu users.

Conversely, I don't see the need to add the huge pile of stuff that comes with UEFI/OVMF/whatever to qemu either. One might argue that

having any BIOS callbacks in the OS is a huge mistake, and certainly I've learned in practice that this argument is true.

I'll admit that this is a fairly dumb argument to make while we are talking about a QEMU release only a few months from now. But, as UEFI seems to be gaining ground in the industry, I think the sooner QEMU can get this support, the better.

Am Samstag, den 03.10.2009, 15:13 -0700 schrieb Jordan Justen:

Quoted 4 lines

I'll admit that this is a fairly dumb argument to make while we are talking about a QEMU release only a few months from now.  But, as UEFI seems to be gaining ground in the industry, I think the sooner QEMU can get this support, the better.

This smells like self-fulfulling prophecy: Let QEmu support EFI in the hope that EFI actually gains ground (for example by better testability due to available emulation environments)

So you want QEmu as a marketing device - nothing wrong with saying that, right?

Quoted 3 lines

So you want QEmu as a marketing device - nothing wrong with saying that, right?

I'm not in marketing. :) But, I do work for Intel, on tianocore.org and thus UEFI. I've been working with Tiano/EFI since ~2003, when Intel started converting its desktop motherboards over to a Tiano code base. So yes, I have a bias.

I think (but no, I cannot back this up) that tens of millions of UEFI compatible motherboards are shipping out each year now. Microsoft has implemented UEFI support in Vista and Win7. Several Linux vendor have or are enabling UEFI support now. Mac OS X implements UEFI.

My point? Well, while I think QEMU support for UEFI is still valuable to help support UEFI adoption, I think it could have done a lot more for UEFI if it was done several years ago. :)

Am Samstag, den 03.10.2009, 15:13 -0700 schrieb Jordan Justen:

Quoted 4 lines

I'll admit that this is a fairly dumb argument to make while we are talking about a QEMU release only a few months from now. But, as UEFI seems to be gaining ground in the industry, I think the sooner QEMU can get this support, the better.

This smells like self-fulfulling prophecy: Let QEmu support EFI in the hope that EFI actually gains ground (for example by better testability due to available emulation environments)

So you want QEmu as a marketing device - nothing wrong with saying that, right?

Regards,
Patrick

EFI actually DOES HAVE ground.

Itanium machines are still worldwide used, manufactured and sold.

Intel-based Macintoshes used them and ALL of their facilities.

The disadvantages can be a lot.
The advantages that I see, are the following (implemented by Apple's EFI):
Hardware drivers, so the OS loader can use ANY hardware present.

Hardware testing easily programable, as you can use the EFI to call the hardware (unlike PC diagnostics that makes direct and real mode calls to the hardware, making them imposible to test different hardware without implementing all variations -SCSI cards, wifi cards, so on-)

Filesystem independent bootloader (it just expects the EFI to have the filesystem and disk driver, then searches the disk for the OS loader)

Upgradable drivers without firmware patching (so if I add a wifi card, I can put a driver for netbooting it)

Extensive input devices support (USB keyb and mice, bluetooth keyb and mice and infrared remote)

I think that this is impossible (if not nearly to) make using BIOS. I think it is only possible with OpenFirmware or EFI.

And I prefer EFI for the matters of being programable in C (personal distaste for Forth)

and the EFI System Partition usability.

A bootloader can fuckinly easy put a good splash without limiting to 12 colors or making calls to the VGA system (for example). What will happen with the GRUB if tomorrow VGA disappears? What a mess...

And I don't work for Intel, IBM, HP, Apple, etc.

In QEMU' side I see that maybe not for 0.12.0 but anyway, EFI is a MUST have if we want to emulate, support, test, patch, anything that uses it, starting with Itanium, continuing with Intel Macintosh, and finishing with all the thousands of PCs (not only from Intel, as I've seen a bunch from Asus and Asrock) that instead of using a BIOS are using a hidden EFI with a boot-by-default CSM.

I'm sure, EFI will prevail over BIOS and sooner or later, also over OpenFirmware.

And we need to move with the world, not against it.

Hi Natalia,

I tried to understand your points. Please correct me where I'm wrong.

On 04.10.2009 06:10, Natalia Portillo wrote:

Quoted 3 lines

EFI actually DOES HAVE ground.

Itanium machines are still worldwide used, manufactured and sold.

Unless I'm mistaken, Itanium is not x86. Now if we consider EFI for x86 because it has solid standing on Itanium, shouldn't we also consider U-Boot and OpenFirmware for x86 because they have solid standing on ARM/PowerPC?

Quoted 2 lines

Intel-based Macintoshes used them and ALL of their facilities.

True.

Quoted 4 lines

The disadvantages can be a lot.
The advantages that I see, are the following (implemented by Apple's EFI):
Hardware drivers, so the OS loader can use ANY hardware present.

Same for BIOS.

Quoted 6 lines

Hardware testing easily programable, as you can use the EFI to call the hardware (unlike PC diagnostics that makes direct and real mode calls to the hardware, making them imposible to test different hardware without implementing all variations -SCSI cards, wifi cards, so on-)

Same for BIOS (the only difference is that the interface to the BIOS provides a 16bit interface and EFI provides a 32bit interface).

Quoted 3 lines

Filesystem independent bootloader (it just expects the EFI to have the filesystem and disk driver, then searches the disk for the OS loader)

Burn a BIOS together with DOS in the ROM and you get the same functionality. I have seen youtube videos of such systems.

Quoted 3 lines

Upgradable drivers without firmware patching (so if I add a wifi card, I can put a driver for netbooting it)

Install DOS with a boot agent on the disk and you get exactly the same feature.

Quoted 2 lines

Extensive input devices support (USB keyb and mice, bluetooth keyb and mice and infrared remote)

AFAIK USB keyboard+mouse is supported in pretty much every BIOS out there. Not sure about bluetooth keyboard and infrared remote, but the boards with builtin bluetooth support probably also have a bluetooth keyboard driver in the BIOS.

Quoted 3 lines

I think that this is impossible (if not nearly to) make using BIOS. I think it is only possible with OpenFirmware or EFI.

Except for bluetooth keyboard and infrared remote which I'm not sure about, everything you listed above is the same for BIOS and EFI.

Quoted 2 lines

And I prefer EFI for the matters of being programable in C (personal distaste for Forth)

Now the Forth argument is something I can agree with. ;-)

Quoted 1 line

and the EFI System Partition usability.

I don't understand what an EFI System Partition is used for. The wikipedia entry suggests that it is essentially a special partition where bootloaders live (instead of using a normal partition for that) and where EFI loads drivers and DOS/Windows PE commandline executables from. That sounds suspiciously like a DOS partition with NTLDR and loadlin.

Quoted 4 lines

A bootloader can fuckinly easy put a good splash without limiting to 12 colors or making calls to the VGA system (for example). What will happen with the GRUB if tomorrow VGA disappears? What a mess...

I've seen quite a few GRUB installations with 24-bit color splash screens in high resolutions. Graphics hardware won't disappear from computers any time soon, only the interfaces may change.

Quoted 9 lines

And I don't work for Intel, IBM, HP, Apple, etc.

In QEMU' side I see that maybe not for 0.12.0 but anyway, EFI is a MUST have if we want to emulate, support, test, patch, anything that uses it, starting with Itanium, continuing with Intel Macintosh, and finishing with all the thousands of PCs (not only from Intel, as I've seen a bunch from Asus and Asrock) that instead of using a BIOS are using a hidden EFI with a boot-by-default CSM.

Interesting. Do these boards still have a 20-30 second wait time from poweron to bootloader? I've seen coreboot+SeaBIOS achieve 1 second from poweron to bootloader on real x86 hardware, so I always wonder what the commercial BIOSes do during the time we wait for them.

Quoted 3 lines

I'm sure, EFI will prevail over BIOS and sooner or later, also over OpenFirmware.

I'm not so sure about that. EFI is "cool", but so were Linux netbooks and nowadays people buy mostly Windows netbooks.

Quoted 1 line

And we need to move with the world, not against it.

Fully agreed.

Many years ago, some EFI proponent told me: "EFI is great. It is a 32-bit BIOS with builtin 32-bit DOS. Now every operating system can use EFI drivers in compatibility mode like Windows 95 used DOS drivers in compatibility mode."
That statement had a lasting impression on me and I've yet to see any explanation why that statement would be incorrect. I'm very willing to learn, so if I got something wrong, please educate me.

Quoted 7 lines · Carl-Daniel Hailfinger

Hi Natalia,

I tried to understand your points. Please correct me where I'm wrong.

Quoted 2 lines · Natalia Portillo

EFI actually DOES HAVE ground.

Quoted 6 lines

Quoted 1 line

Itanium machines are still worldwide used, manufactured and sold.

Unless I'm mistaken, Itanium is not x86. Now if we consider EFI for x86 because it has solid standing on Itanium, shouldn't we also consider U-Boot and OpenFirmware for x86 because they have solid standing on ARM/PowerPC?

EFI is used in x86 as well, contrary to OF.

Quoted 6 lines

Quoted 4 lines

The disadvantages can be a lot.
The advantages that I see, are the following (implemented by Apple's EFI):
Hardware drivers, so the OS loader can use ANY hardware present.

Same for BIOS.

BIOS searches for them in ROM space.
No manufacturer ROM, no driver.

Quoted 8 lines

Quoted 5 lines

Hardware testing easily programable, as you can use the EFI to call the hardware (unlike PC diagnostics that makes direct and real mode calls to the hardware, making them imposible to test different hardware without implementing all variations -SCSI cards, wifi cards, so on-)

Same for BIOS (the only difference is that the interface to the BIOS provides a 16bit interface and EFI provides a 32bit interface).

Not only a 16 bit interface, REAL MODE interface.
Your driver can crash my driver easily.

Quoted 5 lines

Quoted 2 lines

Filesystem independent bootloader (it just expects the EFI to have the filesystem and disk driver, then searches the disk for the OS loader)

Burn a BIOS together with DOS in the ROM and you get the same functionality. I have seen youtube videos of such systems.

DOS is not a filesystem independent bootloader.

And filesystem drivers for DOS are not drivers, hacks and traps to INT 21h.

Quoted 7 lines

Quoted 2 lines

Extensive input devices support (USB keyb and mice, bluetooth keyb and mice and infrared remote)

AFAIK USB keyboard+mouse is supported in pretty much every BIOS out there. Not sure about bluetooth keyboard and infrared remote, but the boards with builtin bluetooth support probably also have a bluetooth keyboard driver in the BIOS.

Take any of it, no support.
I know a couple of Asus and Abit motherboards with integrated Wifi and Bluetooth.

Yet to see wifi netboot in ANY wifi card with bios. Seen EFI netboot in Apple wifis.

Quoted 6 lines

I don't understand what an EFI System Partition is used for. The wikipedia entry suggests that it is essentially a special partition where bootloaders live (instead of using a normal partition for that) and where EFI loads drivers and DOS/Windows PE commandline executables from. That sounds suspiciously like a DOS partition with NTLDR and loadlin.

The partition is empty by default.
EFI expects a directory structure and loads drivers from this partition if present, without loading any bootloader.
That allows it to load a bootloader from an ext3 filesystem, a direct ELF kernel from XFS, or a netboot updated driver for a WiMAX (for example).

Quoted 7 lines

Quoted 3 lines

A bootloader can fuckinly easy put a good splash without limiting to 12 colors or making calls to the VGA system (for example). What will happen with the GRUB if tomorrow VGA disappears? What a mess...

I've seen quite a few GRUB installations with 24-bit color splash screens in high resolutions. Graphics hardware won't disappear from computers any time soon, only the interfaces may change.

So, say me how!

Quoted 4 lines

Interesting. Do these boards still have a 20-30 second wait time from poweron to bootloader? I've seen coreboot+SeaBIOS achieve 1 second from poweron to bootloader on real x86 hardware, so I always wonder what the commercial BIOSes do during the time we wait for them.

Nope. The only "wait" is the normal Apple POST. Then the CSM takes 1 or 2 seconds to load the OS.

Quoted 7 lines

Many years ago, some EFI proponent told me: "EFI is great. It is a 32-bit BIOS with builtin 32-bit DOS. Now every operating system can use EFI drivers in compatibility mode like Windows 95 used DOS drivers in compatibility mode."
That statement had a lasting impression on me and I've yet to see any explanation why that statement would be incorrect. I'm very willing to learn, so if I got something wrong, please educate me.

EFI is not a 32-bit BIOS, it is a 32 or 64 bit protected mode hardware initialization and booting interface.

I'm not going to take a side on this matter. But, I think what will be more important is what is used in the majority of OS's and systems.
This is why we still put the 16-bit legacy BIOS as the #1 priority
after ~30 years. But, like I mention, I think there are signs that this may shift towards UEFI at some point.

Quoted 22 lines · Natalia Portillo

Quoted 12 lines · Jordan Justen

Quoted 11 lines · ron minnich

I use qemu for a lot of coreboot work. I really depend on qemu for many things I do, not just coreboot related. The qemu target in coreboot has been very heavily used by us to test out new ideas.

That said, I don't see a compelling need to augment seabios with coreboot on qemu *in the standard distribution*. If seabios gets the job done, and gets OSes booted, I think that's sufficient. I don't see a need to complicate anyone's life with something that is, after all, a sideshow for qemu users.

Quoted 6 lines

Quoted 2 lines

Conversely, I don't see the need to add the huge pile of stuff that comes with UEFI/OVMF/whatever to qemu either. One might argue that

This is a valid argument right now. OS X is the only OS today that targets UEFI, and specifically not legacy BIOS. But, in 5 ~ 10 years that might not be the case.

Quoted 5 lines

Wrong case.

Itanium architecture ONLY uses UEFI.
Having already got UEFI compatibility in QEMU is a step forware implemente ia64-machine.

Good point. For OVMF we only support i386/x86-64 right now, but, we would like to add ia64 support. I think the lack of ia64 support in QEMU is one of the big things holding this back.

Correct me if I'm wrong but most the IA64 opcodes can be derived from IA32 emulation isn't it why both are "Intel Architecture"?

Quoted 11 lines · Jordan Justen

Quoted 3 lines · Anthony Liguori

So I think the best way forward is to hold off on UEFI in mainline until we can provide a single unified stack.

While it is true that a separate machine type could potentially be viewed as increasing the testing requirements, I am not so sure.  If QEMU has a UEFI+CSM solution, wouldn't we still have to test both UEFI based OS boot and the CSM based legacy OS boot?

Given that you can easily pack coreboot+SeaBIOS+UEFI into one ROM and have coreboot boot either SeaBIOS (BIOS interface) or UEFI (EFI interface), wouldn't that solve all problems in one go? You get an unified stack and don't have to mess around with different firmware files because coreboot can read in a Qemu machine config option (or NVRAM/CMOS) and select the interface based on that.

The hardware init part would be identical for all variants, only the interface would differ. coreboot works now and has the benefit of supporting real hardware as well, so the difference between a setup inside Qemu and a setup on real hardware is minimal.

Quoted 3 lines

Quoted 3 lines

Quoted 3 lines

I don't know of any efforts to create an open source CSM.  Is someone working on converting SeaBIOS to act as a CSM?

The coreboot solution would also avoid converting SeaBIOS because SeaBIOS already works as coreboot payload (that's how coreboot developers call the CSM).

Quoted 7 lines

I do know that we don't intend to start an open source CSM project on tianocore.org at this time.

But, if such a project was started, I think we would support development via email on our edk2 dev list, and potentially even host the project on tianocore.org.

Quoted 7 lines

So, I would invite anyone thinking of tackling such a task to join the edk2 dev list to discuss your work.  Furthermore, if you'd like to request the project to be hosted on tianocore.org, feel free to ask our administrators.  (As mentioned previously, we usually deal with the BSD license on tianocore.org, but please don't let this hold you back from asking questions, or requesting hosting of your project.)

I do invite everyone who is interested in having one single ROM file with EFI+SeaBIOS to join the coreboot list as well. We already have coreboot+SeaBIOS running (both on hardware and inside Qemu) and coreboot+tianocore works as well. It has the advantage of requiring zero changes in coreboot, zero changes in SeaBIOS and (I think) zero changes in Tianocore.

As a nice side benefit for the EFI specialists, you get the ability to run EFI on every coreboot supported platform.

If you have any questions, please ask.

Regards,
Carl-Daniel

Carl-Daniel,

Interesting. So, where can I download a QEMU bios rom to boot a UEFI OS with this solution?

Can you explain what coreboot+tianocore means? Which parts of tianocore do you use in this situation?

If your solution can accomplish all of what you say, I'm not sure how OVMF would be able to compete against in terms of being included with QEMU. Part of the reason for starting OVMF was to help enable UEFI support within VM's such as QEMU. If OVMF was utilized by QEMU it definitely would have been a bit of a boost for our efforts, but if QEMU accomplishes UEFI support via another path, then overall we will still be happy.

Carl-Daniel,

Interesting. So, where can I download a QEMU bios rom to boot a UEFI OS with this solution?

Can you explain what coreboot+tianocore means? Which parts of tianocore do you use in this situation?

If your solution can accomplish all of what you say, I'm not sure how OVMF would be able to compete against in terms of being included with QEMU. Part of the reason for starting OVMF was to help enable UEFI support within VM's such as QEMU. If OVMF was utilized by QEMU it definitely would have been a bit of a boost for our efforts, but if QEMU accomplishes UEFI support via another path, then overall we will still be happy.

Thanks,

-Jordan

Quoted 19 lines · Carl-Daniel Hailfinger

Quoted 13 lines · Jordan Justen

Quoted 4 lines · Anthony Liguori

So I think the best way forward is to hold off on UEFI in mainline until we can provide a single unified stack.

While it is true that a separate machine type could potentially be viewed as increasing the testing requirements, I am not so sure. If QEMU has a UEFI+CSM solution, wouldn't we still have to test both UEFI based OS boot and the CSM based legacy OS boot?

Given that you can easily pack coreboot+SeaBIOS+UEFI into one ROM and have coreboot boot either SeaBIOS (BIOS interface) or UEFI (EFI interface), wouldn't that solve all problems in one go? You get an

Quoted 10 lines · Anthony Liguori

Quoted 5 lines · Natalia Portillo

for target-i386 and target-x86_64:

o EFI firmware emulation (as a machine or command line option)
o OS/2 1.x support

AFAIK, no one is actually working on these. EFI is also not terribly useful without a CSM and I suspect porting SeaBIOS to tiano core is going to be a big effort.

Several Linux distributions are getting UEFI support ironed out now, so UEFI can be used without a CSM. QEMU support for UEFI could help the Linux distributions to enable UEFI support...

Windows Vista and 7 support UEFI on x64-64, but I've not been able to boot either with the OVMF project yet. (I suspect that the video driver might not be initializing correctly, but the root cause of the failure is not yet known.)

Quoted 9 lines

--
Regards,

Anthony Liguori

integrating target-alpha and target-zxspectrum in the branch tree

a new official os support list design (please, give me ideas, css, help, anything!) with a better captcha

Quoted 1 line · Natalia Portillo

integrating target-alpha and target-zxspectrum in the branch tree

Heh. :)

z80-softmmu now has MSX1 emulation, with thanks to Juha Riihimäki, and I've also added the beginnings of ZX Spectrum 128K and SAM Coupé emulation. It still needs quite a few cleanups, though.

When you speak of target-alpha, I presume you're referring to Tristan Gingold's system emulation patches... it would certainly be nice to see these applied.

Same also for Laurent Vivier's m68k patches -- I would certainly have a go at implementing Atari ST emulation, and would also be interested in helping with Amiga, 68K Mac and perhaps even NeXT, SUN3, etc.

Quoted 2 lines

a new official os support list design (please, give me ideas, css, help, anything!) with a better captcha

For the OS support list, I'd perhaps suggest something similar to the Wine AppDB -- one page per OS version per architecture, containing details of each tester's QEMU configuration (KVM/TCG, host OS/arch, emulated devices, etc.) and guest OS configuration (drivers/kernel used, service packs applied, etc.), together with any extra notes.

If it's of any use: http://source.winehq.org/git/appdb.git/

Hi Stuart!

Nice to read you!

Quoted 4 lines

z80-softmmu now has MSX1 emulation, with thanks to Juha Riihimäki, and I've also added the beginnings of ZX Spectrum 128K and SAM Coupé emulation. It still needs quite a few cleanups, though.

Great great!!!
If you need any software for testing them just email me.

Could you please email me the public domain ROMs?
I've lost them

Quoted 3 lines

Same also for Laurent Vivier's m68k patches -- I would certainly have a go at implementing Atari ST emulation, and would also be interested in helping with Amiga, 68K Mac and perhaps even NeXT, SUN3, etc.

If operating systems or real systems are need for that I have almost all. Atari ST512FM, ST1024, Amiga 500, a pletory of Macs.
And of course, Atari TOS, Amiga Kickstart, OS and Workbench, Mac OS, A/UX, NeXTStep and OpenStep, and SunOS/sun3.

Quoted 3 lines

For the OS support list, I'd perhaps suggest something similar to the Wine AppDB [...]
If it's of any use: http://source.winehq.org/git/appdb.git/

A GREAT idea I'll check on it.

Quoted 6 lines

Quoted 4 lines

For the OS support list, I'd perhaps suggest something similar to the Wine AppDB [...]
If it's of any use: http://source.winehq.org/git/appdb.git/

A GREAT idea I'll check on it.

Thanks! I'm eager to hear what conclusion you come to. :-)

Hi,

Now that 0.11.0 is behind us, it's time to start thinking about 0.12.0.

I'd like to do a few things different this time around. I don't think the -rc process went very well as I don't think we got more testing out of it. I'd like to shorten the timeline for 0.12.0 a good bit. The 0.10 stable tree got pretty difficult to maintain toward the end of the cycle. We also had a pretty huge amount of change between 0.10 and 0.11 so I think a shorter cycle is warranted.

I think aiming for early to mid-December would give us roughly a 3 month cycle and would align well with some of the Linux distribution cycles. I'd like to limit things to a single -rc that lasted only for about a week. This is enough time to fix most of the obvious issues I think.

I'd also like to try to enumerate some features for this release. Here's a short list of things I expect to see for this release (target-i386 centric). Please add or comment on items that you'd either like to see in the release or are planning on working on.

o VMState conversion -- I expect most of the pc target to be completed
o qdev conversion -- I hope that we'll get most of the pc target completely converted to qdev
o storage live migration
o switch to SeaBIOS (need to finish porting features from Bochs)
o switch to gPXE (need to resolve slirp tftp server issue)
o KSM integration
o in-kernel APIC support for KVM
o guest SMP support for KVM
o updates to the default pc machine type

Please add to this list and I'll collect it all and post it somewhere.

Thanks!

* VDI block driver

* Update eepro100 driver

* Fix VNC support for Windows

Quoted 2 lines · Stefan Weil

* VDI block driver

If a new rev is submitted in time, sure.

Quoted 3 lines

* Update eepro100 driver

I have some of the patches that Blue Swirl didn't apply in my staging tree so they should get included assuming all goes well.

Quoted 3 lines

* Fix VNC support for Windows

I wasn't aware this was broken, can you file a bug report?

Regards,

Anthony Liguori

Hi,

Now that 0.11.0 is behind us, it's time to start thinking about 0.12.0.

I'd like to do a few things different this time around. I don't think the -rc process went very well as I don't think we got more testing out of it. I'd like to shorten the timeline for 0.12.0 a good bit. The 0.10 stable tree got pretty difficult to maintain toward the end of the cycle. We also had a pretty huge amount of change between 0.10 and 0.11 so I think a shorter cycle is warranted.

I think aiming for early to mid-December would give us roughly a 3 month cycle and would align well with some of the Linux distribution cycles. I'd like to limit things to a single -rc that lasted only for about a week. This is enough time to fix most of the obvious issues I think.

I'd also like to try to enumerate some features for this release. Here's a short list of things I expect to see for this release (target-i386 centric). Please add or comment on items that you'd either like to see in the release or are planning on working on.

o VMState conversion -- I expect most of the pc target to be completed
o qdev conversion -- I hope that we'll get most of the pc target completely converted to qdev
o storage live migration
o switch to SeaBIOS (need to finish porting features from Bochs)
o switch to gPXE (need to resolve slirp tftp server issue)
o KSM integration
o in-kernel APIC support for KVM
o guest SMP support for KVM
o updates to the default pc machine type

Please add to this list and I'll collect it all and post it somewhere.

Thanks!

Signature · 3 lines

Regards,

Anthony Liguori

o VMState conversion -- I expect most of the pc target to be completed
o qdev conversion -- I hope that we'll get most of the pc target
completely converted to qdev
o storage live migration
o switch to SeaBIOS (need to finish porting features from Bochs)
o switch to gPXE (need to resolve slirp tftp server issue)
o KSM integration
o in-kernel APIC support for KVM
o guest SMP support for KVM
o updates to the default pc machine type

Please add to this list and I'll collect it all and post it somewhere.

Quoted 11 lines · Anthony Liguori

o VMState conversion -- I expect most of the pc target to be completed
o qdev conversion -- I hope that we'll get most of the pc target
completely converted to qdev
o storage live migration
o switch to SeaBIOS (need to finish porting features from Bochs)
o switch to gPXE (need to resolve slirp tftp server issue)
o KSM integration
o in-kernel APIC support for KVM
o guest SMP support for KVM
o updates to the default pc machine type

Quoted 2 lines

Please add to this list and I'll collect it all and post it somewhere.

What about Or Gerlitz' raw backend driver ? I did not see it go in yet, or did I miss something ?

Quoted 8 lines · Jens Osterkamp

Quoted 2 lines · Anthony Liguori

Please add to this list and I'll collect it all and post it somewhere.

What about Or Gerlitz' raw backend driver ? I did not see it go in yet, or did I miss something ?

The patch seems to have not been updated after the initial posting and the first feedback cycle.

I'm generally inclined to oppose the functionality as I don't think it offers any advantages over the existing backends.

There are two reasons why I think this backend is important:

- As an easy way to provide isolation between guests (private ethernet port aggregator, PEPA) and external enforcement of network priviledges (virtual ethernet port aggregator, VEPA) using the macvlan subsystem.

Quoted 6 lines · Arnd Bergmann

There are two reasons why I think this backend is important:

- As an easy way to provide isolation between guests (private ethernet port aggregator, PEPA) and external enforcement of network priviledges (virtual ethernet port aggregator, VEPA) using the macvlan subsystem.

Can't this all be done with tap and a bridge?

Regards,

Anthony Liguori

Quoted 10 lines · Anthony Liguori

Quoted 8 lines · Jens Osterkamp

Quoted 2 lines · Anthony Liguori

Please add to this list and I'll collect it all and post it somewhere.

What about Or Gerlitz' raw backend driver ? I did not see it go in yet, or did I miss something ?

Quoted 5 lines

The patch seems to have not been updated after the initial posting and the first feedback cycle.

I'm generally inclined to oppose the functionality as I don't think it offers any advantages over the existing backends.

There are two reasons why I think this backend is important:

- As an easy way to provide isolation between guests (private ethernet port aggregator, PEPA) and external enforcement of network priviledges (virtual ethernet port aggregator, VEPA) using the macvlan subsystem.

- As a counterpart to the vhost_net driver, providing an identical user interface with or without vhost_net acceleration in the kernel.

	Arnd <><

Quoted 17 lines · Jens Osterkamp

Quoted 13 lines · Anthony Liguori

o VMState conversion -- I expect most of the pc target to be completed
o qdev conversion -- I hope that we'll get most of the pc target
completely converted to qdev
o storage live migration
o switch to SeaBIOS (need to finish porting features from Bochs)
o switch to gPXE (need to resolve slirp tftp server issue)
o KSM integration
o in-kernel APIC support for KVM
o guest SMP support for KVM
o updates to the default pc machine type

Please add to this list and I'll collect it all and post it somewhere.

Quoted 3 lines

What about Or Gerlitz' raw backend driver ? I did not see it go in yet, or did I miss something ?

The patch seems to have not been updated after the initial posting and the first feedback cycle.

I'm generally inclined to oppose the functionality as I don't think it offers any advantages over the existing backends.

Looks like Or has abandoned it. I have an updated version which works with new APIs, etc. Let me post it and we'll go from there.

Quoted 2 lines

I'm generally inclined to oppose the functionality as I don't think it offers any advantages over the existing backends.

I patch it in and use it all the time. It's much easier to setup on a random machine than a bridged config.

Quoted 8 lines · Michael S. Tsirkin

Looks like Or has abandoned it. I have an updated version which works with new APIs, etc. Let me post it and we'll go from there.

Quoted 3 lines

I'm generally inclined to oppose the functionality as I don't think it offers any advantages over the existing backends.

Quoted 3 lines

I patch it in and use it all the time. It's much easier to setup on a random machine than a bridged config.

Having two things that do the same thing is just going to lead to user confusion.

If the problem is tap is too hard to setup, we should try to simplify tap configuration.

Regards,

Anthony Liguori

Quoted 13 lines · Anthony Liguori

Quoted 11 lines · Michael S. Tsirkin

Looks like Or has abandoned it. I have an updated version which works with new APIs, etc. Let me post it and we'll go from there.

Quoted 3 lines

I'm generally inclined to oppose the functionality as I don't think it offers any advantages over the existing backends.

I patch it in and use it all the time. It's much easier to setup on a random machine than a bridged config.

Quoted 2 lines

Having two things that do the same thing is just going to lead to user confusion.

They do not do the same thing. With raw socket you can use windows update without a bridge in the host, with tap you can't.

Quoted 2 lines

If the problem is tap is too hard to setup, we should try to simplify tap configuration.

The problem is bridge is too hard to setup. Simplifying that is a good idea, but outside the scope of the qemu project.

Quoted 17 lines · Michael S. Tsirkin

Quoted 15 lines · Anthony Liguori

Quoted 11 lines · Michael S. Tsirkin

Looks like Or has abandoned it. I have an updated version which works with new APIs, etc. Let me post it and we'll go from there.

Quoted 3 lines

I'm generally inclined to oppose the functionality as I don't think it offers any advantages over the existing backends.

I patch it in and use it all the time. It's much easier to setup on a random machine than a bridged config.

Having two things that do the same thing is just going to lead to user confusion.

Quoted 2 lines

They do not do the same thing. With raw socket you can use windows update without a bridge in the host, with tap you can't.

On the other hand, with raw socket, guest Windows can't access files on the host's Samba share can it? So it's not that useful even for Windows guests.

Quoted 6 lines

Quoted 2 lines

If the problem is tap is too hard to setup, we should try to simplify tap configuration.

The problem is bridge is too hard to setup. Simplifying that is a good idea, but outside the scope of the qemu project.

I venture it's important enough for qemu that it's worth working on that. Something that looks like the raw socket but behaves like an automatically instantiated bridge attached to the bound interface would be a useful interface.

I don't have much time, but I'll help anybody who wants to do that.

-- Jamie

Quoted 18 lines · Jamie Lokier

Quoted 17 lines · Michael S. Tsirkin

Quoted 15 lines · Anthony Liguori

Quoted 11 lines · Michael S. Tsirkin

Looks like Or has abandoned it. I have an updated version which works with new APIs, etc. Let me post it and we'll go from there.

Quoted 3 lines

I'm generally inclined to oppose the functionality as I don't think it offers any advantages over the existing backends.

I patch it in and use it all the time. It's much easier to setup on a random machine than a bridged config.

Having two things that do the same thing is just going to lead to user confusion.

Quoted 6 lines

Quoted 2 lines

They do not do the same thing. With raw socket you can use windows update without a bridge in the host, with tap you can't.

On the other hand, with raw socket, guest Windows can't access files on the host's Samba share can it? So it's not that useful even for Windows guests.

I guess this depends on whether you use the same host for samba :)

Quoted 7 lines

Quoted 6 lines

Quoted 2 lines

If the problem is tap is too hard to setup, we should try to simplify tap configuration.

The problem is bridge is too hard to setup. Simplifying that is a good idea, but outside the scope of the qemu project.

Quoted 4 lines

I venture it's important enough for qemu that it's worth working on that. Something that looks like the raw socket but behaves like an automatically instantiated bridge attached to the bound interface would be a useful interface.

I agree, that would be good to have.

Quoted 4 lines

I don't have much time, but I'll help anybody who wants to do that.

-- Jamie

--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Can't we bind the raw socket to the tap interface instead of the physical interface and allow the bridge config to work.

So at this point, I think it's a mistake to include raw socket support. If the goal is to improve networking usability such that it just works as a root user, let's incorporate a default network script that creates a bridge or something like that. There are better ways to achieve that goal.

I would be much more inclined to consider
taking raw and improving the performance long term if guest<->host networking worked. This appears to be a fundamental limitation though and I think it's something that will forever plague users if we include this feature.

Quoted 2 lines · Anthony Liguori

I would be much more inclined to consider taking raw and improving the performance long term if guest<->host networking worked. This appears to be a fundamental limitation though and I think it's something that will forever plague users if we include this feature.

In fact, I think it's fixable with a raw socket bound to a macvlan. Would that be enough?

Quoted 8 lines · Michael S. Tsirkin

Quoted 5 lines · Anthony Liguori

I would be much more inclined to consider taking raw and improving the performance long term if guest<->host networking worked. This appears to be a fundamental limitation though and I think it's something that will forever plague users if we include this feature.

Quoted 3 lines

In fact, I think it's fixable with a raw socket bound to a macvlan. Would that be enough?

What setup does that entail on the part of a user? Wouldn't we be back to square one wrt users having to run archaic networking commands in order to set things up?

Regards,

Anthony Liguori

Quoted 12 lines · Anthony Liguori

Quoted 10 lines · Michael S. Tsirkin

Quoted 3 lines · Anthony Liguori

I would be much more inclined to consider taking raw and improving the performance long term if guest<->host networking worked. This appears to be a fundamental limitation though and I think it's something that will forever plague users if we include this feature.

In fact, I think it's fixable with a raw socket bound to a macvlan. Would that be enough?

Quoted 2 lines

What setup does that entail on the part of a user? Wouldn't we be back to square one wrt users having to run archaic networking commands in order to set things up?

Unlike bridge, qemu could set up macvlan without disrupting host networking. The only issue would be cleanup if qemu is killed.

Quoted 20 lines · Michael S. Tsirkin

Quoted 17 lines · Anthony Liguori

Quoted 11 lines · Michael S. Tsirkin

Quoted 6 lines · Anthony Liguori

I would be much more inclined to consider taking raw and improving the performance long term if guest<->host networking worked. This appears to be a fundamental limitation though and I think it's something that will forever plague users if we include this feature.

In fact, I think it's fixable with a raw socket bound to a macvlan. Would that be enough?

What setup does that entail on the part of a user? Wouldn't we be back to square one wrt users having to run archaic networking commands in order to set things up?

Quoted 4 lines

Unlike bridge, qemu could set up macvlan without disrupting host networking. The only issue would be cleanup if qemu is killed.

But this would require additional features in macvlan, correct?

This also only works if a guest uses the mac address assigned to it, correct? If a guest was bridging the virtual nic, this would all come apart?

Regards,

Anthony Liguori

Not sure: what is the "this" that you are talking about.

Quoted 2 lines · Michael S. Tsirkin

Not sure: what is the "this" that you are talking about.

I meant, fixing guest<->host traffic.

You're former argument hinged on
being able to having networking that Just Worked without adding new things. If this doesn't work today with macvlan, then the argument is invalid.

Regards,

Anthony Liguori

Quoted 21 lines · Anthony Liguori

Quoted 20 lines · Michael S. Tsirkin

Quoted 17 lines · Anthony Liguori

Quoted 11 lines · Michael S. Tsirkin

Quoted 6 lines · Anthony Liguori

I would be much more inclined to consider taking raw and improving the performance long term if guest<->host networking worked. This appears to be a fundamental limitation though and I think it's something that will forever plague users if we include this feature.

In fact, I think it's fixable with a raw socket bound to a macvlan. Would that be enough?

What setup does that entail on the part of a user? Wouldn't we be back to square one wrt users having to run archaic networking commands in order to set things up?

Quoted 6 lines

Quoted 4 lines

Unlike bridge, qemu could set up macvlan without disrupting host networking. The only issue would be cleanup if qemu is killed.

But this would require additional features in macvlan, correct?

Not sure: what is the "this" that you are talking about. It can already be set up without disturbing host networking.

Quoted 3 lines

This also only works if a guest uses the mac address assigned to it, correct? If a guest was bridging the virtual nic, this would all come apart?

Hmm, you could enable promisc mode, but generally this is true:
if you require bridging, use a bridge.

Quoted 3 lines

Regards,

Anthony Liguori

Quoted 40 lines · Michael S. Tsirkin

Quoted 18 lines · Jamie Lokier

Quoted 17 lines · Michael S. Tsirkin

Quoted 15 lines · Anthony Liguori

Quoted 11 lines · Michael S. Tsirkin

Looks like Or has abandoned it. I have an updated version which works with new APIs, etc. Let me post it and we'll go from there.

Quoted 3 lines

I'm generally inclined to oppose the functionality as I don't think it offers any advantages over the existing backends.

I patch it in and use it all the time. It's much easier to setup on a random machine than a bridged config.

Having two things that do the same thing is just going to lead to user confusion.

Quoted 6 lines

Quoted 2 lines

They do not do the same thing. With raw socket you can use windows update without a bridge in the host, with tap you can't.

On the other hand, with raw socket, guest Windows can't access files on the host's Samba share can it? So it's not that useful even for Windows guests.

I guess this depends on whether you use the same host for samba :)

Quoted 11 lines

Quoted 6 lines

Quoted 2 lines

If the problem is tap is too hard to setup, we should try to simplify tap configuration.

The problem is bridge is too hard to setup. Simplifying that is a good idea, but outside the scope of the qemu project.

I venture it's important enough for qemu that it's worth working on that. Something that looks like the raw socket but behaves like an automatically instantiated bridge attached to the bound interface would be a useful interface.

Quoted 1 line

I agree, that would be good to have.

Can't we bind the raw socket to the tap interface instead of the physical interface and allow the bridge config to work.

Thanks
Sridhar

Quoted 8 lines

Quoted 4 lines

I don't have much time, but I'll help anybody who wants to do that.

-- Jamie

--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Hi,

Now that 0.11.0 is behind us, it's time to start thinking about 0.12.0.

o storage live migration