x86/microcode: support for microcode update in Xen dom0

From original posting:

From: Jeremy Fitzhardinge <[email protected]>

Hi all,

I'm proposing this for the next merge window v3.2.

I originally posted this early this year, and it prompted a debate about what the "proper" way that Linux should do microcode updates, with the general concensus being "earlier", ideally in the bootloader (or in the case of Xen, as the hypervisor boots before starting any domains). However, as far as I know there has been no progress along those lines.

I would like to therefore merge this so that a Linux kernel booting as dom0 under Xen can update the microcode in the same manner as a kernel booting natively. When we work out how boot-time microcode updates can be done, then we'll look at modifying Xen accordingly. In the meantime, we should have a functional parity.

The only change to this code from the previous posting is some patch restructuring so that regardless of how the platform.h ABI header gets merged (since there are some other pending branches containing it), it will be identical and cause no merge headaches.

From original posting:

This series adds a new "Xen" microcode update type, in addition to Intel and AMD.

The Xen hypervisor is responsible for performing the actual microcode update (since only it knows what physical CPUs are in the system and has sufficient privilege to access them), but it requires the dom0 kernel to provide the actual microcode update data.

Xen update mechanism is uniform independent of the CPU type, but the driver must know where to find the data file, which depends on the CPU type. And since the update hypercall updates all CPUs, we only need to execute it once on any CPU - but for simplicity it just runs it only on (V)CPU 0.

Thanks,
J

Jeremy Fitzhardinge (2):
xen: add dom0_op hypercall
xen: add CPU microcode update driver

Yu Ke (1):
xen/acpi: Domain0 acpi parser related platform hypercall

 arch/ia64/include/asm/xen/interface.h |    1 +
 arch/x86/include/asm/microcode.h      |    9 +
 arch/x86/include/asm/xen/hypercall.h  |    8 +
 arch/x86/include/asm/xen/interface.h  |    1 +
 arch/x86/kernel/Makefile              |    1 +
 arch/x86/kernel/microcode_core.c      |    5 +-
 arch/x86/kernel/microcode_xen.c       |  198 ++++++++++++++++++++
 arch/x86/xen/Kconfig                  |    4 +
 include/xen/interface/platform.h      |  320 +++++++++++++++++++++++++++++++++
 include/xen/interface/xen.h           |    1 +
 10 files changed, 547 insertions(+), 1 deletions(-)
 create mode 100644 arch/x86/kernel/microcode_xen.c
 create mode 100644 include/xen/interface/platform.h

Ping?

Quoted 12 lines · Jeremy Fitzhardinge

From: Jeremy Fitzhardinge <[email protected]>

Hi all,

I'm proposing this for the next merge window v3.2.

I originally posted this early this year, and it prompted a debate about what the "proper" way that Linux should do microcode updates, with the general concensus being "earlier", ideally in the bootloader (or in the case of Xen, as the hypervisor boots before starting any domains). However, as far as I know there has been no progress along those lines.

Quoted 15 lines · Jeremy Fitzhardinge

Ping?

Quoted 12 lines · Jeremy Fitzhardinge

From: Jeremy Fitzhardinge <[email protected]>

Hi all,

I'm proposing this for the next merge window v3.2.

I originally posted this early this year, and it prompted a debate about what the "proper" way that Linux should do microcode updates, with the general concensus being "earlier", ideally in the bootloader (or in the case of Xen, as the hypervisor boots before starting any domains). However, as far as I know there has been no progress along those lines.

Err,

wait a sec, last time the suggestion was that _because_ xen supports the multiboot protocol, implementing an early solution would be even easier for you:

http://marc.info/?l=linux-kernel&m=129667641316947&w=2

Why would a xen microcode solution would even be contingent on how upstream Linux solves it (and when)?

AFAICT, you want to hand off the microcode image from the bootloader to the hypervisor and update ucode there - no need for dom0 changes...

Why would a xen microcode solution would even be contingent on how upstream Linux solves it (and when)?

Quoted 3 lines · Borislav Petkov

Why would a xen microcode solution would even be contingent on how upstream Linux solves it (and when)?

Agreed... especially since Xen is "special" when it comes to booting anyway.

While doing the whole boot time multiboot thing may offer some small hypothetical technical advantages, it has the significant cost of just complicating the whole deployment and use story.

Quoted 3 lines · Jeremy Fitzhardinge

While doing the whole boot time multiboot thing may offer some small hypothetical technical advantages, it has the significant cost of just complicating the whole deployment and use story.

You simply can't call the need to apply ucode as early as possible a "hypothetical techical advantage." Other issues like how to handle ucode images and how to put them together and how distros distribute them and whether xen minimizes the amount of "specialness" or not are only secondary.

Why is it paramount to do it as early as possible? As in, even doing it before Linux kernel is invoked is preferred than during initrd runtime?

Quoted 2 lines · Konrad Rzeszutek Wilk

Why is it paramount to do it as early as possible? As in, even doing it before Linux kernel is invoked is preferred than during initrd runtime?

It is paramount to do it as early as possible *because the CPU is broken*. That's why there is a microcode update at all. It is *supposed* to be installed by BIOS, but for whatever reason it wasn't (including user doesn't want to update the BIOS), so the very fact that this is done in the OS at all is a bit of a fail.

Doing it in the bootloader is messy because bootloaders typically aren't SMP-aware (and really shouldn't need to be), which leaves the OS. On native hardware it should ideally be done as early in the processor bringup as possible.

The stickiest part of all of this is where to leave the microcode so the kernel can get to it way early. We could use the linked list, but that would mean bootloader enabling; another idea was to stick it as a prefix to the initramfs identified by a signature.

Quoted 4 lines · H. Peter Anvin

The stickiest part of all of this is where to leave the microcode so the kernel can get to it way early. We could use the linked list, but that would mean bootloader enabling; another idea was to stick it as a prefix to the initramfs identified by a signature.

I think it would be easiest for the multiboot case if it were a separate file, but I don't feel I'm really up on all the details.

Quoted 6 lines · Jeremy Fitzhardinge

Quoted 4 lines · H. Peter Anvin

The stickiest part of all of this is where to leave the microcode so the kernel can get to it way early. We could use the linked list, but that would mean bootloader enabling; another idea was to stick it as a prefix to the initramfs identified by a signature.

Quoted 2 lines

I think it would be easiest for the multiboot case if it were a separate file, but I don't feel I'm really up on all the details.

For the multiboot case, yes of course.

Some bootloaders allow initramfs to be concatenated from multiple files; unfortunately I'm not sure if Grub ever grew that capability. Sigh.

Presumably we'd want some way to make all possible microcode files available to the hypervisor/kernel so that there's no need to construct the bootloader config for a specific CPU vendor (or worse, CPU model). So that would need either some way of specifying multiple files, or a wrapper which can contain multiple files?

Quoted 7 lines · Jeremy Fitzhardinge

Presumably we'd want some way to make all possible microcode files available to the hypervisor/kernel so that there's no need to construct the bootloader config for a specific CPU vendor (or worse, CPU model). So that would need either some way of specifying multiple files, or a wrapper which can contain multiple files?

I have been thinking a "bundle" format with a per-vendor header followed by the microcode in the appropriate format for each vendor.

Quoted 9 lines · Jeremy Fitzhardinge

Quoted 7 lines

Presumably we'd want some way to make all possible microcode files available to the hypervisor/kernel so that there's no need to construct the bootloader config for a specific CPU vendor (or worse, CPU model). So that would need either some way of specifying multiple files, or a wrapper which can contain multiple files?

Quoted 2 lines

I have been thinking a "bundle" format with a per-vendor header followed by the microcode in the appropriate format for each vendor.

Yes, makes sense. Keep it as simple as possible.

The stickiest part of all of this is where to leave the microcode so the kernel can get to it way early. We could use the linked list, but that would mean bootloader enabling;

another idea was to stick it as a prefix
to the initramfs identified by a signature.

Bottomline is, extending initrd handling to deal with multiple initrd images might turn out to be easier to do than the linked list deal.

Quoted 2 lines · Borislav Petkov

Bottomline is, extending initrd handling to deal with multiple initrd images might turn out to be easier to do than the linked list deal.

Alternatively and IMHO, we could avoid the bootloader enabling by making the ucode part of the initramfs and pull up some of the prepare_namespace() work in kernel_init() before smp_init() so that we can have it ready for when bootstrapping the cores.

Quoted 2 lines · Borislav Petkov

Bottomline is, extending initrd handling to deal with multiple initrd images might turn out to be easier to do than the linked list deal.

Alternatively and IMHO, we could avoid the bootloader enabling by making the ucode part of the initramfs and pull up some of the prepare_namespace() work in kernel_init() before smp_init() so that we can have it ready for when bootstrapping the cores.

<scratches his head>
You are still using the microcode API (the existing one) to program the CPUs right? It is just that you are fetching the images much much earlier than it currently is done?

Quoted 5 lines · Konrad Rzeszutek Wilk

<scratches his head>
You are still using the microcode API (the existing one) to program the CPUs right? It is just that you are fetching the images much much earlier than it currently is done?

No, the goal is to ditch the existing API and load the CPUs very early in the start path.

Quoted 6 lines · Konrad Rzeszutek Wilk

<scratches his head>
You are still using the microcode API (the existing one) to program the CPUs right? It is just that you are fetching the images much much earlier than it currently is done?

No, the goal is to ditch the existing API and load the CPUs very early in the start path.

-hpa

I think this approach is good to get the microcode applied as early as possible at boot time. But on servers you usually do not want to reboot the machine unless you do a BIOS update which will apply the new microcode anyway. So for applying the microcode update at runtime I would like to keep the existing API.

Quoted 5 lines · Christoph Egger

I think this approach is good to get the microcode applied as early as possible at boot time. But on servers you usually do not want to reboot the machine unless you do a BIOS update which will apply the new microcode anyway. So for applying the microcode update at runtime I would like to keep the existing API.

No, you don't want to keep the existing API and you know it. I've explained to you last week why.

No, you don't want to keep the existing API and you know it. I've explained to you last week why.

Quoted 3 lines · Jeremy Fitzhardinge

While doing the whole boot time multiboot thing may offer some small hypothetical technical advantages, it has the significant cost of just complicating the whole deployment and use story.

You simply can't call the need to apply ucode as early as possible a "hypothetical techical advantage."

Other issues like how to handle ucode
images and how to put them together and how distros distribute them and whether xen minimizes the amount of "specialness" or not are only secondary.

The current scheme has worked pretty well so far; there doesn't seem to be a huge concern about it. Have there been actual observed failures with the current mechanism, or is the drive to make it earlier driven by an aesthetic desire to make it "as it should be"?

Quoted 6 lines · Jeremy Fitzhardinge

The current scheme has worked pretty well so far; there doesn't seem to be a huge concern about it. Have there been actual observed failures with the current mechanism, or is the drive to make it earlier driven by an aesthetic desire to make it "as it should be"?

Yes, there has. There is at least one case where we have to print an error message saying "update your BIOS" because our current microcode update is too late.

Quoted 4 lines

Yes, there has. There is at least one case where we have to print an error message saying "update your BIOS" because our current microcode update is too late.

Is that K8 errata 93 or something else?

Quoted 8 lines · H. Peter Anvin

Quoted 6 lines · Jeremy Fitzhardinge

The current scheme has worked pretty well so far; there doesn't seem to be a huge concern about it. Have there been actual observed failures with the current mechanism, or is the drive to make it earlier driven by an aesthetic desire to make it "as it should be"?

Quoted 3 lines

Yes, there has. There is at least one case where we have to print an error message saying "update your BIOS" because our current microcode update is too late.

Is that K8 errata 93 or something else?

If there were a general shift to "this is how we're going to do microcode in the future", then Xen will happily go along for the ride.

But for right now, this patch seems like the pragmatic solution.

I think the real question is where there's something objectionable about the patch itself?

Quoted 2 lines · Jeremy Fitzhardinge

If there were a general shift to "this is how we're going to do microcode in the future", then Xen will happily go along for the ride.

It *is* how we want to do microcode in the future. There is a prototype for the Intel hardware side here; we just haven't had time to finalize it partly because I got pulled onto the kernel.org situation.

Quoted 4 lines

It *is* how we want to do microcode in the future. There is a prototype for the Intel hardware side here; we just haven't had time to finalize it partly because I got pulled onto the kernel.org situation.

OK, great! Has there been any public discussion about this?

Quoted 1 line

But for right now, this patch seems like the pragmatic solution.

No.

From: Jeremy Fitzhardinge <[email protected]>

Hi all,

I'm proposing this for the next merge window v3.2.

I originally posted this early this year, and it prompted a debate about what the "proper" way that Linux should do microcode updates, with the general concensus being "earlier", ideally in the bootloader (or in the case of Xen, as the hypervisor boots before starting any domains). However, as far as I know there has been no progress along those lines.

I would like to therefore merge this so that a Linux kernel booting as dom0 under Xen can update the microcode in the same manner as a kernel booting natively. When we work out how boot-time microcode updates can be done, then we'll look at modifying Xen accordingly. In the meantime, we should have a functional parity.

The only change to this code from the previous posting is some patch restructuring so that regardless of how the platform.h ABI header gets merged (since there are some other pending branches containing it), it will be identical and cause no merge headaches.

Quoted 1 line

From original posting:

This series adds a new "Xen" microcode update type, in addition to Intel and AMD.

The Xen hypervisor is responsible for performing the actual microcode update (since only it knows what physical CPUs are in the system and has sufficient privilege to access them), but it requires the dom0 kernel to provide the actual microcode update data.

Xen update mechanism is uniform independent of the CPU type, but the driver must know where to find the data file, which depends on the CPU type. And since the update hypercall updates all CPUs, we only need to execute it once on any CPU - but for simplicity it just runs it only on (V)CPU 0.

Thanks,
J

Jeremy Fitzhardinge (2):
xen: add dom0_op hypercall
xen: add CPU microcode update driver

Yu Ke (1):
xen/acpi: Domain0 acpi parser related platform hypercall

 arch/ia64/include/asm/xen/interface.h |    1 +
 arch/x86/include/asm/microcode.h      |    9 +
 arch/x86/include/asm/xen/hypercall.h  |    8 +
 arch/x86/include/asm/xen/interface.h  |    1 +
 arch/x86/kernel/Makefile              |    1 +
 arch/x86/kernel/microcode_core.c      |    5 +-
 arch/x86/kernel/microcode_xen.c       |  198 ++++++++++++++++++++
 arch/x86/xen/Kconfig                  |    4 +
 include/xen/interface/platform.h      |  320
+++++++++++++++++++++++++++++++++
 include/xen/interface/xen.h           |    1 +
 10 files changed, 547 insertions(+), 1 deletions(-)
 create mode 100644 arch/x86/kernel/microcode_xen.c
 create mode 100644 include/xen/interface/platform.h

--
1.7.6.2

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel

After pulling in this change from your tree, I found the following bug, when checking an enum value, which should be considered before inclusion:

arch/x86/kernel/microcode_xen.c +1 -1

index 9d2a06b..6a73957 100644
@@ -157,7 +157,7 @@ static enum ucode_state
 xen_request_microcode_user(int cpu,
 ret = UCODE_OK;
 
 out:
-if (ret == 0)
+if (ret == UCODE_OK)
 uci->mc = uc;
 else

vfree(uc);