kvm: bios: Put AP boot up code to 0x1000

From ba05b3c821400127074f65ee4d172fabbe3524cd Mon Sep 17 00:00:00 2001 From: Sheng Yang <[email protected]>
Date: Thu, 31 Jul 2008 10:39:24 +0800
Subject: [PATCH] kvm: bios: Put AP boot up code to 0x1000

Rather than 0x10000, which can be overrided by userspace program like grub.

Signed-off-by Sheng Yang <[email protected]>

---
 bios/rombios32.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

bios/rombios32.c +1 -2

index 2dc1d25..f7a4234 100755
@@ -60,7 +60,7 @@ typedef unsigned long long uint64_t;
 
 #define APIC_ENABLED 0x0100
 
-#define AP_BOOT_ADDR 0x10000
+#define AP_BOOT_ADDR 0x1000
 
 #define MPTABLE_MAX_SIZE  0x00002000
 #define SMI_CMD_IO_ADDR   0xb2
--

1.5.6

Quoted 8 lines · Sheng

From ba05b3c821400127074f65ee4d172fabbe3524cd Mon Sep 17 00:00:00 2001 From: Sheng Yang <[email protected]>
Date: Thu, 31 Jul 2008 10:39:24 +0800
Subject: [PATCH] kvm: bios: Put AP boot up code to 0x1000

Rather than 0x10000, which can be overrided by userspace program like grub.

Quoted 6 lines · Sheng

Quoted 5 lines · Sheng

From ba05b3c821400127074f65ee4d172fabbe3524cd Mon Sep 17 00:00:00 2001 From: Sheng Yang <[email protected]>
Date: Thu, 31 Jul 2008 10:39:24 +0800
Subject: [PATCH] kvm: bios: Put AP boot up code to 0x1000

Quoted 2 lines

Quoted 2 lines

Rather than 0x10000, which can be overrided by userspace program like grub.

How about 0xex000? And avoid copying the bios to low memory?

Gleb tried it and it worked. We'd have to change qemu to mark the memory as RAM, not ROM, but that's what real hardware does as well (shadowing the ROM into RAM).

Quoted 14 lines · On Thursday 31 July 2008 14:41:04 Avi Kivity wrote:

Quoted 10 lines · Sheng

Quoted 8 lines · Sheng

From ba05b3c821400127074f65ee4d172fabbe3524cd Mon Sep 17 00:00:00 2001 From: Sheng Yang <[email protected]>
Date: Thu, 31 Jul 2008 10:39:24 +0800
Subject: [PATCH] kvm: bios: Put AP boot up code to 0x1000

Rather than 0x10000, which can be overrided by userspace program like grub.

How about 0xex000? And avoid copying the bios to low memory?

Quoted 4 lines

Gleb tried it and it worked. We'd have to change qemu to mark the memory as RAM, not ROM, but that's what real hardware does as well (shadowing the ROM into RAM).

I tried 0xea000, and it's fine. But avoid copying is better.

But I don't understand the RAM/ROM issue in qemu here...

And do you mean you have patch on hand now? :)

IIRC the rombios32.c writes to the memory it is in, so it expects RAM, not ROM.

kvm doesn't support ROM, so it would work. Qemu doesn't, so it would fail.

Rather than 0x10000, which can be overrided by userspace program like grub.

Quoted 2 lines · Sheng

Rather than 0x10000, which can be overrided by userspace program like grub.

Could you please explain this problem?
If i don't misunderstand this smp_ap_boot_code_start from rombios32start.S gets copied to AP_BOOT_CODE and then there is a broadcast INIT and SIPI. After the code is run it's not used anymore so there should be no need to care where it's located.

- Sebastian

Please refer to my another patch comment. Basicly, AP is running HLT in 0x10038, but it can exit due to some reason, here QEmu want to raise it up . But after AP exit from HLT, it can't return to it because the code was overrided by grub and no HLT loop there. Then AP had gone away...

Quoted 5 lines · Sheng

Please refer to my another patch comment. Basicly, AP is running HLT in 0x10038, but it can exit due to some reason, here QEmu want to raise it up . But after AP exit from HLT, it can't return to it because the code was overrided by grub and no HLT loop there. Then AP had gone away...

Thank you for pointing me to "[PATCH] KVM: Fix QEmu interrupted HLT emulation". You wrote there:
"Though I also sent a patch for BIOS, it's necessary to get correct behavior here."

Would i be correct saying the problem is kvm/qemu only and with that patch applied the bios fix is not needed?

If you still want to change the bios code i would suggest jumping to the original smp_ap_boot_code location in rombios32.bin which should not get overwritten. The following patch is based on the bochs version.

Quoted 11 lines

Quoted 5 lines

Please refer to my another patch comment. Basicly, AP is running HLT in 0x10038, but it can exit due to some reason, here QEmu want to raise it up . But after AP exit from HLT, it can't return to it because the code was overrided by grub and no HLT loop there. Then AP had gone away...

Thank you for pointing me to "[PATCH] KVM: Fix QEmu interrupted HLT emulation". You wrote there:
"Though I also sent a patch for BIOS, it's necessary to get correct behavior here."

Quoted 2 lines

Would i be correct saying the problem is kvm/qemu only and with that patch applied the bios fix is not needed?

No... I think more important is get bios fixed. Anyway, copy BIOS part of running code to a place that may overwritten by userspace program is wrong, if you expect AP still running when userspace program is running.

In fact, if BIOS code in right place, we won't need fix HLT, for HLT have a assumption that it might be interrupt by some event, so usually it is wrapped by a loop, as current BIOS did.

Quoted 5 lines

If you still want to change the bios code i would suggest jumping to the original smp_ap_boot_code location in rombios32.bin which should not get overwritten. The following patch is based on the bochs version.

Yeah, I just curious why we need to copy BIOS here and there in the beginning...

Thanks for your patch, but is it based on bochs upstream?

I found the patch don't apply...

I also want to know why we need copy AP boot code, if anyone know the detail...

The AP enters HALT state with IF=0. What can make it resume at the following "jmp 1b"? NMI? SMI?

I think i read something about a need for 4kb alignment for the startup vector in the mp specs. AP_BOOT_ADDR is currently at 0x10000 (16 x 4kb).

- Sebastian

rombios32start.S Wed Aug 6 21:39:17 2008 renamed from rombios32start.S.orig Wed Aug 6 21:37:22 2008 +0 -0

@@ -40,6 +40,7 @@ _start:
 
  .code16

smp_ap_boot_code_start:

rombios32start.S Wed Aug 6 21:39:17 2008 renamed from rombios32start.S.orig Wed Aug 6 21:37:22 2008 +1 -0

+  cli
  xor %ax, %ax
  mov %ax, %ds
  xor %eax, %eax

@@ -60,6 +61,7 @@ smp_ap_boot_code_start:

12:

incw CPU_COUNT_ADDR
+ ljmp $0xe000, $(1f-_start)
1:
hlt
jmp 1b

Quoted 4 lines

The AP enters HALT state with IF=0. What can make it resume at the following "jmp 1b"? NMI? SMI?

Yes, but also INIT.

We might ignore IF=0 for hlt (which would be a bug).

Quoted 4 lines

I think i read something about a need for 4kb alignment for the startup vector in the mp specs. AP_BOOT_ADDR is currently at 0x10000 (16 x 4kb).

Yes, the SIPI address is 4K x sipi vector.

Quoted 6 lines · Avi Kivity

Quoted 4 lines

The AP enters HALT state with IF=0. What can make it resume at the following "jmp 1b"? NMI? SMI?

Yes, but also INIT.

Are you sure the AP will continue after the hlt instruction on INIT? I think it will start to execute code from the reset vector (0xfffffff0).

Jump to rombios before executing the halt loop.

Signed-off-by Sebastian Herbszt <[email protected]>

diff -uNrp a/bios/rombios32start.S b/bios/rombios32start.S

bios/rombios32start.S +2 -1

@@ -40,6 +40,7 @@ _start:
 
   .code16
 smp_ap_boot_code_start:
+  cli
   xor %ax, %ax
   mov %ax, %ds
   xor %eax, %eax

@@ -60,6 +61,7 @@ smp_ap_boot_code_start:
 12:
 
   incw CPU_COUNT_ADDR
+  ljmp $0xe000, $(1f-_start)
 1:
   hlt
   jmp 1b
 
--

To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 1 line · Sebastian Herbszt

Jump to rombios before executing the halt loop.

Why? More importantly, why is this specific to KVM?

Regards,

Anthony Liguori

Quoted 23 lines

Signed-off-by Sebastian Herbszt <[email protected]>

diff -uNrp a/bios/rombios32start.S b/bios/rombios32start.S

bios/rombios32start.S +2 -1

@@ -40,6 +40,7 @@ _start:
 
   .code16
 smp_ap_boot_code_start:
+  cli
   xor %ax, %ax
   mov %ax, %ds
   xor %eax, %eax

@@ -60,6 +61,7 @@ smp_ap_boot_code_start:
 12:
 
   incw CPU_COUNT_ADDR
+  ljmp $0xe000, $(1f-_start)
 1:
   hlt
   jmp 1b
 
--

Quoted 4 lines

To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at
http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 4 lines · On Monday 18 August 2008 10:33:11 Anthony Liguori wrote:

Quoted 2 lines · Sebastian Herbszt

Jump to rombios before executing the halt loop.

Quoted 2 lines

Why? More importantly, why is this specific to KVM?

The bios copy AP boot up code to 0x10000 now in KVM, so it can be overwrite by userspace program like grub. I found it caused stop/cont/info cpus in grub corrupt AP. Please refer to "
KVM: Fix exiting from HLT emulation with MP_STATE_HALTED"
(well, I found the patch duplicated with Marcelo's patch later, then drop it...)

Yeah, I am agree this should not be a specific issue to KVM.

Jump to rombios before executing the halt loop.

 
   .code16

smp_ap_boot_code_start:
+ cli

Quoted 4 lines

   .code16

smp_ap_boot_code_start:
+ cli

Redundant (but no harm done).