Skip to contentSkip to conversation

Add rudimentary Hyper-V guest support

PATCH In review Alexander Graf 6 participants 41 messages May 15, 2009 – May 18, 2009

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

cover Add rudimentary Hyper-V guest support

Now that we have nested SVM in place, let's make use of it and virtualize something non-kvm.
The first interesting target that came to my mind here was Hyper-V.

This patchset makes Windows Server 2008 boot with Hyper-V, which runs the "dom0" in virtualized mode already. I haven't been able to run a second VM within for now though, but maybe I just wasn't patient enough ;-).

Alexander Graf (6):
Add definition for IGNNE MSR
MMU: don't bail on PAT bits in PTE
Emulator: Inject #PF when page was not found
Implement Hyper-V MSRs
Nested SVM: Implement INVLPGA
Nested SVM: Improve interrupt injection

 arch/x86/include/asm/msr-index.h |    1 +
 arch/x86/kvm/mmu.c               |    2 +-
 arch/x86/kvm/svm.c               |   59 +++++++++++++++++++++++++++----------
 arch/x86/kvm/x86.c               |    7 +++-
 4 files changed, 50 insertions(+), 19 deletions(-)
#1 lore raw

PATCH 1/6 Add definition for IGNNE MSR

Hyper-V tries to access MSR_IGNNE, so let's at least have a definition for it in our headers.

---
 arch/x86/include/asm/msr-index.h |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)
arch/x86/include/asm/msr-index.h +1 -0
index ec41fc1..e273549 100644
@@ -372,6 +372,7 @@
 /* AMD-V MSRs */
 
 #define MSR_VM_CR                       0xc0010114
+#define MSR_VM_IGNNE                    0xc0010115
 #define MSR_VM_HSAVE_PA                 0xc0010117
 
 #endif /* _ASM_X86_MSR_INDEX_H */
Signature · 1 line
1.6.0.2
#2 lore raw

PATCH 2/6 MMU: don't bail on PAT bits in PTE

A 64bit PTE can have bit7 set to 1 which means "Use this bit for the PAT". Currently KVM's MMU code treats this bit as reserved, even though it's not.

As long as we're not required to make use of the PAT bits which is only required for DMA/MMIO from my understanding, we can safely ignore it.

Hyper-V uses this bit for kernel PTEs.

---
 arch/x86/kvm/mmu.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
arch/x86/kvm/mmu.c +1 -1
index 8fcdae9..cce055a 100644
@@ -2169,7 +2169,7 @@ static void reset_rsvds_bits_mask(struct kvm_vcpu *vcpu, int level)
 		context->rsvd_bits_mask[1][1] = exb_bit_rsvd |
 			rsvd_bits(maxphyaddr, 51) |
 			rsvd_bits(13, 20);		/* large page */
-		context->rsvd_bits_mask[1][0] = ~0ull;
+		context->rsvd_bits_mask[1][0] = 0ull;
 		break;
 	}
 }
Signature · 1 line
1.6.0.2
#3 lore raw

PATCH 3/6 Emulator: Inject #PF when page was not found

If we couldn't find a page on read_emulated, it might be a good idea to tell the guest about that and inject a #PF.

We do the same already for write faults. I don't know why it was not implemented for reads.

---
 arch/x86/kvm/x86.c |    7 +++++--
 1 files changed, 5 insertions(+), 2 deletions(-)
arch/x86/kvm/x86.c +5 -2
index 5fcde2c..5aa1219 100644
@@ -2131,10 +2131,13 @@ static int emulator_read_emulated(unsigned long addr,
 		goto mmio;
 
 	if (kvm_read_guest_virt(addr, val, bytes, vcpu)
-				== X86EMUL_CONTINUE)
+				== X86EMUL_CONTINUE) {
 		return X86EMUL_CONTINUE;
-	if (gpa == UNMAPPED_GVA)
+	}
+	if (gpa == UNMAPPED_GVA) {
+		kvm_inject_page_fault(vcpu, addr, 0);
 		return X86EMUL_PROPAGATE_FAULT;
+	}
 
 mmio:
 	/*
Signature · 1 line
1.6.0.2
#4 lore raw

PATCH 4/6 Implement Hyper-V MSRs

Hyper-V uses some MSRs, some of which are actually reserved for BIOS usage.

But let's be nice today and have it its way, because otherwise it fails terribly.

For MSRs where I could find a name I used the name, otherwise they're just added in their hex form for now.

---
 arch/x86/kvm/svm.c |    5 +++++
 1 files changed, 5 insertions(+), 0 deletions(-)
arch/x86/kvm/svm.c +5 -0
index ef43a18..30e6b43 100644
@@ -1932,6 +1932,7 @@ static int svm_get_msr(struct kvm_vcpu *vcpu, unsigned ecx, u64 *data)
 		*data = svm->hsave_msr;
 		break;
 	case MSR_VM_CR:
+	case 0x40000081:
 		*data = 0;
 		break;
 	case MSR_IA32_UCODE_REV:
@@ -2034,6 +2035,10 @@ static int svm_set_msr(struct kvm_vcpu *vcpu, unsigned ecx, u64 data)
 	case MSR_VM_HSAVE_PA:
 		svm->hsave_msr = data;
 		break;
+	case MSR_VM_CR:
+	case MSR_VM_IGNNE:
+	case MSR_K8_HWCR:
+		break;
 	default:
 		return kvm_set_msr_common(vcpu, ecx, data);
 	}
Signature · 1 line
1.6.0.2
#5 lore raw

PATCH 5/6 Nested SVM: Implement INVLPGA

SVM adds another way to do INVLPG by ASID which Hyper-V makes use of, so let's implement it!

For now we just do the same thing invlpg does, as asid switching means we flush the mmu anyways. That might change one day though.

---
 arch/x86/kvm/svm.c |   14 +++++++++++++-
 1 files changed, 13 insertions(+), 1 deletions(-)
arch/x86/kvm/svm.c +13 -1
index 30e6b43..b2c6cf3 100644
@@ -1785,6 +1785,18 @@ static int clgi_interception(struct vcpu_svm *svm, struct kvm_run *kvm_run)
 	return 1;
 }
 
+static int invlpga_interception(struct vcpu_svm *svm, struct kvm_run *kvm_run)
+{
+	struct kvm_vcpu *vcpu = &svm->vcpu;
+	nsvm_printk("INVLPGA\n");
+	svm->next_rip = kvm_rip_read(&svm->vcpu) + 3;
+	skip_emulated_instruction(&svm->vcpu);
+
+	kvm_mmu_reset_context(vcpu);
+	kvm_mmu_load(vcpu);
+	return 1;
+}
+
 static int invalid_op_interception(struct vcpu_svm *svm,
 				   struct kvm_run *kvm_run)
 {
@@ -2130,7 +2142,7 @@ static int (*svm_exit_handlers[])(struct vcpu_svm *svm,
 	[SVM_EXIT_INVD]                         = emulate_on_interception,
 	[SVM_EXIT_HLT]				= halt_interception,
 	[SVM_EXIT_INVLPG]			= invlpg_interception,
-	[SVM_EXIT_INVLPGA]			= invalid_op_interception,
+	[SVM_EXIT_INVLPGA]			= invlpga_interception,
 	[SVM_EXIT_IOIO] 		  	= io_interception,
 	[SVM_EXIT_MSR]				= msr_interception,
 	[SVM_EXIT_TASK_SWITCH]			= task_switch_interception,
Signature · 1 line
1.6.0.2
#6 lore raw

PATCH 6/6 Nested SVM: Improve interrupt injection

While trying to get Hyper-V running, I realized that the interrupt injection mechanisms that are in place right now are not 100% correct.

This patch makes nested SVM's interrupt injection behave more like on a real machine.

---
 arch/x86/kvm/svm.c |   40 +++++++++++++++++++++++++---------------
 1 files changed, 25 insertions(+), 15 deletions(-)
arch/x86/kvm/svm.c +25 -15
index b2c6cf3..1d22d46 100644
@@ -1517,7 +1517,8 @@ static int nested_svm_vmexit_real(struct vcpu_svm *svm, void *arg1,
 	/* Kill any pending exceptions */
 	if (svm->vcpu.arch.exception.pending == true)
 		nsvm_printk("WARNING: Pending Exception\n");
-	svm->vcpu.arch.exception.pending = false;
+	kvm_clear_exception_queue(&svm->vcpu);
+	kvm_clear_interrupt_queue(&svm->vcpu);
 
 	/* Restore selected save entries */
 	svm->vmcb->save.es = hsave->save.es;
@@ -1585,7 +1586,8 @@ static int nested_svm_vmrun(struct vcpu_svm *svm, void *arg1,
 	svm->nested_vmcb = svm->vmcb->save.rax;
 
 	/* Clear internal status */
-	svm->vcpu.arch.exception.pending = false;
+	kvm_clear_exception_queue(&svm->vcpu);
+	kvm_clear_interrupt_queue(&svm->vcpu);
 
 	/* Save the old vmcb, so we don't need to pick what we save, but
 	   can restore everything when a VMEXIT occurs */
@@ -2276,21 +2278,15 @@ static inline void svm_inject_irq(struct vcpu_svm *svm, int irq)
 		((/*control->int_vector >> 4*/ 0xf) << V_INTR_PRIO_SHIFT);
 }
 
-static void svm_queue_irq(struct kvm_vcpu *vcpu, unsigned nr)
-{
-	struct vcpu_svm *svm = to_svm(vcpu);
-
-	svm->vmcb->control.event_inj = nr |
-		SVM_EVTINJ_VALID | SVM_EVTINJ_TYPE_INTR;
-}
-
 static void svm_set_irq(struct kvm_vcpu *vcpu, int irq)
 {
 	struct vcpu_svm *svm = to_svm(vcpu);
 
-	nested_svm_intr(svm);
+	if(!(svm->vcpu.arch.hflags & HF_GIF_MASK))
+		return;
 
-	svm_queue_irq(vcpu, irq);
+	svm->vmcb->control.event_inj = irq |
+		SVM_EVTINJ_VALID | SVM_EVTINJ_TYPE_INTR;
 }
 
 static void update_cr8_intercept(struct kvm_vcpu *vcpu, int tpr, int irr)
@@ -2318,13 +2314,25 @@ static int svm_interrupt_allowed(struct kvm_vcpu *vcpu)
 	struct vmcb *vmcb = svm->vmcb;
 	return (vmcb->save.rflags & X86_EFLAGS_IF) &&
 		!(vmcb->control.int_state & SVM_INTERRUPT_SHADOW_MASK) &&
-		(svm->vcpu.arch.hflags & HF_GIF_MASK);
+		(svm->vcpu.arch.hflags & HF_GIF_MASK) &&
+		!is_nested(svm);
 }
 
 static void enable_irq_window(struct kvm_vcpu *vcpu)
 {
-	svm_set_vintr(to_svm(vcpu));
-	svm_inject_irq(to_svm(vcpu), 0x0);
+	struct vcpu_svm *svm = to_svm(vcpu);
+	nsvm_printk("Trying to open IRQ window\n");
+
+	nested_svm_intr(svm);
+
+	/* In case GIF=0 we can't rely on the CPU to tell us when
+	 * GIF becomes 1, because that's a separate STGI/VMRUN intercept.
+	 * The next time we get that intercept, this function will be
+	 * called again though and we'll get the vintr intercept. */
+	if (svm->vcpu.arch.hflags & HF_GIF_MASK) {
+		svm_set_vintr(svm);
+		svm_inject_irq(svm, 0x0);
+	}
 }
 
 static void enable_nmi_window(struct kvm_vcpu *vcpu)
@@ -2392,6 +2400,8 @@ static void svm_complete_interrupts(struct vcpu_svm *svm)
 	case SVM_EXITINTINFO_TYPE_EXEPT:
 		/* In case of software exception do not reinject an exception
 		   vector, but re-execute and instruction instead */
+		if (is_nested(svm))
+			break;
 		if (vector == BP_VECTOR || vector == OF_VECTOR)
 			break;
 		if (exitintinfo & SVM_EXITINTINFO_VALID_ERR) {
Signature · 1 line
1.6.0.2
#7 lore raw

Re: [PATCH 6/6] Nested SVM: Improve interrupt injection

arch/x86/kvm/svm.c static inline void svm_inject_irq(struct vcpu_svm *svm, int irq) · line 2286

Quoted 8 lines · Alexander Graf

static void svm_set_irq(struct kvm_vcpu *vcpu, int irq) {

arch/x86/kvm/svm.c +2 -1
 	struct vcpu_svm *svm = to_svm(vcpu);
 
-	nested_svm_intr(svm);
+	if(!(svm->vcpu.arch.hflags & HF_GIF_MASK))
+		return;

Why would this function be called if HF_GIF_MASK is not set? This check is done in svm_interrupt_allowed().

arch/x86/kvm/svm.c static inline void svm_inject_irq(struct vcpu_svm *svm, int irq) · line 2290

Quoted 5 lines
arch/x86/kvm/svm.c +2 -1
-	svm_queue_irq(vcpu, irq);
+	svm->vmcb->control.event_inj = irq |
+		SVM_EVTINJ_VALID | SVM_EVTINJ_TYPE_INTR;
 }

--
Gleb.

#8 lore raw

Re: [PATCH 6/6] Nested SVM: Improve interrupt injection

Quoted 11 lines · Gleb Natapov

arch/x86/kvm/svm.c static inline void svm_inject_irq(struct vcpu_svm *svm, int irq) · line 2286

Quoted 8 lines · Alexander Graf

static void svm_set_irq(struct kvm_vcpu *vcpu, int irq)

{
   struct vcpu_svm *svm = to_svm(vcpu);
arch/x86/kvm/svm.c +2 -1
-    nested_svm_intr(svm);
+    if(!(svm->vcpu.arch.hflags & HF_GIF_MASK))
+        return;

Why would this function be called if HF_GIF_MASK is not set? This check is done in svm_interrupt_allowed().

I agree it shouldn't but I don't remember why exactly it triggered here. I'll try and get a backtrace tomorrow.

Either way, if it's not a return, it's a BUG().

Alex

Quoted 10 lines
Quoted 7 lines
arch/x86/kvm/svm.c +2 -1
-    svm_queue_irq(vcpu, irq);
+    svm->vmcb->control.event_inj = irq |
+        SVM_EVTINJ_VALID | SVM_EVTINJ_TYPE_INTR;

}

--
Gleb.

#9 lore raw

Re: [PATCH 6/6] Nested SVM: Improve interrupt injection

In reply to Gleb Natapov
Quoted 11 lines · Gleb Natapov

arch/x86/kvm/svm.c static inline void svm_inject_irq(struct vcpu_svm *svm, int irq) · line 2286

Quoted 8 lines · Alexander Graf

static void svm_set_irq(struct kvm_vcpu *vcpu, int irq)

{
	struct vcpu_svm *svm = to_svm(vcpu);
arch/x86/kvm/svm.c +2 -1
-	nested_svm_intr(svm);
+	if(!(svm->vcpu.arch.hflags & HF_GIF_MASK))
+		return;

Why would this function be called if HF_GIF_MASK is not set? This check is done in svm_interrupt_allowed().

Looks like I was doing something odd - WARN_ON doesn't trigger (which is reasonable).

I think I put the check in when I still had nested_svm_intr() before, because that would unset GIF.

Alex

#10 lore raw

Re: [PATCH 5/6] Nested SVM: Implement INVLPGA

In reply to Alexander Graf
Quoted 21 lines · Alexander Graf

SVM adds another way to do INVLPG by ASID which Hyper-V makes use of, so let's implement it!

For now we just do the same thing invlpg does, as asid switching means we flush the mmu anyways. That might change one day though.

---
 arch/x86/kvm/svm.c |   14 +++++++++++++-
 1 files changed, 13 insertions(+), 1 deletions(-)
arch/x86/kvm/svm.c +2 -0
index 30e6b43..b2c6cf3 100644
@@ -1785,6 +1785,18 @@ static int clgi_interception(struct vcpu_svm *svm, struct kvm_run *kvm_run)
 	return 1;
 }
 
+static int invlpga_interception(struct vcpu_svm *svm, struct kvm_run *kvm_run)
+{

arch/x86/kvm/svm.c static int clgi_interception(struct vcpu_svm *svm, struct kvm_run *kvm_run) · line 1799

Quoted 10 lines
arch/x86/kvm/svm.c +10 -0
+	struct kvm_vcpu *vcpu = &svm->vcpu;
+	nsvm_printk("INVLPGA\n");
+	svm->next_rip = kvm_rip_read(&svm->vcpu) + 3;
+	skip_emulated_instruction(&svm->vcpu);
+
+	kvm_mmu_reset_context(vcpu);
+	kvm_mmu_load(vcpu);
+	return 1;
+}
+

Hmm, since we flush the TLB on every nested-guest entry I think we can make this function a nop.

Quoted 16 lines
arch/x86/kvm/svm.c +1 -1
 static int invalid_op_interception(struct vcpu_svm *svm,
 				   struct kvm_run *kvm_run)
 {
@@ -2130,7 +2142,7 @@ static int (*svm_exit_handlers[])(struct vcpu_svm *svm,
 	[SVM_EXIT_INVD]                         = emulate_on_interception,
 	[SVM_EXIT_HLT]				= halt_interception,
 	[SVM_EXIT_INVLPG]			= invlpg_interception,
-	[SVM_EXIT_INVLPGA]			= invalid_op_interception,
+	[SVM_EXIT_INVLPGA]			= invlpga_interception,
 	[SVM_EXIT_IOIO] 		  	= io_interception,
 	[SVM_EXIT_MSR]				= msr_interception,
 	[SVM_EXIT_TASK_SWITCH]			= task_switch_interception,
Signature · 1 line
1.6.0.2
Signature · 6 lines
           | Advanced Micro Devices GmbH
 Operating | Karl-Hammerschmidt-Str. 34, 85609 Dornach bei München
 System    | 
 Research  | Geschäftsführer: Thomas M. McCoy, Giuliano Meroni
 Center    | Sitz: Dornach, Gemeinde Aschheim, Landkreis München
           | Registergericht München, HRB Nr. 43632
#11 lore raw

Re: [PATCH 5/6] Nested SVM: Implement INVLPGA

Quoted 9 lines · Joerg Roedel
Quoted 7 lines · Alexander Graf

SVM adds another way to do INVLPG by ASID which Hyper-V makes use of, so let's implement it!

For now we just do the same thing invlpg does, as asid switching means we flush the mmu anyways. That might change one day though.

Quoted 17 lines

arch/x86/kvm/svm.c static int clgi_interception(struct vcpu_svm *svm, struct kvm_run *kvm_run) · line 1799

Quoted 13 lines
arch/x86/kvm/svm.c +12 -0
+static int invlpga_interception(struct vcpu_svm *svm, struct kvm_run *kvm_run)
+{
+	struct kvm_vcpu *vcpu = &svm->vcpu;
+	nsvm_printk("INVLPGA\n");
+	svm->next_rip = kvm_rip_read(&svm->vcpu) + 3;
+	skip_emulated_instruction(&svm->vcpu);
+
+	kvm_mmu_reset_context(vcpu);
+	kvm_mmu_load(vcpu);
+	return 1;
+}
+

Hmm, since we flush the TLB on every nested-guest entry I think we can make this function a nop.

I think, unless it specified ASID 0? In that case you need a local tlb flush.

(the kvm_mmu_reset_context() and kvm_mmu_load() are total overkills in any case).

Signature · 1 line
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
#12 lore raw

Re: [PATCH 5/6] Nested SVM: Implement INVLPGA

Quoted 9 lines · Avi Kivity
Quoted 4 lines

Hmm, since we flush the TLB on every nested-guest entry I think we can make this function a nop.

I think, unless it specified ASID 0? In that case you need a local tlb flush.

(the kvm_mmu_reset_context() and kvm_mmu_load() are total overkills in any case).

Oh, but we do need to resync OOS pages, here for ASID 0 and on guest entry. Marcelo?

Signature · 1 line
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
#13 lore raw

Re: [PATCH 5/6] Nested SVM: Implement INVLPGA

Quoted 9 lines · Avi Kivity
Quoted 8 lines · Avi Kivity
Quoted 4 lines

Hmm, since we flush the TLB on every nested-guest entry I think we can make this function a nop.

I think, unless it specified ASID 0? In that case you need a local tlb flush.

Quoted 6 lines
Quoted 3 lines

(the kvm_mmu_reset_context() and kvm_mmu_load() are total overkills in any case).

Oh, but we do need to resync OOS pages, here for ASID 0 and on guest entry. Marcelo?

Right, call kvm_mmu_invlpg() with the linear address passed to INVLPGA.

#14 lore raw

Re: [PATCH 5/6] Nested SVM: Implement INVLPGA

In reply to Joerg Roedel
Quoted 21 lines · Joerg Roedel
Quoted 20 lines · Alexander Graf

SVM adds another way to do INVLPG by ASID which Hyper-V makes use of, so let's implement it!

For now we just do the same thing invlpg does, as asid switching means we flush the mmu anyways. That might change one day though.

---
arch/x86/kvm/svm.c | 14 +++++++++++++-

1 files changed, 13 insertions(+), 1 deletions(-)
arch/x86/kvm/svm.c +0 -0
index 30e6b43..b2c6cf3 100644
@@ -1785,6 +1785,18 @@ static int clgi_interception(struct vcpu_svm  
 *svm, struct kvm_run *kvm_run)
 	return 1;
 }
 
Quoted 16 lines

arch/x86/kvm/svm.c static int clgi_interception(struct vcpu_svm *svm, struct kvm_run *kvm_run) · line 1799

Quoted 13 lines
arch/x86/kvm/svm.c +12 -0
+static int invlpga_interception(struct vcpu_svm *svm, struct  
 kvm_run *kvm_run)
+{
+	struct kvm_vcpu *vcpu = &svm->vcpu;
+	nsvm_printk("INVLPGA\n");
+	svm->next_rip = kvm_rip_read(&svm->vcpu) + 3;
+	skip_emulated_instruction(&svm->vcpu);
+
+	kvm_mmu_reset_context(vcpu);
+	kvm_mmu_load(vcpu);
+	return 1;
+}
+

Hmm, since we flush the TLB on every nested-guest entry I think we can make this function a nop.

Well we flush the TLB on every VMRUN, but this is still 100% within the 2nd level guest, so I think we should do something, no?.

Alex

#15 lore raw

Re: [PATCH 4/6] Implement Hyper-V MSRs

In reply to Alexander Graf
Quoted 9 lines · Alexander Graf

Hyper-V uses some MSRs, some of which are actually reserved for BIOS usage.

But let's be nice today and have it its way, because otherwise it fails terribly.

For MSRs where I could find a name I used the name, otherwise they're just added in their hex form for now.

Most of these are not Hyper-V MSRs. They are x86 MSRs that happen to be hit by Hyper-v.

arch/x86/kvm/svm.c static int svm_get_msr(struct kvm_vcpu *vcpu, unsigned ecx, u64 *data) · line 1938

Quoted 12 lines
arch/x86/kvm/svm.c +1 -0
index ef43a18..30e6b43 100644
@@ -1932,6 +1932,7 @@ static int svm_get_msr(struct kvm_vcpu *vcpu, unsigned ecx, u64 *data)
 		*data = svm->hsave_msr;
 		break;
 	case MSR_VM_CR:
+	case 0x40000081:
 		*data = 0;
 		break;
 

This probably is a Hyper-V MSR, but I don't see how it expects it to be present in real hardware. Are you sure this is really needed?

arch/x86/kvm/svm.c static int svm_set_msr(struct kvm_vcpu *vcpu, unsigned ecx, u64 data) · line 2042

Quoted 9 lines
arch/x86/kvm/svm.c +4 -0
@@ -2034,6 +2035,10 @@ static int svm_set_msr(struct kvm_vcpu *vcpu, unsigned ecx, u64 data)
 	case MSR_VM_HSAVE_PA:
 		svm->hsave_msr = data;
 		break;
+	case MSR_VM_CR:
+	case MSR_VM_IGNNE:
+	case MSR_K8_HWCR:
+		break;
 

Please add a ratelimited printk() if any value is written which would cause behaviour which we do not emulate. This will prevent a guest getting unexpected behaviour silently.

Signature · 1 line
error compiling committee.c: too many arguments to function
#16 lore raw

Re: [PATCH 4/6] Implement Hyper-V MSRs

Quoted 14 lines · Avi Kivity
Quoted 9 lines · Alexander Graf

Hyper-V uses some MSRs, some of which are actually reserved for BIOS usage.

But let's be nice today and have it its way, because otherwise it fails terribly.

For MSRs where I could find a name I used the name, otherwise they're just added in their hex form for now.

Most of these are not Hyper-V MSRs. They are x86 MSRs that happen to be hit by Hyper-v.

Quoted 15 lines

arch/x86/kvm/svm.c static int svm_get_msr(struct kvm_vcpu *vcpu, · line 1938

Quoted 13 lines
arch/x86/kvm/svm.c +1 -0
index ef43a18..30e6b43 100644
@@ -1932,6 +1932,7 @@ static int svm_get_msr(struct kvm_vcpu *vcpu,  
 unsigned ecx, u64 *data)
       *data = svm->hsave_msr;
       break;
   case MSR_VM_CR:
+    case 0x40000081:
       *data = 0;
       break;

This probably is a Hyper-V MSR, but I don't see how it expects it to be present in real hardware. Are you sure this is really needed?

Well hyper-v just crashes/reboots if it get a #gp on that msr, so I suppose yes.

Quoted 14 lines

arch/x86/kvm/svm.c static int svm_set_msr(struct kvm_vcpu · line 2043

Quoted 12 lines
arch/x86/kvm/svm.c +4 -0
@@ -2034,6 +2035,10 @@ static int svm_set_msr(struct kvm_vcpu  
 *vcpu, unsigned ecx, u64 data)
   case MSR_VM_HSAVE_PA:
       svm->hsave_msr = data;
       break;
+    case MSR_VM_CR:
+    case MSR_VM_IGNNE:
+    case MSR_K8_HWCR:
+        break;
 

Please add a ratelimited printk() if any value is written which would cause behaviour which we do not emulate. This will prevent a guest getting unexpected behaviour silently.

Right. Good catch.

Alex

Quoted 5 lines
Signature · 2 lines
-- 
error compiling committee.c: too many arguments to function
#17 lore raw

Re: [PATCH 4/6] Implement Hyper-V MSRs

Quoted 9 lines · Alexander Graf
Quoted 7 lines
Quoted 5 lines

case MSR_VM_CR:
+ case 0x40000081:

       *data = 0;
       break;

This probably is a Hyper-V MSR, but I don't see how it expects it to be present in real hardware. Are you sure this is really needed?

Well hyper-v just crashes/reboots if it get a #gp on that msr, so I suppose yes.

This is suspicious. It won't get this MSR on real hardware.

Maybe this was with cpuid.hypervisor enabled?

Signature · 1 line
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
#18 lore raw

Re: [PATCH 4/6] Implement Hyper-V MSRs

Quoted 7 lines · Avi Kivity
Quoted 6 lines · Alexander Graf
Quoted 6 lines
Quoted 5 lines

case MSR_VM_CR:
+ case 0x40000081:

      *data = 0;
      break;
Quoted 7 lines
Quoted 3 lines
Quoted 1 line

This probably is a Hyper-V MSR, but I don't see how it expects it to be present in real hardware. Are you sure this is really needed?

Well hyper-v just crashes/reboots if it get a #gp on that msr, so I suppose yes.

This is suspicious. It won't get this MSR on real hardware.

Maybe this was with cpuid.hypervisor enabled?

Before I sent out this patch I rechecked if the 0x4 msr is really required because it seemed awkward to me too and it did, but I can recheck for a 3rd time :)

Alex

Quoted 5 lines
Signature · 2 lines
-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
#19 lore raw

Re: [PATCH 4/6] Implement Hyper-V MSRs

In reply to Avi Kivity
Quoted 7 lines · Avi Kivity
Quoted 6 lines · Alexander Graf
Quoted 6 lines
Quoted 5 lines

case MSR_VM_CR:
+ case 0x40000081:

      *data = 0;
      break;
Quoted 7 lines
Quoted 3 lines
Quoted 1 line

This probably is a Hyper-V MSR, but I don't see how it expects it to be present in real hardware. Are you sure this is really needed?

Well hyper-v just crashes/reboots if it get a #gp on that msr, so I suppose yes.

This is suspicious. It won't get this MSR on real hardware.

Maybe this was with cpuid.hypervisor enabled?

Hm - seems to boot fine without. Oh well :).

Alex

Quoted 5 lines
Signature · 2 lines
-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
#20 lore raw

Re: [PATCH 3/6] Emulator: Inject #PF when page was not found

In reply to Alexander Graf
Quoted 5 lines · Alexander Graf

If we couldn't find a page on read_emulated, it might be a good idea to tell the guest about that and inject a #PF.

We do the same already for write faults. I don't know why it was not implemented for reads.

Have you checked that the emulator will never ever do speculative reads? This may be the reason why the fault was not injected here.

Quoted 30 lines
---
 arch/x86/kvm/x86.c |    7 +++++--
 1 files changed, 5 insertions(+), 2 deletions(-)
arch/x86/kvm/x86.c +5 -2
index 5fcde2c..5aa1219 100644
@@ -2131,10 +2131,13 @@ static int emulator_read_emulated(unsigned long addr,
 		goto mmio;
 
 	if (kvm_read_guest_virt(addr, val, bytes, vcpu)
-				== X86EMUL_CONTINUE)
+				== X86EMUL_CONTINUE) {
 		return X86EMUL_CONTINUE;
-	if (gpa == UNMAPPED_GVA)
+	}
+	if (gpa == UNMAPPED_GVA) {
+		kvm_inject_page_fault(vcpu, addr, 0);
 		return X86EMUL_PROPAGATE_FAULT;
+	}
 
 mmio:
 	/*
Signature · 1 line
1.6.0.2
Signature · 6 lines
           | Advanced Micro Devices GmbH
 Operating | Karl-Hammerschmidt-Str. 34, 85609 Dornach bei München
 System    | 
 Research  | Geschäftsführer: Thomas M. McCoy, Giuliano Meroni
 Center    | Sitz: Dornach, Gemeinde Aschheim, Landkreis München
           | Registergericht München, HRB Nr. 43632
#21 lore raw

Re: [PATCH 3/6] Emulator: Inject #PF when page was not found

In reply to Alexander Graf
Quoted 7 lines · Alexander Graf

If we couldn't find a page on read_emulated, it might be a good idea to tell the guest about that and inject a #PF.

We do the same already for write faults. I don't know why it was not implemented for reads.

I can't think why it was done for writes. Normally, a guest page fault would be trapped and reflected a long time before emulation, in FNAME(page_fault)(), after walk_addr().

Can you give some details on the situation? What instruction was executed, and why kvm tried to emulate it?

(I guess it depends on the relative priority of svm instruction intercepts and the page fault intercept?)

Signature · 1 line
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
#22 lore raw

Re: [PATCH 3/6] Emulator: Inject #PF when page was not found

Quoted 4 lines · Avi Kivity
Quoted 3 lines · Alexander Graf

If we couldn't find a page on read_emulated, it might be a good idea to tell the guest about that and inject a #PF.

Quoted 8 lines
Quoted 4 lines

We do the same already for write faults. I don't know why it was not implemented for reads.

I can't think why it was done for writes. Normally, a guest page fault would be trapped and reflected a long time before emulation, in FNAME(page_fault)(), after walk_addr().

Can you give some details on the situation? What instruction was executed, and why kvm tried to emulate it?

I remember it was something about accessing the apic with npt. Maybe the real problem was the restricted bit checking that made the emulated instruction behave differently from the real mmu.

I really need to start writing down why I did things when doing them :).

I can recheck if it still breaks without the inject.

Alex

Quoted 7 lines

(I guess it depends on the relative priority of svm instruction intercepts and the page fault intercept?)

Signature · 1 line
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
#23 lore raw

Re: [PATCH 3/6] Emulator: Inject #PF when page was not found

Quoted 6 lines · Alexander Graf
Quoted 4 lines

I can't think why it was done for writes. Normally, a guest page fault would be trapped and reflected a long time before emulation, in FNAME(page_fault)(), after walk_addr().

Can you give some details on the situation? What instruction was executed, and why kvm tried to emulate it?

I remember it was something about accessing the apic with npt. Maybe the real problem was the restricted bit checking that made the emulated instruction behave differently from the real mmu.

The apic should not be mapped by Hyper-V's shadow page tables, so this should have been handled by page_fault().

Signature · 1 line
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
#24 lore raw

Re: [PATCH 3/6] Emulator: Inject #PF when page was not found

Quoted 9 lines · Avi Kivity
Quoted 6 lines · Alexander Graf
Quoted 4 lines

I can't think why it was done for writes. Normally, a guest page fault would be trapped and reflected a long time before emulation, in FNAME(page_fault)(), after walk_addr().

Can you give some details on the situation? What instruction was executed, and why kvm tried to emulate it?

I remember it was something about accessing the apic with npt. Maybe the real problem was the restricted bit checking that made the emulated instruction behave differently from the real mmu.

The apic should not be mapped by Hyper-V's shadow page tables, so this should have been handled by page_fault().

I think I only had to include this to find out that the restricted bit was checked for, so I got a blue screen in the guest :-). Hyper-V works fine without this patch on NPT.

Alex

#25 lore raw

Re: [PATCH 2/6] MMU: don't bail on PAT bits in PTE

In reply to Alexander Graf
Quoted 18 lines · Alexander Graf

A 64bit PTE can have bit7 set to 1 which means "Use this bit for the PAT". Currently KVM's MMU code treats this bit as reserved, even though it's not.

As long as we're not required to make use of the PAT bits which is only required for DMA/MMIO from my understanding, we can safely ignore it.

Hyper-V uses this bit for kernel PTEs.

---
 arch/x86/kvm/mmu.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
arch/x86/kvm/mmu.c +0 -0
index 8fcdae9..cce055a 100644

arch/x86/kvm/mmu.c static void reset_rsvds_bits_mask(struct kvm_vcpu *vcpu, int level) · line 2175

Quoted 8 lines
arch/x86/kvm/mmu.c +1 -1
@@ -2169,7 +2169,7 @@ static void reset_rsvds_bits_mask(struct kvm_vcpu *vcpu, int level)
 		context->rsvd_bits_mask[1][1] = exb_bit_rsvd |
 			rsvd_bits(maxphyaddr, 51) |
 			rsvd_bits(13, 20);		/* large page */
-		context->rsvd_bits_mask[1][0] = ~0ull;
+		context->rsvd_bits_mask[1][0] = 0ull;
 		break;
 	}
 }

Just to make sure I understand what this does: if guest sets bit7, will bit7 get set in shadow PTEs as well?

Signature · 1 line
MST
#26 lore raw

Re: [PATCH 2/6] MMU: don't bail on PAT bits in PTE

Quoted 27 lines · Michael S. Tsirkin
Quoted 26 lines · Alexander Graf

A 64bit PTE can have bit7 set to 1 which means "Use this bit for the PAT".
Currently KVM's MMU code treats this bit as reserved, even though it's not.

As long as we're not required to make use of the PAT bits which is only
required for DMA/MMIO from my understanding, we can safely ignore it.

Hyper-V uses this bit for kernel PTEs.

---
arch/x86/kvm/mmu.c | 2 +-

1 files changed, 1 insertions(+), 1 deletions(-)
arch/x86/kvm/mmu.c +0 -1
index 8fcdae9..cce055a 100644
@@ -2169,7 +2169,7 @@ static void reset_rsvds_bits_mask(struct  
 kvm_vcpu *vcpu, int level)
 		context->rsvd_bits_mask[1][1] = exb_bit_rsvd |
 			rsvd_bits(maxphyaddr, 51) |
 			rsvd_bits(13, 20);		/* large page */
-		context->rsvd_bits_mask[1][0] = ~0ull;
Quoted 8 lines
Quoted 4 lines
arch/x86/kvm/mmu.c +1 -0
+		context->rsvd_bits_mask[1][0] = 0ull;
 		break;
 	}

}

Just to make sure I understand what this does: if guest sets bit7, will
bit7 get set in shadow PTEs as well?

I don't see any code that interprets bit7, so the shadow PTE should be completely unaffected.

But to be sure I asked Jörg to take a look at it as well, as he's more familiar with the x86 SPT code than I am :-).

Alex

#27 lore raw

Re: [PATCH 2/6] MMU: don't bail on PAT bits in PTE

Quoted 39 lines · Alexander Graf
Quoted 27 lines · Michael S. Tsirkin
Quoted 26 lines · Alexander Graf

A 64bit PTE can have bit7 set to 1 which means "Use this bit for the PAT".
Currently KVM's MMU code treats this bit as reserved, even though it's not.

As long as we're not required to make use of the PAT bits which is only
required for DMA/MMIO from my understanding, we can safely ignore it.

Hyper-V uses this bit for kernel PTEs.

---
arch/x86/kvm/mmu.c | 2 +-

1 files changed, 1 insertions(+), 1 deletions(-)
arch/x86/kvm/mmu.c +0 -1
index 8fcdae9..cce055a 100644
@@ -2169,7 +2169,7 @@ static void reset_rsvds_bits_mask(struct  
 kvm_vcpu *vcpu, int level)
 		context->rsvd_bits_mask[1][1] = exb_bit_rsvd |
 			rsvd_bits(maxphyaddr, 51) |
 			rsvd_bits(13, 20);		/* large page */
-		context->rsvd_bits_mask[1][0] = ~0ull;
Quoted 8 lines
Quoted 4 lines
arch/x86/kvm/mmu.c +1 -0
+		context->rsvd_bits_mask[1][0] = 0ull;
 		break;
 	}

}

Just to make sure I understand what this does: if guest sets bit7, will
bit7 get set in shadow PTEs as well?

Quoted 5 lines

I don't see any code that interprets bit7, so the shadow PTE should be completely unaffected.

But to be sure I asked Jörg to take a look at it as well, as he's more familiar with the x86 SPT code than I am :-).

The PAT bit is not propagated into the shadow page tables. Anyway, the problem is fixed the wrong way in this patch. The real problem is that a 4kb pte is checked with mask considered for large pages (which do not exist on walker level 0). The attached patch fixes it the better way imho.

From 7530aef3ed580b70a74224f8c04857754501c496 Mon Sep 17 00:00:00 2001
From: Joerg Roedel <[email protected]>
Date: Fri, 15 May 2009 15:14:19 +0200
Subject: [PATCH] kvm/mmu: fix reserved bit checking on 4kb pte level

The reserved bits checking code looks at bit 7 of the pte to determine if it has to use the mask for a large pte or a normal pde. This does not work on 4kb pte level because bit 7 is used there for PAT. Account this in the checking function.

---
 arch/x86/kvm/mmu.c |    6 ++++--
 1 files changed, 4 insertions(+), 2 deletions(-)
arch/x86/kvm/mmu.c +4 -2
index 479e748..8d9552e 100644
@@ -2124,9 +2124,11 @@ static void paging_free(struct kvm_vcpu *vcpu)
 
 static bool is_rsvd_bits_set(struct kvm_vcpu *vcpu, u64 gpte, int level)
 {
-	int bit7;
+	int bit7 = 0;
+
+	if (level != PT_PAGE_TABLE_LEVEL)
+		bit7 = (gpte >> 7) & 1;
 
-	bit7 = (gpte >> 7) & 1;
 	return (gpte & vcpu->arch.mmu.rsvd_bits_mask[bit7][level-1]) != 0;
 }
 
Signature · 10 lines
1.6.2.4


-- 
           | Advanced Micro Devices GmbH
 Operating | Karl-Hammerschmidt-Str. 34, 85609 Dornach bei München
 System    | 
 Research  | Geschäftsführer: Thomas M. McCoy, Giuliano Meroni
 Center    | Sitz: Dornach, Gemeinde Aschheim, Landkreis München
           | Registergericht München, HRB Nr. 43632
#28 lore raw

Re: [PATCH 2/6] MMU: don't bail on PAT bits in PTE

Quoted 12 lines · Joerg Roedel

Subject: [PATCH] kvm/mmu: fix reserved bit checking on 4kb pte level

The reserved bits checking code looks at bit 7 of the pte to determine if it has to use the mask for a large pte or a normal pde. This does not work on 4kb pte level because bit 7 is used there for PAT. Account this in the checking function.

static bool is_rsvd_bits_set(struct kvm_vcpu *vcpu, u64 gpte, int level) {

arch/x86/kvm/mmu.c static void paging_free(struct kvm_vcpu *vcpu) · line 2133

Quoted 9 lines
arch/x86/kvm/mmu.c +4 -2
-	int bit7;
+	int bit7 = 0;
+
+	if (level != PT_PAGE_TABLE_LEVEL)
+		bit7 = (gpte >> 7) & 1;
 
-	bit7 = (gpte >> 7) & 1;
 	return (gpte & vcpu->arch.mmu.rsvd_bits_mask[bit7][level-1]) != 0;
 }

If we make rsvd_bits_mask[1][0] == rsvd_bits_mask[0][0], we don't need the extra check. That's why it is named bit7 and not pse (need to make sure bit 7 is not reserved in this case).

Signature · 1 line
error compiling committee.c: too many arguments to function
#29 lore raw
In reply to Alexander Graf
Quoted 7 lines · Alexander Graf

Now that we have nested SVM in place, let's make use of it and virtualize something non-kvm.
The first interesting target that came to my mind here was Hyper-V.

This patchset makes Windows Server 2008 boot with Hyper-V, which runs the "dom0" in virtualized mode already. I haven't been able to run a second VM within for now though, but maybe I just wasn't patient enough ;-).

In order to find out why things were slow with nested SVM I hacked intercept reporting into debugfs in my local tree and found pretty interesting results (using NPT):

 SVM_EXIT_CLGI          3888080       0
 SVM_EXIT_CPUID            3460       0
 SVM_EXIT_CR0_SEL_WRI         0       0
 SVM_EXIT_ERR                 0       0
 SVM_EXIT_FERR_FREEZE         0       0
 SVM_EXIT_GDTR_READ           0       0
 SVM_EXIT_GDTR_WRITE          0       0
 SVM_EXIT_HLT             40186       0
 SVM_EXIT_ICEBP               0       0
 SVM_EXIT_IDTR_READ           0       0
 SVM_EXIT_IDTR_WRITE          0       0
 SVM_EXIT_INIT                0       0
 SVM_EXIT_INTR           193173       0
 SVM_EXIT_INVD                0       0
 SVM_EXIT_INVLPG              1       0
 SVM_EXIT_INVLPGA        536994       0
 SVM_EXIT_IOIO          3450484       0
 SVM_EXIT_IRET                0       0
 SVM_EXIT_LDTR_READ           0       0
 SVM_EXIT_LDTR_WRITE          0       0
 SVM_EXIT_MONITOR             0       0
 SVM_EXIT_MSR            124614       0
 SVM_EXIT_MWAIT               0       0
 SVM_EXIT_MWAIT_COND          0       0
 SVM_EXIT_NMI                 0       0
 SVM_EXIT_NPF           1040416       0
 SVM_EXIT_PAUSE               0       0
 SVM_EXIT_POPF                0       0
 SVM_EXIT_PUSHF               0       0
 SVM_EXIT_RDPMC               0       0
 SVM_EXIT_RDTSC               0       0
 SVM_EXIT_RDTSCP              0       0
 SVM_EXIT_RSM                 0       0
 SVM_EXIT_SHUTDOWN            0       0
 SVM_EXIT_SKINIT              0       0
 SVM_EXIT_SMI                20       0
 SVM_EXIT_STGI          3888080       0
 SVM_EXIT_SWINT               0       0
 SVM_EXIT_TASK_SWITCH         0       0
 SVM_EXIT_TR_READ             0       0
 SVM_EXIT_TR_WRITE            0       0
 SVM_EXIT_VINTR          402865       0
 SVM_EXIT_VMLOAD        3888096       0
 SVM_EXIT_VMMCALL        767288       0
 SVM_EXIT_VMRUN         3888096       0
 SVM_EXIT_VMSAVE        3888096       0
 SVM_EXIT_WBINVD             64       0

So apparently the most intercepts come from the SVM helper calls (clgi, stgi, vmload, vmsave). I guess I need to get back to the "emulate when GIF=0" approach to get things fast.

Alex

#30 lore raw
Quoted 3 lines · Alexander Graf

In order to find out why things were slow with nested SVM I hacked intercept reporting into debugfs in my local tree and found pretty interesting results (using NPT):

[...]

Quoted 1 line

So apparently the most intercepts come from the SVM helper calls (clgi, stgi, vmload, vmsave). I guess I need to get back to the "emulate when GIF=0" approach to get things fast.

There's only a limited potential here (a factor of three, reducing 6 exits to 2, less the emulation overhead). There's a lot more to be gained from nested npt, since you'll avoid most of the original exits in the first place.

Signature · 1 line
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
#31 lore raw
Quoted 9 lines · Avi Kivity
Quoted 3 lines · Alexander Graf

In order to find out why things were slow with nested SVM I hacked intercept reporting into debugfs in my local tree and found pretty interesting results (using NPT):

[...]

Quoted 1 line

So apparently the most intercepts come from the SVM helper calls (clgi, stgi, vmload, vmsave). I guess I need to get back to the "emulate when GIF=0" approach to get things fast.

There's only a limited potential here (a factor of three, reducing 6 exits to 2, less the emulation overhead). There's a lot more to be gained from nested npt, since you'll avoid most of the original exits in the first place.

I think the reversed is the case. Look at those numbers (w2k8 bootup):

http://pastebin.ca/1423596

The only thing nested NPT would achieve is a reduction of #NPF exits. But they are absolutely in the minority today already. Normal #PF's do get directly passed to the guest already.

Of course, this all depends on the workload. For kernbench style benchmarks nested NPT probably gives you a bigger win, but anything doing IO is slowed down way more than it has to now.

Alex

#32 lore raw
Quoted 8 lines · Alexander Graf
Quoted 2 lines

There's only a limited potential here (a factor of three, reducing 6 exits to 2, less the emulation overhead). There's a lot more to be gained from nested npt, since you'll avoid most of the original exits in the first place.

I think the reversed is the case. Look at those numbers (w2k8 bootup):

http://pastebin.ca/1423596

The only thing nested NPT would achieve is a reduction of #NPF exits. But they are absolutely in the minority today already. Normal #PF's do get directly passed to the guest already.

#NPF exits are caused when guest/host mappings change, which they don't, or by mmio (which happens both for guest and nguest).

I don't understand how you can pass #PFs directly to the guest. Surely the guest has enabled pagefault interception, and you need to set up its vmcb?

Quoted 2 lines

Of course, this all depends on the workload. For kernbench style benchmarks nested NPT probably gives you a bigger win, but anything doing IO is slowed down way more than it has to now.

What is causing 17K pio exits/sec? What port numbers?

Signature · 1 line
error compiling committee.c: too many arguments to function
#33 lore raw
Quoted 6 lines · Avi Kivity
Quoted 5 lines · Alexander Graf
Quoted 2 lines

There's only a limited potential here (a factor of three, reducing 6 exits to 2, less the emulation overhead). There's a lot more to be gained from nested npt, since you'll avoid most of the original exits in the first place.

I think the reversed is the case. Look at those numbers (w2k8 bootup):

Quoted 7 lines
Quoted 3 lines

http://pastebin.ca/1423596

The only thing nested NPT would achieve is a reduction of #NPF exits. But they are absolutely in the minority today already. Normal #PF's do get directly passed to the guest already.

#NPF exits are caused when guest/host mappings change, which they don't, or by mmio (which happens both for guest and nguest).

I don't understand how you can pass #PFs directly to the guest. Surely the guest has enabled pagefault interception, and you need to set up its vmcb?

Ugh - looks like I totally forgot to include #PF exits in my stats, which is why I didn't see them.

Quoted 5 lines
Quoted 3 lines

Of course, this all depends on the workload. For kernbench style benchmarks nested NPT probably gives you a bigger win, but anything doing IO is slowed down way more than it has to now.

What is causing 17K pio exits/sec? What port numbers?

Any hints on how to easily find that out? For someone who's too stupid to get kvmtrace working :-).

Alex

#34 lore raw
Quoted 5 lines · Alexander Graf
Quoted 3 lines
Quoted 1 line

Of course, this all depends on the workload. For kernbench style benchmarks nested NPT probably gives you a bigger win, but anything doing IO is slowed down way more than it has to now.

What is causing 17K pio exits/sec? What port numbers?

Any hints on how to easily find that out? For someone who's too stupid to get kvmtrace working :-).

You can always printk() every 1000 loops, but kvmtrace is actually pretty easy to use. Compile it in, run your guest (pinning to one cpu deconfuses the output), run kvmtrace -o blah, then use './kvmtrace_format formats' as a filter on the binary output.

Signature · 1 line
error compiling committee.c: too many arguments to function
#35 lore raw
In reply to Avi Kivity
Quoted 6 lines · Avi Kivity
Quoted 5 lines · Alexander Graf
Quoted 2 lines

There's only a limited potential here (a factor of three, reducing 6 exits to 2, less the emulation overhead). There's a lot more to be gained from nested npt, since you'll avoid most of the original exits in the first place.

I think the reversed is the case. Look at those numbers (w2k8 bootup):

Quoted 7 lines
Quoted 3 lines

http://pastebin.ca/1423596

The only thing nested NPT would achieve is a reduction of #NPF exits. But they are absolutely in the minority today already. Normal #PF's do get directly passed to the guest already.

#NPF exits are caused when guest/host mappings change, which they don't, or by mmio (which happens both for guest and nguest).

I don't understand how you can pass #PFs directly to the guest. Surely the guest has enabled pagefault interception, and you need to set up its vmcb?

I guess you're right: http://pastebin.ca/1426458

Alex

#36 lore raw
Quoted 11 lines · Alexander Graf
Quoted 8 lines · Avi Kivity
Quoted 7 lines · Alexander Graf
Quoted 2 lines

There's only a limited potential here (a factor of three, reducing 6 exits to 2, less the emulation overhead). There's a lot more to be gained from nested npt, since you'll avoid most of the original exits in the first place.

I think the reversed is the case. Look at those numbers (w2k8 bootup):

http://pastebin.ca/1423596

Quoted 7 lines
Quoted 5 lines
Quoted 1 line

The only thing nested NPT would achieve is a reduction of #NPF exits. But they are absolutely in the minority today already. Normal #PF's do get directly passed to the guest already.

#NPF exits are caused when guest/host mappings change, which they don't, or by mmio (which happens both for guest and nguest).

I don't understand how you can pass #PFs directly to the guest. Surely the guest has enabled pagefault interception, and you need to set up its vmcb?

I guess you're right: http://pastebin.ca/1426458

Any idea where the ioio exits come from? If it's IDE, we can eliminate them by using virtio.

Signature · 1 line
error compiling committee.c: too many arguments to function
#37 lore raw
Quoted 14 lines · Avi Kivity
Quoted 13 lines · Alexander Graf
Quoted 10 lines · Avi Kivity
Quoted 8 lines · Alexander Graf
Quoted 2 lines

There's only a limited potential here (a factor of three, reducing 6 exits to 2, less the emulation overhead). There's a lot more to be gained from nested npt, since you'll avoid most of the original exits in the first place.

I think the reversed is the case. Look at those numbers (w2k8 bootup):

http://pastebin.ca/1423596

The only thing nested NPT would achieve is a reduction of #NPF exits. But they are absolutely in the minority today already. Normal #PF's do get directly passed to the guest already.

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines

#NPF exits are caused when guest/host mappings change, which they don't, or by mmio (which happens both for guest and nguest).

I don't understand how you can pass #PFs directly to the guest. Surely the guest has enabled pagefault interception, and you need to set up its vmcb?

I guess you're right: http://pastebin.ca/1426458

Any idea where the ioio exits come from? If it's IDE, we can eliminate them by using virtio.

I'm still not getting kvmtrace to work. ./kvmtrace -o log only gives me empty files (4 bytes each), even though I did ./configure --with-kvm-trace in the kernel dir.

Alex

#38 lore raw
Quoted 3 lines · Alexander Graf

I'm still not getting kvmtrace to work. ./kvmtrace -o log only gives me empty files (4 bytes each), even though I did ./configure --with-kvm-trace in the kernel dir.

Which kernel dir? kvm-kmod? it worked for me there.

Signature · 1 line
error compiling committee.c: too many arguments to function
#39 lore raw
Quoted 6 lines · Avi Kivity
Quoted 3 lines · Alexander Graf

I'm still not getting kvmtrace to work. ./kvmtrace -o log only gives me empty files (4 bytes each), even though I did ./configure --with-kvm-trace in the kernel dir.

Which kernel dir? kvm-kmod? it worked for me there.

Have things changed again? I used the kvm/kernel directory from qemu-kvm.git.

Are the any dependencies on the host kernel I might not have fulfilled?

Alex

#40 lore raw
Quoted 11 lines · Alexander Graf
Quoted 6 lines · Avi Kivity
Quoted 3 lines · Alexander Graf

I'm still not getting kvmtrace to work. ./kvmtrace -o log only gives me empty files (4 bytes each), even though I did ./configure --with-kvm-trace in the kernel dir.

Which kernel dir? kvm-kmod? it worked for me there.

Have things changed again? I used the kvm/kernel directory from qemu-kvm.git.

They're much simpler. clone kvm-kmod.git (from same directory), 'git submodule update --init', you'll get linux-2.6 underneath. ./configure && make sync && make.

Quoted 3 lines

Are the any dependencies on the host kernel I might not have fulfilled?

CONFIG_MARKERS.

Signature · 1 line
error compiling committee.c: too many arguments to function