KVM: x86: Fix KVM behavior that does not follow spec

According to Intel specifications, PAE and non-PAE does not have any reserved bits. In long-mode, regardless to PCIDE, only the high bits (above the physical address) are reserved.

Signed-off-by Nadav Amit <[email protected]>

---
:100644 100644 7de069af.. e21aee9... M arch/x86/include/asm/kvm_host.h
:100644 100644 205b17e... 1d60374... M arch/x86/kvm/emulate.c
:100644 100644 8b8fc0b... f4d9839... M arch/x86/kvm/x86.c

 arch/x86/include/asm/kvm_host.h |    6 +-----
 arch/x86/kvm/emulate.c          |    4 ----
 arch/x86/kvm/x86.c              |   25 +++++--------------------
 3 files changed, 6 insertions(+), 29 deletions(-)

arch/x86/include/asm/kvm_host.h +1 -5

index 7de069af..e21aee9 100644
@@ -50,11 +50,7 @@
 			  | X86_CR0_ET | X86_CR0_NE | X86_CR0_WP | X86_CR0_AM \
 			  | X86_CR0_NW | X86_CR0_CD | X86_CR0_PG))
 
-#define CR3_PAE_RESERVED_BITS ((X86_CR3_PWT | X86_CR3_PCD) - 1)
-#define CR3_NONPAE_RESERVED_BITS ((PAGE_SIZE-1) & ~(X86_CR3_PWT | X86_CR3_PCD))
-#define CR3_PCID_ENABLED_RESERVED_BITS 0xFFFFFF0000000000ULL
-#define CR3_L_MODE_RESERVED_BITS (CR3_NONPAE_RESERVED_BITS |	\
-				  0xFFFFFF0000000000ULL)
+#define CR3_L_MODE_RESERVED_BITS 0xFFFFFF0000000000ULL
 #define CR4_RESERVED_BITS                                               \
 	(~(unsigned long)(X86_CR4_VME | X86_CR4_PVI | X86_CR4_TSD | X86_CR4_DE\
 			  | X86_CR4_PSE | X86_CR4_PAE | X86_CR4_MCE     \

arch/x86/kvm/emulate.c +0 -4

index 205b17e..1d60374 100644
@@ -3386,10 +3386,6 @@ static int check_cr_write(struct x86_emulate_ctxt *ctxt)
 		ctxt->ops->get_msr(ctxt, MSR_EFER, &efer);
 		if (efer & EFER_LMA)
 			rsvd = CR3_L_MODE_RESERVED_BITS;
-		else if (ctxt->ops->get_cr(ctxt, 4) & X86_CR4_PAE)
-			rsvd = CR3_PAE_RESERVED_BITS;
-		else if (ctxt->ops->get_cr(ctxt, 0) & X86_CR0_PG)
-			rsvd = CR3_NONPAE_RESERVED_BITS;
 
 		if (new_val & rsvd)
 			return emulate_gp(ctxt, 0);

arch/x86/kvm/x86.c +0 -15

index 8b8fc0b..f4d9839 100644
@@ -701,26 +701,11 @@ int kvm_set_cr3(struct kvm_vcpu *vcpu, unsigned long cr3)
 		return 0;
 	}
 
-	if (is_long_mode(vcpu)) {
-		if (kvm_read_cr4_bits(vcpu, X86_CR4_PCIDE)) {
-			if (cr3 & CR3_PCID_ENABLED_RESERVED_BITS)
-				return 1;
-		} else
-			if (cr3 & CR3_L_MODE_RESERVED_BITS)
-				return 1;
-	} else {
-		if (is_pae(vcpu)) {
-			if (cr3 & CR3_PAE_RESERVED_BITS)
-				return 1;
-			if (is_paging(vcpu) &&
-			    !load_pdptrs(vcpu, vcpu->arch.walk_mmu, cr3))
-				return 1;
-		}

From: Jan Kiszka <[email protected]>

Regression of 346874c9: PAE is set in long mode, but that does not mean we have valid PDPTRs.

Signed-off-by Jan Kiszka <[email protected]>

---
 arch/x86/kvm/x86.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

arch/x86/kvm/x86.c +0 -2

index c5582c3..198aac8 100644
@@ -701,10 +701,11 @@ int kvm_set_cr3(struct kvm_vcpu *vcpu, unsigned long cr3)
 		return 0;
 	}
 
-	if (is_long_mode(vcpu) && (cr3 & CR3_L_MODE_RESERVED_BITS))
-		return 1;

When using address-size override prefix with string instructions in long-mode, ESI/EDI/ECX are zero extended if they are affected by the instruction (incremented/decremented). Currently, the KVM emulator does not do so.

In addition, although it is not well-documented, when address override prefix is used with REP-string instruction, RCX high half is zeroed even if ECX was zero on the first iteration. Therefore, the emulator should clear the upper part of RCX in this case, as x86 CPUs do.

Signed-off-by Nadav Amit <[email protected]>

---
:100644 100644 69e2636... a69ed67... M arch/x86/kvm/emulate.c

 arch/x86/kvm/emulate.c |    4 ++++
 1 file changed, 4 insertions(+)

arch/x86/kvm/emulate.c +0 -0

index 69e2636..a69ed67 100644

arch/x86/kvm/emulate.c +0 -0

 			ctxt->eip = ctxt->_eip;
 			goto done;
 		}

Signature · 1 line

1.7.10.4

Quoted 20 lines · Nadav Amit

When using address-size override prefix with string instructions in long-mode, ESI/EDI/ECX are zero extended if they are affected by the instruction (incremented/decremented). Currently, the KVM emulator does not do so.

In addition, although it is not well-documented, when address override prefix is used with REP-string instruction, RCX high half is zeroed even if ECX was zero on the first iteration. Therefore, the emulator should clear the upper part of RCX in this case, as x86 CPUs do.

Signed-off-by Nadav Amit <[email protected]>

---
:100644 100644 69e2636... a69ed67... M arch/x86/kvm/emulate.c

  arch/x86/kvm/emulate.c |    4 ++++
  1 file changed, 4 insertions(+)

arch/x86/kvm/emulate.c +0 -0

index 69e2636..a69ed67 100644

arch/x86/kvm/emulate.c register_address_increment(struct x86_emulate_ctxt *ctxt, unsigned long *reg, in · line 495

Quoted 5 lines

arch/x86/kvm/emulate.c +2 -0

@@ -491,6 +491,8 @@ register_address_increment(struct x86_emulate_ctxt *ctxt, unsigned long *reg, in
  	else
  		mask = ad_mask(ctxt);
  	masked_increment(reg, mask, inc);
+	if (ctxt->ad_bytes == 4)
+		*reg &= 0xffffffff;

*reg=(u32)*reg; and you can do it inside else part.

register_address_increment() is used also by jmp_rel and loop instructions, is this correct for both of those too? Probably yes.

arch/x86/kvm/emulate.c int x86_emulate_insn(struct x86_emulate_ctxt *ctxt) · line 4573

Quoted 9 lines

arch/x86/kvm/emulate.c +2 -0

  }
 
  static void rsp_increment(struct x86_emulate_ctxt *ctxt, int inc)

@@ -4567,6 +4569,8 @@ int x86_emulate_insn(struct x86_emulate_ctxt *ctxt)
  	if (ctxt->rep_prefix && (ctxt->d & String)) {
  		/* All REP prefixes have the same first termination condition */
  		if (address_mask(ctxt, reg_read(ctxt, VCPU_REGS_RCX)) == 0) {
+			if (ctxt->ad_bytes == 4)
+				*reg_write(ctxt, VCPU_REGS_RCX) = 0;

Does zero extension happens even if ECX was zero at the beginning on an instruction or only during ECX modification. If later it is already covered in register_address_increment, no?

Gleb,

Quoted 28 lines · Gleb Natapov

Quoted 20 lines · Nadav Amit

When using address-size override prefix with string instructions in long-mode, ESI/EDI/ECX are zero extended if they are affected by the instruction (incremented/decremented). Currently, the KVM emulator does not do so.

In addition, although it is not well-documented, when address override prefix is used with REP-string instruction, RCX high half is zeroed even if ECX was zero on the first iteration. Therefore, the emulator should clear the upper part of RCX in this case, as x86 CPUs do.

Signed-off-by Nadav Amit <[email protected]>

---
:100644 100644 69e2636... a69ed67... M arch/x86/kvm/emulate.c

 arch/x86/kvm/emulate.c |    4 ++++
 1 file changed, 4 insertions(+)

arch/x86/kvm/emulate.c +0 -0

index 69e2636..a69ed67 100644

arch/x86/kvm/emulate.c register_address_increment(struct x86_emulate_ctxt *ctxt, unsigned long *reg, in · line 495

Quoted 5 lines

arch/x86/kvm/emulate.c +2 -0

@@ -491,6 +491,8 @@ register_address_increment(struct x86_emulate_ctxt *ctxt, unsigned long *reg, in
 	else
 		mask = ad_mask(ctxt);
 	masked_increment(reg, mask, inc);
+	if (ctxt->ad_bytes == 4)
+		*reg &= 0xffffffff;

*reg=(u32)*reg; and you can do it inside else part.

Quoted 3 lines

register_address_increment() is used also by jmp_rel and loop instructions, is this correct for both of those too? Probably yes.

It appears to be so.
Results of 32-bit operations are implicitly zero extended to 64-bit values, and this appears to apply to all 32 bit operations, including implicit ones. Therefore it seems to apply to all these operations.

Quoted 11 lines

arch/x86/kvm/emulate.c int x86_emulate_insn(struct x86_emulate_ctxt *ctxt) · line 4573

Quoted 9 lines

arch/x86/kvm/emulate.c +2 -0

 }
 
 static void rsp_increment(struct x86_emulate_ctxt *ctxt, int inc)

@@ -4567,6 +4569,8 @@ int x86_emulate_insn(struct x86_emulate_ctxt *ctxt)
 	if (ctxt->rep_prefix && (ctxt->d & String)) {
 		/* All REP prefixes have the same first termination condition */
 		if (address_mask(ctxt, reg_read(ctxt, VCPU_REGS_RCX)) == 0) {
+			if (ctxt->ad_bytes == 4)
+				*reg_write(ctxt, VCPU_REGS_RCX) = 0;

Does zero extension happens even if ECX was zero at the beginning on an instruction or only during ECX modification. If later it is already covered in register_address_increment, no?

The observed behaviour of the Sandy-Bridge I use, is that even if ECX is zero on the first iteration, the high half of RCX is zeroed. Therefore, this is a different case, which was not covered in register_address_increment. I agree it is totally undocumented. Following your previous comment - I may have missed the case in which loop instruction is executed with ECX = 0 while RCX != 0 and the address size is 32 bit. I will test this case soon (yet, it is lower on my priority list).

Quoted 51 lines · Nadav Amit

Gleb,

Quoted 28 lines · Gleb Natapov

Quoted 20 lines · Nadav Amit

When using address-size override prefix with string instructions in long-mode, ESI/EDI/ECX are zero extended if they are affected by the instruction (incremented/decremented). Currently, the KVM emulator does not do so.

In addition, although it is not well-documented, when address override prefix is used with REP-string instruction, RCX high half is zeroed even if ECX was zero on the first iteration. Therefore, the emulator should clear the upper part of RCX in this case, as x86 CPUs do.

Signed-off-by Nadav Amit <[email protected]>

---
:100644 100644 69e2636... a69ed67... M arch/x86/kvm/emulate.c

 arch/x86/kvm/emulate.c |    4 ++++
 1 file changed, 4 insertions(+)

arch/x86/kvm/emulate.c +0 -0

index 69e2636..a69ed67 100644

arch/x86/kvm/emulate.c register_address_increment(struct x86_emulate_ctxt *ctxt, unsigned long *reg, in · line 495

Quoted 5 lines

arch/x86/kvm/emulate.c +2 -0

@@ -491,6 +491,8 @@ register_address_increment(struct x86_emulate_ctxt *ctxt, unsigned long *reg, in
 	else
 		mask = ad_mask(ctxt);
 	masked_increment(reg, mask, inc);
+	if (ctxt->ad_bytes == 4)
+		*reg &= 0xffffffff;

*reg=(u32)*reg; and you can do it inside else part.

Quoted 3 lines

register_address_increment() is used also by jmp_rel and loop instructions, is this correct for both of those too? Probably yes.

It appears to be so.
Results of 32-bit operations are implicitly zero extended to 64-bit values, and this appears to apply to all 32 bit operations, including implicit ones. Therefore it seems to apply to all these operations.

Quoted 11 lines

arch/x86/kvm/emulate.c int x86_emulate_insn(struct x86_emulate_ctxt *ctxt) · line 4573

Quoted 9 lines

arch/x86/kvm/emulate.c +2 -0

 }
 
 static void rsp_increment(struct x86_emulate_ctxt *ctxt, int inc)

@@ -4567,6 +4569,8 @@ int x86_emulate_insn(struct x86_emulate_ctxt *ctxt)
 	if (ctxt->rep_prefix && (ctxt->d & String)) {
 		/* All REP prefixes have the same first termination condition */
 		if (address_mask(ctxt, reg_read(ctxt, VCPU_REGS_RCX)) == 0) {
+			if (ctxt->ad_bytes == 4)
+				*reg_write(ctxt, VCPU_REGS_RCX) = 0;

Does zero extension happens even if ECX was zero at the beginning on an instruction or only during ECX modification. If later it is already covered in register_address_increment, no?

Quoted 8 lines

The observed behaviour of the Sandy-Bridge I use, is that even if ECX is zero on the first iteration, the high half of RCX is zeroed. Therefore, this is a different case, which was not covered in register_address_increment. I agree it is totally undocumented. Following your previous comment - I may have missed the case in which loop instruction is executed with ECX = 0 while RCX != 0 and the address size is 32 bit. I will test this case soon (yet, it is lower on my priority list).

In 64-bit mode, the operand size for all near branches (CALL, RET, JCC, JCXZ, JMP, and LOOP) is forced to 64 bits.

These instructions update the 64-bit RIP without the need for a REX operand-size prefix.

The following aspects of near branches are controlled by the effective operand size:
• Truncation of the size of the instruction pointer
...

In 64-bit mode, all of the above actions are forced to 64 bits regardless of operand size prefixes (operand size
prefixes are silently ignored). However, the displacement field for relative branches is still limited to 32 bits and the address size for near branches is not forced in 64-bit mode. Address sizes affect the size of RCX used for JCXZ and LOOP; they also impact the address calculation for memory
indirect branches. Such addresses are 64 bits by default; but they can be overridden to 32 bits by an address size
prefix.

So it seems your patch incorrectly handles "rex call" for example.

Quoted 83 lines · Marcelo Tosatti

Quoted 51 lines · Nadav Amit

Gleb,

Quoted 28 lines · Gleb Natapov

Quoted 20 lines · Nadav Amit

When using address-size override prefix with string instructions in long-mode, ESI/EDI/ECX are zero extended if they are affected by the instruction (incremented/decremented). Currently, the KVM emulator does not do so.

In addition, although it is not well-documented, when address override prefix is used with REP-string instruction, RCX high half is zeroed even if ECX was zero on the first iteration. Therefore, the emulator should clear the upper part of RCX in this case, as x86 CPUs do.

Signed-off-by Nadav Amit <[email protected]>

---
:100644 100644 69e2636... a69ed67... M arch/x86/kvm/emulate.c

  arch/x86/kvm/emulate.c |    4 ++++
  1 file changed, 4 insertions(+)

arch/x86/kvm/emulate.c +0 -0

index 69e2636..a69ed67 100644

arch/x86/kvm/emulate.c register_address_increment(struct x86_emulate_ctxt *ctxt, unsigned long *reg, in · line 495

Quoted 5 lines

arch/x86/kvm/emulate.c +2 -0

@@ -491,6 +491,8 @@ register_address_increment(struct x86_emulate_ctxt *ctxt, unsigned long *reg, in
  	else
  		mask = ad_mask(ctxt);
  	masked_increment(reg, mask, inc);
+	if (ctxt->ad_bytes == 4)
+		*reg &= 0xffffffff;

*reg=(u32)*reg; and you can do it inside else part.

Quoted 3 lines

register_address_increment() is used also by jmp_rel and loop instructions, is this correct for both of those too? Probably yes.

It appears to be so.
Results of 32-bit operations are implicitly zero extended to 64-bit values, and this appears to apply to all 32 bit operations, including implicit ones. Therefore it seems to apply to all these operations.

Quoted 11 lines

arch/x86/kvm/emulate.c int x86_emulate_insn(struct x86_emulate_ctxt *ctxt) · line 4573

Quoted 9 lines

arch/x86/kvm/emulate.c +2 -0

  }
 
  static void rsp_increment(struct x86_emulate_ctxt *ctxt, int inc)

@@ -4567,6 +4569,8 @@ int x86_emulate_insn(struct x86_emulate_ctxt *ctxt)
  	if (ctxt->rep_prefix && (ctxt->d & String)) {
  		/* All REP prefixes have the same first termination condition */
  		if (address_mask(ctxt, reg_read(ctxt, VCPU_REGS_RCX)) == 0) {
+			if (ctxt->ad_bytes == 4)
+				*reg_write(ctxt, VCPU_REGS_RCX) = 0;

Does zero extension happens even if ECX was zero at the beginning on an instruction or only during ECX modification. If later it is already covered in register_address_increment, no?

Quoted 8 lines

The observed behaviour of the Sandy-Bridge I use, is that even if ECX is zero on the first iteration, the high half of RCX is zeroed. Therefore, this is a different case, which was not covered in register_address_increment. I agree it is totally undocumented. Following your previous comment - I may have missed the case in which loop instruction is executed with ECX = 0 while RCX != 0 and the address size is 32 bit. I will test this case soon (yet, it is lower on my priority list).

In 64-bit mode, the operand size for all near branches (CALL, RET, JCC, JCXZ, JMP, and LOOP) is forced to 64 bits.

These instructions update the 64-bit RIP without the need for a REX operand-size prefix.

The following aspects of near branches are controlled by the effective operand size:
• Truncation of the size of the instruction pointer
...

In 64-bit mode, all of the above actions are forced to 64 bits regardless of operand size prefixes (operand size
prefixes are silently ignored). However, the displacement field for relative branches is still limited to 32 bits and the address size for near branches is not forced in 64-bit mode. Address sizes affect the size of RCX used for JCXZ and LOOP; they also impact the address calculation for memory
indirect branches. Such addresses are 64 bits by default; but they can be overridden to 32 bits by an address size
prefix.

Quoted 1 line

So it seems your patch incorrectly handles "rex call" for example.

Err, operand size is forced to 64-bits, not address size.

"The following aspects of near branches are controlled by the effective operand size:
• Truncation of the size of the instruction pointer"

Still, "67h call" should not truncate EIP (which your patch does).

Quoted 9 lines

Err, operand size is forced to 64-bits, not address size.

"The following aspects of near branches are controlled by the effective operand size:
• Truncation of the size of the instruction pointer"

Still, "67h call" should not truncate EIP (which your patch does).

Yes, I missed it.
But if I am not mistaken again, it means that the existing implementation of jmp_rel is broken as well when address-size override prefix is used. In this case, as I see it, the existing masking would cause the carry from the add operation to the lower half of the rip not to be added to the rip higher half.

I guess another patch is needed for that as well.

This series of patches fix various scenarios in which KVM behavior does not follow x86 specifications. Each patch actually deals with a separate bug. These bugs can cause the guest to get stuck (i.e., make no progress), encounter spurious injected exceptions, or cause guest code to misbehave. As a result guest OS can potentially fail.

Thanks for reviewing the patches.

Nadav Amit (5):
KVM: x86: Fix wrong/stuck PMU when guest does not use PMI
KVM: x86: Fix CR3 reserved bits
KVM: x86: IN instruction emulation should ignore REP-prefix
KVM: x86: RSI/RDI/RCX are zero-extended when affected by string ops
KVM: x86: Processor mode may be determined incorrectly