Skip to contentSkip to conversation

KVM: IOAPIC: only access APIC registers one dword at a time

PATCH Superseded Xiao Guangrong 5 participants 12 messages Jul 2, 2010 – Jul 5, 2010 KVM

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 KVM: IOAPIC: only access APIC registers one dword at a time

The IOAPIC spec says:

When accessing these registers, accesses must be done one dword at a time. For example, software should never access byte 2 from the Data register before accessing bytes 0 and 1. The hardware will not attempt to recover from a bad programming model in this case.

So, this patch removes other width access

---
 virt/kvm/ioapic.c |   10 +++-------
 1 files changed, 3 insertions(+), 7 deletions(-)
virt/kvm/ioapic.c +3 -7
index 1149c60..f96816f 100644
@@ -306,14 +306,10 @@ static int ioapic_mmio_read(struct kvm_io_device *this, gpa_t addr, int len,
 	spin_unlock(&ioapic->lock);
 
 	switch (len) {
-	case 8:
-		*(u64 *) val = result;
-		break;
-	case 1:
-	case 2:
 	case 4:
-		memcpy(val, (char *)&result, len);
+		*(u32 *) val = result;
 		break;
+
 	default:
 		printk(KERN_WARNING "ioapic: wrong length %d\n", len);
 	}
@@ -332,7 +328,7 @@ static int ioapic_mmio_write(struct kvm_io_device *this, gpa_t addr, int len,
 		     (void*)addr, len, val);
 	ASSERT(!(addr & 0xf));	/* check alignment */
 
-	if (len == 4 || len == 8)
+	if (len == 4)
 		data = *(u32 *) val;
 	else {
 		printk(KERN_WARNING "ioapic: Unsupported size %d\n", len);
Signature · 1 line
1.6.1.2
#1 lore raw

Hi, Xiao Guangrong

Quoted 6 lines · Xiao Guangrong

The IOAPIC spec says:

When accessing these registers, accesses must be done one dword at a time. For example, software should never access byte 2 from the Data register before accessing bytes 0 and 1. The hardware will not attempt to recover from a bad programming model in this case.

I could understand what you described on the above, but I don't think it is the best method to fix it. Could you consider a nice one?

Quoted 13 lines

So, this patch removes other width access

---
 virt/kvm/ioapic.c |   10 +++-------
 1 files changed, 3 insertions(+), 7 deletions(-)
virt/kvm/ioapic.c +0 -0
index 1149c60..f96816f 100644

virt/kvm/ioapic.c static int ioapic_mmio_read(struct kvm_io_device *this, gpa_t addr, int len, · line 315

Quoted 10 lines
virt/kvm/ioapic.c +0 -6
@@ -306,14 +306,10 @@ static int ioapic_mmio_read(struct kvm_io_device *this, gpa_t addr, int len,
 	spin_unlock(&ioapic->lock);
 
 	switch (len) {
-	case 8:
-		*(u64 *) val = result;
-		break;
-	case 1:
-	case 2:
 	case 4:
-		memcpy(val, (char *)&result, len);

Here the parameter len is not used for reading data from ioapic register, before switch case the value of ioapic register has been read by ioapic_read_indirect().

virt/kvm/ioapic.c static int ioapic_mmio_read(struct kvm_io_device *this, gpa_t addr, int len, · line 314

Quoted 5 lines
virt/kvm/ioapic.c +2 -0
+		*(u32 *) val = result;
 		break;
+
 	default:
 		printk(KERN_WARNING "ioapic: wrong length %d\n", len);

And I think here is not good to print warning message. Maybe it is better to do such kind of checking at the first of this function before ioapic_read_indirect().

virt/kvm/ioapic.c static int ioapic_mmio_write(struct kvm_io_device *this, gpa_t addr, int len, · line 334

Quoted 10 lines
virt/kvm/ioapic.c +1 -1
 	}
@@ -332,7 +328,7 @@ static int ioapic_mmio_write(struct kvm_io_device *this, gpa_t addr, int len,
 		     (void*)addr, len, val);
 	ASSERT(!(addr & 0xf));	/* check alignment */
 
-	if (len == 4 || len == 8)
+	if (len == 4)
 		data = *(u32 *) val;
 	else {
 		printk(KERN_WARNING "ioapic: Unsupported size %d\n", len);

So I hope you could read the source code again. I think you can use a better method to fix it.

Best Regards,
Jin Dongming

#2 lore raw
Quoted 13 lines · Jin Dongming

virt/kvm/ioapic.c static int ioapic_mmio_read(struct kvm_io_device *this, gpa_t addr, int len, · line 315

Quoted 9 lines
virt/kvm/ioapic.c +0 -6
 	switch (len) {
-	case 8:
-		*(u64 *) val = result;
-		break;
-	case 1:
-	case 2:
 	case 4:
-		memcpy(val, (char *)&result, len);

Here the parameter len is not used for reading data from ioapic register, before switch case the value of ioapic register has been read by ioapic_read_indirect().

Yeah, it's right, but it's rarely operation that guest using incorrect width to access the registers, so i don't think it's worthy to move the width judgment before the real registers read.

Quoted 10 lines

virt/kvm/ioapic.c static int ioapic_mmio_read(struct kvm_io_device *this, gpa_t addr, int len, · line 314

Quoted 6 lines
virt/kvm/ioapic.c +2 -0
+		*(u32 *) val = result;
 		break;
+
 	default:
 		printk(KERN_WARNING "ioapic: wrong length %d\n", len);

And I think here is not good to print warning message. Maybe it is better to do such kind of checking at the first of this function before ioapic_read_indirect().

ditto

Thanks for your review, Jin!

#3 lore raw

PATCH v2 1/1 KVM: IOAPIC: only access APIC registers one dword at a time

The IOAPIC spec says:

When accessing these registers, accesses must be done one dword at a time. For example, software should never access byte 2 from the Data register before accessing bytes 0 and 1. The hardware will not attempt to recover from a bad programming model in this case.

So, this patch removes other width access

---
 virt/kvm/ioapic.c |   20 +++++++-------------
 1 files changed, 7 insertions(+), 13 deletions(-)
virt/kvm/ioapic.c +7 -13
index 1149c60..6610d11 100644
@@ -288,6 +288,11 @@ static int ioapic_mmio_read(struct kvm_io_device *this, gpa_t addr, int len,
 	ioapic_debug("addr %lx\n", (unsigned long)addr);
 	ASSERT(!(addr & 0xf));	/* check alignment */
 
+	if (len != 4) {
+		printk(KERN_WARNING "ioapic: wrong length %d\n", len);
+		return 0;
+	}
+
 	addr &= 0xff;
 	spin_lock(&ioapic->lock);
 	switch (addr) {
@@ -305,18 +310,7 @@ static int ioapic_mmio_read(struct kvm_io_device *this, gpa_t addr, int len,
 	}
 	spin_unlock(&ioapic->lock);
 
-	switch (len) {
-	case 8:
-		*(u64 *) val = result;
-		break;
-	case 1:
-	case 2:
-	case 4:
-		memcpy(val, (char *)&result, len);
-		break;
-	default:
-		printk(KERN_WARNING "ioapic: wrong length %d\n", len);
-	}
+	*(u32 *) val = result;
 	return 0;
 }
 
@@ -332,7 +326,7 @@ static int ioapic_mmio_write(struct kvm_io_device *this, gpa_t addr, int len,
 		     (void*)addr, len, val);
 	ASSERT(!(addr & 0xf));	/* check alignment */
 
-	if (len == 4 || len == 8)
+	if (len == 4)
 		data = *(u32 *) val;
 	else {
 		printk(KERN_WARNING "ioapic: Unsupported size %d\n", len);
Signature · 1 line
1.6.1.2
#4 lore raw
Quoted 19 lines · Xiao Guangrong

The IOAPIC spec says:

When accessing these registers, accesses must be done one dword at a time. For example, software should never access byte 2 from the Data register before accessing bytes 0 and 1. The hardware will not attempt to recover from a bad programming model in this case.

So, this patch removes other width access

---
 virt/kvm/ioapic.c |   20 +++++++-------------
 1 files changed, 7 insertions(+), 13 deletions(-)
virt/kvm/ioapic.c +0 -0
index 1149c60..6610d11 100644

virt/kvm/ioapic.c static int ioapic_mmio_read(struct kvm_io_device *this, gpa_t addr, int len, · line 295

Quoted 8 lines
virt/kvm/ioapic.c +5 -0
@@ -288,6 +288,11 @@ static int ioapic_mmio_read(struct kvm_io_device *this, gpa_t addr, int len,
 	ioapic_debug("addr %lx\n", (unsigned long)addr);
 	ASSERT(!(addr & 0xf));	/* check alignment */
 
+	if (len != 4) {
+		printk(KERN_WARNING "ioapic: wrong length %d\n", len);
+		return 0;
+	}
+

Just remove the printks please, as guests can flood hosts dmesg.

#5 lore raw
Quoted 8 lines · Marcelo Tosatti

Review

Quoted 5 lines
(quoted diff) +5 -0
+	if (len != 4) {
+		printk(KERN_WARNING "ioapic: wrong length %d\n", len);
+		return 0;
+	}
+

Just remove the printks please, as guests can flood hosts dmesg.

OK, please review this one, thanks, Marcelo!

Subject: [PATCH v3] KVM: IOAPIC: only access APIC registers one dword at a time

The IOAPIC spec says:
When accessing these registers, accesses must be done one dword at a time. For example, software should never access byte 2 from the Data register before accessing bytes 0 and 1. The hardware will not attempt to recover from a bad programming model in this case.

So, this patch removes other width access

---
 virt/kvm/ioapic.c |   22 ++++++----------------
 1 files changed, 6 insertions(+), 16 deletions(-)
virt/kvm/ioapic.c +6 -16
index 1149c60..aed91fd 100644
@@ -288,6 +288,9 @@ static int ioapic_mmio_read(struct kvm_io_device *this, gpa_t addr, int len,
 	ioapic_debug("addr %lx\n", (unsigned long)addr);
 	ASSERT(!(addr & 0xf));	/* check alignment */
 
+	if (len != 4)
+		return 0;
+
 	addr &= 0xff;
 	spin_lock(&ioapic->lock);
 	switch (addr) {
@@ -305,18 +308,7 @@ static int ioapic_mmio_read(struct kvm_io_device *this, gpa_t addr, int len,
 	}
 	spin_unlock(&ioapic->lock);
 
-	switch (len) {
-	case 8:
-		*(u64 *) val = result;
-		break;
-	case 1:
-	case 2:
-	case 4:
-		memcpy(val, (char *)&result, len);
-		break;
-	default:
-		printk(KERN_WARNING "ioapic: wrong length %d\n", len);
-	}
+	*(u32 *) val = result;
 	return 0;
 }
 
@@ -332,12 +324,10 @@ static int ioapic_mmio_write(struct kvm_io_device *this, gpa_t addr, int len,
 		     (void*)addr, len, val);
 	ASSERT(!(addr & 0xf));	/* check alignment */
 
-	if (len == 4 || len == 8)
+	if (len == 4)
 		data = *(u32 *) val;
-	else {
-		printk(KERN_WARNING "ioapic: Unsupported size %d\n", len);
+	else
 		return 0;
-	}
 
 	addr &= 0xff;
 	spin_lock(&ioapic->lock);
Signature · 1 line
1.6.1.2
#6 lore raw
In reply to Xiao Guangrong
Quoted 10 lines · Xiao Guangrong

The IOAPIC spec says:

When accessing these registers, accesses must be done one dword at a time. For example, software should never access byte 2 from the Data register before accessing bytes 0 and 1. The hardware will not attempt to recover from a bad programming model in this case.

So, this patch removes other width access

The ioapic code also implements the ia64 iosapic. I'm guessing that does support 64-bit accesses. Please check the iosapic documentation.

There might be guests that use incorrect access despite the documentation; if real hardware supports it, it should work. So we need to start with just a warning, and allow the access. Later we can drop the invalid access.

virt/kvm/ioapic.c static int ioapic_mmio_read(struct kvm_io_device *this, gpa_t addr, int len, · line 296

Quoted 10 lines
virt/kvm/ioapic.c +5 -0
@@ -288,6 +288,11 @@ static int ioapic_mmio_read(struct kvm_io_device *this, gpa_t addr, int len,
  	ioapic_debug("addr %lx\n", (unsigned long)addr);
  	ASSERT(!(addr&  0xf));	/* check alignment */
 
+	if (len != 4) {
+		printk(KERN_WARNING "ioapic: wrong length %d\n", len);
+		return 0;
+	}
+
 

Guest triggered, so needs to be rate limited.

Signature · 2 lines
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.
#7 lore raw
Quoted 11 lines · Avi Kivity
Quoted 10 lines · Xiao Guangrong

The IOAPIC spec says:

When accessing these registers, accesses must be done one dword at a time.
For example, software should never access byte 2 from the Data register before
accessing bytes 0 and 1. The hardware will not attempt to recover from a bad
programming model in this case.

Quoted 7 lines
Quoted 3 lines

So, this patch removes other width access

The ioapic code also implements the ia64 iosapic. I'm guessing that does support 64-bit accesses. Please check the iosapic documentation.

The iosapic also using 32-bit to access registers:

All registers are accessed using 32-bit uncacheable loads and stores to a reserved memory location in system memory. This implies that to modify a field (e.g., a bit or a byte) in any register, the whole 32-bit register must be read, the field modified, and the 32 bits written back. Partial register access, or non-aligned register access, are implementation-defined by the I/O xAPIC and will not be compatible across different implementations. Also, registers that are described as 64 bits wide are accessed as multiple independent 32-bit registers.

[ From << Intel® Itanium® Processor Family Interrupt Architecture Guide >>, P2-6 ]

Quoted 4 lines

There might be guests that use incorrect access despite the documentation; if real hardware supports it, it should work. So we need to start with just a warning, and allow the access. Later we can drop the invalid access.

If the OS contravene the spec, i thinks it's the OS's bug, also, i have tested some versions windows/linux guests, it's no broken, can we directly drop the other wide access?

Quoted 6 lines
Quoted 6 lines
virt/kvm/ioapic.c +0 -0
@@ -288,6 +288,11 @@ static int ioapic_mmio_read(struct kvm_io_device
 *this, gpa_t addr, int len,
      ioapic_debug("addr %lx\n", (unsigned long)addr);
      ASSERT(!(addr&  0xf));    /* check alignment */
 
Quoted 8 lines

virt/kvm/ioapic.c static int ioapic_mmio_read(struct kvm_io_device *this, gpa_t addr, int len,

Quoted 6 lines
virt/kvm/ioapic.c +5 -0
+    if (len != 4) {
+        printk(KERN_WARNING "ioapic: wrong length %d\n", len);
+        return 0;
+    }
+
 

Guest triggered, so needs to be rate limited.

Yeah, will using printk_ratelimit cooperate with it.

#8 lore raw
Quoted 8 lines · Xiao Guangrong
Quoted 4 lines

There might be guests that use incorrect access despite the documentation; if real hardware supports it, it should work. So we need to start with just a warning, and allow the access. Later we can drop the invalid access.

If the OS contravene the spec, i thinks it's the OS's bug, also, i have tested some versions windows/linux guests, it's no broken, can we directly drop the other wide access?

If reality contradicts spec we should emulate reality. I vaguely remember some guest using memcpy on apic page for instance.

--
Gleb.

#9 lore raw
In reply to Xiao Guangrong
Quoted 25 lines · Xiao Guangrong
Quoted 12 lines · Avi Kivity
Quoted 10 lines · Xiao Guangrong

The IOAPIC spec says:

When accessing these registers, accesses must be done one dword at a time.
For example, software should never access byte 2 from the Data register before
accessing bytes 0 and 1. The hardware will not attempt to recover from a bad
programming model in this case.

Quoted 8 lines
Quoted 4 lines

So, this patch removes other width access

The ioapic code also implements the ia64 iosapic. I'm guessing that does support 64-bit accesses. Please check the iosapic documentation.

The iosapic also using 32-bit to access registers:

Quoted 9 lines

All registers are accessed using 32-bit uncacheable loads and stores to a reserved memory location in system memory. This implies that to modify a field (e.g., a bit or a byte) in any register, the whole 32-bit register must be read, the field modified, and the 32 bits written back. Partial register access, or non-aligned register access, are implementation-defined by the I/O xAPIC and will not be compatible across different implementations. Also, registers that are described as 64 bits wide are accessed as multiple independent 32-bit registers.

[ From<< Intel® Itanium® Processor Family Interrupt Architecture Guide>>, P2-6 ]

Ok.

Quoted 8 lines
Quoted 5 lines

There might be guests that use incorrect access despite the documentation; if real hardware supports it, it should work. So we need to start with just a warning, and allow the access. Later we can drop the invalid access.

If the OS contravene the spec, i thinks it's the OS's bug, also, i have tested some versions windows/linux guests, it's no broken, can we directly drop the other wide access?

Well, there's the spec and there's real life, but in this case we can try and if we see a problem we'll re-add the other access length.

Signature · 1 line
error compiling committee.c: too many arguments to function
#10 lore raw
Quoted 6 lines · Avi Kivity
Quoted 6 lines
Quoted 6 lines

There might be guests that use incorrect access despite the documentation; if real hardware supports it, it should work. So we need to start with just a warning, and allow the access. Later we can drop the invalid access.

Quoted 9 lines
Quoted 5 lines

If the OS contravene the spec, i thinks it's the OS's bug, also, i have tested some versions
windows/linux guests, it's no broken, can we directly drop the other wide access?

Well, there's the spec and there's real life, but in this case we can try and if we see a problem we'll re-add the other access length.

OK, i'll do it as your suggestion, thanks.

#11 lore raw
Quoted 7 lines · Xiao Guangrong
Quoted 5 lines

Well, there's the spec and there's real life, but in this case we can try and if we see a problem we'll re-add the other access length.

OK, i'll do it as your suggestion, thanks.

I meant that you can drop access length != 4, same as in your original patch. Only change needed is printk rate limit.

Signature · 1 line
error compiling committee.c: too many arguments to function