Skip to contentSkip to conversation

MTRR support on x86, part 1

PATCH In review Carl-Daniel Hailfinger 3 participants 8 messages Dec 4, 2008 – Jan 26, 2009

lore
Share & tools
lore

Carl-Daniel Hailfinger

author
#0 lore raw

PATCH 1/1 MTRR support on x86, part 1

The current codebase ignores MTRR (Memory Type Range Register) configuration writes and reads because Qemu does not implement caching. All BIOS/firmware in know of for x86 do implement a mode called Cache-as-RAM (CAR) which locks down the CPU cache lines and uses the CPU cache like RAM before RAM is enabled. Qemu assumes RAM is accessible from the start, but it would be nice to be able to run real BIOS/firmware in Qemu. For that, we need CAR support and for CAR support we have to support MTRRs.

This patch is a first step in that direction. MTRRs are MSRs supported by all recent x86 CPUs, even old i586. Besides influencing cache, the MTRRs can be written and read back, so discarding MTRR writes violates the expectations of existing code out there.
Handle common x86 MTRR reads and writes, but don't act on them.

One open question remains: Is CPUX86State initialized with zeros or do I have to zero the MTRR settings stored there explicitly?

Signed-off-by Carl-Daniel Hailfinger <[email protected]>
cpu.h +25 -0
Index: target-i386/cpu.h
===================================================================
@@ -261,8 +261,25 @@
 
 #define MSR_IA32_PERF_STATUS            0x198
 
+#define MSR_MTRRphysBase(reg)		(0x200 + 2 * (reg))
+#define MSR_MTRRphysMask(reg)		(0x200 + 2 * (reg) + 1)
+
+#define MSR_MTRRfix64K_00000		0x250
+#define MSR_MTRRfix16K_80000		0x258
+#define MSR_MTRRfix16K_A0000		0x259
+#define MSR_MTRRfix4K_C0000		0x268
+#define MSR_MTRRfix4K_C8000		0x269
+#define MSR_MTRRfix4K_D0000		0x26a
+#define MSR_MTRRfix4K_D8000		0x26b
+#define MSR_MTRRfix4K_E0000		0x26c
+#define MSR_MTRRfix4K_E8000		0x26d
+#define MSR_MTRRfix4K_F0000		0x26e
+#define MSR_MTRRfix4K_F8000		0x26f
+
 #define MSR_PAT                         0x277
 
+#define MSR_MTRRdefType			0x2ff
+
 #define MSR_EFER                        0xc0000080
 
 #define MSR_EFER_SCE   (1 << 0)

@@ -629,6 +646,14 @@
     uint32_t cpuid_ext3_features;
     uint32_t cpuid_apic_id;
 
+    /* MTRRs */
+    uint64_t mtrr_fixed[11];
+    uint64_t mtrr_deftype;
+    struct {
+        uint64_t base;
+        uint64_t mask;
+    } mtrr_var[8];
+
 #ifdef USE_KQEMU
     int kqemu_enabled;
     int last_io_time;
op_helper.c +80 -0

92 lines not on this page · raw message

Signature · 1 line
http://www.hailfinger.org/
#1 lore raw
Quoted 33 lines · Carl-Daniel Hailfinger

The current codebase ignores MTRR (Memory Type Range Register) configuration writes and reads because Qemu does not implement caching. All BIOS/firmware in know of for x86 do implement a mode called Cache-as-RAM (CAR) which locks down the CPU cache lines and uses the CPU cache like RAM before RAM is enabled. Qemu assumes RAM is accessible from the start, but it would be nice to be able to run real BIOS/firmware in Qemu. For that, we need CAR support and for CAR support we have to support MTRRs.

This patch is a first step in that direction. MTRRs are MSRs supported by all recent x86 CPUs, even old i586. Besides influencing cache, the MTRRs can be written and read back, so discarding MTRR writes violates the expectations of existing code out there.
Handle common x86 MTRR reads and writes, but don't act on them.

One open question remains: Is CPUX86State initialized with zeros or do I have to zero the MTRR settings stored there explicitly?

Signed-off-by Carl-Daniel Hailfinger <[email protected]>

Index: target-i386/cpu.h
===================================================================

cpu.h +5 -0
@@ -261,8 +261,25 @@
 
 #define MSR_IA32_PERF_STATUS            0x198
 
+#define MSR_MTRRphysBase(reg)		(0x200 + 2 * (reg))
+#define MSR_MTRRphysMask(reg)		(0x200 + 2 * (reg) + 1)
+
+#define MSR_MTRRfix64K_00000		0x250
+#define MSR_MTRRfix16K_80000		0x258

cpu.h line 278

Quoted 10 lines
cpu.h +9 -0
+#define MSR_MTRRfix16K_A0000		0x259
+#define MSR_MTRRfix4K_C0000		0x268
+#define MSR_MTRRfix4K_C8000		0x269
+#define MSR_MTRRfix4K_D0000		0x26a
+#define MSR_MTRRfix4K_D8000		0x26b
+#define MSR_MTRRfix4K_E0000		0x26c
+#define MSR_MTRRfix4K_E8000		0x26d
+#define MSR_MTRRfix4K_F0000		0x26e
+#define MSR_MTRRfix4K_F8000		0x26f

I'm not a huge fan of the naming convention here.

Quoted 9 lines
cpu.h +2 -0
 #define MSR_PAT                         0x277
 
+#define MSR_MTRRdefType			0x2ff
+
 #define MSR_EFER                        0xc0000080
 
 #define MSR_EFER_SCE   (1 << 0)

@@ -629,6 +646,14 @@
     uint32_t cpuid_ext3_features;

cpu.h line 656

Quoted 10 lines
cpu.h +7 -0
     uint32_t cpuid_apic_id;
 
+    /* MTRRs */
+    uint64_t mtrr_fixed[11];
+    uint64_t mtrr_deftype;
+    struct {
+        uint64_t base;
+        uint64_t mask;
+    } mtrr_var[8];

These have to be saved/restored or else you'll potentially break live migration/savevm/loadvm.

Regards,

Anthony Liguori

author
#2 lore raw
Quoted 39 lines · Anthony Liguori
Quoted 38 lines · Carl-Daniel Hailfinger

The current codebase ignores MTRR (Memory Type Range Register) configuration writes and reads because Qemu does not implement caching. All BIOS/firmware in know of for x86 do implement a mode called Cache-as-RAM (CAR) which locks down the CPU cache lines and uses the CPU cache like RAM before RAM is enabled. Qemu assumes RAM is accessible from the start, but it would be nice to be able to run real BIOS/firmware in Qemu. For that, we need CAR support and for CAR support we have to support MTRRs.

This patch is a first step in that direction. MTRRs are MSRs supported by all recent x86 CPUs, even old i586. Besides influencing cache, the MTRRs can be written and read back, so discarding MTRR writes violates the expectations of existing code out there.
Handle common x86 MTRR reads and writes, but don't act on them.

One open question remains: Is CPUX86State initialized with zeros or do I have to zero the MTRR settings stored there explicitly?

Signed-off-by: Carl-Daniel Hailfinger
<[email protected]>

Index: target-i386/cpu.h
===================================================================

cpu.h (working copy) renamed from cpu.h (revision 5879) +9 -0
@@ -261,8 +261,25 @@
 
 #define MSR_IA32_PERF_STATUS            0x198
 
+#define MSR_MTRRphysBase(reg)        (0x200 + 2 * (reg))
+#define MSR_MTRRphysMask(reg)        (0x200 + 2 * (reg) + 1)
+
+#define MSR_MTRRfix64K_00000        0x250
+#define MSR_MTRRfix16K_80000        0x258
+#define MSR_MTRRfix16K_A0000        0x259
+#define MSR_MTRRfix4K_C0000        0x268
+#define MSR_MTRRfix4K_C8000        0x269
+#define MSR_MTRRfix4K_D0000        0x26a
Quoted 8 lines

cpu.h (working copy) line 278

Quoted 6 lines
cpu.h (working copy) renamed from cpu.h (revision 5879) +5 -0
+#define MSR_MTRRfix4K_D8000        0x26b
+#define MSR_MTRRfix4K_E0000        0x26c
+#define MSR_MTRRfix4K_E8000        0x26d
+#define MSR_MTRRfix4K_F0000        0x26e
+#define MSR_MTRRfix4K_F8000        0x26f

I'm not a huge fan of the naming convention here.

Except the MSR_ prefix, this is the name the MTRRs have in the AMD data sheets. I'm open to alternatives, though. If you have a suggestion, I'll implement it.

Quoted 11 lines
Quoted 11 lines
cpu.h (working copy) renamed from cpu.h (revision 5879) +2 -0
 #define MSR_PAT                         0x277
 
+#define MSR_MTRRdefType            0x2ff
+
 #define MSR_EFER                        0xc0000080
 
 #define MSR_EFER_SCE   (1 << 0)

@@ -629,6 +646,14 @@
     uint32_t cpuid_ext3_features;
     uint32_t cpuid_apic_id;
Quoted 11 lines

cpu.h (working copy) line 656

Quoted 8 lines
cpu.h (working copy) renamed from cpu.h (revision 5879) +7 -0
+    /* MTRRs */
+    uint64_t mtrr_fixed[11];
+    uint64_t mtrr_deftype;
+    struct {
+        uint64_t base;
+        uint64_t mask;
+    } mtrr_var[8];

These have to be saved/restored or else you'll potentially break live migration/savevm/loadvm.

Thanks, I was unaware of that. I'll read up on savevm/loadvm and post a new patch.

Thanks for the review!

Regards,
Carl-Daniel

Signature · 1 line
http://www.hailfinger.org/
author
#3 lore raw
In reply to Anthony Liguori
Quoted 36 lines · Anthony Liguori
Quoted 35 lines · Carl-Daniel Hailfinger

The current codebase ignores MTRR (Memory Type Range Register) configuration writes and reads because Qemu does not implement caching. All BIOS/firmware in know of for x86 do implement a mode called Cache-as-RAM (CAR) which locks down the CPU cache lines and uses the CPU cache like RAM before RAM is enabled. Qemu assumes RAM is accessible from the start, but it would be nice to be able to run real BIOS/firmware in Qemu. For that, we need CAR support and for CAR support we have to support MTRRs.

This patch is a first step in that direction. MTRRs are MSRs supported by all recent x86 CPUs, even old i586. Besides influencing cache, the MTRRs can be written and read back, so discarding MTRR writes violates the expectations of existing code out there.
Handle common x86 MTRR reads and writes, but don't act on them.

Signed-off-by: Carl-Daniel Hailfinger
<[email protected]>

Index: target-i386/cpu.h
===================================================================

cpu.h (working copy) renamed from cpu.h (revision 5879) +9 -0
@@ -261,8 +261,25 @@
 
 #define MSR_IA32_PERF_STATUS            0x198
 
+#define MSR_MTRRphysBase(reg)        (0x200 + 2 * (reg))
+#define MSR_MTRRphysMask(reg)        (0x200 + 2 * (reg) + 1)
+
+#define MSR_MTRRfix64K_00000        0x250
+#define MSR_MTRRfix16K_80000        0x258
+#define MSR_MTRRfix16K_A0000        0x259
+#define MSR_MTRRfix4K_C0000        0x268
+#define MSR_MTRRfix4K_C8000        0x269
+#define MSR_MTRRfix4K_D0000        0x26a
Quoted 8 lines

cpu.h (working copy) line 278

Quoted 6 lines
cpu.h (working copy) renamed from cpu.h (revision 5879) +5 -0
+#define MSR_MTRRfix4K_D8000        0x26b
+#define MSR_MTRRfix4K_E0000        0x26c
+#define MSR_MTRRfix4K_E8000        0x26d
+#define MSR_MTRRfix4K_F0000        0x26e
+#define MSR_MTRRfix4K_F8000        0x26f

I'm not a huge fan of the naming convention here.

The offer to use another naming scheme still stands. If you have any suggestion, I'll act upon it.

Quoted 11 lines
Quoted 11 lines
cpu.h (working copy) renamed from cpu.h (revision 5879) +2 -0
 #define MSR_PAT                         0x277
 
+#define MSR_MTRRdefType            0x2ff
+
 #define MSR_EFER                        0xc0000080
 
 #define MSR_EFER_SCE   (1 << 0)

@@ -629,6 +646,14 @@
     uint32_t cpuid_ext3_features;
     uint32_t cpuid_apic_id;
Quoted 11 lines

cpu.h (working copy) line 656

Quoted 8 lines
cpu.h (working copy) renamed from cpu.h (revision 5879) +7 -0
+    /* MTRRs */
+    uint64_t mtrr_fixed[11];
+    uint64_t mtrr_deftype;
+    struct {
+        uint64_t base;
+        uint64_t mask;
+    } mtrr_var[8];

These have to be saved/restored or else you'll potentially break live migration/savevm/loadvm.

I added save/restore support.
Thanks a lot for giving me the hints I needed to implement this.

Regards,
Carl-Daniel

The current codebase ignores MTRR (Memory Type Range Register) configuration writes and reads because Qemu does not implement caching. All BIOS/firmware in know of for x86 do implement a mode called Cache-as-RAM (CAR) which locks down the CPU cache lines and uses the CPU cache like RAM before RAM is enabled. Qemu assumes RAM is accessible from the start, but it would be nice to be able to run real BIOS/firmware in Qemu. For that, we need CAR support and for CAR support we have to support MTRRs.

This patch is a first step in that direction. MTRRs are MSRs supported by all recent x86 CPUs, even old i586. Besides influencing cache, the MTRRs can be written and read back, so discarding MTRR writes violates the expectations of existing code out there.
Handle common x86 MTRR reads and writes, but don't act on them.

Signed-off-by Carl-Daniel Hailfinger <[email protected]>
machine.c +22 -1
Index: target-i386/machine.c
===================================================================
@@ -134,6 +134,15 @@
     qemu_put_be16s(f, &env->intercept_dr_write);
     qemu_put_be32s(f, &env->intercept_exceptions);
     qemu_put_8s(f, &env->v_tpr);
+
+    /* MTRRs */
+    for(i = 0; i < 11; i++)
+        qemu_put_be64s(f, &env->mtrr_fixed[i]);
+    qemu_put_be64s(f, &env->mtrr_deftype);
+    for(i = 0; i < 8; i++) {
+        qemu_put_be64s(f, &env->mtrr_var[i].base);
+        qemu_put_be64s(f, &env->mtrr_var[i].mask);
+    }
 }
 
 #ifdef USE_X86LDOUBLE

@@ -169,7 +178,7 @@
     int32_t a20_mask;
 
     if (version_id != 3 && version_id != 4 && version_id != 5
-        && version_id != 6 && version_id != 7)
+        && version_id != 6 && version_id != 7 && version_id != 8)
         return -EINVAL;
     for(i = 0; i < CPU_NB_REGS; i++)
         qemu_get_betls(f, &env->regs[i]);

@@ -302,6 +311,18 @@
         qemu_get_be32s(f, &env->intercept_exceptions);
         qemu_get_8s(f, &env->v_tpr);
     }
+
+    if (version_id >= 8) {
+        /* MTRRs */
+        for(i = 0; i < 11; i++)
+            qemu_put_be64s(f, &env->mtrr_fixed[i]);
+        qemu_put_be64s(f, &env->mtrr_deftype);
+        for(i = 0; i < 8; i++) {
+            qemu_put_be64s(f, &env->mtrr_var[i].base);
+            qemu_put_be64s(f, &env->mtrr_var[i].mask);
+        }
+    }
+
     /* XXX: ensure compatiblity for halted bit ? */
     /* XXX: compute redundant hflags bits */
     env->hflags = hflags;
cpu.h +26 -1
Index: target-i386/cpu.h
===================================================================
@@ -261,8 +261,25 @@
 
 #define MSR_IA32_PERF_STATUS            0x198
 
+#define MSR_MTRRphysBase(reg)		(0x200 + 2 * (reg))
+#define MSR_MTRRphysMask(reg)		(0x200 + 2 * (reg) + 1)
+
+#define MSR_MTRRfix64K_00000		0x250
+#define MSR_MTRRfix16K_80000		0x258
+#define MSR_MTRRfix16K_A0000		0x259
+#define MSR_MTRRfix4K_C0000		0x268
+#define MSR_MTRRfix4K_C8000		0x269
+#define MSR_MTRRfix4K_D0000		0x26a
+#define MSR_MTRRfix4K_D8000		0x26b
+#define MSR_MTRRfix4K_E0000		0x26c
+#define MSR_MTRRfix4K_E8000		0x26d
+#define MSR_MTRRfix4K_F0000		0x26e
+#define MSR_MTRRfix4K_F8000		0x26f
+
 #define MSR_PAT                         0x277
 
+#define MSR_MTRRdefType			0x2ff
+
 #define MSR_EFER                        0xc0000080
 
 #define MSR_EFER_SCE   (1 << 0)

@@ -629,6 +646,14 @@
     uint32_t cpuid_ext3_features;
     uint32_t cpuid_apic_id;
 
+    /* MTRRs */
+    uint64_t mtrr_fixed[11];
+    uint64_t mtrr_deftype;
+    struct {
+        uint64_t base;
+        uint64_t mask;
+    } mtrr_var[8];
+
 #ifdef USE_KQEMU
     int kqemu_enabled;
     int last_io_time;

@@ -776,7 +801,7 @@
 #define cpu_signal_handler cpu_x86_signal_handler
 #define cpu_list x86_cpu_list
 
-#define CPU_SAVE_VERSION 7
+#define CPU_SAVE_VERSION 8
 
 /* MMU modes definitions */
 #define MMU_MODE0_SUFFIX _kernel
op_helper.c +80 -0

92 lines not on this page · raw message

author
#4 lore raw
Quoted 78 lines · Carl-Daniel Hailfinger
Quoted 37 lines · Anthony Liguori
Quoted 35 lines · Carl-Daniel Hailfinger

The current codebase ignores MTRR (Memory Type Range Register) configuration writes and reads because Qemu does not implement caching. All BIOS/firmware in know of for x86 do implement a mode called Cache-as-RAM (CAR) which locks down the CPU cache lines and uses the CPU cache like RAM before RAM is enabled. Qemu assumes RAM is accessible from the start, but it would be nice to be able to run real BIOS/firmware in Qemu. For that, we need CAR support and for CAR support we have to support MTRRs.

This patch is a first step in that direction. MTRRs are MSRs supported by all recent x86 CPUs, even old i586. Besides influencing cache, the MTRRs can be written and read back, so discarding MTRR writes violates the expectations of existing code out there.
Handle common x86 MTRR reads and writes, but don't act on them.

Signed-off-by: Carl-Daniel Hailfinger
<[email protected]>

Index: target-i386/cpu.h
===================================================================

cpu.h (working copy) renamed from cpu.h (revision 5879) +9 -0
@@ -261,8 +261,25 @@
 
 #define MSR_IA32_PERF_STATUS            0x198
 
+#define MSR_MTRRphysBase(reg)        (0x200 + 2 * (reg))
+#define MSR_MTRRphysMask(reg)        (0x200 + 2 * (reg) + 1)
+
+#define MSR_MTRRfix64K_00000        0x250
+#define MSR_MTRRfix16K_80000        0x258
+#define MSR_MTRRfix16K_A0000        0x259
+#define MSR_MTRRfix4K_C0000        0x268
+#define MSR_MTRRfix4K_C8000        0x269
+#define MSR_MTRRfix4K_D0000        0x26a
Quoted 9 lines

cpu.h (working copy) line 279

Quoted 7 lines
cpu.h (working copy) renamed from cpu.h (revision 5879) +5 -0
+#define MSR_MTRRfix4K_D8000        0x26b
+#define MSR_MTRRfix4K_E0000        0x26c
+#define MSR_MTRRfix4K_E8000        0x26d
+#define MSR_MTRRfix4K_F0000        0x26e
+#define MSR_MTRRfix4K_F8000        0x26f

I'm not a huge fan of the naming convention here.

The offer to use another naming scheme still stands. If you have any suggestion, I'll act upon it.

Quoted 11 lines
Quoted 11 lines
cpu.h (working copy) renamed from cpu.h (revision 5879) +2 -0
 #define MSR_PAT                         0x277
 
+#define MSR_MTRRdefType            0x2ff
+
 #define MSR_EFER                        0xc0000080
 
 #define MSR_EFER_SCE   (1 << 0)

@@ -629,6 +646,14 @@
     uint32_t cpuid_ext3_features;
     uint32_t cpuid_apic_id;
Quoted 12 lines

cpu.h (working copy) line 657

Quoted 9 lines
cpu.h (working copy) renamed from cpu.h (revision 5879) +7 -0
+    /* MTRRs */
+    uint64_t mtrr_fixed[11];
+    uint64_t mtrr_deftype;
+    struct {
+        uint64_t base;
+        uint64_t mask;
+    } mtrr_var[8];

These have to be saved/restored or else you'll potentially break live migration/savevm/loadvm.

Quoted 3 lines

I added save/restore support.
Thanks a lot for giving me the hints I needed to implement this.

Sorry, I submitted a previous buggy version. New version follows.

The current codebase ignores MTRR (Memory Type Range Register) configuration writes and reads because Qemu does not implement caching. All BIOS/firmware in know of for x86 do implement a mode called Cache-as-RAM (CAR) which locks down the CPU cache lines and uses the CPU cache like RAM before RAM is enabled. Qemu assumes RAM is accessible from the start, but it would be nice to be able to run real BIOS/firmware in Qemu. For that, we need CAR support and for CAR support we have to support MTRRs.

This patch is a first step in that direction. MTRRs are MSRs supported by all recent x86 CPUs, even old i586. Besides influencing cache, the MTRRs can be written and read back, so discarding MTRR writes violates the expectations of existing code out there.
Handle common x86 MTRR reads and writes, but don't act on them.

Signed-off-by Carl-Daniel Hailfinger <[email protected]>
machine.c +22 -1
Index: target-i386/machine.c
===================================================================
@@ -134,6 +134,15 @@
     qemu_put_be16s(f, &env->intercept_dr_write);
     qemu_put_be32s(f, &env->intercept_exceptions);
     qemu_put_8s(f, &env->v_tpr);
+
+    /* MTRRs */
+    for(i = 0; i < 11; i++)
+        qemu_put_be64s(f, &env->mtrr_fixed[i]);
+    qemu_put_be64s(f, &env->mtrr_deftype);
+    for(i = 0; i < 8; i++) {
+        qemu_put_be64s(f, &env->mtrr_var[i].base);
+        qemu_put_be64s(f, &env->mtrr_var[i].mask);
+    }
 }
 
 #ifdef USE_X86LDOUBLE

@@ -169,7 +178,7 @@
     int32_t a20_mask;
 
     if (version_id != 3 && version_id != 4 && version_id != 5
-        && version_id != 6 && version_id != 7)
+        && version_id != 6 && version_id != 7 && version_id != 8)
         return -EINVAL;
     for(i = 0; i < CPU_NB_REGS; i++)
         qemu_get_betls(f, &env->regs[i]);

@@ -302,6 +311,18 @@
         qemu_get_be32s(f, &env->intercept_exceptions);
         qemu_get_8s(f, &env->v_tpr);
     }
+
+    if (version_id >= 8) {
+        /* MTRRs */
+        for(i = 0; i < 11; i++)
+            qemu_get_be64s(f, &env->mtrr_fixed[i]);
+        qemu_get_be64s(f, &env->mtrr_deftype);
+        for(i = 0; i < 8; i++) {
+            qemu_get_be64s(f, &env->mtrr_var[i].base);
+            qemu_get_be64s(f, &env->mtrr_var[i].mask);
+        }
+    }
+
     /* XXX: ensure compatiblity for halted bit ? */
     /* XXX: compute redundant hflags bits */
     env->hflags = hflags;
cpu.h +26 -1
Index: target-i386/cpu.h
===================================================================
@@ -261,8 +261,25 @@
 
 #define MSR_IA32_PERF_STATUS            0x198
 
+#define MSR_MTRRphysBase(reg)		(0x200 + 2 * (reg))
+#define MSR_MTRRphysMask(reg)		(0x200 + 2 * (reg) + 1)
+
+#define MSR_MTRRfix64K_00000		0x250
+#define MSR_MTRRfix16K_80000		0x258
+#define MSR_MTRRfix16K_A0000		0x259
+#define MSR_MTRRfix4K_C0000		0x268
+#define MSR_MTRRfix4K_C8000		0x269
+#define MSR_MTRRfix4K_D0000		0x26a
+#define MSR_MTRRfix4K_D8000		0x26b
+#define MSR_MTRRfix4K_E0000		0x26c
+#define MSR_MTRRfix4K_E8000		0x26d
+#define MSR_MTRRfix4K_F0000		0x26e
+#define MSR_MTRRfix4K_F8000		0x26f
+
 #define MSR_PAT                         0x277
 
+#define MSR_MTRRdefType			0x2ff
+
 #define MSR_EFER                        0xc0000080
 
 #define MSR_EFER_SCE   (1 << 0)

@@ -629,6 +646,14 @@
     uint32_t cpuid_ext3_features;
     uint32_t cpuid_apic_id;
 
+    /* MTRRs */
+    uint64_t mtrr_fixed[11];
+    uint64_t mtrr_deftype;
+    struct {
+        uint64_t base;
+        uint64_t mask;
+    } mtrr_var[8];
+
 #ifdef USE_KQEMU
     int kqemu_enabled;
     int last_io_time;

@@ -776,7 +801,7 @@
 #define cpu_signal_handler cpu_x86_signal_handler
 #define cpu_list x86_cpu_list
 
-#define CPU_SAVE_VERSION 7
+#define CPU_SAVE_VERSION 8
 
 /* MMU modes definitions */
 #define MMU_MODE0_SUFFIX _kernel
op_helper.c +80 -0

92 lines not on this page · raw message

author
#5 lore raw

PATCH 1/1 MTRR support on x86 [resend]

The current codebase ignores MTRR (Memory Type Range Register) configuration writes and reads because Qemu does not implement caching. All BIOS/firmware in know of for x86 do implement a mode called Cache-as-RAM (CAR) which locks down the CPU cache lines and uses the CPU cache like RAM before RAM is enabled. Qemu assumes RAM is accessible from the start, but it would be nice to be able to run real BIOS/firmware in Qemu. For that, we need CAR support and for CAR support we have to support MTRRs.

This patch is a first step in that direction. MTRRs are MSRs supported by all recent x86 CPUs, even old i586. Besides influencing cache, the MTRRs can be written and read back, so discarding MTRR writes violates the expectations of existing code out there.

An added benefit of this patch is that it fixes the following Linux kernel error message present in recent kernels (provided the BIOS has the recent MTRR patches applied):

------------[ cut here ]------------
WARNING: at arch/x86/kernel/cpu/mtrr/main.c:1500 mtrr_trim_uncached_memory+0x382/0x384()
WARNING: strange, CPU MTRRs all blank?
Modules linked in:
Supported: Yes
Pid: 0, comm: swapper Not tainted 2.6.27.7-9-default #1
 [<c0106570>] dump_trace+0x6b/0x249
 [<c01070a5>] show_trace+0x20/0x39
 [<c0343c02>] dump_stack+0x71/0x76
 [<c012acb2>] warn_slowpath+0x6f/0x90
 [<c0542f8f>] mtrr_trim_uncached_memory+0x382/0x384
 [<c053f24d>] setup_arch+0x40d/0x639
 [<c053a6ac>] start_kernel+0x6b/0x31f
 =======================
---[ end trace 4eaa2a86a8e2da22 ]---

Handle common x86 MTRR reads and writes, but don't act on them.

Signed-off-by Carl-Daniel Hailfinger <[email protected]>

Index: target-i386/machine.c
===================================================================

machine.c +22 -1
@@ -134,6 +134,15 @@
     qemu_put_be16s(f, &env->intercept_dr_write);
     qemu_put_be32s(f, &env->intercept_exceptions);
     qemu_put_8s(f, &env->v_tpr);
+
+    /* MTRRs */
+    for(i = 0; i < 11; i++)
+        qemu_put_be64s(f, &env->mtrr_fixed[i]);
+    qemu_put_be64s(f, &env->mtrr_deftype);
+    for(i = 0; i < 8; i++) {
+        qemu_put_be64s(f, &env->mtrr_var[i].base);
+        qemu_put_be64s(f, &env->mtrr_var[i].mask);
+    }
 }
 
 #ifdef USE_X86LDOUBLE

@@ -169,7 +178,7 @@
     int32_t a20_mask;
 
     if (version_id != 3 && version_id != 4 && version_id != 5
-        && version_id != 6 && version_id != 7)
+        && version_id != 6 && version_id != 7 && version_id != 8)
         return -EINVAL;
     for(i = 0; i < CPU_NB_REGS; i++)
         qemu_get_betls(f, &env->regs[i]);

@@ -302,6 +311,18 @@
         qemu_get_be32s(f, &env->intercept_exceptions);
         qemu_get_8s(f, &env->v_tpr);
     }
+
+    if (version_id >= 8) {
+        /* MTRRs */
+        for(i = 0; i < 11; i++)
+            qemu_get_be64s(f, &env->mtrr_fixed[i]);
+        qemu_get_be64s(f, &env->mtrr_deftype);
+        for(i = 0; i < 8; i++) {
+            qemu_get_be64s(f, &env->mtrr_var[i].base);
+            qemu_get_be64s(f, &env->mtrr_var[i].mask);
+        }
+    }
+
     /* XXX: ensure compatiblity for halted bit ? */
     /* XXX: compute redundant hflags bits */
     env->hflags = hflags;
cpu.h +26 -1
Index: target-i386/cpu.h
===================================================================
@@ -261,8 +261,25 @@
 
 #define MSR_IA32_PERF_STATUS            0x198
 
+#define MSR_MTRRphysBase(reg)		(0x200 + 2 * (reg))
+#define MSR_MTRRphysMask(reg)		(0x200 + 2 * (reg) + 1)
+
+#define MSR_MTRRfix64K_00000		0x250
+#define MSR_MTRRfix16K_80000		0x258
+#define MSR_MTRRfix16K_A0000		0x259
+#define MSR_MTRRfix4K_C0000		0x268
+#define MSR_MTRRfix4K_C8000		0x269
+#define MSR_MTRRfix4K_D0000		0x26a
+#define MSR_MTRRfix4K_D8000		0x26b
+#define MSR_MTRRfix4K_E0000		0x26c
+#define MSR_MTRRfix4K_E8000		0x26d
+#define MSR_MTRRfix4K_F0000		0x26e
+#define MSR_MTRRfix4K_F8000		0x26f
+
 #define MSR_PAT                         0x277
 
+#define MSR_MTRRdefType			0x2ff
+
 #define MSR_EFER                        0xc0000080
 
 #define MSR_EFER_SCE   (1 << 0)

@@ -629,6 +646,14 @@
     uint32_t cpuid_ext3_features;
     uint32_t cpuid_apic_id;
 
+    /* MTRRs */
+    uint64_t mtrr_fixed[11];
+    uint64_t mtrr_deftype;
+    struct {
+        uint64_t base;
+        uint64_t mask;
+    } mtrr_var[8];
+
 #ifdef USE_KQEMU
     int kqemu_enabled;
     int last_io_time;

@@ -805,7 +830,7 @@
 #define cpu_signal_handler cpu_x86_signal_handler
 #define cpu_list x86_cpu_list
 
-#define CPU_SAVE_VERSION 7
+#define CPU_SAVE_VERSION 8
 
 /* MMU modes definitions */
 #define MMU_MODE0_SUFFIX _kernel
op_helper.c +80 -0

92 lines not on this page · raw message

Signature · 1 line
http://www.hailfinger.org/
author
#6 lore raw

Re: [Qemu-devel] [PATCH] MTRR support on x86 [resend]

Quoted 33 lines · Carl-Daniel Hailfinger

The current codebase ignores MTRR (Memory Type Range Register) configuration writes and reads because Qemu does not implement caching. All BIOS/firmware in know of for x86 do implement a mode called Cache-as-RAM (CAR) which locks down the CPU cache lines and uses the CPU cache like RAM before RAM is enabled. Qemu assumes RAM is accessible from the start, but it would be nice to be able to run real BIOS/firmware in Qemu. For that, we need CAR support and for CAR support we have to support MTRRs.

This patch is a first step in that direction. MTRRs are MSRs supported by all recent x86 CPUs, even old i586. Besides influencing cache, the MTRRs can be written and read back, so discarding MTRR writes violates the expectations of existing code out there.

An added benefit of this patch is that it fixes the following Linux kernel error message present in recent kernels (provided the BIOS has the recent MTRR patches applied):

------------[ cut here ]------------
WARNING: at arch/x86/kernel/cpu/mtrr/main.c:1500 mtrr_trim_uncached_memory+0x382/0x384()
WARNING: strange, CPU MTRRs all blank?
Modules linked in:
Supported: Yes
Pid: 0, comm: swapper Not tainted 2.6.27.7-9-default #1
 [<c0106570>] dump_trace+0x6b/0x249
 [<c01070a5>] show_trace+0x20/0x39
 [<c0343c02>] dump_stack+0x71/0x76
 [<c012acb2>] warn_slowpath+0x6f/0x90
 [<c0542f8f>] mtrr_trim_uncached_memory+0x382/0x384
 [<c053f24d>] setup_arch+0x40d/0x639
 [<c053a6ac>] start_kernel+0x6b/0x31f
 =======================
---[ end trace 4eaa2a86a8e2da22 ]---
Quoted 4 lines

Handle common x86 MTRR reads and writes, but don't act on them.

Signed-off-by Carl-Daniel Hailfinger <[email protected]>

Just a short heads-up: Since one of the BIOS MTRR support patches is a bit incomplete, this patch is not enough to silence that Linux warning. I'll post a new patch with improved compatibility tomorrow.

Regards,
Carl-Daniel

Signature · 1 line
http://www.hailfinger.org/
Applied
#7 lore raw

Re: [Qemu-devel] [PATCH] MTRR support on x86 [resend]

In reply to Carl-Daniel Hailfinger
Quoted 33 lines · Carl-Daniel Hailfinger

The current codebase ignores MTRR (Memory Type Range Register) configuration writes and reads because Qemu does not implement caching. All BIOS/firmware in know of for x86 do implement a mode called Cache-as-RAM (CAR) which locks down the CPU cache lines and uses the CPU cache like RAM before RAM is enabled. Qemu assumes RAM is accessible from the start, but it would be nice to be able to run real BIOS/firmware in Qemu. For that, we need CAR support and for CAR support we have to support MTRRs.

This patch is a first step in that direction. MTRRs are MSRs supported by all recent x86 CPUs, even old i586. Besides influencing cache, the MTRRs can be written and read back, so discarding MTRR writes violates the expectations of existing code out there.

An added benefit of this patch is that it fixes the following Linux kernel error message present in recent kernels (provided the BIOS has the recent MTRR patches applied):

------------[ cut here ]------------
WARNING: at arch/x86/kernel/cpu/mtrr/main.c:1500 mtrr_trim_uncached_memory+0x382/0x384()
WARNING: strange, CPU MTRRs all blank?
Modules linked in:
Supported: Yes
Pid: 0, comm: swapper Not tainted 2.6.27.7-9-default #1
 [<c0106570>] dump_trace+0x6b/0x249
 [<c01070a5>] show_trace+0x20/0x39
 [<c0343c02>] dump_stack+0x71/0x76
 [<c012acb2>] warn_slowpath+0x6f/0x90
 [<c0542f8f>] mtrr_trim_uncached_memory+0x382/0x384
 [<c053f24d>] setup_arch+0x40d/0x639
 [<c053a6ac>] start_kernel+0x6b/0x31f
 =======================
---[ end trace 4eaa2a86a8e2da22 ]---
Quoted 4 lines

Handle common x86 MTRR reads and writes, but don't act on them.

Signed-off-by Carl-Daniel Hailfinger <[email protected]>

Applied. Thanks.

Regards,

Anthony Liguori

Quoted 2 lines