Skip to contentSkip to conversation

x86: Memory barrier before setting ICR

PATCH In review Nadav Amit 2 participants 4 messages Jun 25, 2019 – Jun 27, 2019 KVM

lore
Share & tools
lore

Nadav Amit

author
#0 lore raw

PATCH 1/1 x86: Memory barrier before setting ICR

The wrmsr that is used in x2apic ICR programming does not behave as a memory barrier. There is a hidden assumption that it is. Add an explicit memory barrier for this reason.

---
 lib/x86/apic.c | 2 ++
 1 file changed, 2 insertions(+)
lib/x86/apic.c +2 -0
index bc2706e..1514730 100644
@@ -2,6 +2,7 @@
 #include "apic.h"
 #include "msr.h"
 #include "processor.h"
+#include "asm/barrier.h"
 
 void *g_apic = (void *)0xfee00000;
 void *g_ioapic = (void *)0xfec00000;
@@ -71,6 +72,7 @@ static void x2apic_write(unsigned reg, u32 val)
 
 static void x2apic_icr_write(u32 val, u32 dest)
 {
+    mb();
     asm volatile ("wrmsr" : : "a"(val), "d"(dest),
                   "c"(APIC_BASE_MSR + APIC_ICR/16));
 }
Signature · 1 line
2.17.1
#1 lore raw
Quoted 22 lines · Nadav Amit

The wrmsr that is used in x2apic ICR programming does not behave as a memory barrier. There is a hidden assumption that it is. Add an explicit memory barrier for this reason.

---
  lib/x86/apic.c | 2 ++
  1 file changed, 2 insertions(+)
lib/x86/apic.c +1 -0
index bc2706e..1514730 100644
@@ -2,6 +2,7 @@
  #include "apic.h"
  #include "msr.h"
  #include "processor.h"
+#include "asm/barrier.h"
 
  void *g_apic = (void *)0xfee00000;
  void *g_ioapic = (void *)0xfec00000;

lib/x86/apic.c static void x2apic_write(unsigned reg, u32 val) · line 78

Quoted 7 lines
lib/x86/apic.c +1 -0
@@ -71,6 +72,7 @@ static void x2apic_write(unsigned reg, u32 val)
 
  static void x2apic_icr_write(u32 val, u32 dest)
  {
+    mb();
      asm volatile ("wrmsr" : : "a"(val), "d"(dest),
                    "c"(APIC_BASE_MSR + APIC_ICR/16));
  }

Regarding non-serializing forms, the SDM mentions,

        "X2APIC MSRs (MSR indices 802H to 83FH)"

(APIC_BASE_MSR + APIC_ICR/16) is a different value. So I am wondering why we need a barrier here.

author
#2 lore raw
Quoted 35 lines · Krish Sadhukhan
Quoted 21 lines · Nadav Amit

The wrmsr that is used in x2apic ICR programming does not behave as a memory barrier. There is a hidden assumption that it is. Add an explicit memory barrier for this reason.

---
 lib/x86/apic.c | 2 ++
 1 file changed, 2 insertions(+)
lib/x86/apic.c +1 -0
index bc2706e..1514730 100644
@@ -2,6 +2,7 @@
 #include "apic.h"
 #include "msr.h"
 #include "processor.h"
+#include "asm/barrier.h"
   void *g_apic = (void *)0xfee00000;
 void *g_ioapic = (void *)0xfec00000;

lib/x86/apic.c static void x2apic_write(unsigned reg, u32 val) · line 77

Quoted 6 lines
lib/x86/apic.c +1 -0
@@ -71,6 +72,7 @@ static void x2apic_write(unsigned reg, u32 val)
   static void x2apic_icr_write(u32 val, u32 dest)
 {
+    mb();
     asm volatile ("wrmsr" : : "a"(val), "d"(dest),
                   "c"(APIC_BASE_MSR + APIC_ICR/16));
 }

Regarding non-serializing forms, the SDM mentions,

Quoted 4 lines

"X2APIC MSRs (MSR indices 802H to 83FH)"

(APIC_BASE_MSR + APIC_ICR/16) is a different value. So I am wondering why we need a barrier here.

What am I missing here?

APIC_BASE_MSR = 0x800
APIC_ICR = 0x300

So 0x800 + (0x300 / 16) = 0x830 , and 0x802 <= 0x830 <= 0x83f

Hence, writes to APIC_ICR do not seem to be serializing. No?

Reviewed-by
#3 lore raw
Quoted 38 lines · Nadav Amit
Quoted 34 lines · Krish Sadhukhan
Quoted 21 lines · Nadav Amit

The wrmsr that is used in x2apic ICR programming does not behave as a memory barrier. There is a hidden assumption that it is. Add an explicit memory barrier for this reason.

---
  lib/x86/apic.c | 2 ++
  1 file changed, 2 insertions(+)
lib/x86/apic.c +1 -0
index bc2706e..1514730 100644
@@ -2,6 +2,7 @@
  #include "apic.h"
  #include "msr.h"
  #include "processor.h"
+#include "asm/barrier.h"
    void *g_apic = (void *)0xfee00000;
  void *g_ioapic = (void *)0xfec00000;

lib/x86/apic.c static void x2apic_write(unsigned reg, u32 val) · line 77

Quoted 6 lines
lib/x86/apic.c +1 -0
@@ -71,6 +72,7 @@ static void x2apic_write(unsigned reg, u32 val)
    static void x2apic_icr_write(u32 val, u32 dest)
  {
+    mb();
      asm volatile ("wrmsr" : : "a"(val), "d"(dest),
                    "c"(APIC_BASE_MSR + APIC_ICR/16));
  }

Regarding non-serializing forms, the SDM mentions,

Quoted 4 lines

"X2APIC MSRs (MSR indices 802H to 83FH)"

(APIC_BASE_MSR + APIC_ICR/16) is a different value. So I am wondering why we need a barrier here.

Quoted 8 lines

What am I missing here?

APIC_BASE_MSR = 0x800
APIC_ICR = 0x300

So 0x800 + (0x300 / 16) = 0x830 , and 0x802 <= 0x830 <= 0x83f

Hence, writes to APIC_ICR do not seem to be serializing. No?

Sorry, my bad !  I missed the "to" in the quote from the SDM and thought only two MSRs were non-serializing.