Skip to contentSkip to conversation

x86: MCE: Re-implement MCE log ring buffer as per-CPU ring buffer

PATCH In review Huang Ying 4 participants 9 messages Jun 8, 2009 – Jun 11, 2009 x86

lore
Share & tools
lore

Huang Ying

author
#0 lore raw

PATCH 1/1 x86: MCE: Re-implement MCE log ring buffer as per-CPU ring buffer

Re-implement MCE log ring buffer as per-CPU ring buffer for better scalability.

AK: the main reason to add the patch is less that performance (normally there shouldn't be that many errors, so it doesn't matter too much), but that it scales to a larger number of CPUs. The old mce log ring had a fixed sized buffer that was hard to extend and didn't scale on larger systems, which could cause overflows and other issues. Also this fixes a long standing known livelock in the old code where a very high continuous error rate could livelock the reader.

Basic design is as follow:

- One ring buffer for each CPU

+ MCEs are added to corresponding local per-CPU buffer, instead of one big global buffer. Contention/unfairness between CPUs is eleminated.

+ MCE records are read out and removed from per-CPU buffers by mutex protected global reader function. Because there are no many readers in system to contend in most cases.

- Per-CPU ring buffer data structure

+ An array is used to hold MCE records. integer "head" indicates next writing position and integer "tail" indicates next reading position.

+ To distinguish buffer empty and full, head and tail wrap to 0 at MCE_LOG_LIMIT instead of MCE_LOG_LEN. Then the real next writing position is head % MCE_LOG_LEN, and real next reading position is tail % MCE_LOG_LEN. If buffer is empty, head == tail, if buffer is full, head % MCE_LOG_LEN == tail % MCE_LOG_LEN and head != tail.

- Lock-less for writer side

+ MCE log writer may come from NMI, so the writer side must be lock-less. For per-CPU buffer of one CPU, writers may come from process, IRQ or NMI context, so "head" is increased with cmpxchg_local() to allocate buffer space.

+ Reader side is protected with a mutex to guarantee only one reader is active in the whole system.

Performance test show that the throughput of per-CPU mcelog buffer can reach 430k records/s compared with 5.3k records/s for original implementation on a 2-core 2.1GHz Core2 machine.

ChangeLog:

v5:

- Rebased on latest x86-tip.git/mce4.

- Fix parsing mce_log data structure in dumping kernel.

v4:

- Rebased on x86-tip.git x86/mce3 branch

- Fix a synchronization issue about mce.finished

- Fix comment style

v3:

- Use DEFINE_PER_CPU to allocate per_cpu mcelog buffer.

- Use cond_resched() to prevent possible system not reponsing issue for large user buffer.

v2:

- Use alloc_percpu() to allocate per_cpu mcelog buffer.

- Use ndelay to implement witer timeout.

---
 arch/x86/include/asm/mce.h       |   18 +-
 arch/x86/kernel/cpu/mcheck/mce.c |  303 +++++++++++++++++++++++----------------
 2 files changed, 196 insertions(+), 125 deletions(-)
arch/x86/include/asm/mce.h +13 -5
@@ -68,20 +68,28 @@ struct mce {
  * is set.
  */
 
-#define MCE_LOG_LEN 32
+#define MCE_LOG_LEN		32
+#define MCE_LOG_LIMIT		(MCE_LOG_LEN * 2 - 1)
+
+static inline int mce_log_index(int n)
+{
+	return n >= MCE_LOG_LEN ? n - MCE_LOG_LEN : n;
+}
+
+struct mce_log_cpu;
 
 struct mce_log {
-	char signature[12]; /* "MACHINECHECK" */
+	char signature[12]; /* "MACHINECHEC2" */
 	unsigned len;	    /* = MCE_LOG_LEN */
-	unsigned next;
 	unsigned flags;
 	unsigned recordlen;	/* length of struct mce */
-	struct mce entry[MCE_LOG_LEN];
+	unsigned nr_mcelog_cpus;
+	struct mce_log_cpu **mcelog_cpus;
 };
 
 #define MCE_OVERFLOW 0		/* bit 0 in flags means overflow */
 
-#define MCE_LOG_SIGNATURE	"MACHINECHECK"
+#define MCE_LOG_SIGNATURE	"MACHINECHEC2"
 
 #define MCE_GET_RECORD_LEN   _IOR('M', 1, int)
 #define MCE_GET_LOG_LEN      _IOR('M', 2, int)
arch/x86/kernel/cpu/mcheck/mce.c +183 -120

389 lines not on this page · raw message

#1 lore raw
Quoted 2 lines · Huang Ying

Re-implement MCE log ring buffer as per-CPU ring buffer for better scalability.

this should be done using the generic ring-buffer code - see include/linux/ring_buffer.h. Having a private implementation in architecture code is not really acceptable.

Ingo

author
#2 lore raw
Quoted 8 lines · Ingo Molnar
Quoted 2 lines · Huang Ying

Re-implement MCE log ring buffer as per-CPU ring buffer for better scalability.

this should be done using the generic ring-buffer code - see include/linux/ring_buffer.h. Having a private implementation in architecture code is not really acceptable.

MCE log ring buffer need to be NMI-safe, while current generic ring-buffer code is not NMI safe. Maybe we can use the generic ring-buffer code after it goes NMI-safe. But for now, I think this patch is better.

Best Regards,
Huang Ying

#3 lore raw
Quoted 6 lines · Huang Ying
Quoted 5 lines · Ingo Molnar
Quoted 2 lines · Huang Ying

Re-implement MCE log ring buffer as per-CPU ring buffer for better scalability.

Quoted 6 lines
Quoted 3 lines

this should be done using the generic ring-buffer code - see include/linux/ring_buffer.h. Having a private implementation in architecture code is not really acceptable.

MCE log ring buffer need to be NMI-safe, while current generic ring-buffer code is not NMI safe. [...]

Then please work with Steve to make it NMI safe (or do so yourself)
- i've Cc:-ed him - that way we help the core kernel: one of its essential facilities will get one more user. We dont put private, open-coded ring-buffer facilities into architecture code.

Ingo

author
#4 lore raw
Quoted 12 lines · Ingo Molnar
Quoted 10 lines · Huang Ying
Quoted 8 lines · Ingo Molnar
Quoted 2 lines · Huang Ying

Re-implement MCE log ring buffer as per-CPU ring buffer for better scalability.

this should be done using the generic ring-buffer code - see include/linux/ring_buffer.h. Having a private implementation in architecture code is not really acceptable.

Quoted 7 lines
Quoted 2 lines

MCE log ring buffer need to be NMI-safe, while current generic ring-buffer code is not NMI safe. [...]

Then please work with Steve to make it NMI safe (or do so yourself)
- i've Cc:-ed him - that way we help the core kernel: one of its essential facilities will get one more user. We dont put private, open-coded ring-buffer facilities into architecture code.

Steve have said he has a NMI-safe solution, but it is under patent application. It seems that patent application will take very long time, so we decide to enhance mce log buffer instead of waiting.

Best Regards,
Huang Ying

#5 lore raw
Quoted 16 lines · Huang Ying
Quoted 15 lines · Ingo Molnar
Quoted 12 lines · Huang Ying
Quoted 8 lines · Ingo Molnar
Quoted 2 lines · Huang Ying

Re-implement MCE log ring buffer as per-CPU ring buffer for better scalability.

this should be done using the generic ring-buffer code - see include/linux/ring_buffer.h. Having a private implementation in architecture code is not really acceptable.

MCE log ring buffer need to be NMI-safe, while current generic ring-buffer code is not NMI safe. [...]

Quoted 8 lines
Quoted 4 lines

Then please work with Steve to make it NMI safe (or do so yourself)
- i've Cc:-ed him - that way we help the core kernel: one of its essential facilities will get one more user. We dont put private, open-coded ring-buffer facilities into architecture code.

Steve have said he has a NMI-safe solution, but it is under patent application. It seems that patent application will take very long time, so we decide to enhance mce log buffer instead of waiting.

Nah - if patent applications held up releases of code and products up to the point where a patent issues then Intel wouldnt be able to release a new CPU for a decade or so ;-)

Anyway, i'll let Steve clarify the status of the NMI-safe ringbuffer patches.

Ingo

#6 lore raw
In reply to Huang Ying
Quoted 4 lines · Huang Ying

Steve have said he has a NMI-safe solution, but it is under patent application. It seems that patent application will take very long time, so we decide to enhance mce log buffer instead of waiting.

Note, the application was finished last week. The wait now is for me to port the code back to the latest kernel (things has changed since then). I've just been too busy trying to get other things working. I'll get the code out this week.

-- Steve

#7 lore raw
In reply to Huang Ying
Quoted 2 lines · Huang Ying

Re-implement MCE log ring buffer as per-CPU ring buffer for better scalability.

It seems your patch implicitly includes some bug fix for tip/mce3. Still discussion of generic lockless ring buffer is going on, so how about dividing the patch into 2 pieces, for improvement and for bug fix?

Maybe improvement part need to be reworked, so I pulled out the bug fix part from your patch.

If it's all right, I'd like to put this on top of tip/mce3 asap.

Thanks,
H.Seto

==

[PATCH] x86, mce: fix mce printing

- Add print_mce_head() instead of first flag
- Make header to be printed always
- Stop double printing of corrected errors

[ This portion is originate from Huang Ying's patch ]

---
 arch/x86/kernel/cpu/mcheck/mce.c |   26 +++++++++++++++-----------
 1 files changed, 15 insertions(+), 11 deletions(-)
arch/x86/kernel/cpu/mcheck/mce.c +15 -11
index d4e7b59..6a3127e 100644
@@ -180,12 +180,8 @@ void mce_log(struct mce *mce)
 	set_bit(0, &notify_user);
 }
 
-static void print_mce(struct mce *m, int *first)
+static void print_mce(struct mce *m)
 {
-	if (*first) {
-		printk(KERN_EMERG "\n" KERN_EMERG "HARDWARE ERROR\n");
-		*first = 0;
-	}
 	printk(KERN_EMERG
 	       "CPU %d: Machine Check Exception: %16Lx Bank %d: %016Lx\n",
 	       m->extcpu, m->mcgstatus, m->bank, m->status);
@@ -208,6 +204,11 @@ static void print_mce(struct mce *m, int *first)
 			m->apicid);
 }
 
+static void print_mce_head(void)
+{
+	printk(KERN_EMERG "\n" KERN_EMERG "HARDWARE ERROR\n");
+}
+
 static void print_mce_tail(void)
 {
 	printk(KERN_EMERG "This is not a software problem!\n"
@@ -234,7 +235,6 @@ static void wait_for_panic(void)
 static void mce_panic(char *msg, struct mce *final, char *exp)
 {
 	int i;
-	int first = 1;
 
 	/*
 	 * Make sure only one CPU runs in machine check panic
@@ -245,23 +245,27 @@ static void mce_panic(char *msg, struct mce *final, char *exp)
 
 	bust_spinlocks(1);
 	console_verbose();
+	print_mce_head();
 	/* First print corrected ones that are still unlogged */
 	for (i = 0; i < MCE_LOG_LEN; i++) {
 		struct mce *m = &mcelog.entry[i];
-		if ((m->status & MCI_STATUS_VAL) &&
-			!(m->status & MCI_STATUS_UC))
-			print_mce(m, &first);
+		if (!(m->status & MCI_STATUS_VAL))
+			continue;
+		if (!(m->status & MCI_STATUS_UC))
+			print_mce(m);
 	}
 	/* Now print uncorrected but with the final one last */
 	for (i = 0; i < MCE_LOG_LEN; i++) {
 		struct mce *m = &mcelog.entry[i];
 		if (!(m->status & MCI_STATUS_VAL))
 			continue;
+		if (!(m->status & MCI_STATUS_UC))
+			continue;
 		if (!final || memcmp(m, final, sizeof(struct mce)))
-			print_mce(m, &first);
+			print_mce(m);
 	}
 	if (final)
-		print_mce(final, &first);
+		print_mce(final);
 	if (cpu_missing)
 		printk(KERN_EMERG "Some CPUs didn't answer in synchronization\n");
 	print_mce_tail();
Signature · 1 line
1.6.3
author
#8 lore raw
Quoted 4 lines · Hidetoshi Seto
Quoted 2 lines · Huang Ying

Re-implement MCE log ring buffer as per-CPU ring buffer for better scalability.

Quoted 8 lines

It seems your patch implicitly includes some bug fix for tip/mce3. Still discussion of generic lockless ring buffer is going on, so how about dividing the patch into 2 pieces, for improvement and for bug fix?

Maybe improvement part need to be reworked, so I pulled out the bug fix part from your patch.

If it's all right, I'd like to put this on top of tip/mce3 asap.

It is OK for me.

Best Regards,
Huang Ying