Skip to contentSkip to conversation

x86: MCE: Re-implement MCE log ring buffer as per-CPU ring buffer

PATCH In review Huang Ying 4 participants 7 messages Apr 28, 2009 – Apr 29, 2009 x86

lore
Share & tools
lore

Huang Ying

author
#0 lore raw

PATCH 1/1 x86: MCE: Re-implement MCE log ring buffer as per-CPU ring buffer

Re-implement MCE log ring buffer as per-CPU ring buffer for better scalability. Basic design is as follow:

- One ring buffer for each CPU

+ MCEs are added to corresponding local per-CPU buffer, instead of one big global buffer. Contention/unfairness between CPUs is eleminated.

+ MCE records are read out and removed from per-CPU buffers by mutex protected global reader function. Because there are no many readers in system to contend in most cases.

- Per-CPU ring buffer data structure

+ An array is used to hold MCE records. integer "head" indicates next writing position and integer "tail" indicates next reading position.

+ To distinguish buffer empty and full, head and tail wrap to 0 at MCE_LOG_LIMIT instead of MCE_LOG_LEN. Then the real next writing position is head % MCE_LOG_LEN, and real next reading position is tail % MCE_LOG_LEN. If buffer is empty, head == tail, if buffer is full, head % MCE_LOG_LEN == tail % MCE_LOG_LEN and head != tail.

- Lock-less for writer side

+ MCE log writer may come from NMI, so the writer side must be lock-less. For per-CPU buffer of one CPU, writers may come from process, IRQ or NMI context, so "head" is increased with cmpxchg_local() to allocate buffer space.

+ Reader side is protected with a mutex to guarantee only one reader is active in the whole system.

Performance test show that the throughput of per-CPU mcelog buffer can reach 430k records/s compared with 5.3k records/s for original implementation on a 2-core 2.1GHz Core2 machine.

ChangeLog:

v2:

- Use alloc_percpu() to allocate per_cpu mcelog buffer

- Use ndelay to implement witer timeout

---
 arch/x86/include/asm/mce.h          |   13 +
 arch/x86/kernel/cpu/mcheck/mce_64.c |  257 ++++++++++++++++++++++--------------
 2 files changed, 169 insertions(+), 101 deletions(-)
arch/x86/include/asm/mce.h +10 -3
@@ -50,15 +50,22 @@ struct mce {
  * is set.
  */
 
-#define MCE_LOG_LEN 32
+#define MCE_LOG_LEN		32
+#define MCE_LOG_LIMIT		(MCE_LOG_LEN * 2 - 1)
+
+static inline int mce_log_index(int n)
+{
+	return n >= MCE_LOG_LEN ? n - MCE_LOG_LEN : n;
+}
+
+struct mce_log_cpu;
 
 struct mce_log {
 	char signature[12]; /* "MACHINECHECK" */
 	unsigned len;	    /* = MCE_LOG_LEN */
-	unsigned next;
 	unsigned flags;
 	unsigned pad0;
-	struct mce entry[MCE_LOG_LEN];
+	struct mce_log_cpu *mcelog_cpus;
 };
 
 #define MCE_OVERFLOW 0		/* bit 0 in flags means overflow */
arch/x86/kernel/cpu/mcheck/mce_64.c +159 -98

326 lines not on this page · raw message

#1 lore raw
Quoted 6 lines · Huang Ying

ChangeLog:

v2:

- Use alloc_percpu() to allocate per_cpu mcelog buffer

Sorry, why didn't you just use DEFINE_PER_CPU ? That should work as well and will be shorter.

Another thing I noticed. the "MACHINECHECK" signature was originally for crash dump tools to find the log. If you change the format you should change it to MACHINECHEC2 or so.

arch/x86/kernel/cpu/mcheck/mce_64.c static int mce_release(struct inode *ino · line 824

Quoted 6 lines
arch/x86/kernel/cpu/mcheck/mce_64.c +6 -0
+	size_t usize_limit;
+
+	/* Too large user buffer size may cause system not response */
+	usize_limit = num_possible_cpus() * MCE_LOG_LEN * sizeof(struct mce);
+	if (usize > usize_limit)
+		usize = usize_limit;

Did you ever track down what happens here? I still find it worrying

Otherwise looks good.

-Andi

Signature · 1 line
[email protected] -- Speaking for myself only.
author
#2 lore raw
Quoted 10 lines · Andi Kleen
Quoted 6 lines · Huang Ying

ChangeLog:

v2:

- Use alloc_percpu() to allocate per_cpu mcelog buffer

Sorry, why didn't you just use DEFINE_PER_CPU ? That should work as well and will be shorter.

OK. I can do that.

Quoted 3 lines

Another thing I noticed. the "MACHINECHECK" signature was originally for crash dump tools to find the log. If you change the format you should change it to MACHINECHEC2 or so.

Oh, thanks, I will change this.

Quoted 8 lines

arch/x86/kernel/cpu/mcheck/mce_64.c static int mce_release(struct inode *ino · line 824

Quoted 6 lines
arch/x86/kernel/cpu/mcheck/mce_64.c +6 -0
+	size_t usize_limit;
+
+	/* Too large user buffer size may cause system not response */
+	usize_limit = num_possible_cpus() * MCE_LOG_LEN * sizeof(struct mce);
+	if (usize > usize_limit)
+		usize = usize_limit;

Did you ever track down what happens here? I still find it worrying

Maybe the comment is a little confusing. What we do is limit the size of user buffer size to prevent system from not response. If the user buffer size is limited, the time taken by mce_read() can be reasonable, and will not cause system not to response.

But maybe num_possible_cpus() * MCE_LOG_LEN is not a good upper limit, because possible cpus can be fairly large. Maybe we can add another limit, something as follow:

#define MCE_READ_RECORD_LIMIT 512

/*
* Too large user buffer size may cause system not response, so constrain
* the effective size of user buffer
*/
usize_limit = num_possible_cpus() * MCE_LOG_LEN * sizeof(struct mce); if (usize_limit > MCE_RECOARD_LIMIT * sizeof(struct mce))

	usize_limit = MCE_RECOARD_LIMIT * sizeof(struct mce);
if (usize > usize_limit)
	usize = usize_limit;

Best Regards,
Huang Ying

#3 lore raw
Quoted 2 lines

Maybe the comment is a little confusing. What we do is limit the size of user buffer size to prevent system from not response. If the user buffer

Ah. The standard way to handle this is to add cond_resched()s where appropiate. Maybe that would be an alternative or is that not possible? You have to drop locks of course.

-Andi

Signature · 1 line
[email protected] -- Speaking for myself only.
author
#4 lore raw
Quoted 6 lines · Andi Kleen
Quoted 2 lines

Maybe the comment is a little confusing. What we do is limit the size of user buffer size to prevent system from not response. If the user buffer

Ah. The standard way to handle this is to add cond_resched()s where appropiate. Maybe that would be an alternative or is that not possible? You have to drop locks of course.

Yes, I will use cond_resched() instead.

Best Regards,
Huang Ying

#5 lore raw
In reply to Huang Ying
Quoted 32 lines · Huang Ying

Re-implement MCE log ring buffer as per-CPU ring buffer for better scalability. Basic design is as follow:

- One ring buffer for each CPU

+ MCEs are added to corresponding local per-CPU buffer, instead of one big global buffer. Contention/unfairness between CPUs is eleminated.

+ MCE records are read out and removed from per-CPU buffers by mutex protected global reader function. Because there are no many readers in system to contend in most cases.

- Per-CPU ring buffer data structure

+ An array is used to hold MCE records. integer "head" indicates next writing position and integer "tail" indicates next reading position.

+ To distinguish buffer empty and full, head and tail wrap to 0 at MCE_LOG_LIMIT instead of MCE_LOG_LEN. Then the real next writing position is head % MCE_LOG_LEN, and real next reading position is tail % MCE_LOG_LEN. If buffer is empty, head == tail, if buffer is full, head % MCE_LOG_LEN == tail % MCE_LOG_LEN and head != tail.

- Lock-less for writer side

+ MCE log writer may come from NMI, so the writer side must be lock-less. For per-CPU buffer of one CPU, writers may come from process, IRQ or NMI context, so "head" is increased with cmpxchg_local() to allocate buffer space.

Quoted 7 lines

+ Reader side is protected with a mutex to guarantee only one reader is active in the whole system.

Performance test show that the throughput of per-CPU mcelog buffer can reach 430k records/s compared with 5.3k records/s for original implementation on a 2-core 2.1GHz Core2 machine.

We're talking about Machine Check Exceptions here, right? Is there a valid scenario where you care about performance? I always thought that an MCE meant something seriously went wrong, log the event and reboot the machine -- possibly start ordering replacement parts.

But now you're saying we want to be able to record more than 5.3k events a second on this? Sounds daft to me.

Also, it sounds like something that might fit the ftrace ringbuffer thingy.

#6 lore raw
Quoted 2 lines

We're talking about Machine Check Exceptions here, right?

No, this is actually more about machine check events (CMCI or poll), not int #18.

Quoted 2 lines

Is there a
valid scenario where you care about performance?

Yes corrected errors. They just get reported while the system keep running.

In practice they can happen in bursts for a relatively short time. Another case was dumb QA setups where people using high frequency injection of corrected errors (that was one case the old code didn't handle too well). Yes I would agree with you that long term high frequency is not an interesting case still.

However there's another reason why this patchkit is very useful. The old code didn't scale to a large number of CPUs, that is why the per CPU buffers are needed.

Quoted 2 lines

Also, it sounds like something that might fit the ftrace ringbuffer thingy.

We went over this before. Please read this relevant threads.

-Andi