Skip to contentSkip to conversation

x86: Intel microcode loader performance improvement

PATCH In review Dimitri Sivanich 4 participants 5 messages Mar 5, 2010 – Mar 11, 2010 x86

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 x86: Intel microcode loader performance improvement

We've noticed that on large SGI UV system configurations, running microcode.ctl can take very long periods of time. This is due to the large number of vmalloc/vfree calls made by the Intel generic_load_microcode() logic.

By reusing allocated space, the following patch reduces the time to run microcode.ctl on a 1024 cpu system from approximately 80 seconds down to 1 or 2 seconds.

---

 arch/x86/kernel/microcode_intel.c |   22 ++++++++++++++++------
 1 file changed, 16 insertions(+), 6 deletions(-)
arch/x86/kernel/microcode_intel.c +16 -6
Index: linux/arch/x86/kernel/microcode_intel.c
===================================================================
@@ -343,10 +343,11 @@ static enum ucode_state generic_load_mic
 				int (*get_ucode_data)(void *, const void *, size_t))
 {
 	struct ucode_cpu_info *uci = ucode_cpu_info + cpu;
-	u8 *ucode_ptr = data, *new_mc = NULL, *mc;
+	u8 *ucode_ptr = data, *new_mc = NULL, *mc = NULL;
 	int new_rev = uci->cpu_sig.rev;
 	unsigned int leftover = size;
 	enum ucode_state state = UCODE_OK;
+	unsigned int curr_mc_size = 0;
 
 	while (leftover) {
 		struct microcode_header_intel mc_header;
@@ -361,9 +362,15 @@ static enum ucode_state generic_load_mic
 			break;
 		}
 
-		mc = vmalloc(mc_size);
-		if (!mc)
-			break;
+		/* For performance reasons, reuse mc area when possible */
+		if (!mc || mc_size > curr_mc_size) {
+			if (mc)
+				vfree(mc);
+			mc = vmalloc(mc_size);
+			if (!mc)
+				break;
+			curr_mc_size = mc_size;
+		}
 
 		if (get_ucode_data(mc, ucode_ptr, mc_size) ||
 		    microcode_sanity_check(mc) < 0) {
@@ -376,13 +383,16 @@ static enum ucode_state generic_load_mic
 				vfree(new_mc);
 			new_rev = mc_header.rev;
 			new_mc  = mc;
-		} else
-			vfree(mc);
+			mc = NULL;	/* trigger new vmalloc */
+		}
 
 		ucode_ptr += mc_size;
 		leftover  -= mc_size;
 	}
 
+	if (mc)
+		vfree(mc);
+
 	if (leftover) {
 		if (new_mc)
 			vfree(new_mc);
Acked-by on v1 1/1
#1 lore raw
Quoted 10 lines · Dimitri Sivanich

We've noticed that on large SGI UV system configurations, running microcode.ctl can take very long periods of time. This is due to the large number of vmalloc/vfree calls made by the Intel generic_load_microcode() logic.

By reusing allocated space, the following patch reduces the time to run microcode.ctl on a 1024 cpu system from approximately 80 seconds down to 1 or 2 seconds.

This approach seems reasonable in the scope of the current framework.

However, I think a better approach would be to have some kind of shared storage for loaded microcode updates. Given that for the majority of SMP systems all the cpus are normally updated to the very same new instance of microcode, it should be enough to do a search for the first cpu, cache the instance of microcode and then reuse it for others.

-- Dmitry

#2 lore raw
Quoted 15 lines · Dmitry Adamushko
Quoted 11 lines · Dimitri Sivanich

We've noticed that on large SGI UV system configurations, running microcode.ctl can take very long periods of time. This is due to the large number of vmalloc/vfree calls made by the Intel generic_load_microcode() logic.

By reusing allocated space, the following patch reduces the time to run microcode.ctl on a 1024 cpu system from approximately 80 seconds down to 1 or 2 seconds.

This approach seems reasonable in the scope of the current framework.

Quoted 10 lines

However, I think a better approach would be to have some kind of shared storage for loaded microcode updates. Given that for the majority of SMP systems all the cpus are normally updated to the very same new instance of microcode, it should be enough to do a search for the first cpu, cache the instance of microcode and then reuse it for others.

And/or update processors in parallel.

Signature · 1 line
error compiling committee.c: too many arguments to function
#3 lore raw
In reply to Dmitry Adamushko
Quoted 14 lines · Dmitry Adamushko
Quoted 10 lines · Dimitri Sivanich

We've noticed that on large SGI UV system configurations, running microcode.ctl can take very long periods of time. This is due to the large number of vmalloc/vfree calls made by the Intel generic_load_microcode() logic.

By reusing allocated space, the following patch reduces the time to run microcode.ctl on a 1024 cpu system from approximately 80 seconds down to 1 or 2 seconds.

This approach seems reasonable in the scope of the current framework.

Quoted 9 lines

However, I think a better approach would be to have some kind of shared storage for loaded microcode updates. Given that for the majority of SMP systems all the cpus are normally updated to the very same new instance of microcode, it should be enough to do a search for the first cpu, cache the instance of microcode and then reuse it for others.

The assumption that all CPUs are the same is not always true in practice, people buy a system and don't always fully populate initially, and when they add processors, they have a more recent stepping. So reusing microcode or updating in parallel would add complexity, and 2 sec for 1024 CPUs puts a pretty low upper bound on possible improvement. Does more improvement to a one time small delay justify additional complexity?

Systems that size are probably not booted all that often. Something to consider before putting a lot of effort into it, I think.

Signature · 3 lines
Bill Davidsen <[email protected]>
  "We have more to fear from the bungling of the incompetent than from
the machinations of the wicked."  - from Slashdot
#4 lore raw

[tip:x86/microcode] x86: Improve Intel microcode loader performance

In reply to Dimitri Sivanich
Commit-ID:  938179b4f8cf8a4f11234ebf2dff2eb48400acfe
Gitweb:     http://git.kernel.org/tip/938179b4f8cf8a4f11234ebf2dff2eb48400acfe
Author:     Dimitri Sivanich <[email protected]>
AuthorDate: Fri, 5 Mar 2010 11:42:03 -0600
Committer:  Ingo Molnar <[email protected]>
CommitDate: Thu, 11 Mar 2010 13:49:06 +0100

x86: Improve Intel microcode loader performance

We've noticed that on large SGI UV system configurations, running microcode.ctl can take very long periods of time. This is due to the large number of vmalloc/vfree calls made by the Intel generic_load_microcode() logic.

By reusing allocated space, the following patch reduces the time to run microcode.ctl on a 1024 cpu system from approximately 80 seconds down to 1 or 2 seconds.

LKML-Reference: <[email protected]>

---
 arch/x86/kernel/microcode_intel.c |   22 ++++++++++++++++------
 1 files changed, 16 insertions(+), 6 deletions(-)
arch/x86/kernel/microcode_intel.c +16 -6
index 85a343e..3561702 100644
@@ -343,10 +343,11 @@ static enum ucode_state generic_load_microcode(int cpu, void *data, size_t size,
 				int (*get_ucode_data)(void *, const void *, size_t))
 {
 	struct ucode_cpu_info *uci = ucode_cpu_info + cpu;
-	u8 *ucode_ptr = data, *new_mc = NULL, *mc;
+	u8 *ucode_ptr = data, *new_mc = NULL, *mc = NULL;
 	int new_rev = uci->cpu_sig.rev;
 	unsigned int leftover = size;
 	enum ucode_state state = UCODE_OK;
+	unsigned int curr_mc_size = 0;
 
 	while (leftover) {
 		struct microcode_header_intel mc_header;
@@ -361,9 +362,15 @@ static enum ucode_state generic_load_microcode(int cpu, void *data, size_t size,
 			break;
 		}
 
-		mc = vmalloc(mc_size);
-		if (!mc)
-			break;
+		/* For performance reasons, reuse mc area when possible */
+		if (!mc || mc_size > curr_mc_size) {
+			if (mc)
+				vfree(mc);
+			mc = vmalloc(mc_size);
+			if (!mc)
+				break;
+			curr_mc_size = mc_size;
+		}
 
 		if (get_ucode_data(mc, ucode_ptr, mc_size) ||
 		    microcode_sanity_check(mc) < 0) {
@@ -376,13 +383,16 @@ static enum ucode_state generic_load_microcode(int cpu, void *data, size_t size,
 				vfree(new_mc);
 			new_rev = mc_header.rev;
 			new_mc  = mc;
-		} else
-			vfree(mc);
+			mc = NULL;	/* trigger new vmalloc */
+		}
 
 		ucode_ptr += mc_size;
 		leftover  -= mc_size;
 	}
 
+	if (mc)
+		vfree(mc);
+
 	if (leftover) {
 		if (new_mc)
 			vfree(new_mc);